Skip to content

bgc_data_processing.comparison.metrics

Metrics to evaluate Simulations against observations.

BaseMetric(variables_to_evaluate)

Bases: ABC

Base class to implement metrics.

Parameters:

Name Type Description Default
variables_to_evaluate list[str]

List of the names of the variables to use to evaluate.

 required
Source code in src/bgc_data_processing/comparison/metrics.py 
23
24
def __init__(self, variables_to_evaluate: list[str]) -> None:
    self._eval_vars = variables_to_evaluate

metric_name: str instance-attribute

evaluate(observations, simulations)

Evaluate observations dataframe against simulations.

Parameters:

Name Type Description Default
observations DataFrame

Observations dataframe.

 required
simulations DataFrame

Simulations dataframe.

 required

Returns:

Type Description
Series

Evaluation result, for every column.
Source code in src/bgc_data_processing/comparison/metrics.py 
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
def evaluate(
    self,
    observations: pd.DataFrame,
    simulations: pd.DataFrame,
) -> pd.Series:
    """Evaluate observations dataframe against simulations.

    Parameters
    ----------
    observations : pd.DataFrame
        Observations dataframe.
    simulations : pd.DataFrame
        Simulations dataframe.

    Returns
    -------
    pd.Series
        Evaluation result, for every column.
    """
    result = self._eval(observations=observations, simulations=simulations)
    result.name = self.metric_name
    return result

evaluate_storers(observations_storer, simulations_storer)

Evaluate two storers against each other.

Parameters:

Name Type Description Default
observations_storer Storer

Observations storer.

 required
simulations_storer Storer

Simulations storer.

 required

Returns:

Type Description
Series

Result for every column.

Raises:

Type Description
IncomparableStorersError

If the storers have different shapes.
Source code in src/bgc_data_processing/comparison/metrics.py 
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
def evaluate_storers(
    self,
    observations_storer: Storer,
    simulations_storer: Storer,
) -> pd.Series:
    """Evaluate two storers against each other.

    Parameters
    ----------
    observations_storer : Storer
        Observations storer.
    simulations_storer : Storer
        Simulations storer.

    Returns
    -------
    pd.Series
        Result for every column.

    Raises
    ------
    IncomparableStorersError
        If the storers have different shapes.
    """
    obs_vars = observations_storer.variables
    obs_eval_labels = [obs_vars.get(name).label for name in self._eval_vars]
    sim_vars = simulations_storer.variables
    sim_eval_labels = [sim_vars.get(name).label for name in self._eval_vars]
    obs_df = observations_storer.data.filter(obs_eval_labels, axis=1)
    sim_df = simulations_storer.data.filter(sim_eval_labels, axis=1)

    if obs_df.shape != sim_df.shape:
        error_msg = (
            f"DataFrame shapes don't match(observations: {obs_df.shape} "
            "- simulations: {sim_df.shape}) -> make sure both storer have "
            "the variables to evaluate on"
            f" (variables: {self._eval_vars})"
        )
        raise IncomparableStorersError(error_msg)

    nans = obs_df.isna().all(axis=1) | sim_df.isna().all(axis=1)
    return self.evaluate(observations=obs_df[~nans], simulations=sim_df[~nans])

RMSE(variables_to_evaluate)

Bases: BaseMetric

Root-Mean Square Error (RMSE).

See Also

https://en.wikipedia.org/wiki/Root-mean-square_deviation

Parameters:

Name Type Description Default
variables_to_evaluate list[str]

List of the names of the variables to use to evaluate.

 required
Source code in src/bgc_data_processing/comparison/metrics.py 
116
117
def __init__(self, variables_to_evaluate: list[str]) -> None:
    super().__init__(variables_to_evaluate)

metric_name = 'RMSE' class-attribute instance-attribute

Bias(variables_to_evaluate)

Bases: BaseMetric

Bias.

See Also

https://en.wikipedia.org/wiki/Bias_of_an_estimator

Parameters:

Name Type Description Default
variables_to_evaluate list[str]

List of the names of the variables to use to evaluate.

 required
Source code in src/bgc_data_processing/comparison/metrics.py 
157
158
def __init__(self, variables_to_evaluate: list[str]) -> None:
    super().__init__(variables_to_evaluate)

metric_name = 'Bias' class-attribute instance-attribute