credit.metrics.anomaly

credit.metrics.anomaly#

Anomaly-based verification metrics for the Gen 2 metrics framework.

Implements the anomaly correlation coefficient (ACC) and forecast activity (SDAF) as described in Bonavita & Geer (2026), “Forecast verification using information and noise”, Q. J. R. Meteorol. Soc., 152, e70109, https://doi.org/10.1002/qj.70109.

Both metrics require a climatology field — a per-variable spatial mean representing the long-term average state. The climatology is subtracted from forecast and truth before computing anomalies, and the area-weighted mean of the anomaly is removed (debiasing) following the operational practice described in Appendix A of the paper.

Climatology sources (in priority order):

  1. ``climatology_path`` — a path to an xarray-readable dataset (netCDF/Zarr) whose data variables are named by the short name of each var_key (the last path component, e.g. "temperature" for "ERA5/prognostic/3d/temperature"). The dataset must have latitude and longitude coordinates; 3-D variables additionally need a level (or levels) coordinate matching the data config.

  2. Validation data (default) — when no climatology_path is given the climatology is accumulated as a running mean of the target (y_target_processed) fields seen during validation. On the first batch the climatology equals that batch’s target mean; it converges to the full validation mean as more batches are processed. This is an online approximation — for exact results either provide a climatology file or ensure the metric sees all validation data before the scores are logged.

  3. ``climatology`` — a pre-computed {var_key: torch.Tensor} dict supplied programmatically (each tensor broadcastable to the variable’s spatial shape). Overrides both climatology_path and online accumulation.

Config example (combined metric with ACC and activity):

metrics:
  type: combined
  args:
    metrics:
      rmse: {}
      acc:
        climatology_path: /path/to/climatology.nc
      activity:
        climatology_path: /path/to/climatology.nc
    var_weighting: "none"
    use_latitude_weights: true
    latitude_weights: /path/to/static.zarr

Code example:

from credit.metrics.anomaly import AnomalyCorrelationCoefficientMetric

metric = AnomalyCorrelationCoefficientMetric(
    metric_name="acc",
    var_weighting="none",
    use_latitude_weights=True,
    latitude_weights="/path/to/static.zarr",
)
scores = metric(full_data_dict)
# {"acc/ERA5/.../T": 0.95, "acc": 0.92, ...}

Mathematical definitions (following Bonavita & Geer 2026, Appendix A) with area weights w_i normalised to mean 1:

  • Debiased forecast anomaly: d_f = x_f - x_c - mean_w(x_f - x_c)

  • Debiased truth anomaly: d_t = x_t - x_c - mean_w(x_t - x_c)

  • Forecast activity (SDAF): SDAF = sqrt(mean_w(d_f^2))

  • Truth activity (SDAV): SDAV = sqrt(mean_w(d_t^2))

  • ACC: ACC = mean_w(d_f * d_t) / (SDAF * SDAV)

Classes#

AnomalyCorrelationCoefficientMetric

Anomaly correlation coefficient (ACC) per variable.

ForecastActivityMetric

Forecast activity (SDAF) per variable.

Module Contents#

class credit.metrics.anomaly.AnomalyCorrelationCoefficientMetric(*args, climatology_path: str | None = None, climatology: dict | None = None, accumulate_climatology: bool = True, **kwargs)#

Bases: _AnomalyMetricBase

Anomaly correlation coefficient (ACC) per variable.

ACC measures the cosine of the angle between the debiased forecast and truth anomaly vectors in the area-weighted state space (Bonavita & Geer 2026, eq. A8). It is bounded by ±1 and is insensitive to bias and forecast activity.

  • ACC = 1: perfect anomaly pattern.

  • ACC = 0: no correlation with the observed anomaly.

  • ACC < 0: negatively correlated (worse than climatology).

Requires a climatology (see _AnomalyMetricBase).

scale_power = 0#
compute_variable(pred: torch.Tensor, target: torch.Tensor) → torch.Tensor#

Return the elementwise (un-reduced) error tensor for one variable.

The tensor must broadcast against pred (typically the same shape); forward() applies latitude weighting and takes the spatial mean. For example, MSE returns (pred - target) ** 2.

class credit.metrics.anomaly.ForecastActivityMetric(*args, climatology_path: str | None = None, climatology: dict | None = None, accumulate_climatology: bool = True, **kwargs)#

Bases: _AnomalyMetricBase

Forecast activity (SDAF) per variable.

SDAF is the area-weighted standard deviation of the debiased forecast anomaly (Bonavita & Geer 2026, eq. A5). It quantifies how much the forecast deviates from climatology — i.e. how “active” the forecast is. Unrealistically smooth forecasts (e.g. from ML emulators or ensemble averaging) have reduced SDAF.

Requires a climatology (see _AnomalyMetricBase).

scale_power = 1#
compute_variable(pred: torch.Tensor, target: torch.Tensor) → torch.Tensor#

Return the elementwise (un-reduced) error tensor for one variable.

The tensor must broadcast against pred (typically the same shape); forward() applies latitude weighting and takes the spatial mean. For example, MSE returns (pred - target) ** 2.