credit.preblock.fill_values

Attributes

logger
_OPS

Classes

FillValues

Replace values matching a set of rules with constant fill values for selected variables.

Module Contents

credit.preblock.fill_values.logger

credit.preblock.fill_values._OPS

class credit.preblock.fill_values.FillValues(rules: list[dict], variables: list[str] = None, data_types: list[str] = None)

Bases: credit.preblock.base.BasePreblock

Replace values matching a set of rules with constant fill values for selected variables.

Walks a nested batch dict of the form batch[data_type][source][var_key] and applies each rule as a search-and-replace pass. All masks are computed on the original tensor before any replacement (simultaneous semantics), so earlier rules do not affect later rules’ matches.

Each rule is a dict with:

• search: the string "nan" (matches NaN) or a float (used with op).

• op: comparison operator — "==", "!=", "<", "<=", ">", ">=" (default "=="; ignored when search is "nan").

• fill: the replacement value.

Numeric ops (search is a float) never match NaN positions — use search: "nan" to explicitly target NaN values.

If two rules match the same position, the last rule in the list wins. A warning is logged on the first forward pass if any overlap is detected.

Config example:

type: "fill_values"
args:
    rules:
        - search: nan        # NaN       → -1.0
          fill: -1.0
        - search: 0.0        # == 0.0    → 1.0e-4
          op: "=="
          fill: 1.0e-4
        - search: 0.0        # < 0.0     → 0.0  (clamp negatives)
          op: "<"
          fill: 0.0
    variables:               # optional — defaults to all variables
        - "era5/prognostic/3d/Q"
    data_types:              # optional — defaults to ['input', 'target']
        - "input"
        - "target"

rules

variables = []

variables_expanded = False

_overlap_checked = False

data_types = ['input', 'target']

_check_overlaps(masks: list[tuple]) → None

Warn if any two masks overlap (both True at the same position).

Called once on the first forward pass. When rules overlap, the last matching rule wins because replacements are applied sequentially.

forward(batch: dict) → dict