credit.transforms

credit.transforms#

Submodules#

Attributes#

logger

Classes#

`NormalizeLES`
`ToTensorLES`
`NormalizeWRF`
`ToTensorWRF`
`Normalize_ERA5_and_Forcing`	Class to normalize ERA5 and Forcing Datasets.
`ToTensor_ERA5_and_Forcing`	Class to convert ERA5 and Forcing Datasets to torch tensor.
`BridgescalerScaleState`	Convert to rescaled tensor using Bridgescaler.
`NormalizeState_Quantile_Bridgescalar`	Class to use the bridgescaler Quantile functionality.
`ToTensor_BridgeScaler`	Convert to reshaped tensor.

Functions#

load_transforms(conf[, scaler_only])

Load transforms.

Package Contents#

class credit.transforms.NormalizeLES(conf)#

mean_ds#

std_ds#

mean_tensors#

std_tensors#

levels#

varname_upper_air#

num_upper_air#

flag_surface#

flag_dyn_forcing#

flag_diagnostic#

flag_forcing#

flag_static#

__call__(sample, inverse: bool = False)#

transform_array(x: torch.Tensor) → torch.Tensor#: This function applies to y_pred, so there won’t be dynamic forcing, forcing, and static variables.

transform(sample: Dict[str, numpy.ndarray]) → Dict[str, numpy.ndarray]#

This function transforms training batches, it handles forcing & static as follows:

forcing & static don’t need to be transformed; users should transform them and save them to the file
other variables (upper-air, surface, dynamic forcing, diagnostics) need to be transformed

inverse_transform(x: torch.Tensor) → torch.Tensor#: This function applies to y_pred, so there won’t be dynamic forcing, forcing, and static vars

inverse_transform_input(x: torch.Tensor) → torch.Tensor#: Inverse transform for input x. Forcing and static variables are not transformed (they were not transformed in the transform function).

class credit.transforms.ToTensorLES(conf)#

conf#

output_dtype = Ellipsis#

hist_len#

for_len#

flag_surface#

flag_dyn_forcing#

flag_diagnostic#

flag_forcing#

flag_static#

varname_upper_air#

num_forcing_static = 0#

__call__(sample)#

class credit.transforms.NormalizeWRF(conf)#

mean_ds#

std_ds#

mean_tensors#

std_tensors#

levels#

varname_upper_air#

num_upper_air#

flag_surface#

flag_dyn_forcing#

flag_diagnostic#

flag_forcing#

flag_static#

mean_ds_outside#

std_ds_outside#

mean_tensors_outside#

std_tensors_outside#

levels_outside#

varname_upper_air_outside#

num_upper_air_outside#

flag_surface_outside#

__call__(sample, inverse: bool = False)#

transform_array(x: torch.Tensor) → torch.Tensor#: This function applies to y_pred, so there won’t be boundary input, forcing, and static variables.

transform(sample: Dict[str, numpy.ndarray]) → Dict[str, numpy.ndarray]#

This function transforms training batches

forcing & static don’t need to be transformed; users should transform them and save them to the file
other variables need to be transformed

inverse_transform(x: torch.Tensor) → torch.Tensor#: This function applies to y_pred, so there won’t be dynamic forcing, forcing, and static vars

class credit.transforms.ToTensorWRF(conf)#

conf#

output_dtype = Ellipsis#

hist_len#

for_len#

flag_surface#

flag_dyn_forcing#

flag_diagnostic#

flag_forcing#

flag_static#

varname_upper_air#

num_forcing_static = 0#

hist_len_outside#

for_len_outside#

flag_surface_outside#

varname_upper_air_outside#

__call__(sample)#

class credit.transforms.Normalize_ERA5_and_Forcing(conf)#

Class to normalize ERA5 and Forcing Datasets.

mean_ds#

std_ds#

mean_tensors#

std_tensors#

levels#

varname_upper_air#

num_upper_air#

flag_surface#

flag_dyn_forcing#

flag_diagnostic#

flag_forcing#

flag_static#

__call__(sample: credit.data.Sample, inverse: bool = False) → credit.data.Sample#

Normalize ERA5 and Forcing.

Parameters:

sample – batch.
inverse – whether to transform or inverse transform the sample.

Returns:

transformed and normalized sample.

Return type:

torch.tensor

transform_dataset(DS: xarray.Dataset) → xarray.Dataset#

transform_array(x: torch.Tensor) → torch.Tensor#

Transform of y_pred.

Transform via provided scaler file/s of the prediction variable. Dynamic forcing, forcing, and static vars not transformed.

Parameters:: x – batch.
Returns:: transformed x.

transform(sample: Dict[str, numpy.ndarray]) → Dict[str, numpy.ndarray]#

Transform training batches.

Transform handles forcing & static as follows: - forcing & static don’t need to be transformed; users should transform them and save them to the file - other variables (upper-air, surface, dynamic forcing, diagnostics) need to be transformed

Parameters:: sample – batch.
Returns:: transformed sample.

inverse_transform(x: torch.Tensor) → torch.Tensor#

Inverse transform of y_pred.

Inverse transform of prediction variable. Dynamic forcing, forcing, and static vars not transformed.

Parameters:: x – batch.
Returns:: inverse transformed x.

inverse_transform_input(x: torch.Tensor) → torch.Tensor#

Inverse transform for input x.

Forcing and static variables are not transformed (they were not transformed in the transform function).

Parameters:: x – batch.
Returns:: transformed x.

_align_coords(DS: xarray.Dataset, ref_ds: xarray.Dataset, rtol: float = 1e-06, atol: float = 1e-08) → xarray.Dataset#: Ensure DS has the same lat/lon as ref_ds (to within tolerance), then re-assign DS.coords to exactly match ref_ds.coords.

inverse_transform_dataset(DS: xarray.Dataset, rtol: float = 1e-06, atol: float = 1e-08) → xarray.Dataset#: Inverse‐transform DS by (DS * std_ds) + mean_ds, after aligning its coordinates to mean_ds/std_ds.

class credit.transforms.ToTensor_ERA5_and_Forcing(conf)#

Class to convert ERA5 and Forcing Datasets to torch tensor.

conf#

output_dtype = Ellipsis#

hist_len#

for_len#

flag_surface#

flag_dyn_forcing#

flag_diagnostic#

flag_forcing#

flag_static#

varname_upper_air#

flag_upper_air = True#

num_forcing_static = 0#

__call__(sample: credit.data.Sample) → credit.data.Sample#

Convert variables to input/output torch tensors.

Parameters:: sample (interator) – batch.
Returns:: converted torch tensor.
Return type:: torch.tensor

class credit.transforms.BridgescalerScaleState(conf)#

Bases: object

Convert to rescaled tensor using Bridgescaler.

scaler_file#

variables#

surface_variables#

n_levels#

var_levels = []#

n_surface_variables#

n_3dvar_levels#

scaler_df#

scaler_3d#

scaler_surf#

inverse_transform(x: torch.Tensor) → torch.Tensor#

Inverse transform.

Parameters:: x – batch.
Returns:: inverse transformed batch.

transform_array(x: torch.Tensor) → torch.Tensor#

Transform.

Parameters:: x – batch.
Returns:: transformed batch.

transform(sample: Dict[str, numpy.ndarray]) → Dict[str, numpy.ndarray]#

Transform.

Parameters:: sample – batch.
Returns:: transformed batch.

class credit.transforms.NormalizeState_Quantile_Bridgescalar(conf)#

Class to use the bridgescaler Quantile functionality.

Some hoops have to be jumped thorugh, and the efficiency could be improved if we were to retrain the bridgescaler.

scaler_file#

variables#

surface_variables#

levels#

scaler_df#

scaler_3ds#

scaler_surfs#

scaler_3d#

scaler_surf#

__call__(sample: credit.data.Sample, inverse: bool = False) → credit.data.Sample#

Normalize via quantile transform with bridgescaler.

Normalize via provided scaler file/s.

Parameters:: sample (iterator) – batch.
Returns:: transformed torch tensor.
Return type:: torch.tensor

inverse_transform(x: torch.Tensor) → torch.Tensor#

Inverse transform.

Inverse transform via provided scaler file/s.

Parameters:: x – batch.
Returns:: inverse transformed torch tensor.

transform(sample)#

Transform.

Transform via provided scaler file/s.

Parameters:: sample (iterator) – batch.
Returns:: transformed torch tensor.
Return type:: torch.Tensor

class credit.transforms.ToTensor_BridgeScaler(conf)#

Convert to reshaped tensor.

conf#

hist_len#

for_len#

variables#

surface_variables#

allvars#

static_variables#

latN#

lonN#

levels#

one_shot#

__call__(sample: credit.data.Sample) → credit.data.Sample#

Convert to reshaped tensor.

Reshape and convert to torch tensor.

Parameters:: sample (interator) – batch.
Returns:: reshaped torch tensor.
Return type:: torch.tensor

credit.transforms.logger#

credit.transforms.load_transforms(conf, scaler_only=False)#

Load transforms.

Parameters:

conf (str) – path to config
scaler_only (bool) – True –> retrun scaler; False –> return scaler and ToTensor

Returns:

transform

Return type:

tf.tensor