credit.datasets.gen_2.gefs_download

credit.datasets.gen_2.gefs_download#

Download raw GEFS cube-sphere initialization files for local Gen2 loading.

The downloader uses the same source configuration as GEFSDataset and preserves the public GEFS layout:

{base_path}/gefs.YYYYMMDD/HH/atmos/init/{member}/gfs_ctrl.nc
{base_path}/gefs.YYYYMMDD/HH/atmos/init/{member}/gfs_data.tile1.nc
{base_path}/gefs.YYYYMMDD/HH/atmos/init/{member}/sfc_data.tile1.nc

All six atmospheric and six surface tiles are downloaded for every selected member and initialization time. Forecast lead products are not supported;

Command-line usage:

python -m credit.datasets.gen_2.gefs_download -c config/gefs.yml
python -m credit.datasets.gen_2.gefs_download -c config/gefs.yml --num-workers 8 --overwrite

Attributes#

logger

parser

Functions#

download_gefs(→ None)

Download selected GEFS initialization files for local GEFSDataset use.

Module Contents#

credit.datasets.gen_2.gefs_download.logger#
credit.datasets.gen_2.gefs_download.download_gefs(data_config: dict[str, Any], num_workers: int = 4, overwrite: bool = False) → None#

Download selected GEFS initialization files for local GEFSDataset use.

Parameters:

  • data_config – Top-level data configuration with exactly one dataset_type: "gefs" source. The source must define base_path. Omitted members downloads c00; an empty member list discovers all members in the first requested run.

  • num_workers – Number of concurrent file downloads. Defaults to 4.

  • overwrite – Redownload existing files when True. Defaults to False.

Raises:

  • KeyError – If the configuration does not contain a source block.

  • ValueError – If the configuration has multiple sources, the wrong dataset type, no base path, or a forecast lead setting.

  • FileNotFoundError – If any selected member or required tile is missing for a requested initialization.

Example

>>> from credit.datasets.gen_2.gefs_download import download_gefs
>>> download_gefs(config["data"], num_workers=8)
credit.datasets.gen_2.gefs_download.parser#