credit.datasets.load_dataset_and_dataloader
===========================================

.. py:module:: credit.datasets.load_dataset_and_dataloader


Attributes
----------

.. autoapisummary::

   credit.datasets.load_dataset_and_dataloader.dataset_id


Classes
-------

.. autoapisummary::

   credit.datasets.load_dataset_and_dataloader.BatchForecastLenSampler
   credit.datasets.load_dataset_and_dataloader.BatchForecastLenSamplerSamudra
   credit.datasets.load_dataset_and_dataloader.BatchForecastLenDataLoader


Functions
---------

.. autoapisummary::

   credit.datasets.load_dataset_and_dataloader.collate_fn
   credit.datasets.load_dataset_and_dataloader.load_dataset
   credit.datasets.load_dataset_and_dataloader.load_dataloader


Module Contents
---------------

.. py:class:: BatchForecastLenSampler(dataset)

   .. py:attribute:: dataset


   .. py:attribute:: forecast_len


   .. py:attribute:: len


   .. py:method:: __iter__()

      Returns an iterator for the sampler.

      The iterator generates a sequence of zeros with a length equal
      to the calculated `len`. This is primarily a placeholder to
      satisfy the interface requirements of a PyTorch Sampler.

      :returns: An iterator over a sequence of zeros.



   .. py:method:: __len__()

      Returns the length of the sampler.

      The length is the total number of iterations based on the forecast
      length and batches per epoch from the dataset.

      :returns: The total number of iterations.
      :rtype: int



.. py:class:: BatchForecastLenSamplerSamudra(dataset)

   Bases: :py:obj:`BatchForecastLenSampler`


   .. py:attribute:: forecast_len


   .. py:attribute:: len


.. py:class:: BatchForecastLenDataLoader(dataset, offset=1)

   .. py:attribute:: dataset


   .. py:attribute:: forecast_len


   .. py:method:: __iter__()

      Iterates over the dataset.

      This method directly yields samples from the dataset. The forecast
       length is not explicitly handled here; it is assumed to be accounted
       for in the dataset's structure or sampling.

      :Yields: *sample* -- A single sample from the dataset.



   .. py:method:: __len__()

      Returns the length of the DataLoader.

      The length is determined by the forecast length and either the
       dataset's `batches_per_epoch()` method (if available) or the dataset's
       overall length.

      :returns: The total number of samples or iterations.
      :rtype: int



.. py:function:: collate_fn(batch)

   Custom collate function for use with the ERA5_MultiStep_Batcher dataset.

   This function ensures that the time and batch dimensions are not flipped
    during data loading. It assumes that the dataset is structured such that
    the first element of the batch contains the correctly formatted data.

   :param batch: A list of samples from the dataset, where each sample
                 is expected to be identically structured.
   :type batch: list

   :returns:

             The first element of the batch, which contains the correctly
                  formatted data.
   :rtype: Any


.. py:function:: load_dataset(conf, rank=0, world_size=1, is_train=True)

   Load the dataset based on the configuration.

   :param conf: Configuration dictionary containing dataset and training parameters.
   :type conf: dict
   :param rank: Rank of the current process. Default is 0.
   :type rank: int, optional
   :param world_size: Number of processes participating in the job. Default is 1.
   :type world_size: int, optional
   :param is_train: Flag indicating whether the dataset is for training or validation.
                    Default is True.
   :type is_train: bool, optional

   :returns: The loaded dataset.
   :rtype: Dataset


.. py:function:: load_dataloader(conf, dataset, rank=0, world_size=1, is_train=True)

   Load the DataLoader based on the dataset type.

   :param conf: Configuration dictionary containing dataloader parameters.
   :type conf: dict
   :param dataset: The dataset to be used in the DataLoader.
   :type dataset: Dataset
   :param rank: Rank of the current process. Default is 0.
   :type rank: int, optional
   :param world_size: Number of processes participating in the job. Default is 1.
   :type world_size: int, optional
   :param is_train: Flag indicating whether the dataset is for training or validation.
                    Default is True.
   :type is_train: bool, optional

   :returns: The loaded DataLoader.
   :rtype: DataLoader


.. py:data:: dataset_id