credit.scheduler
================

.. py:module:: credit.scheduler


Attributes
----------

.. autoapisummary::

   credit.scheduler.update_on_batch
   credit.scheduler.update_on_epoch
   credit.scheduler.num_epochs


Classes
-------

.. autoapisummary::

   credit.scheduler.LinearWarmupCosineScheduler
   credit.scheduler.CosineAnnealingWarmupRestarts


Functions
---------

.. autoapisummary::

   credit.scheduler.load_scheduler
   credit.scheduler.lr_lambda_phase2
   credit.scheduler.phased_lr_lambda
   credit.scheduler.lr_lambda_phase1
   credit.scheduler.annealed_probability


Module Contents
---------------

.. py:data:: update_on_batch
   :value: ['cosine-annealing-restarts', 'linear-warmup-cosine']


.. py:data:: update_on_epoch
   :value: ['lambda', 'plateau', 'cosine-annealing']


.. py:class:: LinearWarmupCosineScheduler(optimizer, warmup_steps: int = 1000, total_steps: int = 500000, min_lr: float = 1e-05, last_epoch: int = -1)

   Bases: :py:obj:`torch.optim.lr_scheduler.LRScheduler`


   Linear warmup followed by cosine decay to min_lr.

   Steps are batch-level (call scheduler.step() after each optimizer.step()).

   :param optimizer: Wrapped optimizer.
   :param warmup_steps: Number of steps to linearly ramp lr from 0 to base_lr.
   :param total_steps: Total number of steps (warmup + cosine decay).
   :param min_lr: Minimum learning rate at the end of decay (absolute value).
   :param last_epoch: Used by LRScheduler for resume; -1 means start from scratch.


   .. py:attribute:: warmup_steps
      :value: 1000



   .. py:attribute:: total_steps
      :value: 500000



   .. py:attribute:: min_lr
      :value: 1e-05



   .. py:method:: get_lr()

      Compute the next learning rate for each of the optimizer's
      :attr:`~torch.optim.Optimizer.param_groups`.

      :returns: A :class:`list` of learning rates for each of
                the optimizer's :attr:`~torch.optim.Optimizer.param_groups` with the
                same types as their current ``group["lr"]``\s.
      :rtype: list[float | Tensor]

      .. note::
          If you're trying to inspect the most recent learning rate, use
          :meth:`get_last_lr()` instead.

      .. note::
          The returned :class:`~torch.Tensor`\s are copies, and never alias
          the optimizer's ``group["lr"]``\s.



.. py:function:: load_scheduler(optimizer, conf)

   Load a learning rate scheduler based on the configuration.

   Parameters:
   - optimizer: The PyTorch optimizer.
   - conf: The configuration dictionary.

   Returns:
   - scheduler: The PyTorch learning rate scheduler.


.. py:function:: lr_lambda_phase2(step, total_updates_phase2=299000)

.. py:function:: phased_lr_lambda(step, total_updates_phase1=1000, total_updates_phase2=299000)

.. py:function:: lr_lambda_phase1(epoch, num_epochs=100, warmup_epochs=10)

.. py:class:: CosineAnnealingWarmupRestarts(optimizer: torch.optim.Optimizer, first_cycle_steps: int, cycle_mult: float = 1.0, max_lr: float = 0.1, min_lr: float = 0.001, warmup_steps: int = 0, gamma: float = 1.0, last_epoch: int = -1)

   Bases: :py:obj:`torch.optim.lr_scheduler.LRScheduler`


   optimizer (Optimizer): Wrapped optimizer.
   first_cycle_steps (int): First cycle step size.
   cycle_mult(float): Cycle steps magnification. Default: -1.
   max_lr(float): First cycle's max learning rate. Default: 0.1.
   min_lr(float): Min learning rate. Default: 0.001.
   warmup_steps(int): Linear warmup step size. Default: 0.
   gamma(float): Decrease rate of max learning rate by cycle. Default: 1.
   last_epoch (int): The index of last epoch. Default: -1.


   .. py:attribute:: first_cycle_steps


   .. py:attribute:: cycle_mult
      :value: 1.0



   .. py:attribute:: base_max_lr
      :value: 0.1



   .. py:attribute:: max_lr
      :value: 0.1



   .. py:attribute:: min_lr
      :value: 0.001



   .. py:attribute:: warmup_steps
      :value: 0



   .. py:attribute:: gamma
      :value: 1.0



   .. py:attribute:: cur_cycle_steps


   .. py:attribute:: cycle
      :value: 0



   .. py:attribute:: step_in_cycle
      :value: -1



   .. py:method:: init_lr()


   .. py:method:: get_lr()

      Compute the next learning rate for each of the optimizer's
      :attr:`~torch.optim.Optimizer.param_groups`.

      :returns: A :class:`list` of learning rates for each of
                the optimizer's :attr:`~torch.optim.Optimizer.param_groups` with the
                same types as their current ``group["lr"]``\s.
      :rtype: list[float | Tensor]

      .. note::
          If you're trying to inspect the most recent learning rate, use
          :meth:`get_last_lr()` instead.

      .. note::
          The returned :class:`~torch.Tensor`\s are copies, and never alias
          the optimizer's ``group["lr"]``\s.



   .. py:method:: step(epoch=None)

      Step the scheduler.

      :param epoch:
                    .. deprecated:: 1.4
                        If provided, sets :attr:`last_epoch` to ``epoch`` and uses
                        :meth:`_get_closed_form_lr` if it is available. This is not
                        universally supported. Use :meth:`step` without arguments
                        instead.
      :type epoch: int, optional

      .. note::
          Call this method after calling the optimizer's
          :meth:`~torch.optim.Optimizer.step`.



.. py:function:: annealed_probability(epoch, max_epochs=100, min_probability=0.01, max_probability=1.0)

   Anneal the termination probability from 1 to a small value.

   Parameters:
   - epoch: The current epoch.
   - max_epochs: The maximum number of epochs for annealing.
   - min_probability: The minimum termination probability.
   - max_probability: The maximum termination probability.

   Returns:
   - termination_probability: The annealed termination probability.


.. py:data:: num_epochs
   :value: 100