credit.scheduler
================

.. py:module:: credit.scheduler


Attributes
----------

.. autoapisummary::

   credit.scheduler.update_on_batch
   credit.scheduler.update_on_epoch
   credit.scheduler.num_epochs


Classes
-------

.. autoapisummary::

   credit.scheduler.CosineAnnealingWarmupRestarts


Functions
---------

.. autoapisummary::

   credit.scheduler.load_scheduler
   credit.scheduler.lr_lambda_phase2
   credit.scheduler.phased_lr_lambda
   credit.scheduler.lr_lambda_phase1
   credit.scheduler.annealed_probability


Module Contents
---------------

.. py:data:: update_on_batch
   :value: ['cosine-annealing-restarts']


.. py:data:: update_on_epoch
   :value: ['lambda', 'plateau', 'cosine-annealing']


.. py:function:: load_scheduler(optimizer, conf)

   Load a learning rate scheduler based on the configuration.

   Parameters:
   - optimizer: The PyTorch optimizer.
   - conf: The configuration dictionary.

   Returns:
   - scheduler: The PyTorch learning rate scheduler.


.. py:function:: lr_lambda_phase2(step, total_updates_phase2=299000)

.. py:function:: phased_lr_lambda(step, total_updates_phase1=1000, total_updates_phase2=299000)

.. py:function:: lr_lambda_phase1(epoch, num_epochs=100, warmup_epochs=10)

.. py:class:: CosineAnnealingWarmupRestarts(optimizer: torch.optim.Optimizer, first_cycle_steps: int, cycle_mult: float = 1.0, max_lr: float = 0.1, min_lr: float = 0.001, warmup_steps: int = 0, gamma: float = 1.0, last_epoch: int = -1)

   Bases: :py:obj:`torch.optim.lr_scheduler.LRScheduler`


   optimizer (Optimizer): Wrapped optimizer.
   first_cycle_steps (int): First cycle step size.
   cycle_mult(float): Cycle steps magnification. Default: -1.
   max_lr(float): First cycle's max learning rate. Default: 0.1.
   min_lr(float): Min learning rate. Default: 0.001.
   warmup_steps(int): Linear warmup step size. Default: 0.
   gamma(float): Decrease rate of max learning rate by cycle. Default: 1.
   last_epoch (int): The index of last epoch. Default: -1.


   .. py:attribute:: first_cycle_steps


   .. py:attribute:: cycle_mult
      :value: 1.0



   .. py:attribute:: base_max_lr
      :value: 0.1



   .. py:attribute:: max_lr
      :value: 0.1



   .. py:attribute:: min_lr
      :value: 0.001



   .. py:attribute:: warmup_steps
      :value: 0



   .. py:attribute:: gamma
      :value: 1.0



   .. py:attribute:: cur_cycle_steps


   .. py:attribute:: cycle
      :value: 0



   .. py:attribute:: step_in_cycle
      :value: -1



   .. py:method:: init_lr()


   .. py:method:: get_lr()

      Compute learning rate using chainable form of the scheduler.



   .. py:method:: step(epoch=None)

      Perform a step.



.. py:function:: annealed_probability(epoch, max_epochs=100, min_probability=0.01, max_probability=1.0)

   Anneal the termination probability from 1 to a small value.

   Parameters:
   - epoch: The current epoch.
   - max_epochs: The maximum number of epochs for annealing.
   - min_probability: The minimum termination probability.
   - max_probability: The maximum termination probability.

   Returns:
   - termination_probability: The annealed termination probability.


.. py:data:: num_epochs
   :value: 100