credit.scheduler

credit.scheduler#

Attributes#

`update_on_batch`
`update_on_epoch`
`num_epochs`

Classes#

CosineAnnealingWarmupRestarts

optimizer (Optimizer): Wrapped optimizer.

Functions#

`load_scheduler`(optimizer, conf)	Load a learning rate scheduler based on the configuration.
`lr_lambda_phase2`(step[, total_updates_phase2])
`phased_lr_lambda`(step[, total_updates_phase1, ...])
`lr_lambda_phase1`(epoch[, num_epochs, warmup_epochs])
`annealed_probability`(epoch[, max_epochs, ...])	Anneal the termination probability from 1 to a small value.

Module Contents#

credit.scheduler.update_on_batch = ['cosine-annealing-restarts']#

credit.scheduler.update_on_epoch = ['lambda', 'plateau', 'cosine-annealing']#

credit.scheduler.load_scheduler(optimizer, conf)#

Load a learning rate scheduler based on the configuration.

Parameters: - optimizer: The PyTorch optimizer. - conf: The configuration dictionary.

Returns: - scheduler: The PyTorch learning rate scheduler.

credit.scheduler.lr_lambda_phase2(step, total_updates_phase2=299000)#

credit.scheduler.phased_lr_lambda(step, total_updates_phase1=1000, total_updates_phase2=299000)#

credit.scheduler.lr_lambda_phase1(epoch, num_epochs=100, warmup_epochs=10)#

class credit.scheduler.CosineAnnealingWarmupRestarts(optimizer: torch.optim.Optimizer, first_cycle_steps: int, cycle_mult: float = 1.0, max_lr: float = 0.1, min_lr: float = 0.001, warmup_steps: int = 0, gamma: float = 1.0, last_epoch: int = -1)#

Bases: torch.optim.lr_scheduler.LRScheduler

optimizer (Optimizer): Wrapped optimizer. first_cycle_steps (int): First cycle step size. cycle_mult(float): Cycle steps magnification. Default: -1. max_lr(float): First cycle’s max learning rate. Default: 0.1. min_lr(float): Min learning rate. Default: 0.001. warmup_steps(int): Linear warmup step size. Default: 0. gamma(float): Decrease rate of max learning rate by cycle. Default: 1. last_epoch (int): The index of last epoch. Default: -1.

first_cycle_steps#

cycle_mult = 1.0#

base_max_lr = 0.1#

max_lr = 0.1#

min_lr = 0.001#

warmup_steps = 0#

gamma = 1.0#

cur_cycle_steps#

cycle = 0#

step_in_cycle = -1#

init_lr()#

get_lr()#: Compute learning rate using chainable form of the scheduler.

step(epoch=None)#: Perform a step.

credit.scheduler.annealed_probability(epoch, max_epochs=100, min_probability=0.01, max_probability=1.0)#

Anneal the termination probability from 1 to a small value.

Parameters: - epoch: The current epoch. - max_epochs: The maximum number of epochs for annealing. - min_probability: The minimum termination probability. - max_probability: The maximum termination probability.

Returns: - termination_probability: The annealed termination probability.

credit.scheduler.num_epochs = 100#