minerva.optimizers.lars

Classes

LARS	Implements the Layer-wise Adaptive Rate Scaling (LARS) optimizer.

Module Contents

class minerva.optimizers.lars.LARS(params, lr, momentum=0.9, dampening=0, weight_decay=0.9, nesterov=False, trust_coefficient=0.001, eps=1e-08)

Bases: torch.optim.Optimizer

Implements the Layer-wise Adaptive Rate Scaling (LARS) optimizer. Implementation borrowed from lightly SSL library.

Constructs a new LARS optimizer.

Parameters

paramsAny

Parameters to optimize.

lrfloat

Learning rate.

momentumfloat, optional

Momentum factor, by default 0.9

dampeningfloat, optional

Dampening for momentum, by default 0

weight_decayfloat, optional

Weight decay (L2 penalty), by default 0.9

nesterovbool, optional

Enables Nesterov momentum, by default False

trust_coefficientfloat, optional

Trust coefficient for computing learning rate, by default 0.001

epsfloat, optional

Eps for division denominator, by default 1e-8

__setstate__(state)

Parameters:

state (Dict[str, Any])

Return type:

None

step(closure: None = None) → None

step(closure: Callable[[], float]) → float

Performs a single optimization step.

Args:

closure (callable, optional): A closure that reevaluates the model

and returns the loss.

Parameters:

• params (Any)

• lr (float)

• momentum (float)

• dampening (float)

• weight_decay (float)

• nesterov (bool)

• trust_coefficient (float)

• eps (float)