minerva.optimizers

Submodules

Classes

LARS

Implements the Layer-wise Adaptive Rate Scaling (LARS) optimizer.

Package Contents

class minerva.optimizers.LARS(params, lr, momentum=0.9, dampening=0, weight_decay=0.9, nesterov=False, trust_coefficient=0.001, eps=1e-08)[source]

Bases: torch.optim.Optimizer

Implements the Layer-wise Adaptive Rate Scaling (LARS) optimizer. Implementation borrowed from lightly SSL library.

Constructs a new LARS optimizer.

Parameters

paramsAny

Parameters to optimize.

lrfloat

Learning rate.

momentumfloat, optional

Momentum factor, by default 0.9

dampeningfloat, optional

Dampening for momentum, by default 0

weight_decayfloat, optional

Weight decay (L2 penalty), by default 0.9

nesterovbool, optional

Enables Nesterov momentum, by default False

trust_coefficientfloat, optional

Trust coefficient for computing learning rate, by default 0.001

epsfloat, optional

Eps for division denominator, by default 1e-8

__setstate__(state)[source]
Parameters:

state (Dict[str, Any])

Return type:

None

step(closure: None = None) None[source]
step(closure: Callable[[], float]) float

Performs a single optimization step.

Args:
closure (callable, optional): A closure that reevaluates the model

and returns the loss.

Parameters:
  • params (Any)

  • lr (float)

  • momentum (float)

  • dampening (float)

  • weight_decay (float)

  • nesterov (bool)

  • trust_coefficient (float)

  • eps (float)