minerva.models.ssl.fastsiam¶

Classes¶

`FastSiam`	A LightningModule implementation for FastSiam, a self-supervised learning framework.
`SimSiamMLPHead`	A sequential container.

Module Contents¶

class minerva.models.ssl.fastsiam.FastSiam(backbone, in_dim=2048, hid_dim=2048, out_dim=2048, K=3, momentum=0.996, lr=0.001, test_metric=None, num_classes=None)[source]¶

Bases: lightning.LightningModule

A LightningModule implementation for FastSiam, a self-supervised learning framework.

Tris approach for self-supervised learning was proposed by Pototzky et al., (2022) [1] in “FastSiam: Resource-Efficient Self-supervised Learning on a Single GPU”.

[1] Pototzky, D., Sultan, A., Schmidt-Thieme, L. (2022). FastSiam: Resource-Efficient Self-supervised Learning on a Single GPU. In: Andres, B., Bernard, F., Cremers, D., Frintrop, S., Goldlücke, B., Ihrke, I. (eds) Pattern Recognition. DAGM GCPR 2022. Lecture Notes in Computer Science, vol 13485. Springer, Cham. https://doi.org/10.1007/978-3-031-16788-1_4

Parameters¶

backbonenn.Module: The backbone neural network for feature extraction (e.g., ResNet).
in_dimint, optional: Input dimension for the projector network, by default 2048.
hid_dimint, optional: Hidden dimension for the projector and predictor networks, by default 512.
out_dimint, optional: Output dimension for the projector and predictor networks, by default 128.
Kint, optional: Number of target_branch views to generate, by default 3.
momentumfloat, optional: Momentum factor for updating the target_branch, by default 0.996.
lrfloat, optional: Learning rate for the optimizer, by default 1e-3.
test_metricOptional[Callable], optional: A callable to compute the test metric, by default None.
num_classesOptional[int], optional: Number of classes for classification tasks, by default None.

Initialize internal Module state, shared by both nn.Module and ScriptModule.

K = 3¶

_single_step(batch, K, log_prefix)[source]¶

Perform a single training, validation, or test step.

Parameters:

batch (Any)
K (int)
log_prefix (str)

Return type:

torch.Tensor

backbone¶

configure_optimizers()[source]¶

Configure the optimizer for training.

Return type:: torch.optim.Optimizer

ensure_tensor(image)[source]¶

Ensure the input image is a PyTorch tensor with the correct format.

Parameters:: image (torch.Tensor)
Return type:: torch.Tensor

static fastsiam_loss(prediction_branch_pred, target_branch_target)[source]¶

Compute the FastSiam loss (cosine similarity loss).

Parameters:

prediction_branch_pred (torch.Tensor)
target_branch_target (torch.Tensor)

Return type:

torch.Tensor

forward(views)[source]¶

Forward pass through the prediction branch and target branches.

Parameters:: views (Any)
Return type:: tuple[torch.Tensor, torch.Tensor]

global_avg_pool¶

lr = 0.001¶

momentum = 0.996¶

num_classes = None¶

prediction_branch_predictor¶

prediction_branch_projector¶

target_branch_backbone¶

target_branch_projector¶

test_metric = None¶

test_step(batch, batch_idx)[source]¶

Operates on a single batch of data from the test set. In this step you’d normally generate examples or calculate anything of interest such as accuracy.

Args:

batch: The output of your data iterable, normally a DataLoader. batch_idx: The index of this batch. dataloader_idx: The index of the dataloader that produced this batch.

(only if multiple dataloaders used)

Return:

Tensor - The loss tensor
dict - A dictionary. Can include any keys, but must include the key 'loss'.
None - Skip to the next batch.

# if you have one test dataloader:
def test_step(self, batch, batch_idx): ...


# if you have multiple test dataloaders:
def test_step(self, batch, batch_idx, dataloader_idx=0): ...

Examples:

# CASE 1: A single test dataset
def test_step(self, batch, batch_idx):
    x, y = batch

    # implement your own
    out = self(x)
    loss = self.loss(out, y)

    # log 6 example images
    # or generated text... or whatever
    sample_imgs = x[:6]
    grid = torchvision.utils.make_grid(sample_imgs)
    self.logger.experiment.add_image('example_images', grid, 0)

    # calculate acc
    labels_hat = torch.argmax(out, dim=1)
    test_acc = torch.sum(y == labels_hat).item() / (len(y) * 1.0)

    # log the outputs!
    self.log_dict({'test_loss': loss, 'test_acc': test_acc})

If you pass in multiple test dataloaders, test_step() will have an additional argument. We recommend setting the default value of 0 so that you can quickly switch between single and multiple dataloaders.

# CASE 2: multiple test dataloaders
def test_step(self, batch, batch_idx, dataloader_idx=0):
    # dataloader_idx tells you which dataset this is.
    x, y = batch

    # implement your own
    out = self(x)

    if dataloader_idx == 0:
        loss = self.loss0(out, y)
    else:
        loss = self.loss1(out, y)

    # calculate acc
    labels_hat = torch.argmax(out, dim=1)
    acc = torch.sum(y == labels_hat).item() / (len(y) * 1.0)

    # log the outputs separately for each dataloader
    self.log_dict({f"test_loss_{dataloader_idx}": loss, f"test_acc_{dataloader_idx}": acc})

Note:: If you don’t need to test you don’t need to implement this method.
Note:: When the test_step() is called, the model has been put in eval mode and PyTorch gradients have been disabled. At the end of the test epoch, the model goes back to training mode and gradients are enabled.

Parameters:

batch (Any)
batch_idx (int)

Return type:

torch.Tensor

training_step(batch, batch_idx)[source]¶

Here you compute and return the training loss and some additional metrics for e.g. the progress bar or logger.

Args:

batch: The output of your data iterable, normally a DataLoader. batch_idx: The index of this batch. dataloader_idx: The index of the dataloader that produced this batch.

(only if multiple dataloaders used)

Return:

Tensor - The loss tensor
dict - A dictionary which can include any keys, but must include the key 'loss' in the case of automatic optimization.
None - In automatic optimization, this will skip to the next batch (but is not supported for multi-GPU, TPU, or DeepSpeed). For manual optimization, this has no special meaning, as returning the loss is not required.

In this step you’d normally do the forward pass and calculate the loss for a batch. You can also do fancier things like multiple forward passes or something model specific.

Example:

def training_step(self, batch, batch_idx):
    x, y, z = batch
    out = self.encoder(x)
    loss = self.loss(out, x)
    return loss

To use multiple optimizers, you can switch to ‘manual optimization’ and control their stepping:

def __init__(self):
    super().__init__()
    self.automatic_optimization = False


# Multiple optimizers (e.g.: GANs)
def training_step(self, batch, batch_idx):
    opt1, opt2 = self.optimizers()

    # do training_step with encoder
    ...
    opt1.step()
    # do training_step with decoder
    ...
    opt2.step()

Note:: When accumulate_grad_batches > 1, the loss returned here will be automatically normalized by accumulate_grad_batches internally.

Parameters:

batch (Any)
batch_idx (int)

Return type:

torch.Tensor

update_target_branch()[source]¶

Momentum update for the target branch.

Return type:: None

validation_step(batch, batch_idx)[source]¶

Operates on a single batch of data from the validation set. In this step you’d might generate examples or calculate anything of interest like accuracy.

Args:

batch: The output of your data iterable, normally a DataLoader. batch_idx: The index of this batch. dataloader_idx: The index of the dataloader that produced this batch.

(only if multiple dataloaders used)

Return:

Tensor - The loss tensor
dict - A dictionary. Can include any keys, but must include the key 'loss'.
None - Skip to the next batch.

# if you have one val dataloader:
def validation_step(self, batch, batch_idx): ...


# if you have multiple val dataloaders:
def validation_step(self, batch, batch_idx, dataloader_idx=0): ...

Examples:

# CASE 1: A single validation dataset
def validation_step(self, batch, batch_idx):
    x, y = batch

    # implement your own
    out = self(x)
    loss = self.loss(out, y)

    # log 6 example images
    # or generated text... or whatever
    sample_imgs = x[:6]
    grid = torchvision.utils.make_grid(sample_imgs)
    self.logger.experiment.add_image('example_images', grid, 0)

    # calculate acc
    labels_hat = torch.argmax(out, dim=1)
    val_acc = torch.sum(y == labels_hat).item() / (len(y) * 1.0)

    # log the outputs!
    self.log_dict({'val_loss': loss, 'val_acc': val_acc})

If you pass in multiple val dataloaders, validation_step() will have an additional argument. We recommend setting the default value of 0 so that you can quickly switch between single and multiple dataloaders.

# CASE 2: multiple validation dataloaders
def validation_step(self, batch, batch_idx, dataloader_idx=0):
    # dataloader_idx tells you which dataset this is.
    x, y = batch

    # implement your own
    out = self(x)

    if dataloader_idx == 0:
        loss = self.loss0(out, y)
    else:
        loss = self.loss1(out, y)

    # calculate acc
    labels_hat = torch.argmax(out, dim=1)
    acc = torch.sum(y == labels_hat).item() / (len(y) * 1.0)

    # log the outputs separately for each dataloader
    self.log_dict({f"val_loss_{dataloader_idx}": loss, f"val_acc_{dataloader_idx}": acc})

Note:: If you don’t need to validate you don’t need to implement this method.
Note:: When the validation_step() is called, the model has been put in eval mode and PyTorch gradients have been disabled. At the end of validation, the model goes back to training mode and gradients are enabled.

Parameters:

batch (Any)
batch_idx (int)

Return type:

torch.Tensor

Parameters:

backbone (torch.nn.Module)
in_dim (int)
hid_dim (int)
out_dim (int)
K (int)
momentum (float)
lr (float)
test_metric (Optional[Callable])
num_classes (Optional[int])

class minerva.models.ssl.fastsiam.SimSiamMLPHead(layer_sizes, activation_cls=nn.ReLU, batch_norm=False, final_bn=False, final_relu=False, *args, **kwargs)[source]¶

Bases: torch.nn.Sequential

A sequential container.

Modules will be added to it in the order they are passed in the constructor. Alternatively, an OrderedDict of modules can be passed in. The forward() method of Sequential accepts any input and forwards it to the first module it contains. It then “chains” outputs to inputs sequentially for each subsequent module, finally returning the output of the last module.

The value a Sequential provides over manually calling a sequence of modules is that it allows treating the whole container as a single module, such that performing a transformation on the Sequential applies to each of the modules it stores (which are each a registered submodule of the Sequential).

What’s the difference between a Sequential and a torch.nn.ModuleList? A ModuleList is exactly what it sounds like–a list for storing Module s! On the other hand, the layers in a Sequential are connected in a cascading way.

Example:

# Using Sequential to create a small model. When `model` is run,
# input will first be passed to `Conv2d(1,20,5)`. The output of
# `Conv2d(1,20,5)` will be used as the input to the first
# `ReLU`; the output of the first `ReLU` will become the input
# for `Conv2d(20,64,5)`. Finally, the output of
# `Conv2d(20,64,5)` will be used as input to the second `ReLU`
model = nn.Sequential(
    nn.Conv2d(1, 20, 5), nn.ReLU(), nn.Conv2d(20, 64, 5), nn.ReLU()
)

# Using Sequential with OrderedDict. This is functionally the
# same as the above code
model = nn.Sequential(
    OrderedDict(
        [
            ("conv1", nn.Conv2d(1, 20, 5)),
            ("relu1", nn.ReLU()),
            ("conv2", nn.Conv2d(20, 64, 5)),
            ("relu2", nn.ReLU()),
        ]
    )
)

A modular implementation of a multi-layer perceptron (MLP) head, designed for SimSiam-style architectures.

Parameters¶

layer_sizesSequence[int]: Sequence of integers representing the sizes of each layer in the MLP. Must have at least two elements (input and output sizes).
activation_clstype, optional: The class of the activation function to use, by default torch.nn.ReLU. Must be a subclass of torch.nn.Module.
batch_normbool, optional: Whether to include batch normalization after each hidden layer, by default False.
final_bnbool, optional: Whether to include a batch normalization layer after the final layer, by default False.
final_relubool, optional: Whether to include a ReLU activation after the final layer, by default False.
*args, **kwargs :: Additional arguments passed to the activation function.

Raises¶

AssertionError: If layer_sizes has fewer than two elements or contains non-positive integers.
AssertionError: If activation_cls is not a subclass of torch.nn.Module.

Examples¶

>>> head = SimSiamMLPHead([2048, 512, 128], batch_norm=True)
>>> x = torch.randn(32, 2048)  # Batch of 32 samples with input dim 2048
>>> output = head(x)

Parameters:

layer_sizes (Sequence[int])
activation_cls (type)
batch_norm (bool)
final_bn (bool)
final_relu (bool)