minerva.models.nets

Submodules

• minerva.models.nets.base
• minerva.models.nets.cnn_ha_etal
• minerva.models.nets.cnn_pf
• minerva.models.nets.deeplabv3
• minerva.models.nets.imu_transformer
• minerva.models.nets.inception_time
• minerva.models.nets.resnet_1d
• minerva.models.nets.setr
• minerva.models.nets.sfm
• minerva.models.nets.unet
• minerva.models.nets.vit
• minerva.models.nets.wisenet

Classes

DeepLabV3	A DeeplabV3 with a ResNet50 backbone
SETR_PUP	Initializes the SetR model.
SimpleSupervisedModel	Simple pipeline for supervised models.
UNet	This class is a simple implementation of the U-Net model, which is a
WiseNet	Simple pipeline for supervised models.

Package Contents

class minerva.models.nets.DeepLabV3(backbone=None, pred_head=None, loss_fn=None, learning_rate=0.001, num_classes=6, train_metrics=None, val_metrics=None, test_metrics=None)

Bases: minerva.models.nets.base.SimpleSupervisedModel

A DeeplabV3 with a ResNet50 backbone

References

Liang-Chieh Chen, George Papandreou, Florian Schroff, Hartwig Adam. “Rethinking Atrous Convolution for Semantic Image Segmentation”, 2017

Initializes a DeepLabV3 model.

Parameters

backbone: Optional[nn.Module]

The backbone network. Defaults to None.

pred_head: Optional[nn.Module]

The prediction head network. Defaults to None.

loss_fn: Optional[nn.Module]

The loss function. Defaults to None.

learning_rate: float

The learning rate for the optimizer. Defaults to 0.001.

num_classes: int

The number of classes for prediction. Defaults to 6.

train_metrics: Optional[Dict[str, Metric]]

The metrics to be computed during training. Defaults to None.

val_metrics: Optional[Dict[str, Metric]]

The metrics to be computed during validation. Defaults to None.

test_metrics: Optional[Dict[str, Metric]]

The metrics to be computed during testing. Defaults to None.

_loss_func(y_hat, y)

Calculate the loss between the output and the input data.

Parameters

y_hattorch.Tensor

The output data from the forward pass.

ytorch.Tensor

The input data/label.

Returns

torch.Tensor

The loss value.

Parameters:

• y_hat (torch.Tensor)

• y (torch.Tensor)

Return type:

torch.Tensor

configure_optimizers()

forward(x)

Perform a forward pass with the input data on the backbone model.

Parameters

xtorch.Tensor

The input data.

Returns

torch.Tensor

The output data from the forward pass.

Parameters:

x (torch.Tensor)

Return type:

torch.Tensor

Parameters:

• backbone (Optional[torch.nn.Module])

• pred_head (Optional[torch.nn.Module])

• loss_fn (Optional[torch.nn.Module])

• learning_rate (float)

• num_classes (int)

• train_metrics (Optional[Dict[str, torchmetrics.Metric]])

• val_metrics (Optional[Dict[str, torchmetrics.Metric]])

• test_metrics (Optional[Dict[str, torchmetrics.Metric]])

class minerva.models.nets.SETR_PUP(image_size=512, patch_size=16, num_layers=24, num_heads=16, hidden_dim=1024, mlp_dim=4096, encoder_dropout=0.1, num_classes=1000, norm_layer=None, decoder_channels=256, num_convs=4, up_scale=2, kernel_size=3, align_corners=False, decoder_dropout=0.1, conv_norm=None, conv_act=None, interpolate_mode='bilinear', loss_fn=None, train_metrics=None, val_metrics=None, test_metrics=None, aux_output=True, aux_output_layers=[9, 14, 19], aux_weights=[0.3, 0.3, 0.3])

Bases: lightning.LightningModule

Initializes the SetR model.

Parameters

image_sizeint or tuple[int, int]

The input image size. Defaults to 512.

patch_sizeint

The size of each patch. Defaults to 16.

num_layersint

The number of layers in the transformer encoder. Defaults to 24.

num_headsint

The number of attention heads in the transformer encoder. Defaults to 16.

hidden_dimint

The hidden dimension of the transformer encoder. Defaults to 1024.

mlp_dimint

The dimension of the MLP layers in the transformer encoder. Defaults to 4096.

encoder_dropoutfloat

The dropout rate for the transformer encoder. Defaults to 0.1.

num_classesint

The number of output classes. Defaults to 1000.

norm_layernn.Module, optional

The normalization layer to be used in the decoder. Defaults to None.

decoder_channelsint

The number of channels in the decoder. Defaults to 256.

num_convsint

The number of convolutional layers in the decoder. Defaults to 4.

up_scaleint

The scale factor for upsampling in the decoder. Defaults to 2.

kernel_sizeint

The kernel size for convolutional layers in the decoder. Defaults to 3.

align_cornersbool

Whether to align corners during interpolation in the decoder. Defaults to False.

decoder_dropoutfloat

The dropout rate for the decoder. Defaults to 0.1.

conv_normnn.Module, optional

The normalization layer to be used in the convolutional layers of the decoder. Defaults to None.

conv_actnn.Module, optional

The activation function to be used in the convolutional layers of the decoder. Defaults to None.

interpolate_modestr

The interpolation mode for upsampling in the decoder. Defaults to “bilinear”.

loss_fnnn.Module, optional

The loss function to be used during training. Defaults to None.

train_metricsDict[str, Metric], optional

The metrics to be used for training evaluation. Defaults to None.

val_metricsDict[str, Metric], optional

The metrics to be used for validation evaluation. Defaults to None.

test_metricsDict[str, Metric], optional

The metrics to be used for testing evaluation. Defaults to None.

aux_outputbool

Whether to include auxiliary output heads in the model. Defaults to True.

aux_output_layerslist[int] | None

The indices of the layers to output auxiliary predictions. Defaults to [9, 14, 19].

aux_weightslist[float]

The weights for the auxiliary predictions. Defaults to [0.3, 0.3, 0.3].

_compute_metrics(y_hat, y, step_name)

Parameters:

• y_hat (torch.Tensor)

• y (torch.Tensor)

• step_name (str)

_loss_func(y_hat, y)

Calculate the loss between the output and the input data.

Parameters

y_hattorch.Tensor

The output data from the forward pass.

ytorch.Tensor

The input data/label.

Returns

torch.Tensor

The loss value.

Parameters:

• y_hat (torch.Tensor | Tuple[torch.Tensor, torch.Tensor, torch.Tensor, torch.Tensor])

• y (torch.Tensor)

Return type:

torch.Tensor

_single_step(batch, batch_idx, step_name)

Perform a single step of the training/validation loop.

Parameters

batchtorch.Tensor

The input data.

batch_idxint

The index of the batch.

step_namestr

The name of the step, either “train” or “val”.

Returns

torch.Tensor

The loss value.

Parameters:

• batch (torch.Tensor)

• batch_idx (int)

• step_name (str)

configure_optimizers()

forward(x)

Parameters:

x (torch.Tensor)

Return type:

torch.Tensor

predict_step(batch, batch_idx, dataloader_idx=None)

Parameters:

• batch (torch.Tensor)

• batch_idx (int)

• dataloader_idx (int | None)

test_step(batch, batch_idx)

Parameters:

• batch (torch.Tensor)

• batch_idx (int)

training_step(batch, batch_idx)

Parameters:

• batch (torch.Tensor)

• batch_idx (int)

validation_step(batch, batch_idx)

Parameters:

• batch (torch.Tensor)

• batch_idx (int)

Parameters:

• image_size (int | tuple[int, int])

• patch_size (int)

• num_layers (int)

• num_heads (int)

• hidden_dim (int)

• mlp_dim (int)

• encoder_dropout (float)

• num_classes (int)

• norm_layer (Optional[torch.nn.Module])

• decoder_channels (int)

• num_convs (int)

• up_scale (int)

• kernel_size (int)

• align_corners (bool)

• decoder_dropout (float)

• conv_norm (Optional[torch.nn.Module])

• conv_act (Optional[torch.nn.Module])

• interpolate_mode (str)

• loss_fn (Optional[torch.nn.Module])

• train_metrics (Optional[Dict[str, torchmetrics.Metric]])

• val_metrics (Optional[Dict[str, torchmetrics.Metric]])

• test_metrics (Optional[Dict[str, torchmetrics.Metric]])

• aux_output (bool)

• aux_output_layers (list[int] | None)

• aux_weights (list[float])

class minerva.models.nets.SimpleSupervisedModel(backbone, fc, loss_fn, learning_rate=0.001, flatten=True, train_metrics=None, val_metrics=None, test_metrics=None)

Bases: lightning.LightningModule

Simple pipeline for supervised models.

This class implements a very common deep learning pipeline, which is composed by the following steps:

1. Make a forward pass with the input data on the backbone model;

2. Make a forward pass with the input data on the fc model;

3. Compute the loss between the output and the label data;

4. Optimize the model (backbone and FC) parameters with respect to the loss.

This reduces the code duplication for autoencoder models, and makes it easier to implement new models by only changing the backbone model. More complex models, that does not follow this pipeline, should not inherit from this class. Note that, for this class the input data is a tuple of tensors, where the first tensor is the input data and the second tensor is the mask or label.

Initialize the model with the backbone, fc, loss function and metrics. Metrics are used to evaluate the model during training, validation, testing or prediction. It will be logged using lightning logger at the end of each epoch. Metrics should implement the torchmetrics.Metric interface.

Parameters

backbonetorch.nn.Module

The backbone model. Usually the encoder/decoder part of the model.

fctorch.nn.Module

The fully connected model, usually used to classification tasks. Use torch.nn.Identity() if no FC model is needed.

loss_fntorch.nn.Module

The function used to compute the loss.

learning_ratefloat, optional

The learning rate to Adam optimizer, by default 1e-3

flattenbool, optional

If True the input data will be flattened before passing through the fc model, by default True

train_metricsDict[str, Metric], optional

The metrics to be used during training, by default None

val_metricsDict[str, Metric], optional

The metrics to be used during validation, by default None

test_metricsDict[str, Metric], optional

The metrics to be used during testing, by default None

predict_metricsDict[str, Metric], optional

The metrics to be used during prediction, by default None

_compute_metrics(y_hat, y, step_name)

Calculate the metrics for the given step.

Parameters

y_hattorch.Tensor

The output data from the forward pass.

ytorch.Tensor

The input data/label.

step_namestr

Name of the step. It will be used to get the metrics from the self.metrics attribute.

Returns

Dict[str, torch.Tensor]

A dictionary with the metrics values.

Parameters:

• y_hat (torch.Tensor)

• y (torch.Tensor)

• step_name (str)

Return type:

Dict[str, torch.Tensor]

_loss_func(y_hat, y)

Calculate the loss between the output and the input data.

Parameters

y_hattorch.Tensor

The output data from the forward pass.

ytorch.Tensor

The input data/label.

Returns

torch.Tensor

The loss value.

Parameters:

• y_hat (torch.Tensor)

• y (torch.Tensor)

Return type:

torch.Tensor

_single_step(batch, batch_idx, step_name)

Perform a single train/validation/test step. It consists in making a forward pass with the input data on the backbone model, computing the loss between the output and the input data, and logging the loss.

Parameters

batchtorch.Tensor

The input data. It must be a 2-element tuple of tensors, where the first tensor is the input data and the second tensor is the mask.

batch_idxint

The index of the batch.

step_namestr

The name of the step. It will be used to log the loss. The possible values are: “train”, “val” and “test”. The loss will be logged as “{step_name}_loss”.

Returns

torch.Tensor

A tensor with the loss value.

Parameters:

• batch (torch.Tensor)

• batch_idx (int)

• step_name (str)

Return type:

torch.Tensor

configure_optimizers()

forward(x)

Perform a forward pass with the input data on the backbone model.

Parameters

xtorch.Tensor

The input data.

Returns

torch.Tensor

The output data from the forward pass.

Parameters:

x (torch.Tensor)

Return type:

torch.Tensor

predict_step(batch, batch_idx, dataloader_idx=None)

test_step(batch, batch_idx)

Parameters:

• batch (torch.Tensor)

• batch_idx (int)

training_step(batch, batch_idx)

Parameters:

• batch (torch.Tensor)

• batch_idx (int)

validation_step(batch, batch_idx)

Parameters:

• batch (torch.Tensor)

• batch_idx (int)

Parameters:

• backbone (torch.nn.Module)

• fc (torch.nn.Module)

• loss_fn (torch.nn.Module)

• learning_rate (float)

• flatten (bool)

• train_metrics (Optional[Dict[str, torchmetrics.Metric]])

• val_metrics (Optional[Dict[str, torchmetrics.Metric]])

• test_metrics (Optional[Dict[str, torchmetrics.Metric]])

class minerva.models.nets.UNet(n_channels=1, bilinear=False, learning_rate=0.001, loss_fn=None, **kwargs)

Bases: minerva.models.nets.base.SimpleSupervisedModel

This class is a simple implementation of the U-Net model, which is a convolutional neural network used for image segmentation. The model consists of a contracting path (encoder) and an expansive path (decoder). The contracting path follows the typical architecture of a convolutional neural network, with repeated applications of convolutions and max pooling layers. The expansive path consists of up-convolutions and concatenation of feature maps from the contracting path. The model also has skip connections, which allows the expansive path to use information from the contracting path at multiple resolutions. The U-Net model was originally proposed by Ronneberger, Fischer, and Brox in 2015.

This architecture, handles arbitrary input sizes, and returns an output of the same size as the input. The expected input size is (N, C, H, W), where N is the batch size, C is the number of channels, H is the height of the input image, and W is the width of the input image.

Note that, for this implementation, the input batch is a single tensor and not a tuple of tensors (e.g., data and label).

Note that this class wrappers the _UNet class, which is the actual implementation of the U-Net model, into a SimpleReconstructionNet class, which is a simple autoencoder pipeline for reconstruction tasks.

References

Ronneberger, Olaf, Philipp Fischer, and Thomas Brox. “U-net: Convolutional networks for biomedical image segmentation.” Medical Image Computing and Computer-Assisted Intervention-MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18. Springer International Publishing, 2015.

Wrapper implementation of the U-Net model.

Parameters

n_channelsint, optional

The number of channels of the input, by default 1

bilinearbool, optional

If True use bilinear interpolation for upsampling, by default False.

learning_ratefloat, optional

The learning rate to Adam optimizer, by default 1e-3

loss_fntorch.nn.Module, optional

The function used to compute the loss. If None, it will be used the MSELoss, by default None.

kwargsDict

Additional arguments to be passed to the SimpleSupervisedModel class.

Parameters:

• n_channels (int)

• bilinear (bool)

• learning_rate (float)

• loss_fn (Optional[torch.nn.Module])

class minerva.models.nets.WiseNet(in_channels=1, out_channels=1, loss_fn=None, learning_rate=0.001, **kwargs)

Bases: minerva.models.nets.base.SimpleSupervisedModel

Simple pipeline for supervised models.

This class implements a very common deep learning pipeline, which is composed by the following steps:

1. Make a forward pass with the input data on the backbone model;

2. Make a forward pass with the input data on the fc model;

3. Compute the loss between the output and the label data;

4. Optimize the model (backbone and FC) parameters with respect to the loss.

This reduces the code duplication for autoencoder models, and makes it easier to implement new models by only changing the backbone model. More complex models, that does not follow this pipeline, should not inherit from this class. Note that, for this class the input data is a tuple of tensors, where the first tensor is the input data and the second tensor is the mask or label.

Initialize the model with the backbone, fc, loss function and metrics. Metrics are used to evaluate the model during training, validation, testing or prediction. It will be logged using lightning logger at the end of each epoch. Metrics should implement the torchmetrics.Metric interface.

Parameters

backbonetorch.nn.Module

The backbone model. Usually the encoder/decoder part of the model.

fctorch.nn.Module

The fully connected model, usually used to classification tasks. Use torch.nn.Identity() if no FC model is needed.

loss_fntorch.nn.Module

The function used to compute the loss.

learning_ratefloat, optional

The learning rate to Adam optimizer, by default 1e-3

flattenbool, optional

If True the input data will be flattened before passing through the fc model, by default True

train_metricsDict[str, Metric], optional

The metrics to be used during training, by default None

val_metricsDict[str, Metric], optional

The metrics to be used during validation, by default None

test_metricsDict[str, Metric], optional

The metrics to be used during testing, by default None

predict_metricsDict[str, Metric], optional

The metrics to be used during prediction, by default None

_single_step(batch, batch_idx, step_name)

Perform a single train/validation/test step. It consists in making a forward pass with the input data on the backbone model, computing the loss between the output and the input data, and logging the loss.

Parameters

batchtorch.Tensor

The input data. It must be a 2-element tuple of tensors, where the first tensor is the input data and the second tensor is the mask.

batch_idxint

The index of the batch.

step_namestr

The name of the step. It will be used to log the loss. The possible values are: “train”, “val” and “test”. The loss will be logged as “{step_name}_loss”.

Returns

torch.Tensor

A tensor with the loss value.

Parameters:

• batch (torch.Tensor)

• batch_idx (int)

• step_name (str)

Return type:

torch.Tensor

predict_step(batch, batch_idx, dataloader_idx=None)

Parameters:

• in_channels (int)

• out_channels (int)

• loss_fn (torch.nn.Module)

• learning_rate (float)