minerva.models.nets.time_series.imu_transformer

Classes

IMUCNN	A modular Lightning model wrapper for supervised learning tasks.
IMUTransformerEncoder	A modular Lightning model wrapper for supervised learning tasks.
_IMUTransformerEncoder	Base class for all neural network modules.

Module Contents

class minerva.models.nets.time_series.imu_transformer.IMUCNN(input_shape=(6, 60), hidden_dim=64, num_classes=6, dropout_factor=0.1, learning_rate=0.001, *args, **kwargs)

Bases: minerva.models.nets.base.SimpleSupervisedModel

A modular Lightning model wrapper for supervised learning tasks.

This class enables the construction of supervised models by combining a backbone (feature extractor), an optional adapter, and a fully connected (FC) head. It provides a clean interface for setting up custom training, validation, and testing pipelines with pluggable loss functions, metrics, optimizers, and learning rate schedulers.

The architecture is structured as follows:

Backbone Model

v

Adapter (Optional)

(Flatten if needed)

v

Fully Connected Head

v

Loss Function

Training and validation steps comprise the following steps:

1. Forward pass input through the backbone.

2. Pass through adapter (if provided).

3. Flatten the output (if flatten is True) before the FC head.

4. Forward through the FC head.

5. Compute loss with respect to targets.

6. Backpropagate and update parameters.

7. Compute metrics and log them.

8. Return loss. train_loss, val_loss, and test_loss are always logged, along with any additional metrics specified in the train_metrics, val_metrics, and test_metrics dictionaries.

This wrapper is especially useful to quickly set up supervised models for various tasks, such as image classification, object detection, and segmentation. It is designed to be flexible and extensible, allowing users to easily swap out components like the backbone, adapter, and FC head as needed. The model is built with a focus on simplicity and modularity, making it easy to adapt to different use cases and requirements. The model is designed to be used with PyTorch Lightning and is compatible with its training loop.

Note: For more complex architectures that does not follow the above structure should not inherit from this class.

Note: Input batches must be tuples (input_tensor, target_tensor).

Initializes the supervised model with training components and configs.

Parameters

backbonetorch.nn.Module or LoadableModule

The backbone (feature extractor) model.

fctorch.nn.Module or LoadableModule

The fully connected head. Use nn.Identity() if not required.

loss_fntorch.nn.Module

Loss function to optimize during training.

adapterCallable, optional

Function to transform backbone outputs before feeding into fc.

learning_ratefloat, default=1e-3

Learning rate used for optimization.

flattenbool, default=True

If True, flattens backbone outputs before fc.

train_metricsdict, optional

TorchMetrics dictionary for training evaluation.

val_metricsdict, optional

TorchMetrics dictionary for validation evaluation.

test_metricsdict, optional

TorchMetrics dictionary for test evaluation.

freeze_backbonebool, default=False

If True, backbone parameters are frozen during training.

optimizer: type

Optimizer class to be instantiated. By default, it is set to torch.optim.Adam. Should be a subclass of torch.optim.Optimizer (e.g., torch.optim.SGD).

optimizer_kwargsdict, optional

Additional kwargs passed to the optimizer constructor.

lr_schedulertype, optional

Learning rate scheduler class to be instantiated. By default, it is set to None, which means no scheduler will be used. Should be a subclass of torch.optim.lr_scheduler.LRScheduler (e.g., torch.optim.lr_scheduler.StepLR).

lr_scheduler_kwargsdict, optional

Additional kwargs passed to the scheduler constructor.

_calculate_fc_input_features(backbone, input_shape)

Parameters:

• backbone (torch.nn.Module)

• input_shape (Tuple[int, int])

Return type:

int

_create_backbone(input_shape, hidden_dim, dropout_factor)

_create_fc(input_features, hidden_dim, num_classes)

dropout_factor = 0.1

fc_input_channels

hidden_dim = 64

input_shape = (6, 60)

Parameters:

• input_shape (tuple)

• hidden_dim (int)

• num_classes (int)

• dropout_factor (float)

• learning_rate (float)

class minerva.models.nets.time_series.imu_transformer.IMUTransformerEncoder(input_shape=(6, 60), transformer_dim=64, encode_position=True, nhead=8, dim_feedforward=128, transformer_dropout=0.1, transformer_activation='gelu', num_encoder_layers=6, num_classes=6, learning_rate=0.001, *args, **kwargs)

Bases: minerva.models.nets.base.SimpleSupervisedModel

A modular Lightning model wrapper for supervised learning tasks.

This class enables the construction of supervised models by combining a backbone (feature extractor), an optional adapter, and a fully connected (FC) head. It provides a clean interface for setting up custom training, validation, and testing pipelines with pluggable loss functions, metrics, optimizers, and learning rate schedulers.

The architecture is structured as follows:

Backbone Model

v

Adapter (Optional)

(Flatten if needed)

v

Fully Connected Head

v

Loss Function

Training and validation steps comprise the following steps:

1. Forward pass input through the backbone.

2. Pass through adapter (if provided).

3. Flatten the output (if flatten is True) before the FC head.

4. Forward through the FC head.

5. Compute loss with respect to targets.

6. Backpropagate and update parameters.

7. Compute metrics and log them.

8. Return loss. train_loss, val_loss, and test_loss are always logged, along with any additional metrics specified in the train_metrics, val_metrics, and test_metrics dictionaries.

This wrapper is especially useful to quickly set up supervised models for various tasks, such as image classification, object detection, and segmentation. It is designed to be flexible and extensible, allowing users to easily swap out components like the backbone, adapter, and FC head as needed. The model is built with a focus on simplicity and modularity, making it easy to adapt to different use cases and requirements. The model is designed to be used with PyTorch Lightning and is compatible with its training loop.

Note: For more complex architectures that does not follow the above structure should not inherit from this class.

Note: Input batches must be tuples (input_tensor, target_tensor).

Initializes the supervised model with training components and configs.

Parameters

backbonetorch.nn.Module or LoadableModule

The backbone (feature extractor) model.

fctorch.nn.Module or LoadableModule

The fully connected head. Use nn.Identity() if not required.

loss_fntorch.nn.Module

Loss function to optimize during training.

adapterCallable, optional

Function to transform backbone outputs before feeding into fc.

learning_ratefloat, default=1e-3

Learning rate used for optimization.

flattenbool, default=True

If True, flattens backbone outputs before fc.

train_metricsdict, optional

TorchMetrics dictionary for training evaluation.

val_metricsdict, optional

TorchMetrics dictionary for validation evaluation.

test_metricsdict, optional

TorchMetrics dictionary for test evaluation.

freeze_backbonebool, default=False

If True, backbone parameters are frozen during training.

optimizer: type

Optimizer class to be instantiated. By default, it is set to torch.optim.Adam. Should be a subclass of torch.optim.Optimizer (e.g., torch.optim.SGD).

optimizer_kwargsdict, optional

Additional kwargs passed to the optimizer constructor.

lr_schedulertype, optional

Learning rate scheduler class to be instantiated. By default, it is set to None, which means no scheduler will be used. Should be a subclass of torch.optim.lr_scheduler.LRScheduler (e.g., torch.optim.lr_scheduler.StepLR).

lr_scheduler_kwargsdict, optional

Additional kwargs passed to the scheduler constructor.

_create_backbone(input_shape, transformer_dim, encode_position, nhead, dim_feedforward, transformer_dropout, transformer_activation, num_encoder_layers)

_create_fc(transform_dim, num_classes)

input_shape = (6, 60)

Parameters:

• input_shape (tuple)

• transformer_dim (int)

• encode_position (bool)

• nhead (int)

• dim_feedforward (int)

• transformer_dropout (float)

• transformer_activation (str)

• num_encoder_layers (int)

• num_classes (int)

• learning_rate (float)

class minerva.models.nets.time_series.imu_transformer._IMUTransformerEncoder(input_shape=(6, 60), transformer_dim=64, encode_position=True, nhead=8, dim_feedforward=128, transformer_dropout=0.1, transformer_activation='gelu', num_encoder_layers=6, permute=False)

Bases: torch.nn.Module

Base class for all neural network modules.

Your models should also subclass this class.

Modules can also contain other Modules, allowing them to be nested in a tree structure. You can assign the submodules as regular attributes:

import torch.nn as nn
import torch.nn.functional as F

class Model(nn.Module):
    def __init__(self) -> None:
        super().__init__()
        self.conv1 = nn.Conv2d(1, 20, 5)
        self.conv2 = nn.Conv2d(20, 20, 5)

    def forward(self, x):
        x = F.relu(self.conv1(x))
        return F.relu(self.conv2(x))

Submodules assigned in this way will be registered, and will also have their parameters converted when you call to(), etc.

Note

As per the example above, an __init__() call to the parent class must be made before assignment on the child.

Variables:

training (bool) – Boolean represents whether this module is in training or evaluation mode.

Parameters:

• input_shape (tuple)

• transformer_dim (int)

• encode_position (bool)

• nhead (int)

• dim_feedforward (int)

• transformer_dropout (float)

• transformer_activation (str)

• num_encoder_layers (int)

• permute (bool)

input_shape: (tuple) shape of the input data transformer_dim: (int) dimension of the transformer encode_position: (bool) whether to encode position or not nhead: (int) number of attention heads dim_feedforward: (int) dimension of the feedforward network transformer_dropout: (float) dropout rate for the transformer transformer_activation: (str) activation function for the transformer num_encoder_layers: (int) number of transformer encoder layers num_classes: (int) number of output classes permute: bool, optional. If True the input data will be permuted before passing through the model, by default False.

cls_token

encode_position = True

forward(x)

Forward

Parameters

x_type_

A tensor of shape (B, C, S) with B = batch size, C = channels, S = sequence length

input_proj

input_shape = (6, 60)

permute = False

transformer_dim = 64

transformer_encoder