dasf.ml.dl.clusters

Submodules

dasf.ml.dl.clusters.dask

Classes

DaskClusterEnvironment

Create a Dask Cluster environment for workers

Package Contents

class dasf.ml.dl.clusters.DaskClusterEnvironment(metadata=None)[source]

Bases: pytorch_lightning.plugins.environments.ClusterEnvironment

Create a Dask Cluster environment for workers

Parameters

metadatadict: Dictionary containing all data related to workers.

Constructor of the object DaskClusterEnvironment using dict metadata.

_master_port = 23456

detect()[source]

Detect if important data are present into metadata dictionary.

Returns

bool : if they are present or not.

Return type:: bool

property creates_processes_externally: bool

Return True if the cluster is managed (you don’t launch processes yourself).

Return type:: bool

property main_address: str

Return master worker address.

Return type:: str

property main_port: int

Return master worker port.

Return type:: int

creates_children()[source]

Fork children when generate a cluster.

Return type:: bool

world_size()[source]

Return worker world size.

Return type:: int

global_rank()[source]

Return worker global rank.

Return type:: int

local_rank()[source]

Return worker local rank.

Return type:: int

node_rank()[source]

Return worker node rank (which is similar to global rank).

Return type:: int

set_world_size(size)[source]

Set the index of the world size.

Parameters

sizeint: Integer with the world size value.

Parameters:: size (int)
Return type:: None

set_global_rank(rank)[source]

Set the index of the global rank.

Parameters

sizeint: Integer with the global rank value.

Parameters:: rank (int)
Return type:: None