dasf.ml.dl.clusters
Submodules
Classes
Create a Dask Cluster environment for workers |
Package Contents
- class dasf.ml.dl.clusters.DaskClusterEnvironment(metadata=None)
Bases:
pytorch_lightning.plugins.environments.ClusterEnvironmentCreate a Dask Cluster environment for workers
Parameters
- metadatadict
Dictionary containing all data related to workers.
Constructor of the object DaskClusterEnvironment using dict metadata.
- detect()
Detect if important data are present into metadata dictionary.
Returns
bool : if they are present or not.
- Return type:
bool
- property creates_processes_externally: bool
Return True if the cluster is managed (you don’t launch processes yourself).
- Return type:
bool
- property main_address: str
Return master worker address.
- Return type:
str
- property main_port: int
Return master worker port.
- Return type:
int
- creates_children()
Fork children when generate a cluster.
- Return type:
bool
- world_size()
Return worker world size.
- Return type:
int
- global_rank()
Return worker global rank.
- Return type:
int
- local_rank()
Return worker local rank.
- Return type:
int
- node_rank()
Return worker node rank (which is similar to global rank).
- Return type:
int