dasf.pipeline.executors.ray

Ray executor module.

Attributes

USE_RAY

Classes

RayPipelineExecutor

A pipeline executor based on ray data flow.

Module Contents

dasf.pipeline.executors.ray.USE_RAY = True

class dasf.pipeline.executors.ray.RayPipelineExecutor(address=None, port=6379, local=False, use_gpu=False, ray_kwargs=None)

Bases: dasf.pipeline.executors.base.Executor

A pipeline executor based on ray data flow.

Parameters

addressstr

Address of the Dask scheduler, default=None.

portint

Port of the Ray head, default=8786.

localbool

Kicks off a new local Ray cluster, default=False.

use_gpubool

In conjunction with local, it kicks off a local CUDA Ray cluster, default=False.

Constructor of the object RayPipelineExecutor.

property ngpus

Return the number of GPUs in total.

Returns

ngpus : Number of GPUs in total

property is_connected

Return wether the executor is connected or not.

Returns

bool : if the executor is connected.

execute(fn, *args, **kwargs)

Return wether the executor is connected or not.

Parameters

fnCallable

Function to call when executor is performing a task.

Returns

ret : the same return of function fn.

__del__()

Destructor of object.

It also shutdowns Dask on Ray properly with disable_dask_on_ray.