dasf.transforms.base

Definition of the generic operators of the pipeline.

Classes

Operator	Class representing a generic Operator of the pipeline.
Fit	Class representing a Fit operation of the pipeline.
FitPredict	Class representing a Fit with Predict operation of the pipeline.
FitTransform	Class representing a Fit with Transform operation of the pipeline.
Predict	Class representing a Predict operation of the pipeline.
GetParams	Class representing a Get Parameters operation of the pipeline.
SetParams	Class representing a Set Parameters operation of the pipeline.
Transform	Class representing a Transform operation of the pipeline.
TargeteredTransform	Class representing a Targetered Transform operation of the pipeline.
MappedTransform	Class representing a MappedTransform based on Transform
ReductionTransform	Class representing a Reduction based on Transform

Module Contents

class dasf.transforms.base.Operator

Class representing a generic Operator of the pipeline.

get_uuid()

Return the UUID representation of the Operator.

class dasf.transforms.base.Fit

Bases: Operator

Class representing a Fit operation of the pipeline.

abstract _lazy_fit_cpu(X, y=None, **kwargs)

Respective lazy fit mocked function for CPUs.

abstract _lazy_fit_gpu(X, y=None, **kwargs)

Respective lazy fit mocked function for GPUs.

abstract _fit_cpu(X, y=None, **kwargs)

Respective immediate fit mocked function for local CPU(s).

abstract _fit_gpu(X, y=None, **kwargs)

Respective immediate fit mocked function for local GPU(s).

fit(X, y, sample_weight=None, **kwargs)

Generic fit funtion according executor.

static fit_from_model(model, X, y, sample_weight=None, **kwargs)

Return the model of a previous created object.

class dasf.transforms.base.FitPredict

Bases: Operator

Class representing a Fit with Predict operation of the pipeline.

abstract _lazy_fit_predict_cpu(X, y=None, **kwargs)

Respective lazy fit with predict mocked function for CPUs.

abstract _lazy_fit_predict_gpu(X, y=None, **kwargs)

Respective lazy fit with predict mocked function for GPUs.

abstract _fit_predict_cpu(X, y=None, **kwargs)

Respective immediate fit with predict mocked function for local CPU(s).

abstract _fit_predict_gpu(X, y=None, **kwargs)

Respective immediate fit with predict mocked function for local GPU(s).

fit_predict(X, y=None, **kwargs)

Generic fit with predict funtion according executor.

static fit_predict_from_model(model, X, y, sample_weight=None, **kwargs)

Return the model of a previous created object.

class dasf.transforms.base.FitTransform

Bases: Operator

Class representing a Fit with Transform operation of the pipeline.

abstract _lazy_fit_transform_cpu(X, y=None, **kwargs)

Respective lazy fit with transform mocked function for CPUs.

abstract _lazy_fit_transform_gpu(X, y=None, **kwargs)

Respective lazy fit with transform mocked function for GPUs.

abstract _fit_transform_cpu(X, y=None, **kwargs)

Respective immediate fit with transform mocked function for local CPU(s).

abstract _fit_transform_gpu(X, y=None, **kwargs)

Respective immediate fit with transform mocked function for local GPU(s).

fit_transform(X, y=None, **kwargs)

Generic fit with transform funtion according executor.

static fit_transform_from_model(model, X, y, sample_weight=None, **kwargs)

Return the model of a previous created object.

class dasf.transforms.base.Predict

Bases: Operator

Class representing a Predict operation of the pipeline.

abstract _lazy_predict_cpu(X, sample_weight=None, **kwargs)

Respective lazy predict mocked function for CPUs.

abstract _lazy_predict_gpu(X, sample_weight=None, **kwargs)

Respective lazy predict mocked function for GPUs.

abstract _predict_cpu(X, sample_weight=None, **kwargs)

Respective immediate predict mocked function for local CPU(s).

abstract _predict_gpu(X, sample_weight=None, **kwargs)

Respective immediate predict mocked function for local GPU(s).

predict(X, sample_weight=None, **kwargs)

Generic predict funtion according executor.

static predict_from_model(model, X, sample_weight=None, **kwargs)

Return the model of a previous created object.

class dasf.transforms.base.GetParams

Bases: Operator

Class representing a Get Parameters operation of the pipeline.

abstract _lazy_get_params_cpu(deep=True, **kwargs)

Respective lazy get_params mocked function for CPUs.

abstract _lazy_get_params_gpu(deep=True, **kwargs)

Respective lazy get_params mocked function for GPUs.

abstract _get_params_cpu(deep=True, **kwargs)

Respective immediate get_params mocked function for local CPU(s).

abstract _get_params_gpu(deep=True, **kwargs)

Respective immediate get_params mocked function for local GPU(s).

get_params(deep=True, **kwargs)

Generic get_params funtion according executor.

class dasf.transforms.base.SetParams

Bases: Operator

Class representing a Set Parameters operation of the pipeline.

abstract _lazy_set_params_cpu(**params)

Respective lazy set_params mocked function for CPUs.

abstract _lazy_set_params_gpu(**params)

Respective lazy set_params mocked function for GPUs.

abstract _set_params_cpu(**params)

Respective immediate set_params mocked function for local CPU(s).

abstract _set_params_gpu(**params)

Respective immediate set_params mocked function for local GPU(s).

set_params(**params)

Generic set_params funtion according executor.

class dasf.transforms.base.Transform

Bases: Operator

Class representing a Transform operation of the pipeline.

abstract _lazy_transform_cpu(X, **kwargs)

Respective lazy transform mocked function for CPUs.

abstract _lazy_transform_gpu(X, **kwargs)

Respective lazy transform mocked function for GPUs.

abstract _transform_cpu(X, **kwargs)

Respective immediate transform mocked function for local CPU(s).

abstract _transform_gpu(X, **kwargs)

Respective immediate transform mocked function for local GPU(s).

transform(X, **kwargs)

Generic transform funtion according executor.

static transform_from_model(model, X, **kwargs)

Return the model of a previous created object.

class dasf.transforms.base.TargeteredTransform(run_local=None, run_gpu=None)

Bases: Transform

Class representing a Targetered Transform operation of the pipeline.

This specific transform operates according the parameters of the constructor.

Parameters

run_localbool

Define that the operator will run locally and not distributed.

run_gpubool

Define if the operator will use GPU(s) or not.

class dasf.transforms.base.MappedTransform(function, depth=None, boundary=None, trim=True, output_chunk=None, drop_axis=None, new_axis=None)

Bases: Transform

Class representing a MappedTransform based on Transform object.

This object refers to any operation that can be done in blocks. In special, for Dask chunks. There are several ways of doing that. This class tries to simplify how the functions are applied into a block.

Parameters

functionCallable

A function that will be applied in a block.

depthtuple

The value of the boundary elements per axis (the default is None).

boundarystr

The type of the boundary. See Dask boundaries for more examples (the default is None).

trimbool

Option to trim the data after an overlap (the default is True).

output_chunktuple

New shape of the output after computing the function (the default is None).

drop_axistuple

Which axis should be deleted after computing the function (the default is None).

new_axistuple

Which axis represent a new axis after computing the function (the default is None).

__lazy_transform_generic(X, xp, **kwargs)

_lazy_transform_cpu(X, **kwargs)

Respective lazy transform mocked function for CPUs.

_lazy_transform_gpu(X, **kwargs)

Respective lazy transform mocked function for GPUs.

_transform_cpu(X, **kwargs)

Respective immediate transform mocked function for local CPU(s).

_transform_gpu(X, **kwargs)

Respective immediate transform mocked function for local GPU(s).

transform(X, **kwargs)

Generic transform funtion according executor.

class dasf.transforms.base.ReductionTransform(output_size, func_aggregate, func_chunk, func_combine=None)

Bases: Transform

Class representing a Reduction based on Transform object.

This is a simple MapReduction operation using Dask.

Parameters

output_sizetuple

The size of the new output.

func_aggregateCallable

The function called to aggregate the result of each chunk.

func_chunkCallable

The function applied in each chunk.

func_combineCallable

The function to combine each reduction of aggregate (the default is None).

_operation_aggregate_cpu(block, axis=None, keepdims=False)

_operation_aggregate_gpu(block, axis=None, keepdims=False)

_operation_combine_cpu(block, axis=None, keepdims=False)

_operation_combine_gpu(block, axis=None, keepdims=False)

_operation_chunk_cpu(block, axis=None, keepdims=False)

_operation_chunk_gpu(block, axis=None, keepdims=False)

_lazy_transform_cpu(X, *args, **kwargs)

Respective lazy transform mocked function for CPUs.

_lazy_transform_gpu(X, *args, **kwargs)

Respective lazy transform mocked function for GPUs.

_transform_cpu(X, *args, **kwargs)

Respective immediate transform mocked function for local CPU(s).

_transform_gpu(X, *args, **kwargs)

Respective immediate transform mocked function for local GPU(s).

transform(X, *args, **kwargs)

Generic transform funtion according executor.