model.deeplearn.handle package

Submodules

model.deeplearn.handle.dl_callback_builder module

class model.deeplearn.handle.dl_callback_builder.DLCallbackBuilder

Bases: object

Class to handle the building of callbacks for deep learning models.

__init__()

build(mh)

Build the callbacks for the model handled by the given model handler (mh).

Parameters:: mh (DLModelHandler and SimpleDLModelHandler) – The model handler whose callbacks must be built.
Returns:: The built callbacks
Return type:: list

model.deeplearn.handle.dl_class_weighter module

class model.deeplearn.handle.dl_class_weighter.DLClassWeighter

Bases: object

Class to handle class weights for deep learning models.

__init__()

handle_class_weight(mh, y)

Handle the class weight parameter of a deep learning model’s handler.

If no class weight is requested, then class weight will be None.

If automatic class weight is requested (i.e., “auto”), then the class weight is automatically determined from the distribution of expected classes to give a greater weight to less frequent classes and a smaller weight to more frequent classes. More concretely, let \(m\) be the number of samples, \(m_i\) be the number of samples corresponding to class \(i\), and \(n\) be the number of classes. Thus, each class weight will be \(w_i = m/(n m_i)\).

If class weight is a list, tuple or array of weights it will be translated to a dictionary such that the first element is the weight for the first class, and so on.

Parameters:

mh – The model handler whose class weights must be handled.
y (np.ndarray) – The vector of expected point-wise classes.

Returns:

Class weight prepared for the model.

model.deeplearn.handle.dl_label_formatter module

class model.deeplearn.handle.dl_label_formatter.DLLabelFormatter

Bases: object

Class to handle the labels for deep learning models.

__init__()

handle_labels_format(mh, y)

Handles the format in which labels must be given to the model.

For instance, if categorical cross entropy is used, labels must be given using one-hot-encoding. However, if sparse categorical cross entropy is used, labels must be given as an integer.

Parameters:

mh (DLModelHandler) – The model handler whose labels must be formatted.
y – The labels to be formatted.

Returns:

The labels prepared for the model.

model.deeplearn.handle.dl_model_compiler module

class model.deeplearn.handle.dl_model_compiler.DLModelCompiler(compilation_args)

Bases: object

Class to compile deep learning models.

Variables:

comp_args (dict) – The built compilation arguments (do not confuse with the compilation arguments specification typically referred to as compilation_args instead of comp_args).
class_weighter (DLClassWeighter) – The class weighter to handle the model’s class weights, if needed/requested.

__init__(compilation_args)

Initialize a DLModelCompiler.

Parameters:: compilation_args (dict) – The specification on how to compile the model. See dl_model_compiler.DLModelCompiler.build_compilation_args() and SimpleDLModelHandler .

compile(mh, X=None, y=None, y_rf=None, **kwargs)

The method that provides the logic to compile a model.

Parameters:

mh (DLModelHandler) – The model handler to be compiled.
X – Optionally, the coordinates might be used for a better initialization (e.g., automatically derive the number of expected input points, or the dimensionality of the space where the points belong to).
y – Optionally, the labels might be used for a better initialization (e.g., automatically derive the number of classes).
y_rf – The expected values for each receptive field. Can be used to derive class weights.

Returns:

The model handler itself after compiling the architecture, which implies modifying its internal state.

Return type:

DLModelHandler

static build_compilation_args(comp_args, mh=None)

Build the compilation arguments from given spec.

Parameters:

comp_args – The specification to build the compilation arguments.
mh (DLModelHandler or None) – The model handler being compiled. If not given, arguments based on architecture or model handler details will not be built.

Returns:

The dictionary of compilation arguments.

Return type:

dict

model.deeplearn.handle.dl_model_handler module

class model.deeplearn.handle.dl_model_handler.DLModelHandler(arch, **kwargs)

Bases: object

Abstract class to handle deep learning models. Typically, fitting, predicting, and compiling are the main operations supported by deep learning model handlers.

Variables:

arch (Architecture) – The model’s architecture.
class_names (list) – The name for each class involved in the classification problem, if any (it can be ignored by regression models).
compiler – See DLModelCompiler.
compiled – It is None by default, but it will be assigned the compiled model after calling the fit or predict methods.

Ivar:

compilation_args: The key-word specification on how to compile the model.

__init__(arch, **kwargs)

Initialize/instantiate a deep learning model handler.

Parameters:

arch (Architecture) – The architecture to be handled.
kwargs – The key-word specification to instantiate the DLModelHandler.

fit(X, y)

Fit the handled model to given data.

Parameters:

X (np.ndarray) – The structure space matrix, typically the matrix with the x, y, z coordinates as columns.
y (np.ndarray) – The vector of expected labels, the ground-truth from the supervised training perspective.

Returns:

The fit model handler.

Return type:

DLModelHandler

predict(X, y=None, zout=None, plots_and_reports=True)

Compute predictions for the given input data.

Parameters:

X – The input data for the deep learning model, typically the structure space (e.g., the x, y, z, coordinates for a 3D point cloud) and potentially the feature space (e.g., the matrix whose rows give the point-wise features).
y (np.ndarray) – The vector of expected labels, the ground-truth from the supervised training perspective. While it is not necessary to compute predictions, when available it can be given because some models can use it for evaluation and analysis purposes.
zout – It can be given as an empty list in which case its last element after the call will contain the output from the last layer of the neural network, e.g., the softmax scores for a point-wise classification neural network. It can be None, in which case the output from the last layer will not be considered. Note also that zout does not necessarily return the softmax output, it can be defined to consider different output layers or metrics for some potential model.
plots_and_reports (bool) – Control whether to compute and export the plots and reports associated to the computation of predictions (True) or not (False).

Returns:

The predictions.

Return type:

np.ndarray

abstractmethod compile(X=None, y=None, y_rf=None, **kwargs)

The method that provides the logic to compile a model.

Parameters:

X – Optionally, the coordinates might be used for a better initialization (e.g., automatically derive the number of expected input points, or the dimensionality of the space where the points belong to).
y – Optionally, the labels might be used for a better initialization (e.g., automatically derive the number of classes).
y_rf – The expected values for each receptive field. Can be used to derive class weights.

Returns:

The model handler itself after compiling the architecture, which implies modifying its internal state.

Return type:

DLModelHandler

is_compiled()

Check whether the handled model has been compiled (True) or not (False).

Returns:: True if the handled model has been compiled, False otherwise.
Return type:: bool

overwrite_pretrained_model(spec)

Assist the model.Model.overwrite_pretrained_model() method for deep learning models.

See DLPretrainedHandler.

Parameters:: spec (dict) – The key-word specification containing the model’s arguments.

build_callbacks()

Build the callbacks for the model.

By default, the abstract baseline DLModelHandler does not provide an implementation to build callbacks. Derived classes that need to work with model callbacks must override this method to implement the building of any necessary callback.

Returns:: List of built callbacks
Return type:: list

model.deeplearn.handle.dl_model_reporter module

class model.deeplearn.handle.dl_model_reporter.DLModelReporter

Bases: object

Class to handle plots and reports for deep learning models. Note that the plots and reports that are responsibility of this handler are those involved in the operations directly handled by a DLModelHandler or any derived class like the SimpleDLModelHandler.

__init__()

handle_model_summary_report(mh)

Write the model’s summary and also print it through logging system.

This method requires that the model has been compiled so the summary is available.

Parameters:: mh – The model handler whose model’s summary must be reported.
Returns:: Nothing at all.

handle_history_plots_and_reports(mh)

Handle the generation of plots and reports from the training history of the given model handler.

Parameters:: mh – The model handler whose training history must be handled.
Returns:: Nothing at all, but the requested information (plots and reports) is written to the corresponding output files.

handle_receptive_fields_plots_and_reports(mh, X_rf, X=None, y=None)

Handle the plot and reports related to receptive fields including the decision on whether they must be plotted. This method also computes the necessary data (e.g., the model’s probabilities). Note that this method must only be called during fit. To plot and report the data of the receptive fields call the DLModelReporter.do_receptive_fields_plots_and_reports() method directly.

Parameters:

mh (DLModelHandler) – The model handler whose receptive fields’ plots and reports must be done.
X_rf – The input for the model as computed by the model’s pre-processor.
X – The structure space representing the original point cloud (not the receptive fields).
y (np.ndarray) – The vector of expected labels, the ground-truth from the supervised training perspective.

Returns:

Nothing at all.

do_receptive_fields_plots_and_reports(mh, X_rf, zhat_rf, X=None, y=None, training=False)

Do any plot and reports related to the receptive fields when handling a deep learning model.

Parameters:

mh (DLModelHandler) – The model handler whose receptive fields’ plots and reports must be done.
X_rf (np.ndarray) – The receptive fields such that X_rf[i] is the matrix of coordinates representing the points in the i-th receptive field.
zhat_rf (np.ndarray) – The output from the neural network for each receptive field.
X (np.ndarray or None) – The structure space representing the original input point cloud (i.e., not each receptive field). It is not always used, but sometimes it is necessary, e.g., to reduce labels when using a SpConv3DPwiseClassif architecture.
y (np.ndarray) – The expected class for each point (considering original points, i.e., not the receptive fields).
training (bool) – Whether the considered receptive fields are those used for training (True) or not (False).

Returns:

Nothing at all but the plots and reports are exported to the corresponding files.

static prepare_receptive_fields(mh, X_rf)

Prepare the receptive fields for the generation of plots and reports (see DLModelReporter.do_receptive_fields_plots_and_reports()) depending on the underlying model.

Parameters:

mh – The model handler.
X_rf – The original receptive field data.

Returns:

The prepared receptive field data as a tuple where the first element gives the structure space (point-wise coordinates) and the second gives the feature space (point-wise features).

Return type:

tuple

model.deeplearn.handle.dl_path_manager module

class model.deeplearn.handle.dl_path_manager.DLPathManager(**kwargs)

Bases: object

Class to handle the paths for deep learning models.

Variables:

summary_report_path (str) – Path to the file where the summary report will be written, i.e., the report that summarizes the model’s architecture.
training_history_dir (path) – Path to the directory where the training history plots and reports will be exported, i.e., information related to the training along many epochs.
checkpoint_path (str) – The path where the model’s checkpoint will be exported. It is used to keep the best model when using the checkpoint callback strategy during training.
feat_struct_repr_dir (str) – The path where the information relative to FeaturesStructuringLayer layers will be exported.
rbf_feat_extract_repr_dir (str) – The path where the information relative to RBFFeatExtractLayer layers will be exported.
rbf_feat_processing_repr_dir (str) – The path where the information relative to RBFFeatProcessingLayer layers will be exported.
kpconv_representation_dir (str) – The path where the information relative to KPConvLayer layers will be exported.
skpconv_representation_dir (str) – The path where the information relative to StridedKPConvLayer layers will be exported.
lkpconv_representation_dir (str) – The path where the information relative to LightKPConvLayer layers will be exported.
slkpconv_representation_dir (str) – The path where the information relative to StridedLightKPConvLayer layers will be exported.
kpconvx_representation_dir (str) – The path where the information relative to KPConvXLayer layers will be exported.

__init__(**kwargs): Initialize a DLPathManager.

update_paths(model_args, arch)

Consider the current specification of model handling arguments to update the paths.

Parameters:

model_args (dict) – The model arguments from where the new paths must be taken.
arch (Architecture) – The neural network architecture that must be updated, see Architecture.

add_to_state(state)

Add handled paths to state dictionary.

Parameters:: state (dict) – The state dictionary from a __getstate__() method. See simple_dl_model_handler.SimpleDLModelHandler.__getstate__() for an example.
Returns:: Nothing at all.

get_from_state(state)

Get paths to handle from state dictionary.

Parameters:: state (dict) – The state dictionary from a __setstate__() method. See simple_dl_model_handler.SimpleDLModelHandler.__setstate__() for an example.
Returns:: Nothing at all.

model.deeplearn.handle.dl_pretrained_handler module

class model.deeplearn.handle.dl_pretrained_handler.DLPretrainedHandler

Bases: object

Class to handle pretrained deep learning models.

__init__()

overwrite_pretrained_model(mh, spec)

Assist the model.Model.overwrite_pretrained_model() method for deep learning models.

See dl_model_handler.DLModelHandler.overwrite_pretrained_model().

Parameters:

mh (DLModelHandler) – The model handler to be updated.
spec (dict) – The key-word specification containing the model’s arguments.

Returns:

Nothing at all.

model.deeplearn.handle.dl_sequencer_builder module

class model.deeplearn.handle.dl_sequencer_builder.DLSequencerBuilder(sequencer_spec)

Bases: object

Class to handle the building of sequencers for deep learning models.

Variables:: sequencer_spec (dict) – The specification of the sequencer to be built.

__init__(sequencer_spec)

build(mh, X, y_rf, training)

Build the sequencer for the model handled by the given model handler (mh).

Parameters:

mh (DLModelHandler) – Model handler whose sequencer must be built.
X – The input data.
y_rf – The input reference values.
training (bool) – Whether the sequencer must be built for a training context (True) or a predictive context (False).

Returns:

The built sequencer.

Return type:

DLAbstractSequencer

model.deeplearn.handle.dl_training_handler module

class model.deeplearn.handle.dl_training_handler.DLTrainingHandler(freeze_training=None)

Bases: object

Class to handle the training of deep learning models.

Variables:: freeze_training (dict or None) – The freeze training specification, if any.

__init__(freeze_training=None)

__call__(mh, X, y_rf, callbacks)

Run all the fit logic, including the pre-fit and post-fit stages.

Parameters:

X – The pre-processed input.
y_rf – The point-wise labels for each receptive field.
callbacks (list) – The callbacks to be executed during fit.

Returns:

The fit history (also assigned internally to mh.history).

pre_fit(mh, X, y_rf, callbacks)

The pre-fit logic. Typically, it involves creating and populating a cache map that can be used by the underlying architecture (through Architecture.prefit_logic_callback() and the post-fit logic.a

See DLTrainingHandler.__call__() for details about the input arguments.

Returns:: The fit cache map/dictionary.
Return type:: dict

fit(mh, X, y_rf, callbacks)

The fit-logic. Typically, it involves preparing the training data (e.g., building the corresponding sequencer) and fitting the model. The freeze training logic is also handled here, if any.

See DLTrainingHandler.__call__() for details about the input arguments.

Returns:: The training history of the model.

model_fit(mh, callbacks, input_X, input_y)

Call the fit method of the model.

Parameters:

mh (DLModelHandler) – The model handler.
callbacks – See DLTrainingHandler.__call__().
input_X – The input training data ready to be fed into the neural network.
input_y – The input training references ready to be fed into the neural network.

Returns:

The history object representing the concrete training process.

store_fit_backup(mh, X, y_rf)

Store the values of the receptive fields before any sequencing so they can be later restored in exactly the same order as received when calling the DLTrainingHandler.fit() method.

See DLTrainingHandler.fit() and DLTrainingHandler.restore_fit_backup().

Parameters:

mh (DLModelHandler) – The model handler.
X – The input data representing the receptive fields.
y_rf – The references labeling the receptive fields.

Returns:

The file where the backup is stored or None if no backup was stored.

Return type:

file or None

restore_fit_backup(mh, input_X, fit_backup)

Restore the values of the receptive fields after fitting the model to their original values, i.e., those saved by DLTrainingHandler.fit() and DLTrainingHandler.store_fit_backup().

Parameters:

mh (DLModelHandler.) – The model handler.
input_X (DLAbstractSequencer or None) – The sequencer as built during the execution of DLTrainingHandler.fit().
fit_backup (file or None) – The temporary file where the backup is stored, if any.

Returns:

Nothing at all, but the values of the receptive fields at input_X are restored to their original order.

static check_architecture_needs_backup(mh)

Check whether the input receptive fields of the current architecture need to be restored after fitting (True) or not (False).

In general, some architectures (e.g., ConvAutoencPwiseClassif) need a backup of the receptive fields. At least when they are coupled with a input sequencer (see :class:.DLSequencer`).

Parameters:: mh (DLModelHandler) – The model handler.
Returns:: True if a backup of the input receptive fields is necessary. Otherwise, false.
Return type:: bool

static check_input_needs_backup(input_X)

Check whether the input receptive fields needs to be restored after fitting (True) or not (False).

In general, some sequencers (e.g., DLSequencer) need a backup of the receptive fields.

Parameters:: input_X (DLSequencer or list) – The input receptive fields.
Returns:: True if a backup of the input receptive fields is necessary. Otherwise, false.
Return type:: bool

post_fit(mh, X, y_rf, callbacks, fit_cache_map)

The post-fit logic. Typically, it involves digesting the cache map that was generated during the pre-fit logic and running some architecture-dependent logic (see Architecture.posfit_logic_callback())

See DLTrainingHandler.__call__() for details about the input arguments.

Returns:: Nothing at all.

prepare_freeze_training(mh)

Find the iterations at which the training process must be interrupted to apply freeze training-related operations.

See DLTrainingHandler.__call__() for details about the input arguments.

Returns:: The iterations at which a freeze-training related operations must be applied before computing further training iterations. For example, [5, 10, 15, 20] means that the training process must be interrupted at epochs 5, 10, 15, and 20 to apply some operation related to freeze training. Also, the specification of the operations that must be carried out at each relevant epoch. Finally, the initial learning rates, when given.
Return type:: tuple of (list of int, list of dict, and list of float)

handle_round_robin_freezing(layers, interval, strategy)

Extract the freeze training operations and their corresponding epochs following a round-robin strategy.

Parameters:

layers – The names of the layers that must be considered.
layers – list of str
interval (list of int) – The epoch interval between consecutive round-robin updates.
strategy (dict) –
The strategy specification, such that:

– iterative_span
The start (inclusive) and end (exclusive) epochs during which round-robin freeze operations must be applied.

– window_size
How many layers consider at most for each operation.

Returns:

A tuple with the epochs and the operations.

Return type:

tuple of list

handle_random_freezing(layers, interval, strategy)

Extract the freeze training operations and their corresponding epochs following a random strategy.

Parameters:

layers – The names of the layers that must be considered.
layers – list of str
interval (list of int) – The epoch interval between consecutive random updates.
strategy (dict) –
The strategy specification, such that:

– iterative_span
The start (inclusive) and end (exclusive) epochs during which random freeze operations must be applied.

– window_size
How many layers consider at most for each operation.

Returns:

A tuple with the epochs and the operations.

Return type:

tuple of list

model.deeplearn.handle.dl_transfer_handler module

class model.deeplearn.handle.dl_transfer_handler.DLTransferHandler(transfers)

Bases: object

Class to handle transfer learning operations, i.e., transferring the weights from a given model to the current one.

Variables:

weights_path (None or list of str) – Path to the weights that must be transferred to the current model. It can be either a .keras or a .weights.h5 file. When not given, no transfer will take place.
layer_translator (None or list of list of dict) – A list of dictionaries such that each dictionary gives a key representing the name of the layer in the current model and a value representing the name of the layer in the model whose weights must be transferred.
default_to_null (None or list of bool) – If true and no translation is specified in layer_translator, the layer weights will not be updated. If false (default), when the layer name is not included in the layer_translator it will be searched in the model weights assuming exactly the same name.
transfer_count (int) – How many times the weights have been transferred.

__init__(transfers)

transfer(mh)

Transfer the weights. Note that this method will only do the transfer one time, i.e., when transfer_count > 1 it will not transfer again. If the weights must be transferred anyway, it can be done by calling directly the DLTransferHandler.do_transfer() method.

Parameters:: mh (DLModelHandler) – The model handler that contains the model whose weights must be updated.
Returns:: Nothing at all, but the model in the model handler is updated in place.

do_transfer(mh)

Do the actual weight transfer between the handled model and the given weights.

See DLTransferHandler.transfer() for arguments documentation.

transfer_from_model_file(mh, i)

Transfer the weights assuming the file containing the weights is a model file, i.e., it represents the entire model not only the weights.

See DLTransferHandler.transfer() for arguments documentation.

Parameters:: i (int) – The transfer step that must be carried out.

transfer_from_weights_file(mh, i)

Transfer the weights assuming the file only contains the weights, i.e., it does not represent the entire model.

See DLTransferHandler.transfer() for arguments documentation.

Parameters:: i (int) – The transfer step that must be carried out.

can_transfer()

Check whether the DLTransferHandler object can transfer weights or not.

Returns:: True if transfer is possible considering given specification, False otherwise.
Return type:: bool

static extract_neural_network_from_model_handler(mh)

Get the neural network from the model handler.

Parameters:: mh (DLModelHandler) – The model handler whose neural network must be obtained.
Returns:: The neural network itself.

static translate_layer_name(lname, translator, default_to_null=False)

Translate the layer name considering the given translator (dictionary whose keys are original layer names and values are the translated layer name, with None for layers that must not be used).

Parameters:

lname (str) – The original layer name to be translated, if necessary.
translator (dict or None) – The dictionary to translate the original layer name. Keys match original layer names, values give corresponding translated names.
default_to_null (bool) – If true, then layer names that don’t have a corresponding key in the translator will be ignored. If false (default), then layer names that don’t have a corresponding key will be considered as the corresponding translated name.

Returns:

The final layer name or None

Return type:

str or None

update_paths(transfers)

Update the paths of the transfer handler from the given specification.

Parameters:: transfers (dict) – The dictionary specifying a transfer handler configuration.
Returns:: Nothing at all, but the member paths are updated.

model.deeplearn.handle.simple_dl_model_handler module

class model.deeplearn.handle.simple_dl_model_handler.SimpleDLModelHandler(arch, **kwargs)

Bases: DLModelHandler

Class to handle deep learning models in a simple way. It can be seen as the baseline deep learning model handler. See DLModelHandler.

Variables:

path_manager (DLPathManager) – The object that handles the paths involved in model handling.
transfer_handler (DLTransferHandler) – The object that handles transfer learning operations, if any.
out_prefix (str) – The output prefix for path expansions, when necessary.
training_epochs (int) – The number of training epochs for fitting the model.
batch_size (int) – The batch size governing the model’s input.
history (None or keras.callbacks.History) – By default None. It will be updated to contain the training history when calling fit.
checkpoint_monitor (str) – The name of the metric to choose the best model. By default, it is “loss”, which represents the loss function.
learning_rate_on_plateau (dict) – The key-word arguments governing the instantiation of the learning rate on plateau callback.
early_stopping (dict) – The key-word arguments governing the instantiation of the early stopping callback.
sequencer_spec (dict) – The specification on how to build the sequencer for the input data during model training. See SimpleDLModelHandler.build_sequencer().
fit_verbose (str or int) – Whether to use silent mode (0), show a progress bar (1), or print one line per epoch (2). Alternatively, “auto” can be used which typically means (1).
predict_verbose (str or int) – Whether to use silent mode (0), show a progress bar (1) or print one line per epoch (2). Alternatively, “auto” can be used which typically means (1).

__init__(arch, **kwargs)

Initialize/instantiate a simple deep learning model handler.

See DLModelHandler and dl_model_handler.DLModelHandler.__init__().

predict_rf(X_rf)

Compute the predictions on the given receptive fields.

Parameters:: X_rf (list or np.ndarray) – The receptive fields that must be predicted. It can be a list with the different inputs (e.g., for hierarchical models that use a hierarchical pre-processor like HierarchicalFPSPreProcessorPP or HierarchicalSGPreProcessorPP) or directly an input np.ndarray (e.g., input structure space).
Returns:: The predictions as directly computed by the model (e.g., the softmax probabilities from a point-wise classifier).
Return type:: np.ndarray

compile(X=None, y=None, y_rf=None, **kwargs): See DLModelHandler, dl_model_handler.DLModelHandler.compile(), DLModelCompiler, and dl_model_compiler.DLModelCompiler.compile().

build_callbacks(): See dl_model_handler.DLModelHandler.build_callbacks().

update_paths(model_args): Consider the current specification of model handling arguments to update the paths.

build_sequencer(X, y_rf, training)

Build/instantiate a sequencer from the given input data and specification.

Parameters:

X – The input data.
y_rf – The input reference values.
training – Whether the sequencer must be built for a training context (True) or a predictive context (False).

Returns:

The built sequencer.

Return type:

DLSequencer

Module contents

author:: Alberto M. Esmoris Pena

The handle package contains the logic to handle the components involved in deep learning models. For instance, deep learning models have an architecture that must be compiled before training. A handler can take the responsibility of compiling the architecture, so it can be used as a typical model with a fit/train and predict interface.