model package

Subpackages

Submodules

model.classification_model module

class model.classification_model.ClassificationModel(**kwargs)

Bases: Model, ABC

Author:

Alberto M. Esmoris Pena

Abstract class providing some baseline methods for classification models.

static extract_model_args(spec)

Extract the arguments to initialize/instantiate a ClassificationModel from a key-word specification.

Parameters:

spec – The key-word specification containing the arguments.

Returns:

The arguments to initialize/instantiate a ClassificationModel.

__init__(**kwargs)

Initialization for any instance of type ClassificationModel

overwrite_pretrained_model(spec)

See model.Model.overwrite_pretrained_model().

autoval(y, yhat, info=True)

Auto validation during training for classification models.

Parameters:

  • yhat – The predicted classes.

  • y – The expected classes.

  • info – True to log an info message with the auto validation, False otherwise.

Returns:

The results of the auto validation.

Return type:

np.ndarray

on_training_finished(X, y, yhat=None)

See model.Model.on_training_finished().

static autoval_metrics_from_names(names)

See classification_evaluator.ClassificationEvaluator.metrics_from_names() .

model.decorated_model module

class model.decorated_model.DecoratedModel(**kwargs)

Bases: Model

Author:

Alberto M. Esmoris Pena

Abstract decorator for machine learning models that provides the common logic for different types of model decorators.

See Model.

Variables:

  • decorated_model_spec (dict) – The specification of the decorated model.

  • decorated_model (Model) – The decorated model object.

  • undecorated_predictions (bool) – Whether to use the decorated model without decoration when computing the predictions (true) or not (false).

static extract_model_args(spec)

Extract the arguments to initialize/instantiate a DecoratedModel from a key-word specification.

Parameters:

spec – The key-word specification containing the arguments.

Returns:

The arguments to initialize/instantiate a DecoratedModel.

__init__(**kwargs)

Initialization for any instance of type DecoratedModel.

predict(pcloud, X=None)

Decorate the main predictive logic to work on the representation generated by the decorator.

See Model and Model.predict().

prepare_model()

Prepare the decorated model. See Model and Model.prepare_model().

overwrite_pretrained_model(spec)

Overwrite the decorated pretrained model. See Model and Model.overwrite_pretrained_model().

get_input_from_pcloud(pcloud)

Get input from the decorated pretrained model. See Model and Model.get_input_from_pcloud().

is_deep_learning_model()

Check whether the decorated model is a deep learning model. See Model and Model.is_deep_learning_model().

training(X, y, info=True)

Use the training logic of the decorated model. See Model and Model.training().

autoval(y, yhat, info=True)

Auto validation during training through decorated model. See Model and Model.autoval().

train_base(pcloud)

Straightforward training through decorated model. See Model and Model.train_base().

train_autoval(pcloud)

Use autovalidation training strategy from decorated model. See Model and Model.train_autoval().

train_stratified_kfold(pcloud)

Use stratified k-fold training strategy from decorated model. See Model and Model.train_stratified_kfold().

on_training_finished(X, y)

Use on training finished callback from decorated model. See Model and Model.on_training_finished().

abstractmethod decorate_pcloud(pcloud)

Obtain the representation of the input point cloud generated by the decorator.

Parameters:

pcloud (PointCloud) – The input point cloud whose decorated representation must be computed.

Returns:

The decorated representation of the input point cloud.

Return type:

PointCloud

abstractmethod propagate(rf_yhat)

Propagate the values (typically predictions) from the receptive field back to the original point cloud.

Parameters:

rf_yhat (np.ndarray) – The values (typically predictions) in the receptive field representation generated by the decorator.

Returns:

The values (typically predictions) propagated back to the original point cloud.

Return type:

np.ndarray

get_fnames_recursively()

Find through any potential decoration graph until the deepest model is found, then consider its feature names.

Returns:

The feature names of the deepest model in the decoration hierarchy.

Return type:

list of str

property fnames

Getter for feature names to get them from the decorated model. :return: The feature names from the decorated model. :rtype: None or list of str

model.fps_decorated_model module

class model.fps_decorated_model.FPSDecoratedModel(**kwargs)

Bases: DecoratedModel

Author:

Alberto M. Esmoris Pena

Decorator for machine learning models that makes the decorated model work on a FPS-based representation of the point cloud.

The FPS Decorated Model (FPSDecoratedModel) constructs a representation of the point cloud, then it calls the model on this representation and, when used for predicting, it propagates the predictions back to the original point cloud (the one from which the representation was built).

See FPSDecoratorTransformer and DecoratedModel.

Variables:

  • fps_decorator_spec (dict) – The specification of the FPS transformation defining the decorator.

  • fps_decorator (FPSDecoratorTransformer) – The FPS decorator to be applied on input point clouds.

static extract_model_args(spec)

Extract the arguments to initialize/instantiate a FPSDecoratedModel from a key-word specification.

Parameters:

spec – The key-word specification containing the arguments.

Returns:

The arguments to initialize/instantiate a FPSDecoratedModel.

__init__(**kwargs)

Initialization for any instance of type FPSDecoratedModel.

train(pcloud)

Decorate the main training logic to work on the representation. See Model and Model.train().

decorate_pcloud(pcloud)

See DecoratedModel.decorate_pcloud().

propagate(rf_yhat)

See DecoratedModel.propagate().

model.mindist_decorated_model module

class model.mindist_decorated_model.MinDistDecoratedModel(**kwargs)

Bases: DecoratedModel

Author:

Alberto M. Esmoris Pena

Decorator for machine learning models that makes the decorated model work on a minimum distance decimation-based representation of the point cloud, then it calls the model in this representation and, when used for predicting, it propagates the predictions back to the original point cloud (the one from which the representation was built).

See MinDistDecimatorDecorator and DecoratedModel.

Variables:

  • mindist_decorator_spec (dict) – The specification of the minimum distance decimation transformation defining the decorator.

  • mindist_decorator (MinDistDecimatorDecorator) – The minimum distance decimator decorator to be applied on input point clouds.

static extract_model_args(spec)

Extract the arguments to initialize/instantiate a MinDistDecoratedModel from a key-word specification.

Parameters:

spec – The key-word specification containing the arguments.

Returns:

The arguments to initialize/instantiate a MinDistDecoratedModel.

__init__(**kwargs)

Initialization for any instance of type MinDistDecoratedModel.

train(pcloud)

Decorate the main training logic to work on the representation. See Model and Model.train().

decorate_pcloud(pcloud)

See DecoratedModel.decorate_pcloud().

propagate(rf_yhat)

See DecoratedModel.propagate().

model.model module

exception model.model.ModelException(message='')

Bases: VL3DException

Author:

Alberto M. Esmoris Pena

Class for exceptions related to model components. See VL3DException.

__init__(message='')
class model.model.Model(**kwargs)

Bases: object

Author:

Alberto M. Esmoris Pena

Abstract class providing the interface governing any model.

Variables:

  • training_type (str) – The type of training. Either “base”, “autoval”, or “stratified_kfold”.

  • autoval_size (int or float) – The size of the auto validation set (used by model.Model.train_autoval()). It can be given as a ratio in [0, 1] or as the set cardinality in [0, m].

  • shuffle_points (bool) – Flag governing whether to shuffle the points (True) or not (False). It is used by model.Model.train_autoval() and model.Model.train_stratified_kfold().

  • num_folds (int) – The number of folds, i.e. K for K-Folding.

  • imputer (Imputer) – The imputer to deal with missing values (can be None).

  • fnames (list) – The list of feature names (fnames) attribute. These features must correspond to the features in the input point cloud for training and predictions.

  • random_seed (int) – Optional attribute to specify a fixed random seed for the random computations of the model.

  • stratkfold_report_path (str) – The path where the report representing the evaluation of the k-folding procedure must be written.

  • stratkfold_plot_path (str) – The path where the plot representing the evaluation of the k-folding procedure must be written.

  • training_data_pipeline (list of dict) – List of dictionaries such that each dictionary provides a key-word specification of a component to be used to transform or select the training data. Note that the components will be sequentially applied to the data in the same order they are given in the list.

static extract_model_args(spec)

Extract the arguments to initialize/instantiate a Model from a key-word specification.

Parameters:

spec – The key-word specification containing the arguments.

Returns:

The arguments to initialize/instantiate a Model.

__init__(**kwargs)

Root initialization for any instance of type Model.

Parameters:

kwargs – The attributes for the Model.

train(pcloud)

Train the model from a given input point cloud.

Parameters:

pcloud – The input point cloud to train the model with.

Returns:

The trained model.

Return type:

Model

predict(pcloud, X=None)

Use the model to compute predictions on the input point cloud.

Parameters:

  • pcloud – The input point cloud to compute the predictions.

  • X – The input matrix of features where each row represents a point from the point cloud (OPTIONAL). If given, X will be used as point-wise features instead of pcloud. It is often named F in the context of point clouds where the point cloud is a block matrix P = [X|F].

Returns:

The point-wise predictions.

Return type:

np.ndarray

abstractmethod prepare_model()

Prepare the model so its model attribute (i.e., model.model) can be used, for instance by a hyperparameter tuner.

Returns:

The prepared model itself. However, the prepared model must be automatically assigned as an attribute of the object/instance too.

overwrite_pretrained_model(spec)

This method must be called when preparing a pretrained model to overwrite any attribute that must be overriding depending on the model and the given training specification.

Parameters:

spec (dict) – The key-word training specification to continue the training of the pretrained model.

Returns:

Nothing, but the model object internal state is updated.

get_input_from_pcloud(pcloud)

Obtain the model-ready input from the given point cloud.

Parameters:

pcloud – The point cloud containing the data to fit the model.

Returns:

Model-ready input data.

is_deep_learning_model()

Check whether a model is a deep learning model or not.

By default, models are not deep learning models. Those models which are a deep learning model should explicitly overwrite this method to return True. It is necessary for correct pipelines.

Returns:

True if the model is a deep learning model, False otherwise.

abstractmethod training(X, y, info=True)

The fundamental training logic defining the model.

It must be overridden by non-abstract subclasses.

Parameters:

  • X – The input matrix representing the point cloud, e.g., the geometric features matrix.

  • y – The class for each point.

  • info – True to enable info log messages, False otherwise.

Returns:

Nothing, but the model itself is updated.

autoval(y, yhat, info=True)

Auto validation during training.

Any non-abstract subclass must provide an implementation for autoval or avoid calling model.Model.train_autoval() and model.Model.train_stratified_kfold to prevent exceptions/errors.

Parameters:

  • y – The expected values.

  • yhat – The predicted values.

  • info – True to log an info message with the auto validation, False otherwise.

Returns:

The results of the auto validation.

train_base(pcloud)

Straightforward model training.

Parameters:

pcloud – The input point cloud to train the model.

Returns:

The trained model.

Return type:

Model

train_autoval(pcloud)

Auto validation training strategy.

Some part of the data is ignored during training to be used for later validation of the trained model (auto validation).

The subsets are extracted in a stratified way. See model.Model.train_stratified_kfold() for a description of stratification.

Parameters:

pcloud – The input point cloud to train the model.

Returns:

The trained model.

Return type:

Model

train_stratified_kfold(pcloud)

Stratified k-fold training strategy.

Stratification consists of dividing the data into subsets (strata). Test points are taken from each subset, i.e., each stratum of the strata, so they follow a class distribution approximately proportional to the original set. This strategy guarantees that the test points constitute a reliable representation of the original points.

K-folding consists of dividing the data into K different subsets (folds). Then, K iterations are computed such that at each one a different fold is considered as the test set and the other (K-1) folds are used for training.

Stratified K-folding is K-folding with stratified folds, i.e., each fold is also a stratum.

Parameters:

pcloud – The input point cloud to train the model.

Returns:

The trained model.

Return type:

Model

on_training_finished(X, y)

Callback method that must be invoked by any training strategy after finishing the training but before returning the trained model.

Parameters:

  • X – The point-wise features matrix with points as rows and features as columns.

  • y – The point-wise expected classes.

Returns:

Nothing.

optimize_data_types(X=None, y=None)

Optimize the data types, depending on the current configuration of the framework.

Parameters:

  • X – The input data.

  • y – The reference data.

Returns:

The updated data types.

Return type:

tuple of np.ndarray

model.model_op module

exception model.model_op.ModelOpException(message='')

Bases: VL3DException

Author:

Alberto M. Esmoris Pena

Class for exceptions related to model operations. See VL3DException

class model.model_op.ModelOp(model, op)

Bases: object

Author:

Alberto M. Esmoris Pena

Class wrapping a model associated to an operation (useful to handle different model operations like train and predict during pipelines).

Variables:

  • model (Model) – The model.

  • op (enum) – The operation.

class OP(*values)

Bases: Enum

TRAIN = 1
PREDICT = 2
__init__(model, op)

Root initialization for any instance of type ModelOp.

Parameters:

  • model – The model associated to the operation.

  • op – The operation associated to the model.

__call__(pcloud=None, out_prefix=None)

Make the model do the operation.

Parameters:

  • pcloud (PointCloud or None) – Input point cloud. Train and predict operations require it is not None.

  • out_prefix – Optional argument to specify the output prefix for any path in the model that starts with ‘*’.

Returns:

Whatever the operation returns.

update_model_paths(out_prefix)

Update the output paths in the model object.

For each path that starts with “*” it will be updated to replace the “*” by the prefix.

Parameters:

out_prefix (str) – The prefix for the output paths. It is expected to end with “*”.

Returns:

A dictionary with the paths before the update. Note the model is updated in place.

Return type:

dict

update_dl_model_paths(out_prefix, old_paths)

Update the model assuming it is a deep learning model. This method will not check that self.model is a deep learning model. Instead, it assumes it is because otherwise the method MUST NOT be called.

See model_op.ModelOp.update_model_paths().

Parameters:

old_paths (dict) – The output parameter. It must be a dictionary where any modified path (value) will be stored before it is updated, associated to its variable (key).

Returns:

Nothing at all, the old_paths list argument is updated if necessary.

restore_model_paths(old_paths)

Restore previously updated model paths to their original values.

Parameters:

old_paths – The dictionary with the paths before the update.

Returns:

Nothing, the model is restored in place.

restore_dl_model_paths(old_paths)

Restore the model assuming it is a deep learning model. This method will not check that self.model is a deep learning model. Instead, it assumes it is because otherwise the method MSUT NOT be called.

See model_op.ModelOp.restore_model_paths().

Parameters:

old_paths (dict) – The output parameter. It must be a dictionary where any variable (key) that needs to restore its old path (value) is represented.

Returns:

Nothing at all, the old_paths dictionary is usted to restore the variables of the model to its previous value.

static path_needs_update(path: str) → bool

Check whether a model path needs to be updated or not.

A model path needs to be updated if it starts by “*”.

Parameters:

path – The model path to be checked.

Returns:

True if the model path needs to be updated, False otherwise.

Return type:

bool

static merge_path(out_prefix: str, path: str) → str

Merge the output prefix to the model path assuming the output prefix ends by “*” and the model path starts with “*”.

Parameters:

  • out_prefix – The output prefix for the merge.

  • path – The model path for the merge.

Returns:

The merged path.

Return type:

str

model.random_forest_classification_model module

class model.random_forest_classification_model.RandomForestClassificationModel(**kwargs)

Bases: ClassificationModel

Author:

Alberto M. Esmoris Pena

RandomForest model for classification tasks. See Model.

Variables:

  • model_args (dict) – The arguments to initialize a new RandomForest model.

  • model (RandomForestClassifier) – The internal representation of the model.

  • importance_report_path (str) – Path to the file to store the report.

  • importance_report_permutation – Flag to control whether to include the permutation importance in the report (True, default) or not (False).

  • decision_plot_path (str) – Path to the file to store the plots representing the decision trees in the random forest. If only one decision tree is going to be exported, the path is used literally. Otherwise, incrementally updated paths by appending “_n” before the file extension will be considered.

  • decision_plot_trees (int) – The number of decision trees to consider. If -1, then all the decision trees will be considered.

static extract_model_args(spec)

Extract the arguments to initialize/instantiate a RandomForestClassificationModel from a key-word specification.

Parameters:

spec – The key-word specification containing the arguments.

Returns:

The arguments to initialize/instantiate a RandomForestClassificationModel

__init__(**kwargs)

Initialize an instance of RandomForestModel.

Parameters:

kwargs – The attributes for the RandomForestClassificationModel that will also be passed to the parent.

prepare_model()

Prepare a random forest classifier with current model arguments

Returns:

The prepared model itself. Note it is also assigned as the model attribute of the object/instance.

training(X, y, info=True)

The fundamental training logic to train a random forest classifier.

See ClassificationModel and Model. Also see model.Model.training().

on_training_finished(X, y, yhat=None)

See model.Model.on_training_finished().

Module contents

author:

Alberto M. Esmoris Pena

The model package contains the logic to handle models for classification and regression problems on point clouds.