eval.deeplearn package

Submodules

eval.deeplearn.dl_model_evaluation module

class eval.deeplearn.dl_model_evaluation.DLModelEvaluation(**kwargs)

Bases: Evaluation

Author:

Alberto M. Esmoris Pena

Class representing the result of evaluating a deep learning model. See DLModelEvaluator.

Variables:

  • X – The input data.

  • y – Expected values.

  • yhat – Point-wise predictions.

  • zhat – Point-wise outputs (e.g., softmax).

  • activations – Point-wise activations.

  • Fval – The F-values for the point-wise activations wrt the expected values.

  • pval – The p-values for the point-wise acctivations wrt the expected values.

  • class_names – The name for each class.

__init__(**kwargs)

Initialize/instantiate a DLModelEvaluation.

Parameters:

kwargs – The attributes for the DLModelEvaluation.

report(**kwargs)

Transform the DLModelEvaluation into a DLModelReport.

See DLModelReport.

Returns:

The DLModelReport representing the DLModelEvaluation.

Return type:

DLModelReport

can_report()

See Evaluation and evaluation.Evaluation.can_report().

eval.deeplearn.dl_model_evaluator module

class eval.deeplearn.dl_model_evaluator.DLModelEvaluator(**kwargs)

Bases: Evaluator

Author:

Alberto M. Esmoris Pena

Class to evaluate deep learning models.

Variables:

  • dlmodel (Model) – The deep learning model to be evaluated

  • pwise_output_path (str) – Where to export the point-wise output.

  • pwise_activations_path (str) – Where to export the point-wise activations.

  • accept_pipeline_state_predictions (bool) – Whether to accept predictions from a pipeline’s state (True) or not (False).

static extract_eval_args(spec)

Extract the arguments to initialize/instantiate a DLModelEValuator from a key-word specification.

Parameters:

spec – The key-word specification containing the arguments.

Returns:

The arguments to initialize/instantiate a DLModelEvaluator.

__init__(**kwargs)

Initialize/instantiate a DLModelEvaluator.

Parameters:

kwargs – The attributes for the DLModelEvaluator.

eval(X, y=None, **kwargs)

Evaluate the DL model.

Potential evaluations are the point-wise outputs of the model and the point-wise activations of a hidden layer.

Parameters:

  • X – Input data for the evaluation.

  • y – Expected values for the evaluation.

Returns:

The evaluation of the deep learning model.

Return type:

DLModelEvaluation

__call__(x, **kwargs)

Evaluate with extra logic that is convenient for pipeline-based execution.

See evaluator.Evaluator.eval().

lazy_prepare(state)

Prepare the DLModelEvaluator, so it can take the dlmodel from the pipeline’s state that was not available when instantiating the evaluator.

Parameters:

state (SimplePipelineState) – The pipeline’s state.

Returns:

Nothing, but the internal state of the DLModelEvaluator is updated.

eval_args_from_state(state)

Obtain the arguments to call the DLModelEvaluator from the current pipeline’s state.

Parameters:

state (SimplePipelineState) – The pipeline’s state

Returns:

The dictionary of arguments for calling DLModelEvaluator

Return type:

dict

Module contents

author:

Alberto M. Esmoris Pena

The deeplearn evaluation package contains the logic to handle the evaluation of deep learning models.