src.eval.advanced_classification_evaluator

Classes

AdvancedClassificationEvaluator(**kwargs)

class src.eval.advanced_classification_evaluator.AdvancedClassificationEvaluator(**kwargs)

Author:: Alberto M. Esmoris Pena

Class to evaluate classification-like predictions against expected/reference classes in an advanced way, i.e., by computing many evaluations considering different filters.

The arguments of the AdvancedClassificationEvaluator include those of the ClassificationEvaluator. Only those that are introduced by the AdvancedClassificationEvaluator are documented here. See ClassificationEvaluator for details on the common attributes.

Variables:

evaluator (ClassificationEvaluator.) – The ClassificationEvaluator used to compute the filter-wise classification evaluations.

filters (list of dict) –

List of filters such that one evaluation will be carried out for each filter. An example of filter is given below:

{
    "name": "pwe0_1",
    "x": 0.1,
    "conditions": [
        {
            "value_name": "classification",
            "condition_type": "not_equals",
            "value_target": 2,
            "action": "discard"
        },
        {
            "value_name": "PointWiseEntropy",
            "condition_type": "less_than_or_equal_to",
            "value_target": 0.1,
            "action": "preserve"
        }
    ]
}

The filter above will filter out all the points that are classified in the class 2 (third class, note they start at zero) and then it will consider only those that have a point-wise entropy \(\leq 1/10\). The value of the “x” attribute will be used in the figures to represent nodes along the \(x\)-axis and in the output CSV (report) to identify to which evaluation corresponds each row.

domain_name (str) – The name of the variable that constitutes the domain of the advanced evaluation (\(x\)-axis name).

static extract_eval_args(spec)

Extract the arguments to initialize/instantiate an AdvancedClassificationEvaluator from a key-word specification.

See ClassificationEvaluator.extract_eval_args().

Param:: spec The key-word specification containing the arguments.
Returns:: The arguments to initialize/instantiate an AdvancedClassificationEvaluator.

__init__(**kwargs)

Initialize/instantiate an AdvancedClassificationEvaluator.

Parameters:: kwargs – The attributes for the AdvancedClassificationEvaluator.

eval(yhat, y=None, fnames=None, F=None)

Evaluate predicted classes (\(\hat{y}\)) against expected/reference classes (\(y\)) one time for each given filter.

See ClassificationEvaluator.eval().

__call__(pcloud, **kwargs)

Evaluate with extra logic that is convenient for pipeline-based execution.

See Evaluator.eval().

apply_filter(f, yhat, y=None, fnames=None, F=None)

Apply the given filter on the predictions and reference classes to extract the subset of predicted classes that must be evaluated.

Parameters:

f (dict) – The filter to be applied.
yhat (np.ndarray) – The predicted classes to be filtered.
y (np.ndarray) – The reference classes to be filtered.
fnames (list of str) – The names of the features in \(\pmb{F}\).
F (np.ndarray) – The feature space matrix representing the point cloud to be evaluated.

Returns:

Return the domain node (x), filtered predictions, and filtered reference classes as a 3-tuple.

Return type:

tuple

apply_condition(mask, cond, yhat, y=None, fnames=None, F=None, name=None)

Apply the given condition to update the input mask (in place).

Parameters:

mask (np.ndarray of bool) – Boolean mask to be updated. It must specify True for points that must be considered for the evaluation, False otherwise.
cond (dict) – The condition specification.
yhat (np.ndarray) – The predictions.
y (np.ndarray) – The expected/reference classes.
fnames (list of str) – The names of the features in the feature space matrix \(\pmb{F}\).
F (np.ndarray) – The feature space matrix.
name (str) – The name of the filter to which the condition belongs.

get_fnames_from_filters()

Obtain the names of the features that must be considered by the filters.

Returns:: List with the names of the features that must be considered by the filters.
Return type:: list of str

eval_args_from_state(state)

Obtain the arguments to call the AdvancedClassificationEvaluator from the current pipeline’s state.

Parameters:: state (SimplePipelineState) – The pipeline’s state.
Returns:: The dictionary of arguments for calling AdvancedClassificationEvaluator.
Return type:: dict