src.utils.preds.entropic_pred_reduce_strategy
Classes
|
- class src.utils.preds.entropic_pred_reduce_strategy.EntropicPredReduceStrategy(**kwargs)
- Author:
Alberto M. Esmoris Pena
Reduce many predictions per point to a single one by considering the point-wise entropies.
The reduced likelihoods for the \(n_c\) classes of the \(i\)-th point will be as shown below, assuming \(K_i\) values for the reduction.
\[\pmb{\hat{z}}_{i} = \dfrac{ \displaystyle \sum_{k=1}^{K_i}{(1-\hat{e}_{i_k}) \pmb{z}_{i_k}} }{ \displaystyle \sum_{k=1}^{K_i}{(1-\hat{e}_{i_k})} } \in (0, 1)^{n_c}\]In the above equation, \(\pmb{z}_{i_k} \in (0, 1)^{n_c}\) represents the \(k\) vector of likelihoods for the \(i\)-th point where \(k=1,\ldots,K_i\) (\(K_i\) is the number of neighborhoods where the \(i\)-th point appears).
To understand \(\hat{e}_{i_k}\) it is necessary to define the entropy for a classification task of \(n_c\) classes first such that:
\[\mathcal{E}(\pmb{z}_{i_k}) = - \sum_{c=1}^{n_c}{z_{i_kc} \log_2(z_{i_kc})} = \sum_{c=1}^{n_c}{f(z_{i_kc})}\]Now, consider the derivatives of \(f(z) = -z \log_2(z)\):
\[\dfrac{df}{dz} = - \dfrac{1+\ln(z)}{\ln(2)} \;,\quad \dfrac{d^2f}{dz^2} = - \dfrac{1}{z\ln(2)}\]Note that \(\dfrac{df}{dz} = 0 \iff z = e^{-1}\) and \(\dfrac{d^2f}{dz^2} < 0\). For then, \(f(e^{-1})\) will be a maximum, leading to an upper bound \(e^* = -n_ce^{-1}\log_2(e^{-1})\) such that \(\mathcal{E}(\pmb{z}_{i_k}) \leq e^*\). Consequently, it is possible to obtain normalized entropies such that:
\[\hat{e}_{i_k} = \dfrac{\mathcal{E}(\pmb{z}_{i_k})}{e^*} \in (0, 1)\]NOTE that a min clip value \(\epsilon_*\) will be considered to replace all values below it to avoid logarithm of zero or division by zero cases.
See
PredReduceStrategy.- Variables:
min_clip_value (float) – The minimum value that will be allowed. Values below it will be replaced to be the min clip value itself. This is useful to avoid division by zero and logarithm of zero cases. By default it is \(10^{-6}\).
- __init__(**kwargs)
Initialize/instantiate a mean prediction reduction strategy.
- Parameters:
kwargs – The attributes for the EntropicPredReduceStrategy.
- reduce(reducer, npoints, nvals, Z, I)
- reduce_single(reducer, npoints, nvals, Z, I)
Handle the reduce operations when nvals=1. See
EntropicPredReduceStrategy.reduce().
- reduce_many(reducer, npoints, nvals, Z, I)
Handle the reduce operation when nvals>1. See
EntropicPredReduceStrategy.reduce().