src.model.deeplearn.layer.features_structuring_layer

Classes

FeaturesStructuringLayer(*args, **kwargs)

class src.model.deeplearn.layer.features_structuring_layer.FeaturesStructuringLayer(*args, **kwargs)

Author:: Alberto M. Esmoris Pena

A features structuring layer is governed by a features structuring kernel.

A features structuring kernel consists of a structure matrix \(\pmb{Q_X} \in \mathbb{R}^{K \times n_x}\) for a structure space of dimensionality \(n_x\) (typically 3D, i.e., \(x, y, z\)), a features matrix \(\pmb{Q_F} \in \mathbb{R}^{K \times n_f}\) and a kernel distance function \(d_Q(\pmb{q_x}, \pmb{x}) \in \mathbb{R}_{>0}\) that quantifies the distance in the structure space between any kernel point wrt any input point.

A features structuring kernel is generated from three main parameters. First, \(\pmb{r^*} \in \mathbb{R}^{n_x}\) defines the radius along each axis for an ellipsoid, typically in 3D \(\pmb{r^*} = (r^*_x, r^*_y, r^*_z)\) . Second, a radii resolution \(n \in \mathbb{Z}_{> 0}\) defines how many concentric ellipsoids will be considered where the first one is the center point and the last one corresponds to the \(\pmb{r^*}\) radii vector. Finally, \(m_1, \ldots, m_n\) define the angular resolutions such that \(m_k \in \mathbb{Z}_{>0}\) governs how many partitions along the angular domain of the \(\alpha\) and \(\beta\) angles must be considered.

Any features structuring layer expects to receive two inputs in the following order:

A structure space matrix representing a point cloud
A matrix which rows are the point-wise features for the points in 1).

Variables:

max_radii (np.ndarray of float) – The radius of the last ellipsoid along each axis \(\pmb{r}^* \in \mathbb{R}^{n_x}\).
radii_resolution (int) – How many concentric ellipsoids must be considered \(n \in \mathbb{Z}_{>0}\) (the first one is the center point, the last one is the biggest outer ellipsoid).
angular_resolutions (np.ndarray of int) – How many angles consider for each ellipsoid \((m_1, \ldots, m_n)\).
num_kernel_points (int) – The number of points representing the kernel \(K \in \mathbb{Z}_{>0}\).
dim_out (int) – The output dimensionality, it governs the trainable matrix \(\pmb{Q_W} \in \mathbb{R}^{n_f \times n_y}\).
concatenation_strategy (str) – Specify the concatenation strategy defining the output of the layer. It can be “FULL” so the generated output will be passed together with the input or “OPAQUE” in which case the input will not be forwarded as output.
num_features (int) – The number of features per point received as input \(n_f \in \mathbb{Z}_{>0}\).
concatf (func) – The function implementing the concatenation strategy.
QX (tf.Tensor) – The kernel’s structure matrix.
trainable_QX (bool) – Flag to control whether QX is trainable or not.
built_QX (bool) – Flag to control whether QX has been built or not.
omegaD (tf.Tensor) – The kernel’s distance weights.
trainable_omegaD (bool) – Flag to control whether omegaD is trainable or not.
built_omegaD (bool) – Flag to control whether omegaD has been built or not.
omegaF (tf.Tensor) – The kernel’s feature weights.
trainable_omegaF (bool) – Flag to control whether omegaF is trainable or not.
built_omegaF (bool) – Flag to control whether omegaF has been built or not.
QW (tf.Tensor) – The kernel’s weights matrix.
trainable_QW (bool) – Flag to control whether QW is trainable or not.
built_QW (bool) – Flag to control whether QW has been built or not.

__init__(max_radii, radii_resolution=4, angular_resolutions=(1, 2, 4, 8), structure_dimensionality=3, dim_out=4, concatenation_strategy='FULL', trainable_QX=False, trainable_QW=True, trainable_omegaD=True, trainable_omegaF=True, built_QX=False, built_omegaD=False, built_omegaF=False, built_QW=False, **kwargs): See Layer and deeplearn.layer.layer.Layer.__init__().

build(dim_in)

Build the \(\pmb{Q_X} \in \mathbb{R}^{K \times n_x}\) matrix representing the kernel’s structure, and the \(\pmb{Q_W} \in \mathbb{R}^{n_f \times n_y}\) matrix representing the kernel’s weights as well as the \(\pmb{\omega_D} \in \mathbb{R}^{K}\) and \(\pmb{\omega_F} \in \mathbb{R}^{n_f}\) vectors.

The \(\pmb{Q_X}\) matrix represents the disposition of the kernel’s points while the \(\pmb{Q_W}\) is a matrix of weights that defines a potentially trainable transformation on the features.

The \(\pmb{\omega_D}\) vector governs the relevance of the distance wrt to each kernel point when weighting the features, and the \(\pmb{\omega_F}\) vectors governs the relevance of each feature independently of the distance in the structure space.

See Layer and layer.Layer.build().

call(inputs, training=False, mask=False)

The computation of the \(\pmb{Q_F} \in \mathbb{R}^{K \times n_f}\) matrix and the output features \(\pmb{Q_Y}\) obtained after matrix multiplication wrt \(\pmb{Q_W}\).

Let the distance function between the point i of the kernel and the point j of the input point cloud be:

\[d_Q(\pmb{q_{Xi*}}, \pmb{x_{j*}}) = \exp\left[ - \dfrac{\lVert{ \pmb{x_{j*}} - \pmb{q_{Xi*}} }\rVert^2}{ \omega_{Di}^2 } \right]\]

\[\pmb{Q_F} \in \mathbb{R}^{k \times n_f} \;\text{ s.t. }\; q_{Fik} = \sum_{j=1}^{m}{ d_Q(\pmb{q_{Xi*}}, \pmb{x_{j*}}) \omega_{Fk} f_{jk} }\]

Alternatively, \(\pmb{Q_F}\) rows can be expressed as a sum of vectors (where \(\odot\) represents the Hadamard product):

\[\pmb{q_{Fi*}} = \sum_{j=1}^{m}{ d_Q(\pmb{q_{Xi*}}, \pmb{x_{j*}}) ( \pmb{\omega_F} \odot \pmb{f_{j*}} ) }\]

Note also that there exists a distance matrix \(\pmb{Q_D} \in \mathbb{R}^{k \times m}\) describing how close each point from the input point cloud is wrt each kernel’s point. It can be computed as:

\[\begin{split}\pmb{Q_D} = \left[\begin{array}{ccc} q_{D11} & \ldots & q_{D1m} \\ \vdots & \ddots & \vdots \\ q_{Dk1} & \ldots & q_{Dkm} \end{array}\right] \; \text{ s.t. } \; Q_{Dij} = d_Q(\pmb{q_{Xi*}}, \pmb{x_{j*}})\end{split}\]

For then, the output matrix can be calculated as:

\[\pmb{Q_Y} = \pmb{Q_D}^\intercal \pmb{Q_F} \pmb{Q_W}\]

See Layer and layer.Layer.call().

Returns:: The output features. Depending on the concatenation strategy they can be FULL \([\pmb{F}, \pmb{Q_Y}]\) or OPAQUE \([\pmb{Q_Y}]\).

static concatf_full(F, QY)

Returns:: \([\pmb{F}, \pmb{Q_Y}]\)

static concatf_opaque(F, QY)

Returns:: \([\pmb{Q_Y}]\)

assign_concatf()

Assign the concatf function from the current state of the object.

Returns:: Nothing, but self.concatf is updated.

get_config(): Return necessary data to serialize the layer

classmethod from_config(config): Use given config data to deserialize the layer

export_representation(dir_path, out_prefix=None, QXpast=None)

Export a set of files representing the state of the features structuring kernel.

Parameters:

dir_path (str) – The directory where the representation files will be exported.
out_prefix (str) – The output prefix to name the output files.
QXpast (np.ndarray or tf.Tensor or None) – The structure matrix of the layer in the past.

Returns:

Nothing at all, but the representation is exported as a set of files inside the given directory.