src.model.deeplearn.layer.features_downsampling_layer

Classes

FeaturesDownsamplingLayer(*args, **kwargs)

class src.model.deeplearn.layer.features_downsampling_layer.FeaturesDownsamplingLayer(*args, **kwargs)

Author:

Alberto M. Esmoris Pena

A features downsampling layer receives batches of \(m\) points with \(n_f\) features each and downsamples them to \(R\) points with \(n_f\) features each.

It can use a mean-based filter to reduce the features (see FeaturesDownsamplingLayer.mean_filter()), a max-based filter (see FeaturesDownsamplingLayer.max_filter()), a Gaussian-like filter to reduce the features considering the distances between the points (see FeaturesDownsamplingLayer.gaussian_filter()), or a exponential-like filter to reduce the features considering the distances to the contrary as the Gaussian-like filter, i.e., assigning greater weights to points further from the center (see FeaturesDownsamplingLayer.exponential_filter()). Alternatively, a simpler nearest-neighbor filter can be used too (see FeaturesDownsamplingLayer.nearest_filter()).

The input feature space is a tensor \(\mathcal{F} \in \mathbb{R}^{K \times m \times n_f}\) whose slices represent independent receptive fields. The output feature space is a downsampled version of the input one \(\mathcal{Y} \in \mathbb{R}^{K \times R \times n_f}\). Where \(K\) is the batch size.

Variables:

• filter (str) – The name of the filter to be used. Either "mean", "max, "gaussian", "exponential", or "nearest".

• filter_f (Callable) – The method to be used for filtering, derived from the filter attribute.

__init__(filter='mean', **kwargs)

See Layer and layer.Layer.__init__().

build(dim_in)

See Layer and layer.Layer.build().

call(inputs, training=False, mask=False)

Downsample the features from \(m\) input points to \(R\) output points, where \(R \leq m\).

Parameters:

inputs –

The inputs such that:

– inputs[0]

is the structure space tensor before the downsampling.

\[\mathcal{X}_a \in \mathbb{R}^{K \times m \times n_x}\]

– inputs[1]

is the structure space tensor after the downsampling.

\[\mathcal{X}_b \in \mathbb{R}^{K \times R \times n_x}\]

– inputs[2]

is the feature space tensor before the downsampling.

\[\mathcal{F} \in \mathbb{R}^{K \times m \times n_f}\]

– inputs[3]

is the indexing tensor representing the downsampling topology \(\mathcal{N}^D \in \mathbb{Z}^{K \times R \times n_n}\), i.e., for each point in the downsampled space to what points it is connected in the non-downsampled space.

Returns:

The downsampled features \(\mathcal{Y} \in \mathbb{R}^{K \times R \times n_f}\).

static mean_filter(Xa, Xb, Fin, ND)

\[y_{kij} = \dfrac{1}{n_n} \sum_{p=1}^{n_n}{ f_{kn_{kip}^Dj} }\]

Returns:

\(\mathcal{Y} \in \mathbb{R}^{K \times R \times n_f}\)

static max_filter(Xa, Xb, Fin, ND)

\[y_{kij} = \max_{1 \leq p \leq n_n} \quad \left\{ f_{kn_{kip}^{D}j} \right\}\]

Returns:

\(\mathcal{Y} \in \mathbb{R}^{K \times R \times n_f}\)

static gaussian_filter(Xa, Xb, Fin, ND)

\[y_{kij} = \left(\sum_{p=1}^{n_n}{g_{kip}}\right)^{-1} \sum_{p=1}^{n_n}{ g_{kip} f_{kn_{kip}^Dj} }\]

Where:

\[g_{kip} = \exp\left(-\dfrac{ \lVert (\mathcal{X}_a)_{kn_{kip}^D*} - (\mathcal{X}_b)_{ki*} \rVert^2 }{ (d_{ki}^*)^2 }\right)\]

And:

\[d_{ki}^* = \max_{1 \leq p \leq n_n} \; { \lVert (\mathcal{X}_a)_{kn_{kip}^D*} - (\mathcal{X}_b)_{ki*} \rVert }\]

Returns:

\(\mathcal{Y} \in \mathbb{R}^{K \times R \times n_f}\)

static exponential_filter(Xa, Xb, Fin, ND)

\[y_{kij} = \left(\sum_{p=1}^{n_n}{g_{kip}}\right)^{-1} \sum_{p=1}^{n_n}{ g_{kip} f_{kn_{kip}^Dj} }\]

Where:

\[g_{kip} = \exp\left(\dfrac{ \lVert (\mathcal{X}_a)_{kn_{kip}^D*} - (\mathcal{X}_b)_{ki*} \rVert^2 }{ (d_{ki}^*)^2 }\right)\]

And:

\[d_{ki}^* = \max_{1 \leq p \leq n_n} \; { \lVert (\mathcal{X}_a)_{kn_{kip}^D*} - (\mathcal{X}_b)_{ki*} \rVert }\]

Returns:

\(\mathcal{Y} \in \mathbb{R}^{K \times R \times n_f}\)

static nearest_filter(Xa, Xb, Fin, ND)

\[y_{kij} = f_{kn_{ip^*}^Dj}\]

Where:

\[p^* = \operatorname*{argmin}_{1 \leq p \leq n_n} \; { \lVert (\mathcal{X}_a)_{kn_{ip}^D*} - (\mathcal{X}_b)_{ki*} } \rVert^2\]

Returns:

\(\mathcal{Y} \in \mathbb{R}^{K \times R \times n_f}\)

static compute_squared_distances(Xa, Xb, ND)

Compute the squared distances between the downsampled points and those points in the non-downsampled space topologically connected to them (typically because they are the nearest neighbors).

Let \(\pmb{X}_a \in \mathbb{R}^{m \times n_x}\) be a structure space matrix (i.e., a matrix of point-wise coordinates) representing the points before the downsampling and \(\pmb{X}_b \in \mathbb{R}^{R \times n_x}\) be a structure space matrix representing the points after the downsampling. For then, the \(\pmb{N}^D \in \mathbb{Z}^{R \times n_n}\) indexing matrix represents the indices of the neighbors for each point in the downsampled space with respect to the non-downsampled space.

Considering all the previous matrices (that can be seen as the slices of the input tensors), it is possible to compute the distance matrix \(\pmb{D} \in \mathbb{R}^{R \times n_n}\). In this matrix, each element is given by the following expression:

\[d_{ij} = \lVert{ (\pmb{X}_b)_{i*} - (\pmb{X}_a)_{n^D_{ij}*} }\rVert\]

Note that the squared distances matrix is computed instead, which is the same as \(\pmb{D}\) but considering the element-wise squares.

Parameters:

• Xa – The structure space before downsampling.

• Xb – The structure space after downsampling.

• ND – The indexing tensor whose slices are indexing matrices.

Returns:

The tensor with the squared distances for each downsampled point with respect to its neighbors in the non-downsampled space.

Return type:

tf.Tensor

static gather_input_features(Fin, ND)

Gather the input features corresponding to given indexing tensor.

Let \(\mathcal{N}^D \in \mathbb{Z}^{K \times R \times n_n}\) be an indexing tensor of \(K\) batches of \(R\) points with \(n_n\) neighbors each. Also, let \(\mathcal{F} \in \mathbb{R}^{K \times R \times n_f}\) be a features tensor of \(K\) batches of \(m\) points with \(n_f\) features each.

The resulting tensor of gathered input features can be seen as a tensor \(\mathcal{Y} \in \mathbb{R}^{K \times R \times n_n \times n_f}\) of \(K\) batches of \(R\) points each such that for each point \(n_f\) features are gathered for each of its \(n_n\) neighbors. Note that, \(R \leq m\) because \(\mathcal{N}^D\) is an indexing tensor meant to be used for downsampling purposes.

Parameters:

• Fin – The input features.

• ND – The indexing tensor whose slices are indexing matrices.

Returns:

Tensor of gathered input features.

Return type:

tf.Tensor

get_config()

Return necessary data to serialize the layer

classmethod from_config(config)

Use given config data to deserialize the layer