model.deeplearn.layer package

Submodules

model.deeplearn.layer.contextual_point_layer module

class model.deeplearn.layer.contextual_point_layer.ContextualPointLayer(*args, **kwargs)

Bases: Layer

Author:

Alberto M. Esmoris Pena

A contextual point layer receives batches of \(R\) points with \(D_{\text{in}}\) input features, and \(\kappa\) known neighbors in the same space. This layer combines features from local neighborhoods with features from local neighborhoods weighted by spatial distances, and global features for the entire input batch and positional information.

Formally speaking, for a batch size \(B \in \mathbb{Z}_{>0}\) the input consists of a \(n_x\)-dimensional structure space tensor \(\mathcal{X} \in \mathbb{R}^{B \times R \times n_x}\), a \(D_{\text{in}}\)-dimensional feature space tensor \(\mathcal{F} \in \mathbb{R}^{B \times R \times D_{\text{in}}}\), and a tensor indexing the \(\kappa \in \mathbb{Z}_{>0}\) neighbors for each point \(\mathcal{N} \in \mathbb{R}^{B \times R \times \kappa}\).

First, the global information is computed for each element in the batch as a matrix \(\pmb{G} \in \mathbb{R}^{R \times D_H}\) such that

\[\pmb{G} = \sigma_{\gamma} \left(\mathcal{Z}_{\gamma}\left( \pmb{F} \pmb{\Gamma} \oplus \pmb{\gamma} \right)\right) ,\]

where \(D_H \in \mathbb{Z}_{>0}\) is the dimensionality of the hidden feature space, \(\oplus\) represents the vector sum broadcast over a matrix (typical in machine learning contexts), \(\sigma_{\gamma}\) is an activation function (typically a ReLU), \(\mathcal{Z}_{\gamma}\) represents batch normalization, \(\pmb{\Gamma} \in \mathbb{R}^{D_{\text{in}} \times D_H}\) is the matrix of weights, and \(\pmb{\gamma} \in \mathbb{R}^{D_H}\) the vector of weights.

Then, the features from local neighborhoods start by composing the tensor \(\mathcal{\widetilde{F}} \in \mathbb{R}^{R \times \kappa \times D_{\text{in}}}\) that represents the features for the \(\kappa\) neighbors for each of the \(R\) points. Now, the features of the neighbors can be transformed such that

\[\mathcal{H} = \sigma_{\phi}\left( \mathcal{Z}_{\phi}\left( \mathcal{\widetilde{F}} \pmb{\Phi} \oplus \pmb{\phi} \right)\right) ,\]

where \(\sigma_{\phi}\) is an activation function (typically a ReLU), \(\mathcal{Z}_{\phi}\) represents batch normalization, \(\pmb{\Phi} \in \mathbb{R}^{D_{\text{in}} \times D_H}\) is the matrix of weights, and \(\pmb{\phi} \in \mathbb{R}^{D_H}\) is the vector of weights.

Now, we need to compute a matrix of point-wise distances for each local neighborhood \(\pmb{D} \in \mathbb{R}^{R \times \kappa}\). One alternative is to do this with the Euclidean distances or standard vector norms, i.e., \(d_{ij} = \lVert \pmb{x}_{j*} - \pmb{x}_{i*} \rVert\). The other alternative is to consider the squared distances, i.e., \(d_{ij} = \lVert \pmb{x}_{j*} - \pmb{x}_{i*} \rVert^2\). It can be interesting to force an ascending order such that \(d_{ij} \leq d_{i(j+1)}\). With this matrix we can compute a hidden distance matrix \(\pmb{\widetilde{D}} \in \mathbb{R}^{R \times D_H}\)

\[\pmb{\widetilde{D}} = \sigma_{\psi}\left( \mathcal{Z}_{\psi}\left( \pmb{D} \pmb{\Psi} \oplus \pmb{\psi} \right)\right) ,\]

where \(\sigma_{\psi}\) is an activation function (typically a ReLU), \(\mathcal{Z}_{\psi}\) represents batch normalization, \(\pmb{\Psi} \in \mathbb{R}^{\kappa \times D_H}\) is the matrix of weights, and \(\pmb{\psi} \in \mathbb{R}^{D_H}\) is the vector of weights.

The \(\pmb{\widetilde{D}}\) matrix is then reduced again to a \(\kappa\)-dimensional space such that

\[\pmb{\widehat{D}} = \sigma_{\hat{\psi}}\left( \mathcal{Z}_{\hat{\psi}}\left( \pmb{\widetilde{D}} \pmb{\widehat{\Psi}} \oplus \pmb{\hat{\psi}} \right)\right) ,\]

where \(\sigma_{\hat{\psi}}\) is an activation function (typically a ReLU), \(\mathcal{Z}_{\hat{\psi}}\) represents batch normalization, \(\pmb{\widehat{\Psi}} \in \mathbb{R}^{D_H \times \kappa}\) is the matrix of weights, and \(\pmb{\hat{\psi}} \in \mathbb{R}^{\kappa}\) is the vector of weights.

Let us define a tensor \(\mathcal{\widetilde{H}} \in \mathbb{R}^{R \times \kappa \times D_H}\) such that \(\tilde{h}_{ij} = \hat{d}_{ij} \pmb{h}_{ij*}\). At this point, the features from local neighborhoods weighted by spatial distances can be computed as a feature tensor

\[\mathcal{\widehat{H}} = \sigma_{\hat{\phi}}\left( \mathcal{Z}_{\hat{\phi}}\left( \mathcal{\widetilde{H}} \pmb{\widehat{\Phi}} \oplus \pmb{\hat{\phi}} \right)\right) ,\]

where \(\sigma_{\hat{\phi}}\) is an activation function (typically a ReLU), \(\mathcal{Z}_{\hat{\psi}}\) represents batch normalization, \(\pmb{\widehat{\Phi}} \in \mathbb{R}^{D_H \times D_H}\) is the matrix of weights, and \(\pmb{\hat{\phi}} \in \mathbb{R}^{D_H}\) is the vector of weights.

Finally, the output feature space \(\pmb{F} \in \mathbb{R}^{R \times D_{\text{out}}}\) can be calculated as

\[\pmb{\hat{F}} = \sigma_{\theta}\left( \mathcal{Z}_{\theta}\left( \left( \pmb{G} + \mathcal{A}(\mathcal{H}) + \mathcal{A}(\mathcal{\widehat{H}}) \right) \pmb{\Theta} \oplus \pmb{\theta} \right)\right) ,\]

where \(\sigma_{\theta}\) is an activation function (typically a ReLU), \(\mathcal{Z}_{\theta}\) represents batch normalization, \(\pmb{\Theta} \in \mathbb{R}^{D_H \times D_{\text{out}}}\) is the matrix of weights, and \(\pmb{\theta} \in \mathbb{R}^{D_{\text{out}}}\) is the vector of weights. Note that \(\mathcal{A}\) is a symmetric aggregation function (typically max pooling or mean reduction).

Variables:

• hidden_channels (int) – The dimensionality of the hidden feature space.

• output_channels (int) – The dimensionality of the output feature space.

• bn (bool) – Whether to include batch normalization or not.

• bn_momentum (float) – The momentum for the batch normalization layers.

• distance (str) – The distance to be used. It can be either "squared" or "euclidean".

• ascending_order (bool) – Whether to force distance-based ascending order of neighborhoods (True) or not (False).

• aggregation (str) – The aggregation strategy to be used. It can be either "max" or "mean".

• initializer – The initializer for the matrices and vectors of weights.

• regularizer – The regularizer for the matrices and vectors of weights.

• constraint – The constraint for the matrices and vectors of weights.

• bn_along_neighbors (bool) – Whether to normalize neighborhood tensors along the neighborhood axis (True) or along the feature axis (False).

• activation – The activation function for the MLPs.

• BNgamma – The batch normalization layer \(\mathcal{Z}_{\gamma}\) for the gamma MLP.

• BNphi – The batch normalization layer \(\mathcal{Z}_{\phi}\) for the phi MLP.

• BNphiHat – The batch normalization layer \(\mathcal{Z}_{\hat{\phi}}\) for the phiHat MLP.

• BNpsi – The batch normalization layer \(\mathcal{Z}_{\psi}\) for the phi MLP.

• BNpsiHat – The batch normalization layer \(\mathcal{Z}_{\hat{\psi}}\) for the phiHat MLP.

• BNtheta – The batch normalization layer \(\mathcal{Z}_{\theta}\) for the theta MLP.

__init__(hidden_channels=64, output_channels=64, bn=True, bn_momentum=0.95, bn_along_neighbors=True, activation=<class 'keras.src.layers.activations.relu.ReLU'>, distance='euclidean', ascending_order=True, aggregation='max', initializer='glorot_normal', regularizer=None, constraint=None, BNgamma=None, BNphi=None, BNphiHat=None, BNpsi=None, BNpsiHat=None, BNtheta=None, built_Gamma=False, built_gamma=False, built_Phi=False, built_phi=False, built_PhiHat=False, built_phiHat=False, built_Psi=False, built_psi=False, built_PsiHat=False, built_psiHat=False, built_Theta=False, built_theta=False, **kwargs)

See Layer and Layer.__init__().

build(dim_in)

Build the weight vectors and matrices.

See Layer and layer.Layer.build().

call(inputs, training=False, mask=False)

Compute the ContextualPointLayer on an input batch.

Parameters:

inputs –

The input such that:

– inputs[0]

is the structure space tensor representing the geometry of the many receptive fields in the batch.

\[\mathcal{X} \in \mathbb{R}^{B \times R \times n_x}\]

– inputs[1]

is the feature space tensor representing the features of the many receptive fields in the batch.

\[\mathcal{F} \in \mathbb{R}^{B \times R \times n_f}\]

– inputs[2]

is the indexing tensor representing the neighborhoods of \(\kappa\) neighbors for each input point, in the same space.

\[\mathcal{N} \in \mathbb{Z}^{B \times R \times \kappa}\]

Returns:

The output feature space \(\mathcal{\widehat{F}} \in \mathbb{R}^{B \times R \times D_{\mathrm{out}}}\).

normalize(X, bn)

Normalize the tensor X if batch normalization is requested and available.

Parameters:

• X – The input tensor to be normalized.

• bn – The batch normalization layer to be applied, if any.

aggregate(X)

Assist ContextualPointLayer.call in aggregating H and Hhat matrices. Aggregating means that the :math:`B times R times kappa times D_H() tensor will be reduced to a \(B \times R \times D_H\) tensor.

Parameters:

X – The tensor to be aggregated, i.e., one of its axis will be reduced to a single value.

Returns:

The aggregated/reduced tensor.

get_config()

Return necessary data to deserialize the layer.

classmethod from_config(config)

Use given config data to deserialize the layer

model.deeplearn.layer.downsampling_spconv3d_layer module

class model.deeplearn.layer.downsampling_spconv3d_layer.DownsamplingSpConv3DLayer(*args, **kwargs)

Bases: Layer

Author:

Alberto M. Esmoris Pena

The DownsamplingSpConv3DLayer behaves like the SubmanifoldSpConv3DLayer but with the following changes.

1. The indices \(i\) of the active cells are considered for cells from the non-downsampled space, i.e., \(i \in h^D\).

2. The \(\omega(i)\) set is redefined as:

\[\omega(i) = \left\{j_p\right\}_{p=0}^{(w^D)^{n_x}-1} = \left\{j_p\right\}_{p=0}^{(w^D)^3-1}\]

3. The \(j_p = h(j'_p)\) index is computed considering \(h\) from the non-downsampled sparse grid. Also, the \(j'_p(i)\) function is redefined as:

\[j'_p(i) = i + p \mod w^D + \left( \left\lfloor\dfrac{p}{w^D}\right\rfloor \mod w^D \right)n_2 + \left( \left\lfloor\dfrac{p}{(w^D)^2}\right\rfloor \mod w^D \right) n_1 n_2\]

Variables:

• wD – The size (i.e., number of cells) of the downsampling 3D convolutional window.

• f (int) – The number of filters/channels for the convolutions.

• nf (int) – The number of input features per cell.

• ng (int) – The number of output features per cell.

• W_initializer – The initializer for the convolutional filters.

• W_regularizer – The regularizer for the convolutional filters.

• W_constraint – The constraint for the convolutional filters.

• siml (SparseIndexingMapLayer.) – The sparse indexing map layer that is needed to translate sparse indices to sequential indices (i.e., \(h(k) = v\) map).

• wDsq (int) – \((w^D)^2\)

• nwD – \((w^D)^{n_x} = (w^D)^{3}\)

• W (tf.Tensor) – The tensor of weights \(\mathcal{W} \in \mathbb{R}^{f \times n_f \times n_g}\) such that its slices are the matrices representing the trainable parameters for each convolutional filter.

• built_W (bool) – Whether the \(\mathcal{W}\) tensor of weights has been built or not. Initially it is false, but it will be updated once the layer is built.

__init__(wD, f, nf, ng, built_W=False, W_initializer=None, W_regularizer=None, W_constraint=None, siml=None, built_p=False, built_pon1=False, built_ponw=False, built_ponw2=False, cache=None, **kwargs)

Initialize the member attributes of the layer and the internal weights that do not depend on the input dimensionality.

Parameters:

kwargs – The key-word specification to parametrize the layer.

build(dim_in)

Build the \(\mathcal{W} \in \mathbb{R}^{f \times n_f \times n_g}\) tensor representing the weights for each of the \(f\) convolutional filters. Also builds the caches.

See Layer and layer.Layer.build().

call(inputs, training=False, mask=False)

Compute the downsampling convolutions.

See DownsamplingSpConv3DLayer for the maths.

Parameters:

inputs –

The feature space matrices as a 3D tensor with padding, the \(h^D\) vector of indices that gives the indices of the active cells in the non-downsampled sparse grid that are the centers for the downsampling convolutional windows, the matrix \(\pmb{N} \in \mathbb{Z}^{n_x \times K}\) of axis-wise partitions in the non-downsampled space (where \(K\) is the batch size), and the start row indices.

Note that inputs[0] gives the feature spaces, inputs[1] the vector of active cell indices to center the downsampling convolutional window in the non-downsampled sparse grid, inputs[2] the axis-wise partitions, inputs[3] the start row-index for the downsampled space, and inputs[4] the start row-index for the non-downsampled space (source, src). The start row-index allows to handle the padding.

Returns:

The output feature space matrices as a tensor with padding.

Return type:

tf.Tensor

static active_lookup(hD, n, pon1, ponw, ponw2, siml)

Assist the convolve inner function of the DownsamplingSpConv3DLayer.call() method to compute the lookup of the downsampling active cells. See DownsamplingSpConv3DLayer for the meaning of the variables and further details.

static down_spconv3d_on_elem(F, hDomega, W)

Assist the convolve inner function of the DownsamplingSpConv3DLayer.call() method to compute the sparse downsampling 3D convolution. See DownsamplingSpConv3DLayer for the meaning of the variables and further details.

get_config()

Return necessary data to serialize the layer

classmethod from_config(config)

Use given config data to deserialize the layer

class model.deeplearn.layer.downsampling_spconv3d_layer.DownsamplingSpConv3DCache(wD, built_p=False, built_pon1=False, built_ponw=False, built_ponw2=False)

Bases: object

Cache for the DownsamplingSpConv3DLayer. The cache contains certain pre-computations that will be used when computing the layer.

Variables:

• p (tf.Tensor of tf.int32) – The sequential indexing vector representing the upsampling convolutional windows.

• built_p (bool) – Whether the indexing vector representing the upsampling convolutional windows has been built or not. Initially it is false, but it will be updated once the layer is built.

• pon1 (tf.Tensor of tf.int32) – Cache for \(p \mod w^D\).

• built_pon1 (bool) – Whether the cache for \(p \mod w^D\) has been built or not. Initially it is false, but it will be updated once the layer is built.

• ponw (tf.Tensor of tf.int32) – Cache for \(\left(\lfloor\dfrac{p}{w^D}\rfloor \mod w^D\right)\).

• built_ponw (bool) – Whether the cache for \(\left(\lfloor\dfrac{p}{w^D}\rfloor \mod w^D\right)\) has been built or not. Initially it is false, but it will be updated once the layer is built.

• ponw2 (tf.Tensor of tf.int32) – Cache for \(\left(\lfloor\dfrac{p}{(w^D)^2}\rfloor \mod w^D\right)\).

• built_ponw2 (bool) – Whether the cache for \(\left(\lfloor\dfrac{p}{(w^D)^2}\rfloor \mod w^D\right)\) has been built or not. Initially it is false, but it will be updated once the layer is built.

__init__(wD, built_p=False, built_pon1=False, built_ponw=False, built_ponw2=False)

build()

Build the cache for the DownsamplingSpConv3DLayer.

get_config()

Return necessary data to serialize the cache

classmethod from_config(config)

Use given config data to deserialize the cache

model.deeplearn.layer.expansion_layer module

class model.deeplearn.layer.expansion_layer.ExpansionLayer(*args, **kwargs)

Bases: Layer

Author:

Alberto M. Esmoris Pena

Layer wrapping the expand_dims method from keras.ops.

Variables:

axis (int) – The index for the new axis.

__init__(axis, **kwargs)

See Layer and Layer.__init__().

call(x, training=False, mask=False)

Compute the actual expansion.

Parameters:

x – The input tensor

Returns:

The expanded tensor.

get_config()

Return necessary data to deserialize the layer.

classmethod from_config(config)

Use given config data to deserialize the layer

model.deeplearn.layer.features_downsampling_layer module

class model.deeplearn.layer.features_downsampling_layer.FeaturesDownsamplingLayer(*args, **kwargs)

Bases: Layer

Author:

Alberto M. Esmoris Pena

A features downsampling layer receives batches of \(m\) points with \(n_f\) features each and downsamples them to \(R\) points with \(n_f\) features each.

It can use a mean-based filter to reduce the features (see FeaturesDownsamplingLayer.mean_filter()), a max-based filter (see FeaturesDownsamplingLayer.max_filter()), a Gaussian-like filter to reduce the features considering the distances between the points (see FeaturesDownsamplingLayer.gaussian_filter()), or a exponential-like filter to reduce the features considering the distances to the contrary as the Gaussian-like filter, i.e., assigning greater weights to points further from the center (see FeaturesDownsamplingLayer.exponential_filter()). Alternatively, a simpler nearest-neighbor filter can be used too (see FeaturesDownsamplingLayer.nearest_filter()).

The input feature space is a tensor \(\mathcal{F} \in \mathbb{R}^{K \times m \times n_f}\) whose slices represent independent receptive fields. The output feature space is a downsampled version of the input one \(\mathcal{Y} \in \mathbb{R}^{K \times R \times n_f}\). Where \(K\) is the batch size.

Variables:

• filter (str) – The name of the filter to be used. Either "mean", "max, "gaussian", "exponential", or "nearest".

• filter_f (Callable) – The method to be used for filtering, derived from the filter attribute.

__init__(filter='mean', **kwargs)

See Layer and layer.Layer.__init__().

build(dim_in)

See Layer and layer.Layer.build().

call(inputs, training=False, mask=False)

Downsample the features from \(m\) input points to \(R\) output points, where \(R \leq m\).

Parameters:

inputs –

The inputs such that:

– inputs[0]

is the structure space tensor before the downsampling.

\[\mathcal{X}_a \in \mathbb{R}^{K \times m \times n_x}\]

– inputs[1]

is the structure space tensor after the downsampling.

\[\mathcal{X}_b \in \mathbb{R}^{K \times R \times n_x}\]

– inputs[2]

is the feature space tensor before the downsampling.

\[\mathcal{F} \in \mathbb{R}^{K \times m \times n_f}\]

– inputs[3]

is the indexing tensor representing the downsampling topology \(\mathcal{N}^D \in \mathbb{Z}^{K \times R \times n_n}\), i.e., for each point in the downsampled space to what points it is connected in the non-downsampled space.

Returns:

The downsampled features \(\mathcal{Y} \in \mathbb{R}^{K \times R \times n_f}\).

static mean_filter(Xa, Xb, Fin, ND)

\[y_{kij} = \dfrac{1}{n_n} \sum_{p=1}^{n_n}{ f_{kn_{kip}^Dj} }\]

Returns:

\(\mathcal{Y} \in \mathbb{R}^{K \times R \times n_f}\)

static max_filter(Xa, Xb, Fin, ND)

\[y_{kij} = \max_{1 \leq p \leq n_n} \quad \left\{ f_{kn_{kip}^{D}j} \right\}\]

Returns:

\(\mathcal{Y} \in \mathbb{R}^{K \times R \times n_f}\)

static gaussian_filter(Xa, Xb, Fin, ND)

\[y_{kij} = \left(\sum_{p=1}^{n_n}{g_{kip}}\right)^{-1} \sum_{p=1}^{n_n}{ g_{kip} f_{kn_{kip}^Dj} }\]

Where:

\[g_{kip} = \exp\left(-\dfrac{ \lVert (\mathcal{X}_a)_{kn_{kip}^D*} - (\mathcal{X}_b)_{ki*} \rVert^2 }{ (d_{ki}^*)^2 }\right)\]

And:

\[d_{ki}^* = \max_{1 \leq p \leq n_n} \; { \lVert (\mathcal{X}_a)_{kn_{kip}^D*} - (\mathcal{X}_b)_{ki*} \rVert }\]

Returns:

\(\mathcal{Y} \in \mathbb{R}^{K \times R \times n_f}\)

static exponential_filter(Xa, Xb, Fin, ND)

\[y_{kij} = \left(\sum_{p=1}^{n_n}{g_{kip}}\right)^{-1} \sum_{p=1}^{n_n}{ g_{kip} f_{kn_{kip}^Dj} }\]

Where:

\[g_{kip} = \exp\left(\dfrac{ \lVert (\mathcal{X}_a)_{kn_{kip}^D*} - (\mathcal{X}_b)_{ki*} \rVert^2 }{ (d_{ki}^*)^2 }\right)\]

And:

\[d_{ki}^* = \max_{1 \leq p \leq n_n} \; { \lVert (\mathcal{X}_a)_{kn_{kip}^D*} - (\mathcal{X}_b)_{ki*} \rVert }\]

Returns:

\(\mathcal{Y} \in \mathbb{R}^{K \times R \times n_f}\)

static nearest_filter(Xa, Xb, Fin, ND)

\[y_{kij} = f_{kn_{ip^*}^Dj}\]

Where:

\[p^* = \operatorname*{argmin}_{1 \leq p \leq n_n} \; { \lVert (\mathcal{X}_a)_{kn_{ip}^D*} - (\mathcal{X}_b)_{ki*} } \rVert^2\]

Returns:

\(\mathcal{Y} \in \mathbb{R}^{K \times R \times n_f}\)

static compute_squared_distances(Xa, Xb, ND)

Compute the squared distances between the downsampled points and those points in the non-downsampled space topologically connected to them (typically because they are the nearest neighbors).

Let \(\pmb{X}_a \in \mathbb{R}^{m \times n_x}\) be a structure space matrix (i.e., a matrix of point-wise coordinates) representing the points before the downsampling and \(\pmb{X}_b \in \mathbb{R}^{R \times n_x}\) be a structure space matrix representing the points after the downsampling. For then, the \(\pmb{N}^D \in \mathbb{Z}^{R \times n_n}\) indexing matrix represents the indices of the neighbors for each point in the downsampled space with respect to the non-downsampled space.

Considering all the previous matrices (that can be seen as the slices of the input tensors), it is possible to compute the distance matrix \(\pmb{D} \in \mathbb{R}^{R \times n_n}\). In this matrix, each element is given by the following expression:

\[d_{ij} = \lVert{ (\pmb{X}_b)_{i*} - (\pmb{X}_a)_{n^D_{ij}*} }\rVert\]

Note that the squared distances matrix is computed instead, which is the same as \(\pmb{D}\) but considering the element-wise squares.

Parameters:

• Xa – The structure space before downsampling.

• Xb – The structure space after downsampling.

• ND – The indexing tensor whose slices are indexing matrices.

Returns:

The tensor with the squared distances for each downsampled point with respect to its neighbors in the non-downsampled space.

Return type:

tf.Tensor

static gather_input_features(Fin, ND)

Gather the input features corresponding to given indexing tensor.

Let \(\mathcal{N}^D \in \mathbb{Z}^{K \times R \times n_n}\) be an indexing tensor of \(K\) batches of \(R\) points with \(n_n\) neighbors each. Also, let \(\mathcal{F} \in \mathbb{R}^{K \times R \times n_f}\) be a features tensor of \(K\) batches of \(m\) points with \(n_f\) features each.

The resulting tensor of gathered input features can be seen as a tensor \(\mathcal{Y} \in \mathbb{R}^{K \times R \times n_n \times n_f}\) of \(K\) batches of \(R\) points each such that for each point \(n_f\) features are gathered for each of its \(n_n\) neighbors. Note that, \(R \leq m\) because \(\mathcal{N}^D\) is an indexing tensor meant to be used for downsampling purposes.

Parameters:

• Fin – The input features.

• ND – The indexing tensor whose slices are indexing matrices.

Returns:

Tensor of gathered input features.

Return type:

tf.Tensor

get_config()

Return necessary data to serialize the layer

classmethod from_config(config)

Use given config data to deserialize the layer

model.deeplearn.layer.features_structuring_layer module

class model.deeplearn.layer.features_structuring_layer.FeaturesStructuringLayer(*args, **kwargs)

Bases: Layer

Author:

Alberto M. Esmoris Pena

A features structuring layer is governed by a features structuring kernel.

A features structuring kernel consists of a structure matrix \(\pmb{Q_X} \in \mathbb{R}^{K \times n_x}\) for a structure space of dimensionality \(n_x\) (typically 3D, i.e., \(x, y, z\)), a features matrix \(\pmb{Q_F} \in \mathbb{R}^{K \times n_f}\) and a kernel distance function \(d_Q(\pmb{q_x}, \pmb{x}) \in \mathbb{R}_{>0}\) that quantifies the distance in the structure space between any kernel point wrt any input point.

A features structuring kernel is generated from three main parameters. First, \(\pmb{r^*} \in \mathbb{R}^{n_x}\) defines the radius along each axis for an ellipsoid, typically in 3D \(\pmb{r^*} = (r^*_x, r^*_y, r^*_z)\) . Second, a radii resolution \(n \in \mathbb{Z}_{> 0}\) defines how many concentric ellipsoids will be considered where the first one is the center point and the last one corresponds to the \(\pmb{r^*}\) radii vector. Finally, \(m_1, \ldots, m_n\) define the angular resolutions such that \(m_k \in \mathbb{Z}_{>0}\) governs how many partitions along the angular domain of the \(\alpha\) and \(\beta\) angles must be considered.

Any features structuring layer expects to receive two inputs in the following order:

1. A structure space matrix representing a point cloud

2. A matrix which rows are the point-wise features for the points in 1).

Variables:

• max_radii (np.ndarray of float) – The radius of the last ellipsoid along each axis \(\pmb{r}^* \in \mathbb{R}^{n_x}\).

• radii_resolution (int) – How many concentric ellipsoids must be considered \(n \in \mathbb{Z}_{>0}\) (the first one is the center point, the last one is the biggest outer ellipsoid).

• angular_resolutions (np.ndarray of int) – How many angles consider for each ellipsoid \((m_1, \ldots, m_n)\).

• num_kernel_points (int) – The number of points representing the kernel \(K \in \mathbb{Z}_{>0}\).

• dim_out (int) – The output dimensionality, it governs the trainable matrix \(\pmb{Q_W} \in \mathbb{R}^{n_f \times n_y}\).

• concatenation_strategy (str) – Specify the concatenation strategy defining the output of the layer. It can be “FULL” so the generated output will be passed together with the input or “OPAQUE” in which case the input will not be forwarded as output.

• num_features (int) – The number of features per point received as input \(n_f \in \mathbb{Z}_{>0}\).

• concatf (func) – The function implementing the concatenation strategy.

• QX (tf.Tensor) – The kernel’s structure matrix.

• trainable_QX (bool) – Flag to control whether QX is trainable or not.

• built_QX (bool) – Flag to control whether QX has been built or not.

• omegaD (tf.Tensor) – The kernel’s distance weights.

• trainable_omegaD (bool) – Flag to control whether omegaD is trainable or not.

• built_omegaD (bool) – Flag to control whether omegaD has been built or not.

• omegaF (tf.Tensor) – The kernel’s feature weights.

• trainable_omegaF (bool) – Flag to control whether omegaF is trainable or not.

• built_omegaF (bool) – Flag to control whether omegaF has been built or not.

• QW (tf.Tensor) – The kernel’s weights matrix.

• trainable_QW (bool) – Flag to control whether QW is trainable or not.

• built_QW (bool) – Flag to control whether QW has been built or not.

__init__(max_radii, radii_resolution=4, angular_resolutions=(1, 2, 4, 8), structure_dimensionality=3, dim_out=4, concatenation_strategy='FULL', trainable_QX=False, trainable_QW=True, trainable_omegaD=True, trainable_omegaF=True, built_QX=False, built_omegaD=False, built_omegaF=False, built_QW=False, **kwargs)

See Layer and deeplearn.layer.layer.Layer.__init__().

build(dim_in)

Build the \(\pmb{Q_X} \in \mathbb{R}^{K \times n_x}\) matrix representing the kernel’s structure, and the \(\pmb{Q_W} \in \mathbb{R}^{n_f \times n_y}\) matrix representing the kernel’s weights as well as the \(\pmb{\omega_D} \in \mathbb{R}^{K}\) and \(\pmb{\omega_F} \in \mathbb{R}^{n_f}\) vectors.

The \(\pmb{Q_X}\) matrix represents the disposition of the kernel’s points while the \(\pmb{Q_W}\) is a matrix of weights that defines a potentially trainable transformation on the features.

The \(\pmb{\omega_D}\) vector governs the relevance of the distance wrt to each kernel point when weighting the features, and the \(\pmb{\omega_F}\) vectors governs the relevance of each feature independently of the distance in the structure space.

See Layer and layer.Layer.build().

call(inputs, training=False, mask=False)

The computation of the \(\pmb{Q_F} \in \mathbb{R}^{K \times n_f}\) matrix and the output features \(\pmb{Q_Y}\) obtained after matrix multiplication wrt \(\pmb{Q_W}\).

Let the distance function between the point i of the kernel and the point j of the input point cloud be:

\[d_Q(\pmb{q_{Xi*}}, \pmb{x_{j*}}) = \exp\left[ - \dfrac{\lVert{ \pmb{x_{j*}} - \pmb{q_{Xi*}} }\rVert^2}{ \omega_{Di}^2 } \right]\]

\[\pmb{Q_F} \in \mathbb{R}^{k \times n_f} \;\text{ s.t. }\; q_{Fik} = \sum_{j=1}^{m}{ d_Q(\pmb{q_{Xi*}}, \pmb{x_{j*}}) \omega_{Fk} f_{jk} }\]

Alternatively, \(\pmb{Q_F}\) rows can be expressed as a sum of vectors (where \(\odot\) represents the Hadamard product):

\[\pmb{q_{Fi*}} = \sum_{j=1}^{m}{ d_Q(\pmb{q_{Xi*}}, \pmb{x_{j*}}) ( \pmb{\omega_F} \odot \pmb{f_{j*}} ) }\]

Note also that there exists a distance matrix \(\pmb{Q_D} \in \mathbb{R}^{k \times m}\) describing how close each point from the input point cloud is wrt each kernel’s point. It can be computed as:

\[\begin{split}\pmb{Q_D} = \left[\begin{array}{ccc} q_{D11} & \ldots & q_{D1m} \\ \vdots & \ddots & \vdots \\ q_{Dk1} & \ldots & q_{Dkm} \end{array}\right] \; \text{ s.t. } \; Q_{Dij} = d_Q(\pmb{q_{Xi*}}, \pmb{x_{j*}})\end{split}\]

For then, the output matrix can be calculated as:

\[\pmb{Q_Y} = \pmb{Q_D}^\intercal \pmb{Q_F} \pmb{Q_W}\]

See Layer and layer.Layer.call().

Returns:

The output features. Depending on the concatenation strategy they can be FULL \([\pmb{F}, \pmb{Q_Y}]\) or OPAQUE \([\pmb{Q_Y}]\).

static concatf_full(F, QY)

Returns:

\([\pmb{F}, \pmb{Q_Y}]\)

static concatf_opaque(F, QY)

Returns:

\([\pmb{Q_Y}]\)

assign_concatf()

Assign the concatf function from the current state of the object.

Returns:

Nothing, but self.concatf is updated.

get_config()

Return necessary data to serialize the layer

classmethod from_config(config)

Use given config data to deserialize the layer

export_representation(dir_path, out_prefix=None, QXpast=None)

Export a set of files representing the state of the features structuring kernel.

Parameters:

• dir_path (str) – The directory where the representation files will be exported.

• out_prefix (str) – The output prefix to name the output files.

• QXpast (np.ndarray or tf.Tensor or None) – The structure matrix of the layer in the past.

Returns:

Nothing at all, but the representation is exported as a set of files inside the given directory.

model.deeplearn.layer.features_upsampling_layer module

class model.deeplearn.layer.features_upsampling_layer.FeaturesUpsamplingLayer(*args, **kwargs)

Bases: Layer

Author:

Alberto M. Esmoris Pena

A features upsampling layer receives batches of \(R\) points with \(n_f\) features each and upsamples them to \(m\) points with \(n_f\) features each.

It can use a mean-based filter to propagate the features (see FeaturesUpsamplingLayer.mean_filter()), a max-based filter (see FeaturesUpsamplingLayer.max_filter()) a Gaussian-like filter to propagate the features considering the distances between the points (see FeaturesUpsamplingLayer.gaussian_filter()), or an exponential-like filter to propagate the features considering the distances to the contrary as the Gaussian-like filter, i.e., assigning greater weights to points fruther from the center (see FeaturesUpsamplingLayer.exponential_filter()). Alternatively, a simpler nearest-neighbor filter can be used too (see FeaturesUpsamplingLayer.nearest_filter()).

The input feature space is a tensor \(\mathcal{F} \in \mathbb{R}^{K \times R \times n_f}\) whose slices represent independent receptive fields. The output feature space is an upsampled version of the input one \(\mathcal{Y} \in \mathbb{R}^{K \times m \times n_f}\). Where \(K\) is the batch size.

Variables:

• filter (str) – The name of the filter to be used. Either "mean", "gaussian", "exponential", or "nearest".

• filter_f (Callable) – The method to be used for filtering, derived from the filter attribute.

__init__(filter='mean', **kwargs)

See Layer and layer.Layer.__init__().

build(dim_in)

See Layer and layer.Layer.build().

call(inputs, training=False, mask=False)

Upsample the features from \(R\) input points to \(m\) output points, where \(m \geq R\).

Parameters:

inputs –

The inputs such that:

– inputs[0]

is the structure space tensor before the upsampling.

\[\mathcal{X}_a \in \mathbb{R}^{K \times R \times n_x}\]

– inputs[1]

is the structure space tensor after the upsampling.

\[\mathcal{X}_b \in \mathbb{R}^{K \times m \times n_x}\]

– inputs[2]

is the feature space tensor before the upsampling.

\[\mathcal{F} \in \mathbb{R}^{K \times R \times n_f}\]

– inputs[3]

is the indexing tensor representing the upsampling topology \(\mathcal{N}^U \in \mathbb{Z}^{K \times m \times n_n}\), i.e., for each point in the upsampled space to what points it is connected in the non-upsampled space.

Returns:

The upsampled features \(\mathcal{Y} \in \mathbb{R}^{K \times m \times n_f}\)

static mean_filter(Xa, Xb, Fin, NU)

\[y_{kij} = \dfrac{1}{n_n} \sum_{p=1}^{n_n}{ f_{kn_{kip}^Uj} }\]

Returns:

\(\mathcal{Y} \in \mathbb{R}^{K \times m \times n_f}\)

static max_filter(Xa, Xb, Fin, NU)

\[y_{kij} = \max_{1 \leq p \leq n_n} \quad \left\{ f_{kn_{kip}^Uj} \right\}\]

Returns:

\(\mathcal{Y} \in \mathbb{R}^{K \times m \times n_f}\)

static gaussian_filter(Xa, Xb, Fin, NU)

\[y_{kij} = \left(\sum_{p=1}^{n_n}{g_{kip}}\right)^{-1} \sum_{p=1}^{n_n}{ g_{kip} f_{kn_{kip}^Uj} }\]

Where:

\[g_{kip} = \exp\left(- \dfrac{ \lVert (\mathcal{X}_a)_{kn_{kip}^U*} - (\mathcal{X}_b)_{ki*} \rVert^2 }{ (d_{ki}^*)^2 }\right)\]

And:

\[d_{ki}^* = \max_{1 \leq p \leq n_n} \; { \lVert (\mathcal{X}_a)_{kn_{kip}^U*} - (\mathcal{X}_b)_{ki*} \rVert }\]

Returns:

\(\mathcal{Y} \in \mathbb{R}^{K \times m \times n_f}\)

static exponential_filter(Xa, Xb, Fin, NU)

\[y_{kij} = \left(\sum_{p=1}^{n_n}{g_{kip}}\right)^{-1} \sum_{p=1}^{n_n}{ g_{kip} f_{kn_{kip}^Dj} }\]

Where:

\[g_{kip} = \exp\left(\dfrac{ \lVert (\mathcal{X}_a)_{kn_{kip}^D*} - (\mathcal{X}_b)_{ki*} \rVert^2 }{ (d_{ki}^*)^2 }\right)\]

And:

\[d_{ki}^* = \max_{1 \leq p \leq n_n} \; { \lVert (\mathcal{X}_a)_{kn_{kip}^D*} - (\mathcal{X}_b)_{ki*} \rVert }\]

Returns:

\(\mathcal{Y} \in \mathbb{R}^{K \times R \times n_f}\)

static nearest_filter(Xa, Xb, Fin, NU)

\[y_{kij} = f_{kn_{ip^*}^Uj}\]

Where:

\[p^* = \operatorname*{argmin}_{1 \leq p \leq n_n} \; { \lVert (\mathcal{X}_a)_{kn_{ip}^U*} - (\mathcal{X}_b)_{ki*} } \rVert^2\]

Returns:

\(\mathcal{Y} \in \mathbb{R}^{K \times m \times n_f}\)

get_config()

Return necessary data to serialize the layer

classmethod from_config(config)

Use given config data to deserialize the layer

model.deeplearn.layer.geometric_affine_layer module

class model.deeplearn.layer.geometric_affine_layer.GeometricAffineLayer(*args, **kwargs)

Bases: Layer

Author:

Alberto M. Esmoris Pena

A geometric affine layer receives batches of \(R\) points with \(n_f\) input features each, and \(\kappa\) known neighbors in the same space. These inputs are used to compute an output feature space of \(R\) points with \(n_f\) features each. In doing so, the indexing tensor \(\mathcal{N} \in \mathbb{Z}^{B \times R \times \kappa}\) is used to link \(\kappa\) neighbors for each of the \(R\) input points in each of the \(B\) input batches.

The layer applies the following geometric affine transformation:

\[\pmb{\hat{f}_{ik*}} = \pmb{\alpha} \odot \dfrac{ \pmb{f}_{k_i*} - \pmb{f}_{i*} }{ \sigma + \epsilon } + \pmb{\beta}\]

where \(k_i\) refers to the \(k\)-th neighbor of the \(i\)-th point and

\[\sigma = \sqrt{ \left( R \kappa n_f \right)^{-1} \sum_{i=1}^{R}{\sum_{\pmb{x}_{k*} \in \mathcal{N}(\pmb{x}_{i*})}{ \sum_{j=1}^{n_f}{\bigl( f_{kj} - f_{ij} \bigr)^2} }} } .\]

Note that in this model the vectors of weights (i.e., trainable parameters) are \(\pmb{\alpha}, \pmb{\beta} \in \mathbb{R}^{n_f}\). Besides, the \(\epsilon\) value use for numerical stability is often set to \(\epsilon = 10^{-5}\).

The output is a tensor \(\mathcal{\widehat{F}} \in \mathbb{R}^{B \times R \times \kappa \times n_f}\) that contains the batch-wise transformed features for all the points in each local neighborhood.

This layer is inspired in the PointMLP paper (https://doi.org/10.48550/arXiv.2202.07123).

Variables:

• eps (float) – The numerical stability constant \(\epsilon\) that will be added to \(\sigma\) in the denominator. By default it is \(10^{-5}\).

• built_alpha (bool) – Whether the weights vector \(\pmb{\alpha}\) is built. Initially it is false, but it will be updated once the layer is built.

• built_beta (bool) – Whether the weights vector \(\pmb{\beta}\) is built. Initially it is false, but it will be updated once the layer is built.

• alpha_initializer – The initializer for the vector of weights \(\pmb{\alpha}\).

• alpha_regularizer – The regularizer for the vector of weights \(\pmb{\alpha}\).

• alpha_constraint – The constraint for the vector of weights \(\pmb{\alpha}\).

• beta_initializer – The initializer for the vector of weights \(\pmb{\beta}\).

• beta_regularizer – The regularizer for the vector of weights \(\pmb{\beta}\).

• beta_constraint – The constraint for the vector of weights \(\pmb{\beta}\).

__init__(eps=1e-05, built_alpha=False, built_beta=False, alpha_initializer=None, alpha_regularizer=None, alpha_constraint=None, beta_initializer=None, beta_regularizer=None, beta_constraint=None, **kwargs)

See Layer and Layer.__init__().

build(dim_in)

Build the weight vectors.

See Layer and layer.Layer.build().

call(inputs, training=False, mask=False)

Compute the GeometricAffineLayer on an input batch.

Parameters:

inputs –

The input such that:

– inputs[0]

is the structure space tensor representing the geometry of the many receptive fields in the batch.

\[\mathcal{X} \in \mathbb{R}^{K \times R \times n_x}\]

– inputs[1]

is the feature space tensor representing the features of the many receptive fields in the batch.

\[\mathcal{F} \in \mathbb{R}^{K \times R \times n_f}\]

– inputs[2]

is the indexing tensor representing the neighborhoods of \(\kappa\) neighbors for each input point, in the same space.

\[\mathcal{N} \in \mathbb{Z}^{K \times R \times \kappa}\]

Returns:

The output feature space \(\mathcal{\widehat{F}} \in \mathbb{R}^{B \times R \times \kappa \times n_f}\).

compute_sigma(F, N)

Assist the GeometricAffineLayer.call() method by computing \(\sigma\).

compute_output_features(diffs, sigma)

Assist the GeometricAffineLayer.call() method by computing the geometric affine transformation of the neighborhood-wise features.

get_config()

Return necessary data to serialize the layer.

classmethod from_config(config)

Use given config data to deserialize the layer.

model.deeplearn.layer.grouped_point_transformer_layer module

class model.deeplearn.layer.grouped_point_transformer_layer.GroupedPointTransformerLayer(*args, **kwargs)

Bases: Layer

Author:

Alberto M. Esmoris Pena

A grouped point transformer layer receives batches of \(R\) points with \(C\) (read as channels) features each, and \(\kappa\) known neighbors in the same spce. These inputs are used to compute an output feature space of \(R\) points with \(C\) features each. In doing so, the indexing tensor \(\mathcal{N} \in \mathbb{Z}^{B \times R \times \kappa}\) is used to link \(\kappa\) neighbors for each of the \(R\) input points in each of the \(B\) input batches.

The layer applies a feature space transformation on each element of the batch

\[\tilde{f}_{igp} = \sum_{k=1}^{\kappa}{ \tilde{v}_{ikgp} w_{ikg} }\]

that yields a feature tensor \(\pmb{\widetilde{F}} \in \mathbb{R}^{R \times G \times H}\).

The final output is \(\pmb{\widehat{F}} \in \mathbb{R}^{R \times C}\), a reshaped version of the \(\pmb{\widetilde{F}}\) tensor

\[\hat{f}_{i*} = \bigl( \tilde{f}_{i11}, \ldots, \tilde{f}_{i1H}, \tilde{f}_{i21}, \ldots, \tilde{f}_{iGH} \bigr) ,\]

where \(H=C/G\) for a given number of groups \(G\) such that \(G \mid C\).

The previous equations assume the previous computation of many steps. First, let \(\pmb{Q}, \pmb{K}, \pmb{V} \in \mathbb{R}^{C \times C}\) be weight matrices and \(\pmb{q}, \pmb{k}, \pmb{v} \in \mathbb{R}^{C}\) be weight vectors. Now compute

\[\pmb{\widehat{Q}} = \sigma_{Q}\bigl(Z_{Q}\bigl( \pmb{F} \pmb{Q} \oplus \pmb{q} \bigr)\bigr) ,\]

\[\pmb{\widehat{K}} = \sigma_{K}\bigl(Z_{K}\bigl( \pmb{F} \pmb{K} \oplus \pmb{k} \bigr)\bigr) ,\]

and

\[\pmb{\widehat{V}} = \sigma_{V}\bigl(Z_{V}\bigl( \pmb{F} \pmb{V} \oplus \pmb{v} \bigr)\bigr) .\]

Where \(\sigma_{Q}, \sigma_{K}, \sigma_{V}\) are activation functions (typically ReLU), \(Z_{Q}, Z_{K}, Z_{V}\) are batch normalizations, and \(\oplus\) is the broadcast vector summation, i.e., summing a vector to the fibers of a tensor (in this case, summing the vector to all the rows of the corresponding matrix product on the left side). Concerning their dimensionalities note that \(\pmb{\widehat{Q}}, \pmb{\widehat{K}}, \pmb{\widehat{V}} \in \mathbb{R}^{R \times C}\). The input feature space corresponding to a single element in the batch is given by \(\pmb{F} \in \mathbb{R}^{R \times C}\).

Let us consider a relative position encoding tensor for each neighborhood \(\pmb{\widehat{X}} \in \mathbb{R}^{R \times \kappa \times n_x}\) for an \(n_x\)-dimensional structure space such that

\[\pmb{\hat{x}}_{ij*} = \pmb{x}_{j_i*} - \pmb{x}_{i*} ,\]

with \(\pmb{x}_{j_i*} \in \mathcal{N}(\pmb{x}_{i*})\) (i.e., \(\pmb{x}_{j_i*} \in \mathbb{R}^{n_x}\) belongs to the neighborhood of the \(i\)-th point).

Now we can compute two positional encoding tensors

\[\pmb{\Delta_{A}} = \sigma_{A}\bigl(Z_{A}\bigl( \pmb{\widehat{X}} \pmb{\Theta_{A} \oplus \pmb{\theta_{A}}} \bigr)\bigr) \pmb{\widetilde{\Theta}_A} \oplus \pmb{\tilde{\theta}_A}\]

and

\[\pmb{\Delta_{B}} = \sigma_{B}\bigl(Z_{B}\bigl( \pmb{\widehat{X}} \pmb{\Theta_{B} \oplus \pmb{\theta_{B}}} \bigr)\bigr) \pmb{\widetilde{\Theta}_B} \oplus \pmb{\tilde{\theta}_B} .\]

Note that \(\sigma_A, \sigma_B\) are activation functions (typically ReLU) and \(Z_A, Z_B\) correspond to batch normalizations. The weights are represented through the matrices \(\pmb{\Theta_A}, \pmb{\theta_A} \in \mathbb{R}^{n_x \times C}\), \(\pmb{\widetilde{\Theta}_A}, \pmb{\widetilde{\Theta}_B} \in \mathbb{R}^{C \times C}\), and the vectors \(\pmb{\theta_A}, \pmb{\theta_B}, \pmb{\tilde{\theta}_A}, \pmb{\tilde{\theta}_B} \in \mathbb{R}^{C}\) . Concerning the dimensionalities note that \(\pmb{\Delta_A}, \pmb{\Delta_B} \in \mathbb{R}^{R \times \kappa \times C}\) .

Now, we can also compute the differences between keys and queries

\[\pmb{\gamma}_{ij*} = \pmb{\hat{k}}_{j_i*} - \pmb{\hat{q}}_{i*}\]

with (again) \(\pmb{x}_{j_i*} \in \mathcal{N}(\pmb{x}_{i*})\) (i.e., \(\pmb{x}_{j_i*} \in \mathbb{R}^{n_x}\) belongs to the neighborhood of the \(i\)-th point), which leads to the tensor \(\pmb{\Gamma} \in \mathbb{R}^{R \times \kappa \times C}\).

At this point, we can compute

\[\pmb{\widetilde{\Gamma}} = \pmb{\Gamma} \odot \pmb{\Delta_{A}} + \pmb{\Delta_{B}} .\]

Now we can compute the weight matrix \(\pmb{W} \in \mathbb{R}^{R \times \kappa \times G}\), for a number of groups \(G\) such that \(G | C\), that yields the \(w_{ikg}\) terms in the first equation

\[\pmb{W} = \sigma\biggl( \sigma_{\omega}\bigl(Z_{\omega}\bigl( \pmb{\widetilde{\Gamma}} \pmb{\Omega} \odot \pmb{\omega} \bigr)\bigr) \pmb{\widetilde{\Omega}} \oplus \pmb{\tilde{\omega}} \biggr) .\]

In the above equation \(\sigma\) is a softmax activation function that normalizes considering the summation of all the values along the axis of \(G\) groups, \(\sigma_{\omega}\) is an activation function (typically a ReLU), and \(Z_{\omega}\) is a batch normalization.

Finally, we only need to compute the tensor \(\pmb{\widehat{V}} \in \mathbb{R}^{R \times \kappa \times C}\) such that

\[\pmb{\hat{v}}_{ij*} = \pmb{v}_{ij_i*} + \pmb{\delta^{(B)}}_{ij_i*} ,\]

where \(\pmb{\delta^{(B)}}_{ij_i*} \in \mathcal{N}(\pmb{x}_{i*})\) and also \(\pmb{\delta^{(B)}}_{ij_i*}\) represents a row for the \(j\)-th closest neighbor of the \(i\)-th point in the \(\pmb{\Delta_{B}}\) tensor. If we rearrange this tensor such that

\[\tilde{v}_{ijgp} = \hat{v}_{ijc} \,,\quad c=(g-1)H+p\]

we achieve the \(\pmb{\widetilde{V}} \in \mathbb{R}^{R \times \kappa \times G \times H}\) tensor necessary to compute the original equations to yield the output feature space.

This layer is inspired in the PointTransformer v2 paper (https://doi.org/10.48550/arXiv.2210.05666).

Variables:

• groups (int) – The number of groups \(G\) into which the channels will be divided. It must satisfy \(G \mid C\).

• channels (int) – The number of input and output features \(C \in \mathbb{Z}_{>0}\).

• dropout – The dropout layer that can be applied during training to the weight encoding tensor.

• built_Q (bool) – Whether the weights matrix \(\pmb{Q}\) is built. Initially it is false, but it will be updated once the layer is built.

• built_q (bool) – Whether the weights vector \(\pmb{q}\) is built. Initially it is false, but it will be updated once the layer is built.

• built_K (bool) – Whether the weights matrix \(\pmb{K}\) is built. Initially it is false, but it will be updated once the layer is built.

• built_k (bool) – Whether the weights vector \(\pmb{k}\) is built. Initially it is false, but it will be updated once the layer is built.

• built_V (bool) – Whether the weights matrix \(\pmb{V}\) is built. Initially it is false, but it will be updated once the layer is built.

• built_v (bool) – Whether the weights vector \(\pmb{v}\) is built. Initially it is false, but it will be updated once the layer is built.

• built_ThetaA (bool) – Whether the weights matrix \(\pmb{\Theta_A}\) is built. Initially it is false, but it will be updated once the layer is built.

• built_thetaA (bool) – Whether the weights vector \(\pmb{\theta_A}\) is built. Initially it is false, but it will be updated once the layer is built.

• built_ThetaTildeA (bool) – Whether the weights matrix \(\pmb{\widetilde{\Theta}_A}\) is built. Initially it is false, but it will be updated once the layer is built.

• built_thetaTildeA (bool) – Whether the weights vector \(\pmb{\tilde{\theta}_A}\) is built. Initially it is false, but it will be updated once the layer is built.

• built_ThetaB (bool) – Whether the weights matrix \(\pmb{\Theta_B}\) is built. Initially it is false, but it will be updated once the layer is built.

• built_thetaB (bool) – Whether the weights vector \(\pmb{\theta_B}\) is built. Initially it is false, but it will be updated once the layer is built.

• built_ThetaTildeB (bool) – Whether the weights matrix \(\pmb{\widetilde{\Theta}_B}\) is built. Initially it is false, but it will be updated once the layer is built.

• built_thetaTildeB (bool) – Whether the weights vector \(\pmb{\tilde{\theta}_B}\) is built. Initially it is false, but it will be updated once the layer is built.

• built_Omega (bool) – Whether the weights matrix \(\pmb{\Omega}\) is built. Initially it is false, but it will be updated once the layer is built.

• built_omega (bool) – Whether the weights vector \(\pmb{\omega}\) is built. Initially it is false, but it will be updated once the layer is built.

• built_OmegaTilde (bool) – Whether the weights matrix \(\pmb{\widetilde{\Omega}}\) is built. Initially it is false, but it will be updated once the layer is built.

• built_omegaTilde (bool) – Whether the weights vector \(\pmb{\tilde{\omega}}\) is built. Initially it is false, but it will be updated once the layer is built.

• Q_initializer – The initializer for the matrix of weights \(\pmb{Q}\).

• q_initializer – The initializer for the vector of weights \(\pmb{q}\).

• Q_regularizer – The regularizer for the matrix of weights \(\pmb{Q}\).

• q_regularizer – The regularizer for the vector of weights \(\pmb{q}\).

• Q_constraint – The constraint for the matrix of weights \(\pmb{Q}\).

• q_constraint – The constraint for the vector of weights \(\pmb{q}\).

• K_initializer – The initializer for the matrix of weights \(\pmb{K}\).

• k_initializer – The initializer for the vector of weights \(\pmb{k}\).

• K_regularizer – The regularizer for the matrix of weights \(\pmb{K}\).

• k_regularizer – The regularizer for the vector of weights \(\pmb{k}\).

• K_constraint – The constraint for the matrix of weights \(\pmb{K}\).

• k_constraint – The constraint for the vector of weights \(\pmb{k}\).

• V_initializer – The initializer for the matrix of weights \(\pmb{V}\).

• v_initializer – The initializer for the vector of weights \(\pmb{v}\).

• V_regularizer – The regularizer for the matrix of weights \(\pmb{V}\).

• v_regularizer – The regularizer for the vector of weights \(\pmb{v}\).

• V_constraint – The constraint for the matrix of weights \(\pmb{V}\).

• v_constraint – The constraint for the vector of weights \(\pmb{v}\).

• ThetaA_initializer – The initializer for the matrix of weights \(\pmb{\Theta_A}\).

• thetaA_initializer – The initializer for the vector of weights \(\pmb{\theta_A}\).

• ThetaA_regularizer – The regularizer for the matrix of weights \(\pmb{\Theta_A}\).

• thetaA_regularizer – The regularizer for the vector of weights \(\pmb{\theta_A}\).

• ThetaA_constraint – The constraint for the matrix of weights \(\pmb{\Theta_A}\).

• thetaA_constraint – The constraint for the vector of weights \(\pmb{\theta_A}\).

• ThetaB_initializer – The initializer for the matrix of weights \(\pmb{\Theta_B}\).

• thetaB_initializer – The initializer for the vector of weights \(\pmb{\theta_B}\).

• ThetaB_regularizer – The regularizer for the matrix of weights \(\pmb{\Theta_B}\).

• thetaB_regularizer – The regularizer for the vector of weights \(\pmb{\theta_B}\).

• ThetaB_constraint – The constraint for the matrix of weights \(\pmb{\Theta_B}\).

• thetaB_constraint – The constraint for the vector of weights \(\pmb{\theta_B}\).

• Omega_initializer – The initializer for the matrix of weights \(\pmb{\Omega}\).

• omega_initializer – The initializer for the vector of weights \(\pmb{\omega}\).

• Omega_regularizer – The regularizer for the matrix of weights \(\pmb{\Omega}\).

• omega_regularizer – The regularizer for the vector of weights \(\pmb{\omega}\).

• Omega_constraint – The constraint for the matrix of weights \(\pmb{\Omega}\).

• omega_constraint – The constraint for the vector of weights \(\pmb{\omega}\).

• OmegaTilde_initializer – The initializer for the matrix of weights \(\pmb{\widetilde{\Omega}}\).

• omegaTilde_initializer – The initializer for the vector of weights \(\pmb{\tilde{\omega}}\).

• OmegaTilde_regularizer – The regularizer for the matrix of weights \(\pmb{\widetilde{\Omega}}\).

• omegaTilde_regularizer – The regularizer for the vector of weights \(\pmb{\tilde{\omega}}\).

• OmegaTilde_constraint – The constraint for the matrix of weights \(\pmb{\widetilde{\Omega}}\).

• omegaTilde_constraint – The constraint for the vector of weights \(\pmb{\tilde{\omega}}\).

• Q_act – The activation function for the queries.

• Q_bn – The batch normalization layer for the queries.

• K_act – The activation function for the queries.

• K_bn – The batch normalization layer for the queries.

• deltaA_act – The activation function for the positional encoding multiplier.

• deltaA_bn – The batch normalization layer for the positional encoding multiplier.

• deltaB_act – The activation function for the positional encoding bias.

• deltaB_bn – The batch normalization layer for the positional encoding bias.

• omega_act – The activation function for the weight encoding.

• omega_bn – The batch normalization layer for the weight encoding.

• sigma_act – A softmax activation function to apply neighbor-wise normalization.

__init__(groups, channels=None, dropout=None, dropout_rate=0.0, built_Q=False, built_q=False, built_K=False, built_k=False, built_V=False, built_v=False, built_ThetaA=False, built_thetaA=False, built_ThetaTildeA=False, built_thetaTildeA=False, built_ThetaB=False, built_thetaB=False, built_ThetaTildeB=False, built_thetaTildeB=False, built_Omega=False, built_omega=False, built_OmegaTilde=False, built_omegaTilde=False, Q_initializer=None, Q_regularizer=None, Q_constraint=None, K_initializer=None, K_regularizer=None, K_constraint=None, V_initializer=None, V_regularizer=None, V_constraint=None, ThetaA_initializer=None, ThetaA_regularizer=None, ThetaA_constraint=None, ThetaTildeA_initializer=None, ThetaTildeA_regularizer=None, ThetaTildeA_constraint=None, ThetaB_initializer=None, ThetaB_regularizer=None, ThetaB_constraint=None, ThetaTildeB_initializer=None, ThetaTildeB_regularizer=None, ThetaTildeB_constraint=None, Omega_initializer=None, Omega_regularizer=None, Omega_constraint=None, OmegaTilde_initializer=None, OmegaTilde_regularizer=None, OmegaTilde_constraint=None, q_initializer=None, q_regularizer=None, q_constraint=None, k_initializer=None, k_regularizer=None, k_constraint=None, v_initializer=None, v_regularizer=None, v_constraint=None, thetaA_initializer=None, thetaA_regularizer=None, thetaA_constraint=None, thetaTildeA_initializer=None, thetaTildeA_regularizer=None, thetaTildeA_constraint=None, thetaB_initializer=None, thetaB_regularizer=None, thetaB_constraint=None, thetaTildeB_initializer=None, thetaTildeB_regularizer=None, thetaTildeB_constraint=None, omega_initializer=None, omega_regularizer=None, omega_constraint=None, omegaTilde_initializer=None, omegaTilde_regularizer=None, omegaTilde_constraint=None, Q_act=<function relu>, Q_bn=None, Q_bn_momentum=0.98, K_act=<function relu>, K_bn=None, K_bn_momentum=0.98, deltaA_act=<function relu>, deltaA_bn=None, deltaA_bn_momentum=0.98, deltaB_act=<function relu>, deltaB_bn=None, deltaB_bn_momentum=0.98, omega_act=<function relu>, omega_bn=None, omega_bn_momentum=0.98, sigma_act=<function softmax>, **kwargs)

See Layer and Layer.__init__().

Parameters:

dropout_rate (float) – The ratio of units that will be disabled when applying dropout. It must be given inside \([0, 1)\), where \(0\) means no dropout at all and \(1\) means all the units will be deactivated.

build(dim_in)

Build the weight vectors and matrices.

See Layer and layer.Layer.build().

call(inputs, training=False, mask=False)

Compute the GroupedPointTransformerLayer on an input batch.

Parameters:

inputs –

The input such that:

– inputs[0]

is the structure space tensor representing the geometry of the many receptive fields in the batch.

\[\mathcal{X} \in \mathbb{R}^{K \times R \times n_x}\]

– inputs[1]

is the feature space tensor representing the features of the many receptive fields in the batch.

\[\mathcal{F} \in \mathbb{R}^{K \times R \times n_f}\]

– inputs[2]

is the indexing tensor representing the neighborhoods of \(\kappa\) neighbors for each input point, in the same space.

\[\mathcal{N} \in \mathbb{Z}^{K \times R \times \kappa}\]

Returns:

The output feature space \(\mathcal{\widehat{F}} \in \mathbb{R}^{K \times R \times D_{\mathrm{out}}}\).

get_config()

Return necessary data to serialize the layer.

classmethod from_config(config)

Use given config data to deserialize the layer.

model.deeplearn.layer.grouping_point_net_layer module

class model.deeplearn.layer.grouping_point_net_layer.GroupingPointNetLayer(*args, **kwargs)

Bases: Layer

Author:

Alberto M. Esmoris Pena

A grouping point net layer receives batches of \(R\) points with \(n\) features each (typically, \(n=n_x+n_f\), i.e., the sum of the input structure and feature spaces), and \(\kappa\) known neighbors in the same space. These inputs are used to compute an output feature space of \(R\) points with \(D_{\text{out}}\) features each. In doing so, the indexing tensor \(\mathcal{N} \in \mathbb{Z}^{K \times R \times \kappa}\) is used to link \(\kappa\) neighbors for each of the \(R\) input points in each of the \(K\) input batches.

The layer is defined by a matrix \(\pmb{H} \in \mathbb{R}^{D_{\text{out}} \times n}\) that governs the weights for the SharedMLP (also, unitary 1DConv, i.e., win size one and stride one), and a pair (matrix, vector) governing the weights and the bias of the classical MLP (typical Dense layer in Keras):

\[( \pmb{\Gamma} \in \mathbb{R}^{D_{\text{out}} \times D_{\text{out}}}, \pmb{\gamma} \in \mathbb{R}^{D_{\text{out}}} )\]

The grouping PointNet layer applies a PointNet-like operator:

\[\left( \gamma \circ \operatorname*{MAX}_{1 \leq i \leq \kappa} \right)\left( \left\{h(\pmb{p}_{i*})\right\} \right)\]

To each of the neighborhoods, for each point in the batch, for each batch in the input. Where \(\pmb{p}_{i*} \in \mathbb{R}^{n}\) is the vector representation of the \(i\)-th point inside a given group (the neighborhood of a given input point). Note that \(\operatorname{MAX}\) is the component-wise max, as explained in the PointNet paper (https://arxiv.org/abs/1612.00593).

__init__(dim_out, H_activation=None, H_initializer=None, H_regularizer=None, H_constraint=None, gamma_activation=None, gamma_kernel_initializer=None, gamma_kernel_regularizer=None, gamma_kernel_constraint=None, gamma_bias_enabled=True, gamma_bias_initializer=None, gamma_bias_regularizer=None, gamma_bias_constraint=None, **kwargs)

See Layer and layer.Layer.__init__().

build(dim_in)

Build the \(\pmb{H} \in \mathbb{R}^{D_{\text{out}} \times n}\) matrix representing the kernel or weights of the 1DConv or SharedMLP, also the \(\pmb{\Gamma} \in \mathbb{R}^{D_{\text{out}} \times D_{\text{out}}}\) representing the weights of the MLP, and \(\pmb{\gamma} \in \mathbb{R}^{D_{\text{out}}}\) representing its bias (if requested).

See Layer and layer.Layer.build().

call(inputs, training=False, mask=False)

The computation of the \(\mathcal{Y} \in \mathbb{R}^{K \times R \times D_{\text{out}}}\) output feature space.

First, the structure and feature spaces are concatenated to compose the full point cloud matrix \(\pmb{P} = [\pmb{X} | \pmb{F}]\) for each neighborhood in each receptive field in the batch. Then, these \(\pmb{P} \in \mathbb{R}^{\kappa \times n}\) matrices are convolved through a SharedMLP (Unitary-1DConv) such that \((\pmb{P}\pmb{H}^\intercal) \in \mathbb{R}^{\kappa \times D_{\text{out}}}\).

Then, the component-wise max is computed to achieve a vector representation of each point \(\pmb{p}_{i*}^* \in \mathbb{R}^{D_{\text{out}}},\, i=1,\ldots,R\). These representations can be aranged as row-wise vectors in a matrix \(\pmb{P}^{*} \in \mathbb{R}^{R \times D_{\text{out}}}\).

Finally, an MLP must be computed on the \(\pmb{P}^*\) matrix. It can be done without bias:

\[\pmb{Y} = \pmb{P}^{*} \pmb{\Gamma}\]

Or it can be computed with bias:

\[\pmb{Y} = (\pmb{P}^{*} \pmb{\Gamma}) \oplus \pmb{\gamma}\]

Where \(\oplus\) is the broadcast sum typical in machine learning contexts. More concretely, it is a sum of the \(\pmb{\gamma} \in \mathbb{R}^{D_{\text{out}}}\) vector along the rows of the \((\pmb{P}^* \pmb{\Gamma}) \in \mathbb{R}^{R \times D_{\text{out}}}\) matrix.

The output tensor is simply a concatenation along the external axis of the many (\(K\)) \(\pmb{Y}\) matrices.

Parameters:

inputs –

The input such that:

– inputs[0]

is the structure space tensor representing the geometry of the many receptive fields in the batch.

\[\mathcal{X} \in \mathbb{R}^{K \times R \times n_x}\]

– inputs[1]

is the feature space tensor representing the features of the many receptive fields in the batch.

\[\mathcal{F} \in \mathbb{R}^{K \times R \times n_f}\]

– inputs[2]

is the indexing tensor representing the neighborhoods of \(\kappa\) neighbors for each input point, in the same space.

\[\mathcal{N} \in \mathbb{Z}^{K \times R \times \kappa}\]

Returns:

The output feature space \(\mathcal{Y} \in \mathbb{R}^{K \times R \times D_{\text{out}}}\).

calc_full_dimensionality()

Compute the full dimensionality on which the PointNet operator works.

Returns:

The dimensionality of the feature space considered by the PointNet operator. Note it is not necessarily the same that self.feature_dimensionality because the structure space can be concatenated to the feature space before applying the PointNet operator.

Return type:

int

get_config()

Return necessary data to serialize the layer

classmethod from_config(config)

Use given config data to deserialize the layer

model.deeplearn.layer.hourglass_layer module

class model.deeplearn.layer.hourglass_layer.HourglassLayer(*args, **kwargs)

Bases: Layer

Author:

Alberto M. Esmoris Pena

An hourglass layer consists of two unbiased MLPs. The first one uses the weights \(\pmb{W_1} \in \mathbb{R}^{D_{\mathrm{in}} \times D_{\mathrm{h}}}\), to transform the features of \(m \in \mathbb{Z}_{>0}\) points \(\pmb{F} \in \mathbb{R}^{m \times D_{\mathrm{in}}}\) to a reduced feature space of \(D_{h} \in \mathbb{Z}_{>0}\) features. Then, the second unbiased MLP transforms the features from the reduced space to an output feature space of \(D_{\mathrm{out}} \in \mathbb{Z}_{>0}\) features using the weights \(\pmb{W_2} \in \mathbb{R}^{D_h \times D_{\mathrm{out}}}\).

The hourglass layer is a map \(\mathcal{H} : \mathbb{R}^{m \times D_{\mathrm{in}}} \to \mathbb{R}^{m \times D_{\mathrm{out}}}\) that can be mathematically summarized as:

\[\pmb{Y} = \sigma_2(\sigma_1(\pmb{F} \pmb{W_1})\pmb{W_2}) \in \mathbb{R}^{m \times D_{out}}\]

Where \(\sigma_i\) represents an activation function like a ReLU.

A regularization strategy is necessary to circumvent any potential information loss when mapping a high-dimensional feature space to a low-dimensional feature space. This problem can be addressed adding an extra term to the original loss function \(\mathcal{L}\) to obtain a new loss function \(\mathcal{L}' = \mathcal{L} + \beta \mathcal{L}_{h}\) where the hyperparameter \(\beta \in \mathbb{R}\) controls the contribution/impact of the hourglass regularization term to the final loss and:

\[\mathcal{L}_{h} = \left\lVert{ \dfrac{ \pmb{W_1}^{\intercal}\pmb{W_1} }{ \lVert\pmb{W_1}\rVert_2^2 } - \pmb{I} }\right\rVert_{F}\]

Where \(\lVert\cdot\rVert_2\) is the spectral norm of a matrix and \(\lVert\cdot\rVert_F\) is the Frobenius norm.

Further information about the hourglass layer can be read in the SFL-NET paper (https://doi.org/10.1109/TGRS.2023.3313876).

Variables:

• Dh (int) – The requested internal dimensionality, i.e., \(D_{h}\).

• Dout (int) – The requested output dimensionality, i.e., \(D_{\mathrm{out}}\).

• activation (str) – The activation function \(\sigma_1\).

• activation2 (str) – The activation function \(\sigma_2\).

• regularize (bool) – Whether to apply \(+ \beta \mathcal{L}_h\) to the loss function or not.

• beta (float) – The loss factor \(\beta\).

• spectral_strategy (str) – The type of spectral strategy to be used. It can be either “unsafe” (might break during training), “safe” (will not break during training, but it will be twice slower), or “approx” (as fast as unsafe but less prone to break during training).

• built_W1 (bool) – Whether the first matrix of weights \(\pmb{W_1} \in \mathbb{R}^{D_{\mathrm{in}} \times D_{h}}\) is built or not. Initially it is false, but it will be updated once the layer is built.

• built_W2 (bool) – Whether the second matrix of weights \(\pmb{W_2} \in \mathbb{R}^{D_h \times D_{\mathrm{out}}}\) is built or not. Initially it is false, but it will be updated once the layer is built.

• W1_initializer – The initializer for the first matrix of weights \(\pmb{W_1} \in \mathbb{R}^{D_{\mathrm{in}} \times D_{h}}\).

• W1_regularizer – The regularizer for the first matrix of weights \(\pmb{W_1} \in \mathbb{R}^{D_{\mathrm{in}} \times D_{h}}\).

• W1_constraint – The constraint for the first matrix of weights \(\pmb{W_1} \in \mathbb{R}^{D_{\mathrm{in}} \times D_{h}}\).

• W2_initializer – The initializer for the second matrix of weights \(\pmb{W_2} \in \mathbb{R}^{D_h \times D_{\mathrm{out}}}\).

• W2_regularizer – The regularizer for the second matrix of weights \(\pmb{W_2} \in \mathbb{R}^{D_h \times D_{\mathrm{out}}}\).

• W2_constraint – The constraint for the second matrix of weights \(\pmb{W_2} \in \mathbb{R}^{D_h \times D_{\mathrm{out}}}\).

__init__(Dh, Dout, activation='relu', activation2='relu', regularize=True, spectral_strategy='approx', beta=0.1, built_W1=False, built_W2=False, W1_initializer=None, W1_regularizer=None, W1_constraint=None, W2_initializer=None, W2_regularizer=None, W2_constraint=None, sigma=None, sigma2=None, **kwargs)

See Layer and layer.Layer.__init__().

build(dim_in)

Build the \(\pmb{W_1} \in \mathbb{R}^{D_{\mathrm{in}} \times D_h}\) matrix representing the weights of the first MLP and the \(\pmb{W_2} \in \mathbb{R}^{D_h \times D_{\mathrm{out}}}\) matrix representing the weights of the second MLP.

See Layer and layer.Layer.build().

call(inputs, training=False, mask=False)

Compute:

\[\sigma_2\left({ \sigma_1(\pmb{F} \pmb{W_1}) \pmb{W_2} }\right)\]

See HourglassLayer for more details.

Parameters:

inputs – The feature space tensor representing the batch of structure space matrices.

Returns:

The output feature space tensor \(\mathcal{Y} \in \mathbb{R}^{K \times m \times D_{\mathrm{out}}}\). Besides, it might potentially add a term to the loss function.

do_hourglass_regularization()

Apply the hourglass regularization described in HourglassLayer.

do_no_regularization()

Do not apply any regularization at all.

compute_spectral_unsafe(cov)

Because the spectral norm is computed for the W_1^T W_1 matrix, it is guaranteed to be hermitian and positive semidefinite. Algebraically speaking, its eigenvalues can be derived with the eigh routine that uses heevd (hermitian eigenvalue decomposition) from LAPACK. However, sometimes, during training, the resulting matrix might present numerical issues causing the heevd routine to fail.

See HourglassLayer.compute_spectral_safe() and HourglassLayer.compute_spectral_approx().

compute_spectral_safe(cov)

A safe but two times slower alternative to the HourglassLayer.compute_spectral_unsafe() function. It uses the singular value decomposition approach instead of the eigh function. Consequently, it is robust to numerical issues and it will not fail during training.

See HourglassLayer.compute_spectral_unsafe() and HourglassLayer.compute_spectral_approx().

compute_spectral_approx(cov)

A compromise solution that has the speed of the unsafe alternative but is much less likely to break the training process. It considers \(\pmb{W_1}^T \pmb{W_1} + \lambda \pmb{I}\) with \(\lambda \to 0\) instead of \(\pmb{W_1}^T \pmb{W_1}\) to prevent numerical issues.

See HourglassLayer.compute_spectral_safe() and HourglassLayer.compute_spectral_unsafe().

get_config()

Return necessary data to serialize the layer

classmethod from_config(config)

Use given config data to deserialize the layer

model.deeplearn.layer.interdimensional_point_transformer_layer module

class model.deeplearn.layer.interdimensional_point_transformer_layer.InterdimensionalPointTransformerLayer(*args, **kwargs)

Bases: PointTransformerLayer

Author:

Alberto M. Esmoris Pena

Interdimensional version of the PointTransformerLayer layer. Instead of transforming \(R\) input points with \(D_{\text{in}}\) features into \(R\) output points with \(D_{\text{out}}\) features, it transforms \(R_1\) input points with \(D_{\text{in}}\) features into \(R_2\) output points with \(D_{\text{out}}\) features.

See PointTransformerLayer.

__init__(**kwargs)

See Layer and Layer.__init__(). Also see PointTransformerLayer and PointTransformerLayer.__init__().

build(dim_in)

See PointTransformerLayer and PointTransformerLayer.build().

call(inputs, training=False, mask=False)

Compute the interdimensional version of PointTransformer. The main difference with respect to the class PointTransformerLayer

Parameters:

inputs –

The input such that:

– inputs[0]

is the structure space tensor representing the geometry of the many original-dimensional receptive fields in the batch.

\[\mathcal{X_a} \in \mathbb{R}^{K \times R_1 \times n_x}\]

– inputs[1]

is the structure space tensor representing the geometry of the many new-dimensional receptive fields in the batch.

\[\mathcal{X_b} \in \mathbb{R}^{K \times R_2 \times n_x}\]

– inputs[2]

is the feature space tensor representing the features of the many original-dimensional receptive fields in the batch.

\[\mathcal{F} \in \mathbb{R}^{K \times R_1 \times n_f}\]

– inputs[3]

is the indexing tensor representing the neighborhoods of \(\kappa\) neighbors in the original-dimensional space for each point in the new-dimensional space.

\[\mathcal{N} \in \mathbb{Z}^{K \times R_2 \times \kappa}\]

Returns:

The output feature space \(\mathcal{\hat{F}} \in \mathbb{R}^{K \times R_2 \times D_{\mathrm{out}}}\).

model.deeplearn.layer.kpconv_layer module

class model.deeplearn.layer.kpconv_layer.KPConvLayer(*args, **kwargs)

Bases: Layer

Author:

Alberto M. Esmoris Pena

A kernel point convolution layer receives batches of \(R\) points with \(D_{\mathrm{in}}\) features each and \(\kappa\) known neighbors in the same space. These inputs are used to compute an output feature space of \(R\) points with \(D_{\mathrm{out}}\) features each. In doing so, the indexing tensor \(\mathcal{N} \in \mathbb{Z}^{K \times R \times \kappa}\) is used to link \(\kappa\) neighbors for each of the \(R\) input points in each of the \(K\) input batches.

The layer is defined by a structure space matrix \(\pmb{Q} \in \mathbb{R}^{m_q \times n_x}\) representing \(m_q\) points in a \(n_x\)-dimensional Euclidean space, and a tensor of weights \(\mathcal{W} \in \mathbb{R}^{m_q \times D_{\mathrm{in}} \times D_{\mathrm{out}}}\) who has \(m_q\) slices corresponding to a matrix of weights \(\pmb{W}_k \in \mathbb{R}^{D_{\mathrm{in}} \times D_{\mathrm{out}}}\) associated to a different kernel point each.

Further details on the KPConv operator can be read in the corresponding paper (https://arxiv.org/abs/1904.08889).

Variables:

• sigma (float) – The influence distance governing the kernel (\(\sigma\)).

• kernel_radius (float) – The radius of the ball where the kernel is defined.

• num_kernel_points (int) – How many points (\(m_q\)) compose the kernel.

• deformable (bool) – Whether the \(m_q\) kernel points must be optimized (True) or not (False, def.) by the neural network.

• Dout (int) – The output dimensionality of the convolution.

• built_Q (bool) – Whether the structure space matrix \(\pmb{Q} \in \mathbb{R}^{m_q \times n_x}\) is built. Initially it is false, but it will be updated once the layer is built.

• built_W (bool) – Whether the \(m_q\) weight matrices \(\pmb{W} \in \mathbb{R}^{D_{\mathrm{in}} \times D_{\mathrm{out}}}\) corresponding to each kernel point are built or not. Initially it is false, but it will be updated once the layer is built.

• W_initializer – The initializer for the tensor of weights \(\mathcal{W} \in \mathbb{R}^{m_q \times D_{\mathrm{in}} \times D_{\mathrm{out}}}\).

• W_regularizer – The regularizer for the tensor of weights \(\mathcal{W} \in \mathbb{R}^{m_q \times D_{\mathrm{in}} \times D_{\mathrm{out}}}\).

• W_constraint – The constraints for the tensor of weights \(\mathcal{W} \in \mathbb{R}^{m_q \times D_{\mathrm{in}} \times D_{\mathrm{out}}}\).

__init__(sigma=1.0, kernel_radius=1.0, num_kernel_points=19, deformable=False, Dout=320, built_Q=False, built_W=False, W_initializer=None, W_regularizer=None, W_constraint=None, **kwargs)

See Layer and layer.Layer.__init__().

build(dim_in)

Build the \(\pmb{Q} \in \mathbb{R}^{m_q \times n_x}\) matrix representing the kernel’s structure space. Typically, the initialization is delegated to a KernelPointBallInitializer object. Also, build the \(\mathcal{W} \in \mathbb{R}^{m_q \times D_{\mathrm{in}} \times D_{\mathrm{out}}}\) tensor of weights.

See Layer and layer.Layer.build().

call(inputs, training=False, mask=False)

Let \(\mathcal{Q} = (\pmb{Q}, \mathcal{W})\) represent the kernel including both the structure and weights information. Also, let \(\pmb{P} = [\pmb{X} | \pmb{F}]\) be the full matrix representation of the input receptive field including both the structure and the feature spaces.

For then, the output for each receptive field in the input batch can be computed as a matrix \(\pmb{Y} \in \mathbb{R}^{R \times D_{\mathrm{out}}}\) that arises by applying point-wise convolutions as shown below:

\[\begin{split}\pmb{Y} = \left[\begin{array}{ccc} \rule[.5ex]{3.0ex}{0.6pt} & (\pmb{P} * \mathcal{Q})(\pmb{x}_{1*}) & \rule[.5ex]{3.0ex}{0.6pt} \\ \vdots & \vdots & \vdots \\ \rule[.5ex]{3.0ex}{0.6pt} & (\pmb{P} * \mathcal{Q})(\pmb{x}_{R*}) & \rule[.5ex]{3.0ex}{0.6pt} \end{array}\right]\end{split}\]

Each point-wise convolution is defined as:

\[\left(\pmb{P} * \mathcal{Q}\right) (\pmb{x}_{i*}) = \sum_{\pmb{x}_{j*} \in \mathcal{N}_{\pmb{x}_{i*}}}{ \Biggl[{ \sum_{k=1}^{m_q} \max \; \biggl\{ 0, 1 - \dfrac{ \lVert \pmb{x}_{j*} - \pmb{x}_{i*} - \pmb{q}_{k*} \rVert }{ \sigma } \biggr\} } \pmb{W}_{k}^\intercal \Biggr] \pmb{f}_{j*} }\]

Where \(\mathcal{N}_{\pmb{x}_{i*}}\) is the neighborhood of the \(\kappa\) closest neighbors of \(\pmb{x}_{i*}\) in the input receptive field \(\pmb{X}\).

Parameters:

inputs –

The input such that:

– inputs[0]

is the structure space tensor representing the geometry of the many receptive fields in the batch.

\[\mathcal{X} \in \mathbb{R}^{K \times R \times n_x}\]

– inputs[1]

is the feature space tensor representing the features of the many receptive fields in the batch.

\[\mathcal{F} \in \mathbb{R}^{K \times R \times n_f}\]

– inputs[2]

is the indexing tensor representing the neighborhoods of \(\kappa\) neighbors for each input point, in the same space.

\[\mathcal{N} \in \mathbb{Z}^{K \times R \times \kappa}\]

Returns:

The output feature space \(\mathcal{Y} \in \mathbb{R}^{K \times R \times D_{\mathrm{out}}}\).

gather_neighborhoods(N, X, F)

Assist the KPConvLayer.call() method by gathering the data representing the structure and feature spaces of each neighborhood.

compute_output_features(X, NX, NF)

Assist the KPConvLayer.call() method by applying the point-wise convolutions to compute the output features.

build_W(Din)

Assist the KPConvLayer.build() method in building the \(\mathcal{W} \in \mathbb{R}^{K \times D_{\mathrm{in}} \times D_{\mathrm{out}}}\) tensor, i.e., the convolution weights.

Parameters:

Din (int) – The dimensionality of the input feature space \(D_{\mathrm{in}}\).

get_config()

Return necessary data to serialize the layer

classmethod from_config(config)

Use given config data to deserialize the layer

export_representation(dir_path, out_prefix=None, Wpast=None)

Export a set of files representing the state of the kernel for both the structure (Q) and the weights (W).

Parameters:

• dir_path (str) – The directory where the representation files will be exported.

• out_prefix (str) – The output prefix to name the output files.

• Wpast (np.ndarray or tf.Tensor or None) – The weights of the kernel in the past.

Returns:

Nothing at all, but the representation is exported as a set of files inside the given directory.

model.deeplearn.layer.kpconvx_layer module

class model.deeplearn.layer.kpconvx_layer.KPConvXLayer(*args, **kwargs)

Bases: Layer

Author:

Alberto M. Esmoris Pena

A kernel point convolution with attention (KPConvX) receives batches of \(R\) points with \(D_{\mathrm{in}}\) features each and \(\kappa\) known neighbors in the same space. These inputs are used to compute an output feature space of \(R\) points with \(D_{\text{in}}\) features each (i.e., the output dimensionality matches the input one). In doing so, the indexing tensor \(\mathcal{N} \in \mathbb{Z}^{B \times R \times \kappa}\) is used to link \(\kappa\) neighbors for each of the \(R\) input points in each of the \(B\) input batches.

The KPConvX layer is composed of many consecutive blocks. From now one, a single block is mathematically detailed for a single element in the batch.

The kernel point convolution with attention can be defined as an operator such that

\[\begin{split}\pmb{\tilde{f}_{i*}} = \left(\pmb{P} * \mathcal{Q}\right)(\pmb{x}_{i*}) & = \sum_{\pmb{x}_{j*} \in \mathcal{N}(\pmb{x}_{i*})}{ \max \; \left\{ 0, 1 - \dfrac{ \lVert \pmb{x}_{j*} - \pmb{x}_{i*} - \pmb{q}_{k_j*} \rVert }{ \sigma } \right\} \left( \pmb{a}_{k_j*} \bowtie \pmb{w}_{k_j*} \right) \odot \pmb{f}_{j*} } \\ & = \sum_{\pmb{x}_{j*} \in \mathcal{N}(\pmb{x}_{i*})}{ h_{ij} \bigl(\pmb{a}_{k_j*} \bowtie \pmb{w}_{k_j*}\bigr) \odot \pmb{f}_{j*} } .\end{split}\]

Note that \(\mathcal{N}(\pmb{x}_{i*})\) is the neighborhood of the \(i\)-th input point, \(\pmb{q}_{k_j*}\) represents the kernel point that is closest to the \(j\)-th neighbor of the \(i\)-th input point, \(\bowtie\) is a grouped Hadamard product, \(\pmb{W} \in \mathbb{R}^{m_q \times D_{\text{in}}}\) is the matrix of weights (for \(m_q\) kernel points and \(D_{\text{in}}\) features), and

\[\alpha(\pmb{f}_{i*}) = \sigma_{\alpha} \biggl( \sigma_{\phi} \Bigl( \mathcal{Z}_{\phi} \bigl( \pmb{f}_{i*} \pmb{\Phi} \oplus \pmb{\phi} \bigr) \Bigr) \pmb{\Psi} \oplus \pmb{\psi} \biggr) \in \mathbb{R}^{m_q D_{\text{in}}/G} ,\]

which leads to the matrix \(\pmb{A} \in \mathbb{R}^{m_q \times D_{\text{in}}/G}\) with rows

\[\pmb{a}_{k*} = \alpha(\pmb{f}_{i*})((k-1)D_{\text{in}}/G : kD_{\text{in}}/G) .\]

Note that the rows \(\pmb{\tilde{f}}_{i*} \in \mathbb{R}^{D_{\text{in}}}\) lead to the hidden feature space matrix \(\pmb{\widetilde{F}} \in \mathbb{R}^{R \times D_{\text{in}}}\).

In the equation of the vector of modulations \(\alpha(\pmb{f}_{i*}) \in \mathbb{R}^{m_q D_{\text{in}}/G}\) for a group size \(G \in \mathbb{Z}_{>0}\) satisfying \(G \mid D_{\text{in}}\), \(\mathcal{Z}_{\phi}\) represents batch normalization, \(\pmb{\Phi} \in \mathbb{R}^{D_{\text{in}} \times D_{\text{in}}}\) is a matrix of weights, \(\pmb{\phi} \in \mathbb{R}^{D_\text{in}}\) is its corresponding vector of weights, \(\pmb{\Psi} \in \mathbb{R}^{D_{\text{in}} \times m_q D_{\text{in}}/G}\) is another matrix of weights, and \(\pmb{\psi} \in \mathbb{R}^{m_q D_{\text{in}}/G}\) is its corresponding vector of weights, \(\sigma_{\phi}\) is an activation function (typically a leaky ReLU), and \(\sigma_{\alpha}\) is a sigmoid function.

For the sake of understanding, let us look at the following example about how to compute a grouped Hadamard product for \(D_{\text{in}}=6\) input features and a group size of \(G=2\):

\[(a_1, a_2, a_3) \bowtie (f_1, f_2, f_3, f_4, f_5, f_6) = (a_1f_1, a_1f_2, a_2f_3, a_2f_4, a_3f_5, a_3f_6)\]

Now, the KPConvX operator is followed by an upsampling MLP

\[\pmb{\tilde{h}}(\pmb{\tilde{f}}_{i*}) = \sigma_{\tilde{h}}\Bigl(\mathcal{Z}_{\tilde{h}}\bigl( \pmb{\tilde{f}}_{i*} \pmb{\widetilde{A}} + \pmb{\tilde{b}} \bigr)\Bigr) \leadsto \pmb{\widetilde{H}} = \sigma_{\tilde{h}}\Bigl( \mathcal{Z}_{\tilde{h}}\bigl( \pmb{\widetilde{F}} \pmb{\widetilde{A}} \oplus \pmb{\tilde{b}} \bigr)\Bigr) \in \mathbb{R}^{R \times D_H}\]

where \(\pmb{\widetilde{A}} \in \mathbb{R}^{D_{\text{in}} \times D_H}\) is the matrix of weights, \(\pmb{\tilde{b}} \in \mathbb{R}^{D_H}\) is the vector of weights, \(\mathcal{Z}_{\tilde{h}}\) represents batch normalization, \(\sigma_{\tilde{h}}\) is an activation function (typically a leaky ReLU), and \(D_H \in \mathbb{Z}_{>0}\) is the dimensionality of the hidden feature space which is set typically to \(4 D_{\text{in}}\).

Then, a downsampling MLP restores the input dimensionality (that will be also the output dimensionality) such that

\[\pmb{\hat{h}}(\pmb{\tilde{h}}_{i*}) = \sigma_{\hat{h}}\Bigl(\mathcal{Z}_{\hat{h}}\bigl( \pmb{\tilde{h}}_{i*} \pmb{\widehat{A}} + \pmb{\hat{b}} \bigr)\Bigr) \leadsto \pmb{\widehat{H}} = \sigma_{\hat{h}}\Bigl( \mathcal{Z}_{\hat{h}}\bigl( \pmb{\widetilde{H}} \pmb{\widehat{A}} \oplus \pmb{\hat{b}} \bigr)\Bigr) \in \mathbb{R}^{R \times D_{\text{in}}} ,\]

where \(\pmb{\widehat{A}} \in \mathbb{R}^{D_H \times D_{\text{in}}}\) is the matrix of weights, \(\pmb{\hat{b}} \in \mathbb{R}^{D_{\text{in}}}\) is the vector of weights, \(\mathcal{Z}_{\hat{h}}\) represents batch normalization, and \(\sigma_{\hat{h}}\) is an activation function (typically a leaky ReLU).

Finally, the output feature space for a single block will sum the input as a residual contribution to the downsampled hidden feature space \(\pmb{\widehat{H}} \in \mathbb{R}^{R \times D_{\text{in}}}\) as follows

\[\pmb{\widehat{F}} = \pmb{F} + \pmb{\widehat{H}}\]

Further details on the KPConvX operator can be read in the corresponding paper (https://doi.org/10.48550/arXiv.2405.13194).

Variables:

• blocks (int) – The number of blocks in the layer.

• drop_path (float) – The probability inside \([0, 1]\) of dropping a given block. Note that if zero is given, no drop path will b applied at all.

• sigma (float) – The influence distance for each kernel point.

• shell_radii (list or np.ndarray of float) – The radius for each spherical shell.

• shell_points (list or np.ndarray of int) – The number of points at each spherical shell.

• mq (int) – The number of kernel points.

• DH (int) – The number of hidden channels.

• groups (int) – The number of groups \(G\). Note that it must divide the number of input features.

• deformable (bool) – Whether the structure space of the kernel (i.e., the kernel points) can be optimized or not.

• Q_initializer (FibonacciShellInitializer) – The initializer for the kernel’s structure space.

• initializer – The initializer for each tensor of weights.

• regularizer – The regularizer for each tensor of weights.

• constraint – The constraint for each tensor of weights.

• sigmoid – The sigmoid activation function.

• activation – The activation layer (typically Leaky ReLU).

• bn_momentum (float) – The momentum for the batch normalization layers.

• bn_phi (list of keras.layers.BatchNormalization) – The batch normalization layers for the computation of alpha.

• bn_htilde (list of keras.layers.BatchNormalization) – The batch normalization layers for the computation of the upsampling MLP.

• bn_hhat (list of keras.layers.BatchNormalization) – The batch normalization layers for the computation of the downsampling MLP.

• built_Q (bool) – Whether the structure space matrix \(\pmb{Q} \in \mathbb{R}^{m_q \times n_x}\) is built. Initially it is false, but it will be updated once the layer is built.

• built_W (bool) – Whether the tensors of weights for each kernel point at each depth \(\mathcal{W} \in \mathbb{R}^{\text{blocks} \times m_q \times D_{\text{in}}}\) are built.

• built_Phi (bool) – Whether the tensor of matrix-like weights for the first MLP in the alpha function \(\mathcal{\Phi} \in \mathbb{R}^{\text{blocks} \times D_{\text{in}} \times D_{\text{in}}}\) are built.

• built_phi (bool) – Whether the tensor of vector-like weights for the first MLP in the alpha function \(\mathcal{\phi} \in \mathbb{R}^{\text{blocks} \times D_{\text{in}}}\) are built.

• built_Psi (bool) – Whether the tensor of matrix-like weights for the second MLP in the alpha function \(\Psi \in \mathbb{R}^{\text{blocks} \times D_{in} \times m_q D_{\text{in}}/G}\) are built.

• built_psi (bool) – Whether the tensor of vector-like weights for the second MLP in the alpha function \(\psi \in \mathbb{R}^{\text{blocks} \times m_q D_{\text{in}}/G}\) are built.

• built_Atilde (bool) – Whether the tensor of matrix-like weights for the upsampling MLP \(\mathcal{\widetilde{A}} \in \mathbb{R}^{\text{blocks} \times D_{\text{in}} \times D_{\text{in}}}\) are built.

• built_btilde (bool) – Whether the tensor of vector-like weights for the upsampling MLP \(\tilde{b} \in \mathbb{R}^{\text{\blocks} \times D_H}\) are built.

• built_Ahat (bool) – Whether the tensor of matrix-like weights for the downsampling MLP \(\mathcal{\widehat{A}} \in \mathbb{R}^{\text{blocks} \times D_H \times D_{\text{in}}}\) are built.

• built_bhat (bool) – Whether the tensor of vector-like weights for the downsampling MLP \(\hat{b} \in \mathbb{R}^{\text{blocks} \times D_{\text{in}}}\)

__init__(blocks=1, drop_path=0, sigma=1.0, shell_radii=[0, 0.5, 1.0], shell_points=[1, 7, 13], DH=384, groups=8, deformable=False, initializer=None, regularizer=None, constraint=None, bn_momentum=0.9, bn_phi=None, bn_htilde=None, bn_hhat=None, activation=<LeakyReLU name=leaky_re_lu_1, built=True>, built_Q=False, built_W=False, built_Phi=False, built_phi=False, built_Psi=False, built_psi=False, built_Atilde=False, built_btilde=False, built_Ahat=False, built_bhat=False, **kwargs)

See Layer and layer.Layer.__init__().

build(dim_in)

Build the \(\pmb{Q} \in \mathbb{R}^{m_q \times n_x}\) matrix representing the kernel’s structure space. Typically, the initialization is delegated to a FibonacciShellInitializer object. Also, build the many weights matrices and vectors.

See Layer and layer.Layer.build().

call(inputs, training=False, mask=False)

Compute the KPConvXLayer on an input batch.

Parameters:

inputs –

The input such that:

– inputs[0]

is the structure space tensor representing the geometry of the many receptive fields in the batch.

\[\mathcal{X} \in \mathbb{R}^{B \times R \times n_x}\]

– inputs[1]

is the feature space tensor representing the features of the many receptive fields in the batch.

\[\mathcal{F} \in \mathbb{R}^{B \times R \times D_{\text{in}}}\]

– inputs[2]

is the indexing tensor representing the neighborhoods of \(\kappa\) neighbors for each input point, in the same space.

\[\mathcal{N} \in \mathbb{Z}^{B \times R \times \kappa}\]

Returns:

The output feature space \(\mathcal{\widehat{F}} \in \mathbb{R}^{B \times R \times D_{\mathrm{in}}}\).

get_config()

Return necessary data to serialize the layer

classmethod from_config(config)

Use given config data to deserialize the layer

export_representation(dir_path, out_prefix=None, Wpast=None)

Export a set of files representing the state of the kernel for both the structure (Q) and the weights (W).

Parameters:

• dir_path (str) – The directory where the representation files will be exported.

• out_prefix (str) – The output prefix to name the output files.

• Wpast (np.ndarray or tf.Tensor or None) – The weights of the kernel in the past.

Returns:

Nothing at all, but the representation is exported as a set of files inside the given directory.

model.deeplearn.layer.layer module

class model.deeplearn.layer.layer.Layer(*args, **kwargs)

Bases: Layer

Author:

Alberto M. Esmoris Pena

A layer can be seen as a map \(f\) from an arbitrary tuple of input tensors \(\mathcal{X}_{1}, \ldots, \mathcal{X}_{m}\) to an arbitrary tuple of output tensors \(\mathcal{Y}_{1}, \ldots, \mathcal{Y}_{n}\), i.e., \(f(\mathcal{X}_1, \ldots, \mathcal{X}_m) = (\mathcal{Y}_1, \ldots, \mathcal{Y}_n)\).

The Layer class provides an interface that must be realized by any class that must assume the role of a layer inside a neural network.

__init__(**kwargs)

Initialize the member attributes of the layer and the internal weights that do not depend on the input dimensionality.

Parameters:

kwargs – The key-word specification to parametrize the layer.

build(dim_in)

Logic to build the layer before the first call is executed.

This method can be overloaded by any derived class to either change or extend the logic.

Parameters:

dim_in – The dimensionality of the input tensor.

call(inputs, training=False, mask=False)

The layer’s computation logic.

Parameters:

• inputs – The input tensor or the list of input tensors.

• training (bool) – True when the layer is called during training, False otherwise.

• mask – Boolean mask (one boolean per input timestep).

Returns:

The output tensor.

get_config()

Obtain the dictionary specifying how to serialize the layer.

Returns:

The dictionary with the necessary data to serialize the layer.

Return type:

dict

classmethod from_config(config)

Deserialize a layer from given specification.

Parameters:

config – The dictionary specifying how to deserialize the layer.

Returns:

The deserialized layer.

Return type:

Layer or derived

model.deeplearn.layer.light_kpconv_layer module

class model.deeplearn.layer.light_kpconv_layer.LightKPConvLayer(*args, **kwargs)

Bases: KPConvLayer

Author:

Alberto M. Esmoris Pena

A light kernel point convolution layer redefines the typical KPConvLayer to work with a single matrix of weights (instead of one per kernel point) and supporting an optional scale factor for each kernel point and input feature pair. The new tensor of weights is thus a matrix \(\pmb{W} \in \mathbb{R}^{D_{\mathrm{in}} \times D_{\mathrm{out}}}\). Also, all the scale factors can be represented as a matrix \(\pmb{A} \in \mathbb{R}^{m_q \times D_{\mathrm{in}}}\).

See KPConvLayer.

Variables:

• A_trainable (bool) – Whether the scale factors are trainable (i.e., can be updated through backpropagation) or not.

• built_A (bool) – Whether the matrix of scale factors \(\pmb{A} \in \mathbb{R}^{m_q \times D_{\mathrm{in}}}\) is built. Initially it is false, but it will be updated once the layer is built.

• A_initializer – The initializer for the scale factors matrix.

• A_regularizer – The regularizer for the scale factors matrix.

• A_constraint – The constraint for the scale factors matrix.

__init__(A_trainable=True, built_A=False, A_initializer='ones', A_regularizer=None, A_constraint=None, **kwargs)

See KPConvLayer and KPConvLayer.__init__().

build(dim_in)

Build the whole Light KPConv layer by calling the parent’s build method (see KPConvLayer.build()) with the derived LightKPConvLayer.build_W() logic and also build the \(\pmb{A} \in \mathbb{R}^{m_q \times D_{\mathrm{in}}}\) matrix.

See KPConvLayer and KPConvLayer.build()).

compute_output_features(X, NX, NF)

Assist the KPConvLayer.call() method by applying the point-wise convolutions with the single matrix of weights \(\pmb{W} \in \mathbb{R}^{D_{\mathrm{in}} \times D_{\mathrm{out}}}\) and the scale factors in \(\pmb{A} \in \mathbb{R}^{m_q \times D_{\mathrm{in}}}\).

The new operation consists of redefining the kernel point convolution to work with the light kernel (where \(\mathcal{Q} = (\pmb{Q}, \pmb{W}))\) instead of \(\mathcal{Q} = (\pmb{Q}, \mathcal{W}))\)) such that:

\[\begin{split}\begin{aligned} \left(\pmb{P} * \mathcal{Q} \right) (\pmb{x}_{i*}) &=\; \sum_{\pmb{x}_{j*} \in \mathcal{N}_{\pmb{x}_{i*}}} \left[ \sum_{k=1}^{m_q}{ \max \; \left\{ 0,\, 1 - \dfrac{ \lVert{ \pmb{x}_{j*} - \pmb{x}_{i*} - \pmb{q}_{k*} }\rVert }{ \sigma } \right\} \biggl( \operatorname{diag}\left(\pmb{a}_{k*}\right) \pmb{W} \biggr)^{\intercal} } \right] \pmb{f}_{j*} \\ &=\; \sum_{\pmb{x}_{j*} \in \mathcal{N}_{\pmb{x}_{i*}}} \left(\operatorname{diag}\left[\sum_{k=1}^{m_q}{ \max \; \left\{ 0, 1 - \dfrac{ \lVert \pmb{x}_{j*} - \pmb{x}_{i*} - \pmb{q}_{k*} \rVert }{ \sigma } \right\} \pmb{a}_{k*} } \right] \pmb{W}\right)^{\intercal} \pmb{f}_{j*} \end{aligned}\end{split}\]

See KPConvLayer.compute_output_features() and KPConvLayer.call() for further details.

build_W(Din)

Assist the LightKPConvLayer.build() method in building the \(\mathcal{W} \in \mathbb{R}^{D_{\mathrm{in}} \times D_{\mathrm{out}}}\) tensor, i.e., the convolution weights.

See KPConvLayer and KPConvLayer.build().

Parameters:

Din (int) – The dimensionality of the input feature space \(D_{\mathrm{in}}\).

get_config()

Return necessary data to serialize the layer

classmethod from_config(config)

Use given config data to deserialize the layer

export_representation(dir_path, out_prefix=None, Wpast=None, Apast=None)

Export a set of files representing the state of the kernel for both the structure (Q) and the weights (W).

Parameters:

• dir_path (str) – The directory where the representation files will be exported.

• out_prefix (str) – The output prefix to name the output files.

• Wpast (np.ndarray or tf.Tensor or None) – The weights of the kernel in the past.

• Apast (np.ndarray or tf.Tensor or None) – The scale factors in the past.

Returns:

Nothing at all, but the representation is exported as a set of files inside the given directory.

model.deeplearn.layer.point_mlp_layer module

class model.deeplearn.layer.point_mlp_layer.PointMLPLayer(*args, **kwargs)

Bases: Layer

Author:

Alberto M. Esmoris Pena

A PointMLP layer receives batches of \(R\) points with \(\kappa\) neighbors each and \(D_{\text{in}}\) input features. These inputs are reduced to \(D_{\text{out}}\) point-wise features for each of the \(R\) points leading to the output feature space. More formally, the input feature space is a 4-tensor \(\mathcal{F} \in \mathbb{R}^{B \times R \times \kappa \times D_{\text{in}}}\) with \(B\) elements per batch, while the output feature space is a 3-tensor \(\mathcal{\widehat{F}} \in \mathbb{R}^{B \times R \times D_{\text{out}}}\) .

The PointMLP layer can be mathematically described for a single element as

\[\pmb{\hat{f}}_{i*} = \Psi \left( \mathcal{A}\left( \Phi\left(\pmb{F}_{i**}\right) \right) \right)\]

where \(\Phi\) represents a block of residual shared MLPs, \(\mathcal{A}\) represents a symmetric aggregation function (e.g., max pooling), and \(\Psi\) represents the final residual block that transforms the aggregated features to the final output feature space.

This layer is inspired in the PointMLP paper (https://doi.org/10.48550/arXiv.2202.07123).

Variables:

• Dout (int) – The dimensionality of the output feature space.

• groups (int) – Features can be grouped such that each group is associated to a different set of weights. Note that the input and output dimensionalities must be divisible by the number of groups.

• Phi_blocks (int) – The number of blocks for the residual shared MLPs.

• Phi_residual_expansion (int) – The factor multiplying the number of output features in the internal representations computed by the blocks of \(\Phi\).

• Phi_initializer – The initializer for the MLPs in \(\Phi\).

• Phi_regularizer – The regularizer for the MLPs in \(\Phi\).

• Phi_constraint – The constraint for the MLPs in \(\Phi\).

• Phi_bn (bool) – Whether to include batch normalization before the activations in \(\Phi\).

• Phi_bn_momentum (float) – The momentum for the batch normalization operations in \(\Phi\).

• Phi_activation – The activation function for \(\Phi\) (typically a ReLU).

• Psi_blocks (int) – The number of blocks for the final residual MLPs.

• Psi_residual_expansion (int) – The factor multiplying the number of output features in the internal representations computed by the blocks of \(\Psi\).

• Psi_initializer – The initializer for the MLPs in \(\Psi\).

• Psi_regularizer – The regularizer for the MLPs in \(\Psi\).

• Psi_constraint – The constraint for the MLPs in \(\Psi\).

• Psi_bn (bool) – Whether to include batch normalization before the activations in \(\Psi\).

• Psi_bn_momentum (float) – The momentum for the batch normalization operations in \(\Psi\).

• Psi_activation – The activation function for \(\Psi\) (typically a ReLU).

__init__(Dout, groups=1, Phi_blocks=2, Phi_residual_expansion=1, built_Phi=False, built_Psi=False, Phi_initializer=None, Phi_regularizer=None, Phi_constraint=None, Phi_bn=True, Phi_bn_momentum=0.9, Phi_activation=<function relu>, Phi=[None], PhiRes=[None], Psi_blocks=2, Psi_residual_expansion=1, Psi_initializer=None, Psi_regularizer=None, Psi_constraint=None, Psi_bn=True, Psi_bn_momentum=0.9, Psi_activation=<function relu>, PsiRes=[None], **kwargs)

See Layer and Layer.__init__().

build(dim_in)

Build the internal components of the layer.

See Layer and layer.Layer.build().

build_res_block(residual_expansion, initializer, regularizer, constraint, bn_momentum, activation)

Assist the PointMLPLayer.build() method in building the residual MLP blocks.

call(inputs, training=False, mask=False)

Compute the PointMLPLayer on an input batch.

Parameters:

inputs –

The input such that:

– inputs[0]

is the structure space tensor representing the geometry of the many receptive fields in the batch.

\[\mathcal{X} \in \mathbb{R}^{K \times R \times n_x}\]

– inputs[1]

is the feature space tensor representing the features of the many receptive fields in the batch.

\[\mathcal{F} \in \mathbb{R}^{K \times R \times n_f}\]

– inputs[2]

is the indexing tensor representing the neighborhoods of \(\kappa\) neighbors for each input point, in the same space.

\[\mathcal{N} \in \mathbb{Z}^{K \times R \times \kappa}\]

Returns:

The output feature space \(\mathcal{\widehat{F}} \in \mathbb{R}^{K \times R \times D_{\mathrm{out}}}\).

get_config()

Return necessary data to serialize the layer.

classmethod from_config(config)

Use given config data to deserialize the layer.

model.deeplearn.layer.point_transformer_layer module

class model.deeplearn.layer.point_transformer_layer.PointTransformerLayer(*args, **kwargs)

Bases: Layer

Author:

Alberto M. Esmoris Pena

A point transformer layer receives batches of \(R\) points with \(D_{\text{in}}\) features each, and \(\kappa\) known neighbors in the same space. These inputs are used to compute an output feature space of \(R\) points with \(D_{\text{out}}\) features each. In doing so, the indexing tensor \(\mathcal{N} \in \mathbb{Z}^{K \times R \times \kappa}\) is used to link \(\kappa\) neighbors for each of the \(R\) input points in each of the \(K\) input batches.

The layer applies a feature space transformation

\[\pmb{\hat{f}}_{i*} = \sum_{\pmb{x}_{j*} \in \mathcal{N}(\pmb{x}_{i*})}{ \sigma \left( \gamma\left( \psi(\pmb{f}_{j*}) - \phi(\pmb{f}_{i*}) + \delta(\pmb{x}_{i*}, \pmb{x}_{j*}) \right) \right) \odot \left( \alpha(\pmb{f}_{j*}) + \delta(\pmb{x}_{i*}, \pmb{x}_{j*}) \right) } ,\]

where the positional encoding is

\[\delta(\pmb{x}_{i*}, \pmb{x}_{j*}) = \tilde{\sigma}_{\theta} \left( \sigma_{\theta}\left( \left(\pmb{x}_{j*} - \pmb{x}_{i*}\right) \pmb{\Theta} \oplus \pmb{\theta} \right) \pmb{\widetilde{\Theta}} \oplus \pmb{\tilde{\theta}} \right) ,\]

\(\odot\) is the Hadamard product, \(\oplus\) is the vector broadcast sum,

\[\phi(\pmb{f}_{i*}) = \sigma_{\phi}\left( \pmb{f}_{i*} \pmb{\Phi} \oplus \pmb{\phi} \right) ,\]

\[\psi(\pmb{f}_{j*}) = \sigma_{\psi}\left( \pmb{f}_{i*} \pmb{\Psi} \oplus \pmb{\psi} \right) ,\]

\[\alpha(\pmb{f}_{j*}) = \sigma_{\alpha}\left( \pmb{f}_{j*} \pmb{A} \oplus \pmb{a} \right) ,\]

\[\gamma(\pmb{x}) = \sigma_{\gamma} \left( \pmb{x} \pmb{\Gamma} \oplus \pmb{\gamma} \right) ,\]

with \(\sigma_{\phi}, \sigma_{\psi}, \sigma_{\alpha}, \sigma_{\gamma}, \sigma_{\theta}, \tilde{\sigma}_{\theta}\) typically a component-wise ReLU activation, and

\[\sigma(\pmb{x})_p = \dfrac{ e^{x_p} }{ \sum_{q=1}^{D_{\text{out}}}{e^{x_q}} }\]

a softmax activation.

The parameter space of the layer is represented through

\[\pmb{\Phi}, \pmb{\Psi}, \pmb{A} \in \mathbb{R}^{D_{\text{in}} \times D_{\text{out}}} \,,\; \pmb{\Gamma} \in \mathbb{R}^{D_{\text{out}} \times D_{\text{out}}}\]

weight matrices,

\[\pmb{\Theta} \in \mathbb{R}^{n_x \times D_{\text{out}}} \,,\; \pmb{\widetilde{\Theta}} \in \mathbb{R}^{D_{\text{out}} \times D_{\text{out}}}\]

positional weight matrices (with \(n_x\) being the dimensionality of the structure space, e.g., \(n_x=3\) for a 3D point cloud), and

\[\pmb{\phi}, \pmb{\psi}, \pmb{a}, \pmb{\gamma}, \pmb{\theta}, \pmb{\tilde{\theta}} \in \mathbb{R}^{D_{\text{out}}}\]

weight and positional weight vectors.

Note that the structure space of the point cloud is represented through \(\pmb{X} \in \mathbb{R}^{R \times n_x}\), whose rows \(\pmb{x}_{i*} \in \mathbb{R}^{n_x}\) represent point-wise coordinates in an \(n_x\)-dimensional Euclidean space. Besides, the feature space is represented with \(\pmb{F} \in \mathbb{R}^{R \times n_f}\) (where \(n_f = D_{\text{in}}\)), whose rows \(\pmb{f}_{i*} \in \mathbb{R}^{n_f}\) represent point-wise features.

This layer is inspired in the PointTransformer v1 paper (https://doi.org/10.48550/arXiv.2012.09164).

Variables:

• Dout (int) – The output dimensionality of the point transformer.

• built_Phi (bool) – Whether the weights matrix \(\pmb{\Phi}\) is built. Initially it is false, but it will be updated once the layer is built.

• built_Psi (bool) – Whether the weights matrix \(\pmb{\Psi}\) is built. Initially it is false, but it will be updated once the layer is built.

• built_A (bool.) – Whether the weights matrix \(\pmb{A}\) is built. Initially it is false, but it will be updated once the layer is built.

• built_Gamma (bool) – Whether the weights matrix \(\pmb{\Gamma}\) is built. Initially it is false, but it will be updated once the layer is built.

• built_Theta (bool) – Whether the weights matrix \(\pmb{\Theta}\) is built. Initially it is false, but it will be updated once the layer is built.

• built_ThetaTilde (bool) – Whether the weights matrix \(\pmb{\widetilde{\Theta}}\) is built. Initially it is false, but it will be updated once the layer is built.

• built_phi (bool) – Whether the weights vector \(\pmb{\phi}\) is built. Initially it is false, but it will be updated once the layer is built.

• built_psi (bool) – Whether the weights vector \(\pmb{\psi}\) is built. Initially it is false, but it will be updated once the layer is built.

• built_a (bool.) – Whether the weights vector \(\pmb{a}\) is built. Initially it is false, but it will be updated once the layer is built.

• built_gamma (bool) – Whether the weights vector \(\pmb{\gamma}\) is built. Initially it is false, but it will be updated once the layer is built.

• built_theta (bool) – Whether the weights vector \(\pmb{\theta}\) is built. Initially it is false, but it will be updated once the layer is built.

• built_thetaTilde (bool) – Whether the weights vector \(\pmb{\tilde{\theta}}\) is built. Initially it is false, but it will be updated once the layer is built.

• Phi_initializer – The initializer for the matrix of weights \(\pmb{\Phi}\).

• Phi_regularizer – The regularizer for the matrix of weights \(\pmb{\Phi}\).

• Phi_constraint – The constraint for the matrix of weights \(\pmb{\Phi}\).

• Psi_initializer – The initializer for the matrix of weights \(\pmb{\Psi}\).

• Psi_regularizer – The regularizer for the matrix of weights \(\pmb{\Psi}\).

• Psi_constraint – The constraint for the matrix of weights \(\pmb{\Psi}\).

• A_initializer – The initializer for the matrix of weights \(\pmb{A}\).

• A_regularizer – The regularizer for the matrix of weights \(\pmb{A}\).

• A_constraint – The constraint for the matrix of weights \(\pmb{A}\).

• Gamma_initializer – The initializer for the matrix of weights \(\pmb{\Gamma}\).

• Gamma_regularizer – The regularizer for the matrix of weights \(\pmb{\Gamma}\).

• Gamma_constraint – The constraint for the matrix of weights \(\pmb{\Gamma}\).

• Theta_initializer – The initializer for the matrix of weights \(\pmb{\Theta}\).

• Theta_regularizer – The regularizer for the matrix of weights \(\pmb{\Theta}\).

• Theta_constraint – The constraint for the matrix of weights \(\pmb{\Theta}\).

• ThetaTilde_initializer – The initializer for the matrix of weights \(\pmb{\widetilde{\Theta}}\).

• ThetaTilde_regularizer – The regularizer for the matrix of weights \(\pmb{\widetilde{\Theta}}\).

• ThetaTilde_constraint – The constraint for the matrix of weights \(\pmb{\widetilde{\Theta}}\).

• phi_initializer – The initializer for the vector of weights \(\pmb{\phi}\).

• phi_regularizer – The regularizer for the vector of weights \(\pmb{\phi}\).

• phi_constraint – The constraint for the vector of weights \(\pmb{\phi}\).

• psi_initializer – The initializer for the vector of weights \(\pmb{\psi}\).

• psi_regularizer – The regularizer for the vector of weights \(\pmb{\psi}\).

• psi_constraint – The constraint for the vector of weights \(\pmb{\psi}\).

• a_initializer – The initializer for the vector of weights \(\pmb{a}\).

• a_regularizer – The regularizer for the vector of weights \(\pmb{a}\).

• a_constraint – The constraint for the vector of weights \(\pmb{a}\).

• gamma_initializer – The initializer for the vector of weights \(\pmb{\gamma}\).

• gamma_regularizer – The regularizer for the vector of weights \(\pmb{\gamma}\).

• gamma_constraint – The constraint for the vector of weights \(\pmb{\gamma}\).

• theta_initializer – The initializer for the vector of weights \(\pmb{\theta}\).

• theta_regularizer – The regularizer for the vector of weights \(\pmb{\theta}\).

• theta_constraint – The constraint for the vector of weights \(\pmb{\theta}\).

• thetaTilde_initializer – The initializer for the vector of weights \(\pmb{\widetilde{\theta}}\).

• thetaTilde_regularizer – The regularizer for the vector of weights \(\pmb{\widetilde{\theta}}\).

• thetaTilde_constraint – The constraint for the vector of weights \(\pmb{\widetilde{\theta}}\).

__init__(Dout=320, built_Phi=False, built_Psi=False, built_A=False, built_Gamma=False, built_Theta=False, built_ThetaTilde=False, built_phi=False, built_psi=False, built_a=False, built_gamma=False, built_theta=False, built_thetaTilde=False, Phi_initializer=None, Phi_regularizer=None, Phi_constraint=None, Psi_initializer=None, Psi_regularizer=None, Psi_constraint=None, A_initializer=None, A_regularizer=None, A_constraint=None, Gamma_initializer=None, Gamma_regularizer=None, Gamma_constraint=None, Theta_initializer=None, Theta_regularizer=None, Theta_constraint=None, ThetaTilde_initializer=None, ThetaTilde_regularizer=None, ThetaTilde_constraint=None, phi_initializer=None, phi_regularizer=None, phi_constraint=None, psi_initializer=None, psi_regularizer=None, psi_constraint=None, a_initializer=None, a_regularizer=None, a_constraint=None, gamma_initializer=None, gamma_regularizer=None, gamma_constraint=None, theta_initializer=None, theta_regularizer=None, theta_constraint=None, thetaTilde_initializer=None, thetaTilde_regularizer=None, thetaTilde_constraint=None, sigma_activation=<function softmax>, sigma_phi_activation=<function relu>, sigma_psi_activation=<function relu>, sigma_alpha_activation=<function relu>, sigma_gamma_activation=<function relu>, sigma_theta_activation=<function relu>, sigma_theta_tilde_activation=<function relu>, **kwargs)

See Layer and Layer.__init__().

build(dim_in)

Build the weight vectors and matrices.

See Layer and layer.Layer.build().

call(inputs, training=False, mask=False)

Compute the PointTransformerLayer on an input batch.

Parameters:

inputs –

The input such that:

– inputs[0]

is the structure space tensor representing the geometry of the many receptive fields in the batch.

\[\mathcal{X} \in \mathbb{R}^{K \times R \times n_x}\]

– inputs[1]

is the feature space tensor representing the features of the many receptive fields in the batch.

\[\mathcal{F} \in \mathbb{R}^{K \times R \times n_f}\]

– inputs[2]

is the indexing tensor representing the neighborhoods of \(\kappa\) neighbors for each input point, in the same space.

\[\mathcal{N} \in \mathbb{Z}^{K \times R \times \kappa}\]

Returns:

The output feature space \(\mathcal{\widehat{F}} \in \mathbb{R}^{K \times R \times D_{\mathrm{out}}}\).

get_config()

Return necessary data to serialize the layer.

classmethod from_config(config)

Use given config data to deserialize the layer.

model.deeplearn.layer.rbf_feat_extract_layer module

class model.deeplearn.layer.rbf_feat_extract_layer.RBFFeatExtractLayer(*args, **kwargs)

Bases: Layer

Author:

Alberto M. Esmoris Pena

A RBF feature extraction layer is governed by a matrix \(\pmb{Q}\) representing the kernel’s structure space in a \(n_x\)-dimensional Euclidean space such that:

\[\pmb{Q} \in \mathbb{R}^{K \times n_x}\]

For 3D point clouds, \(n_x=3\) and the matrix \(\pmb{Q}\) represents the coordinates of the kernel points in a 3D space centered at zero.

On top of that, a RBF feature extraction layer is also governed by a \(\pmb{\omega} \in \mathbb{R}^K\) vector that parametrizes the curvature of each radial basis function.

The output of a RBF feature extraction layer when considering a Gaussian kernel is given by the expression below:

\[\pmb{Y} \in \mathbb{R}^{m \times K} \;\text{s.t.}\; y_{ij} = \exp\biggl({ - \dfrac{ \lVert{\pmb{x_{i*}}-\pmb{q_{j*}}}\rVert^2 }{ \omega_{j}^2 }}\biggr)\]

The \(\pmb{Y}\) matrix can be understood as a feature matrix extracted from the \(m\) input points through its interaction with the kernel.

Variables:

• structure_initialization_type (str) – The type of initialization strategy for the kernel’s structure space.

• max_radii (np.ndarray of float) – The radius of the last ellipsoid along each axis \(\pmb{r}^* \in \mathbb{R}^{n_x}\).

• radii_resolution (int) – How many concentric ellipsoids must be considered \(n \in \mathbb{Z}_{>0}\) (the first one is the center point, the last one is the biggest outer ellipsoid).

• angular_resolutions (np.ndarray of int) – How many angles consider for each ellipsoid \((m_1, \ldots, m_n)\).

• kernel_function_type (str) – The type of kernel function to be used (e.g., “Gaussian” or “Markov”).

• kernel_function (function) – The kernel function to be used.

• num_kernel_points (int) – The number of points representing the kernel \(K \in \mathbb{Z}_{>0}\).

• Q (tf.Tensor) – The kernel’s structure space matrix

• trainable_Q (bool) – Flag to control whether Q is trainable or not.

• built_Q (bool) – Flag to control whether Q has been built or not.

• omega (tf.Tensor) – The vector of kernel’s sizes (can be thought as a curvature).

• trainable_omega (bool) – Flag to control whether omega is trainable or not.

• built_omega (bool) – Flag to control whether omega has been built or not.

__init__(max_radii, radii_resolution=4, angular_resolutions=(1, 2, 4, 8), kernel_function_type='Gaussian', structure_initialization_type='concentric_ellipsoids', structure_dimensionality=3, trainable_Q=True, trainable_omega=True, built_Q=False, built_omega=False, **kwargs)

See Layer and layer.Layer.__init__().

build(dim_in)

Build the \(\pmb{Q} \in \mathbb{K \times n_x}\) matrix representing the kernel’s structure, and the \(\pmb{\omega}\) vector representing the kernel’s sizes (think about curvature).

The \(\pmb{Q}\) matrix represents the disposition of the kernel’s points while the \(\pmb{\omega}\) vector represents the size of each radial basis function. The size of a radial basis function for typical Gaussian kernels can be seen as the parameter governing the curvature of the exponential.

See Layer and layer.Layer.build().

call(inputs, training=False, mask=False)

The computation of the \(\pmb{Y} \in \mathbb{R}^{m \times K}\) matrix of output features. For example, for a Gaussian kernel this matrix would be:

\[y_{ij} = \exp\left(-\dfrac{ \lVert{\pmb{x_{i*}} - \pmb{q_{j*}}}\rVert^2 }{ \omega_j^2 }\right)\]

Where \(\pmb{x_{i*}}\) is the i-th input point and \(\pmb{q_{j*}}\) is the j-th kernel point.

Returns:

The extracted output features.

Return type:

tf.Tensor

compute_gaussian_kernel(D_squared)

Compute a Gaussian kernel function.

\[y_{ij} = \exp\left( - \dfrac{ \lVert{\pmb{x_{i*}}} - \pmb{q_{j*}}\rVert^2 }{ \omega_{j}^2 } \right)\]

Returns:

The computed Gaussian kernel function.

Return type:

tf.Tensor

compute_markov_kernel(D_squared)

Compute a Markov kernel function.

\[y_{ij} = \exp\left( - \dfrac{ \lVert{\pmb{x_{i*}}} - \pmb{q_{j*}}\rVert }{ \omega_{j}^2 } \right)\]

Returns:

The computed Markov kernel function.

Return type:

tf.Tensor

get_config()

Return necessary data to serialize the layer

classmethod from_config(config)

Use given config data to deserialize the layer

export_representation(dir_path, out_prefix=None, Qpast=None)

Export a set of files representing the state of the kernel’s structure.

Parameters:

• dir_path (str) – The directory where the representation files will be exported.

• out_prefix (str) – The output prefix to name the output files.

• Qpast (np.ndarray or tf.Tensor or None) – The structure matrix of the layer in the past.

Returns:

Nothing at all, but the representation is exported as a set of files inside the given directory.

model.deeplearn.layer.rbf_feat_processing_layer module

class model.deeplearn.layer.rbf_feat_processing_layer.RBFFeatProcessingLayer(*args, **kwargs)

Bases: Layer

Author:

Alberto M. Esmoris Pena

A RBF feature processing layer is governed by a matrix \(\pmb{M} \in \mathbb{R}^{K \times n_f}\) representing the \(K\) kernels for each of the \(n_f\) features, and a matrix \(\pmb{\Omega} \in \mathbb{R}^{K \times n_f}\).

Each column \(\pmb{\mu}_{*k}\) of the matrix \(\pmb{M}\) defines \(K\) kernels for a given input feature, together with each column \(\pmb{\omega}_{*k}\) of the matrix \(\pmb{\Omega}\).

The output of a RBFFeatProcessingLayer consists of a matrix \(\pmb{Y}\in\mathbb{R}^{m \times K n_f}\) with \(K \times n_f\) output features for each of the \(m\) input points.

Let \(\mathcal{Y} \in \mathbb{R}^{K \times m \times n_f}\) be a tensor that can be sliced into \(K\) matrices of \(m\) rows and \(n_f\) columns representing the point-wise output features derived from a given kernel. For then, any cell of this tensor can be defined as follows (assuming a Gaussian RBF):

\[\mathcal{y}_{kij} = \exp\left[-\dfrac{ (f_{ij} - \mu_{kj})^2 }{ \omega_{kj}^2 }\right]\]

Now, the output matrix \(\pmb{Y}\) can be simply defined by reorganizing the tensor \(\mathcal{Y}\) as a matrix such that:

\[\begin{split}\pmb{Y} = \left[\begin{array}{ccccc} | & & | & & | \\ \pmb{y}_{1*1} & \cdots & \pmb{y}_{K*1} & \cdots & \pmb{y}_{K*n_f} \\ | & & | & & | \end{array}\right]\end{split}\]

Technically, it is also convenient to express the output matrix like a component-wise exponential \(\pmb{Y} = \exp\left[- \pmb{D} \odot \pmb{D}\right]\), where \(\odot\) is the Hadamard product and \(\pmb{D} \in \mathbb{R}^{m \times K n_f}\) is a matrix of scaled differences as defined below:

\[\begin{split}\pmb{D} = \left[\begin{array}{ccccc} | & & | & & | \\ \dfrac{\pmb{f}_{*1}-\mu_{11}}{\omega_{11}} & \cdots & \dfrac{\pmb{f}_{*1}-\mu_{K1}}{\omega_{K1}} & \cdots & \dfrac{\pmb{f}_{*n_f}-\mu_{Kn_f}}{\omega_{Kn_f}} \\ | & & | & & | \end{array}\right]\end{split}\]

For initialization, the mean for each \(j\)-th feature \(\mu_j\) is assumed, together with its standard deviation \(\sigma_j\). With this information, it is possible to initialize the columns of the matrix \(\pmb{M}\) by taking \(K\) linearly-spaced samples from the interval \([\mu_j - 3\sigma_j, \mu_j + 3\sigma_j]\). Besides, the rows of the matrix \(\pmb{\Omega}\) can be initialized considering samples from a uniform distribution \(x \sim U(-b, b)\) such that \(\forall 1 < k \leq K,\, \omega_{kj} = a + \sigma_j(b+x)\). Typically, \(a=10^{-2}\) and \(b=1\).

Variables:

• means (list or tuple or np.ndarray) – The mean value for each feature to be processed.

• stdevs (list or tuple or np.ndarray) – The standard deviation for each feature to be processed.

• num_feats (int) – The number of features.

• num_kernels (int) – The number of kernels per feature.

• a (float) – The offset or intercept for the kernel’s sizes.

• b (float) – The parameter governing the uniform distribution.

• trainable_M (bool) – Whether the matrix of centers is trainable (True) or not (False).

• trainable_Omega (bool) – Whether the matrix of kernel’s sizes is trainable (True) or not (False).

• kernel_function_type (str) – The type of kernel function. Supported functions are “Gaussian” and “Markov”.

• M (tf.Tensor) – The matrix of kernel’s centers.

• built_M (bool) – Whether the matrix of kernel’s centers has been built (True) or not (False).

• Omega (tf.Tensor) – The matrix of kernel’s sizes.

• built_Omega – Whether the matrix of kernel’s sizes has been built (True) or not (False).

Vartpye built_Omega:

bool

__init__(num_kernels, means, stdevs, a=0.01, b=1, kernel_function_type='Gaussian', trainable_M=True, trainable_Omega=True, built_M=False, built_Omega=False, **kwargs)

See Layer layer.Layer.__init__().

build(dim_in)

Build the \(\pmb{M} \in \mathbb{R}^{K \times n_f}\) and \(\pmb{\Omega} \in \mathbb{R}^{K \times n_f}\) matrices representing the feature processing kernel’s centers and sizes (curvatures), respectively.

See Layer and layer.Layer.build().

call(inputs, training=False, mask=False)

The computation of the \(\pmb{Y} \in \mathbb{R}^{m \times Kn_f}\) output matrix.

Returns:

The processed output features.

Return type:

tf.Tensor

compute_gaussian_kernel(F)

Compute a Gaussian kernel function.

\[y_{ip} = \exp\left[ - \dfrac{(f_{ij} - \mu_{kj})^2}{\omega_{kj}^2} \right]\]

Parameters:

F – The feature space matrix.

Returns:

The computed Gaussian kernel function.

compute_markov_kernel(F)

Compute a Markov kernel function.

\[y_{ip} = \exp\left[ - \dfrac{\lvert{f_{ij} - \mu_{kj}}\rvert}{\omega_{kj}^2} \right]\]

Parameters:

F – The feature space matrix.

Returns:

The computed Markov kernel function.

get_config()

Return necessary data to serialize the layer

classmethod from_config(config)

Use given config data to deserialize the layer

export_representation(dir_path, out_prefix=None)

Export a set of files representing the state of the kernel.

Parameters:

• dir_path (str) – The directory where the representation files will be exported.

• out_prefix (str) – The output prefix to name the output files.

Returns:

Nothing at all, but the representation is exported as a set of files inside the given directory.

model.deeplearn.layer.shadow_activation_layer module

class model.deeplearn.layer.shadow_activation_layer.ShadowActivationLayer(*args, **kwargs)

Bases: Layer

Shadow version of an activation layer, i.e., it works with tensors with padding, where the padding is used to represent shadow values/points/cells/elements, i.e., those that must not have an impact on the computations.

See tf.Tensor and keras.layers.Activation.

__init__(activation, act=None, offset=0, **kwargs)

See Layer and layer.Layer.__init__().

build(dim_in)

Build the underlying keras.layers.Activation layer.

See Layer, layer.Layer.build(), and keras.layers.Activation.

call(inputs)

Call the shadow Activation layer.

See Layer and layer.Layer.call().

get_config()

Return necessary data to serialize the layer

classmethod from_config(config)

Use given config data to deserialize the layer

model.deeplearn.layer.shadow_batch_normalization_layer module

class model.deeplearn.layer.shadow_batch_normalization_layer.ShadowBatchNormalizationLayer(*args, **kwargs)

Bases: Layer

Shadow version of a batch normalization layer, i.e., it works with tensors with padding, where the padding is used to represent shadow values/points/cells/elements, i.e., those that must not have an impact on the computations.

See tf.Tensor and keras.layers.BatchNormalization.

A classical batch normalization uses the following four variables \(\mu_b\) (mean of batch), \(\sigma^2_b\) (variance of batch), \(\mu_m\) (mean for moving average), and \(\sigma^2_m\) (variance for moving average) such that whether \(x=\mu\) or \(x=\sigma^2\) they can be updated as follows:

\[x'_m = (x_m - x_b) M + x_b\]

Where \(M \in \mathbb{R}\) is the momentum for the moving average and \(x'_m\) is the updated value for the variable governing the moving average.

Note that contrary to classical batch normalization where during training the \(\mu_b, \sigma^2_b\) variables are used while for predictions the \(\mu_m, \sigma^2_m\) variables are used, the shadow batch normalization layer always considers the batch, i.e., it uses \(\mu_b\) and \(\sigma^2_b\) during training and for predictions. Thus, the final normalization is always computed like:

\[\hat{b} = \dfrac{ \gamma (b - \mu_b) }{ \sqrt{\sigma^2_b + \epsilon} } + \beta\]

Where, \(\beta\), \(\gamma\), and \(\epsilon\) are the beta, gamma, and epsilon parameters of the layer. Note that \(b\) represents the values from the batch and \(\hat{b}\) its normalized version.

__init__(bn=None, axis=-1, momentum=0.99, epsilon=0.001, center=True, scale=True, beta_initializer='zeros', gamma_initializer='ones', moving_mean_initializer='zeros', moving_variance_initializer='ones', beta_regularizer=None, gamma_regularizer=None, beta_constraint=None, gamma_constraint=None, synchronized=False, offset=0, **kwargs)

See Layer and layer.Layer.__init__().

build(dim_in)

Build the underlying keras.layers.BatchNormalization layer.

See Layer, layer.Layer.build(), and keras.layers.BatchNormalization.

call(inputs, training=False, mask=False)

Call the shadow BatchNormalization layer.

See Layer and layer.Layer.call().

get_config()

Return necessary data to serialize the layer

classmethod from_config(config)

Use given config data to deserialize the layer

model.deeplearn.layer.shadow_conv1d_layer module

class model.deeplearn.layer.shadow_conv1d_layer.ShadowConv1DLayer(*args, **kwargs)

Bases: Layer

Shadow version of a 1D convolutional layer, i.e., it works with tensors with padding, where the padding is used to represent shadow values/points/cells/elements, i.e., those that must not have an impact on the computations.

See tf.Tensor and keras.layers.Conv1D.

__init__(filters, kernel_size, conv1D=None, strides=1, padding='valid', data_format='channels_last', dilation_rate=1, groups=1, activation=None, use_bias=True, kernel_initializer='glorot_uniform', bias_initializer='zeros', kernel_regularizer=None, bias_regularizer=None, activity_regularizer=None, kernel_constraint=None, bias_constraint=None, offset=0, **kwargs)

See Layer and layer.Layer.__init__().

build(dim_in)

Build the underlying keras.layers.Conv1D layer.

See Layer, layer.Layer.build(), and keras.layers.Conv1D.

call(inputs)

Call the shadow Conv1D layer.

See Layer and layer.Layer.call().

get_config()

Return necessary data to serialize the layer

classmethod from_config(config)

Use given config data to deserialize the layer

model.deeplearn.layer.shadow_dense_layer module

class model.deeplearn.layer.shadow_dense_layer.ShadowDenseLayer(*args, **kwargs)

Bases: Layer

Shadow version of a dense layer, i.e., it works with tensors with padding, where the padding is used to represent shadow values/points/cells/elements, i.e., those that must not have an impact on the computations.

See tf.Tensor and keras.layers.Dense.

__init__(units, activation=None, use_bias=True, kernel_initializer='glorot_uniform', bias_initializer='zeros', kernel_regularizer=None, bias_regularizer=None, activity_regularizer=None, kernel_constraint=None, bias_constraint=None, offset=0, built_kernel=False, built_bias=False, **kwargs)

See Layer and layer.Layer.__init__().

build(dim_in)

Build the kernel and the bias of the layer.

See Layer and layer.Layer.build().

call(inputs)

Call the shadow dense layer.

See Layer and layer.Layer.call().

get_config()

Return necessary data to serialize the layer

classmethod from_config(config)

Use given config data to deserialize the layer

model.deeplearn.layer.sparse_indexing_map_layer module

class model.deeplearn.layer.sparse_indexing_map_layer.SparseIndexingMapLayer(*args, **kwargs)

Bases: Layer

Author:

Alberto M. Esmoris Pena

A sparse indexing map layer handles the building of a tf.MutableHashTable to govern the indexing of sparse grids. More concretely, it enables the computation of \(h(k) = v\), a map that connects the potentially sparse indices of the active cells on a sparse grid (keys, k) to their corresponding sequential indexing (values, v).

The rationale behind the map \(h\) is that neighborhoods on sparse grids must only consider active cells (other cells don’t have useful information). To achieve this, the cell-wise features are codified as a non-sparse matrix of features \(\pmb{F} \in \mathbb{R}^{R \times n_f}\) that represents the \(n_f\) features of the \(R\) active cells. Then, the map \(h\) is used to link the sparse index \(k\) to its corresponding row in \(\pmb{F}\), i.e., \(v\).

Note that the explicitly given sequential indices \(v\) are expected to start at one because zero is used to represent the ground vector, i.e., the vector of features that represents any non-active cell (typically the zero vector).

Variables:

• h (tf.lookup.experimental.MutableHashTable.) – The tf.lookup.experimental.MutableHashTable used to represent the \(h(k) = v\) map.

• built_h (bool) – Whether he mutable hash table has been built or not. Initially it is false, but it will be updated once the layer is built.

__init__(built_h=False, **kwargs)

See Layer and layer.Layer.__init__().

build(dim_in)

Build the mutable hash table object that will be used to represent the \(h(k) = v\) sparse indexing map.

See Layer and layer.Layer.build()

call(inputs)

Remove old values from the mutable hash table and insert the new ones.

Parameters:

inputs –

The keys and values of the sparse indexing map \(h\) as two tensors with padding constituting a vectors of keys (sparse indices) and a vector of values (sequential indices), and the feature space as another tensor.

Note that inputs[0] gives the tensor with keys, inputs[1] the tensor with values, and inputs[2] gives the feature space tensor.

Note also that the input feature space is given but not modified. It is only requested for convenience to define the computational graph of the neural network in such a way that the sparse indexing map layer is for sure called before a certain point, e.g., before computing the first submanifold sparse convolution (SubmanifoldSpConv3DLayer) in a sparse convolutional neural network (SpConv3DPwiseClassif).

Returns:

The given input feature space without modification.

Return type:

tf.Tensor

lookup(k)

Look-up for the corresponding sequential indices (v) for given input sparse indices (k).

Parameters:

k – The input sparse indices as keys (k).

Returns:

The corresponding sequential indices as values (v).

get_config()

Return necessary data to serialize the layer

classmethod from_config(config)

Use given config data to deserialize the layer

model.deeplearn.layer.spconv3d_decoding_layer module

class model.deeplearn.layer.spconv3d_decoding_layer.SpConv3DDecodingLayer(*args, **kwargs)

Bases: Layer

Author:

Alberto M. Esmoris Pena

The sparse convolutional 3D decoding layer is a combination of other layers. For the details about the involved operations see SubmanifoldSpConv3DLayer, UpsamplingSpConv3DLayer, ShadowBatchNormalizationLayer, ShadowActivationLayer, ShadowConv1DLayer, and HierarchicalSHPreProcessorPP. For a formal overview of this layer in terms of compositions see the equation below:

\[\begin{split}\left( \operatorname{SkipLink} \circ \operatorname{UpConv} \circ \operatorname{postwrap} \circ \left[ \ldots \circ \operatorname{SpConv}_k \circ \ldots \right]_{k=1}^{n_\text{spconv}} \circ \operatorname{prewrap} \right)( \pmb{F}, \pmb{h_k}, \pmb{h^D}, \pmb{n} ) = \\ \left( \operatorname{SharedMLP} \circ \operatorname{ReLU} \circ \operatorname{Norm} \circ \operatorname{UpConv} \circ \operatorname{ReLU} \circ \operatorname{Norm} \circ \operatorname{SharedMLP} \circ \left[ \ldots \circ \operatorname{SpConv}_k \circ \ldots \right]_{k=1}^{n_\text{spconv}} \circ \operatorname{ReLU} \circ \operatorname{Norm} \circ \operatorname{SharedMLP} \right)( \pmb{F}, \pmb{h_k}, \pmb{h^D}, \pmb{n} )\end{split}\]

Where \(n_{\text{spconv}}\) is the number of requested 3D sparse submanifold convolutions (SubmanifoldSpConv3DLayer), \(\pmb{F}\) are the input features, \(\pmb{h_k}\) are the keys of the submanifold sparse-to-sequential indexing map, \(\pmb{h^U}\) is the vector of sparse indices involved in the upsampling convolution, and \(\pmb{n} \in \mathbb{Z}^{3}_{>0}\) is the vector of axis-wise partitions.

The \(\operatorname{SharedMLP} \in \mathbb{R}^{m \times b}\) is computed as described in the documentation of SpConv3DEncodingLayer. The residual block works also as documented in SpConv3DEncodingLayer.

Variables:

• w – The submanifold convolutional window (as a half size).

• uw – The upsampling convolutional window (as a full size).

• f – The number of submanifold convolution filters or channels.

• uf – The number of upsampling convolution filters or channels.

• nfa – The number of features per point of the feature space after the pre-wrap transform.

• nfb – The number of features per point of the feature space after the sparse submanifold convolutions.

• nfc – The number of features per point of the feature space after the post-wrap.

• nfd – The number of features per point of the feature space after the sparse upsampling convolutions.

• num_spconvs – How many sparse submanifold convolutions in the sparse submanifold convolutional block, i.e., \(n_{\text{spconvs}}\).

• W_initializers – The kernel initializer for each sparse submanifold convolution.

• W_regularizers – The kernel regularizer for each sparse submanifold convolution.

• W_constraints – The kernel constraints for each sparse submanifold convolution.

• uW_initializer – The kernel initializer for the sparse upsampling convolution.

• uW_regularizer – The kernel regularizer for the sparse upsampling convolution.

• uW_constraint – The kernel constraint for the sparse upsampling convolution.

• siml – The sparse indexing map layer (SparseIndexingMapLayer) for the sparse-to-sequential index lookups.

• spconv_bn – The batch normalization layer for each sparse submanifold convolution.

• spconv_act – The activation layer for each sparse submanifold convolution.

• preact_bn – The batch normalization layer for the pre-activation.

• preact_act – The activation layer for the pre-activation.

• prewrap_bn – The batch normalization layer for the pre-wrapper.

• postwrap_bn – The batch normalization layer for the post-wrapper.

• prewrap_act – The activation layer for the pre-wrapper.

• prewrap_activation – The activation function for the pre-wrapper.

• prewrap_kernel_initializer – The kernel initializer for the pre-wrapper.

• prewrap_kernel_regularizer – The kernel regularizer for the pre-wrapper.

• prewrap_kernel_constraint – The kernel constraint for the pre-wrapper.

• postwrap_act – The activation layer for the pre-wrapper.

• postwrap_activation – The activation function for the pre-wrapper.

• postwrap_kernel_initializer – The kernel initializer for the pre-wrapper.

• postwrap_kernel_regularizer – The kernel regularizer for the pre-wrapper.

• postwrap_kernel_constraint – The kernel constraint for the pre-wrapper.

• residual_strategy – The residual strategy, if any. It can be either "ssc3d" to use a sparse submanifold convolution in the residual block, "conv1d" or "sharedmlp" to use a Shared MLP, or "null" or None to don’t use a residual block.

• res_activation – The activation function for the residual block.

• res_kernel_initializer – The kernel initializer for the residual block.

• res_kernel_regularizer – The kernel regularizer for the residual block.

• res_kernel_constraint – The kernel constraint for the residual block.

• res_spconv_bn – The batch normalization layer for the sparse convolutional residual block.

• res_spconv_act – The activation layer for the sparse convolutional residual block.

• postres_act – The activation layer for the post-residual.

• postres_bn – The batch normalization layer for the post-residual.

• postres_kernel_activation – The activation function for the post-residual SharedMLP.

• postres_kernel_regularizer – The regularizer layer for the post-residual SharedMLP.

• postres_kernel_initializer – The initializer layer for the post-residual SharedMLP.

• postres_kernel_constraint – The constraint layer for the post-residual SharedMLP.

• up_act – The activation layer just before the upsampling.

• up_bn – The batch normalization layer just before the upsampling.

• sl_bn – The batch normalization layer for the skip link processing.

• sl_act – The activation layer for the skip link processing.

• sl_kernel_activation – The activation function for the kernel of the SharedMLP in the skip link processing block.

• sl_kernel_initializer – The initializer for the kernel of the SharedMLP in the skip link processing block.

• sl_kernel_regularizer – The regularizer for the kernel of the SharedMLP in the skip link processing block.

• sl_kernel_constraint – The constraint for the kernel of the SharedMLP in the skip link processing block.

__init__(w, uw, f, uf, nfa, nfb, nfc, nfd, num_spconvs=1, W_initializers=[None], W_regularizers=[None], W_constraints=[None], uW_initializer=None, uW_regularizer=None, uW_constraint=None, siml=None, spconv_bn=[None], spconv_bn_momentum=0.9, spconv_act=[None], spconv_activation=<function relu>, preact_bn=None, preact_act=None, prewrap_bn=None, postwrap_bn=None, wrap_bn_momentum=0.99, prewrap_act=None, postwrap_act=None, prewrap_activation=None, prewrap_kernel_initializer='glorot_uniform', prewrap_kernel_regularizer=None, prewrap_kernel_constraint=None, postwrap_activation=None, postwrap_kernel_initializer='glorot_uniform', postwrap_kernel_regularizer=None, postwrap_kernel_constraint=None, residual_strategy=None, res_activation=None, res_kernel_initializer='glorot_uniform', res_kernel_regularizer=None, res_kernel_constraint=None, res_spconv_activation=<function relu>, res_spconv_bn_momentum=0.9, res_spconv_bn=None, res_spconv_act=None, post_residual_shared_mlp=False, postres_activation=<function relu>, postres_bn_momentum=0.99, postres_act=None, postres_bn=None, postres_kernel_activation=None, postres_kernel_initializer='glorot_uniform', postres_kernel_regularizer=None, postres_kernel_constraint=None, up_activation=<function relu>, up_act=None, up_bn_momentum=0.9, up_bn=None, sl_bn=None, sl_bn_momentum=0.99, sl_act=None, sl_kernel_activation=<function relu>, sl_kernel_initializer='glorot_uniform', sl_kernel_regularizer=None, sl_kernel_constraint=None, submanifold_cache=None, upsampling_cache=None, built_W=False, built_uW=False, built_p=False, built_pon1=False, built_ponw=False, built_ponw2=False, built_up=False, built_upon1=False, built_uponw=False, built_uponw2=False, built_prewrap_kernel=False, built_prewrap_bias=False, built_postwrap_kernel=False, built_postwrap_bias=False, built_res_kernel=False, built_res_bias=False, built_postres_kernel=False, built_postres_bias=False, built_sl_kernel=False, built_sl_bias=False, **kwargs)

See Layer and layer.Layer.__init__().

build(dim_in)

Build the many kernels, biases, and caches for the 3D sparse convolutional decoder.

Parameters:

dim_in – The dimensionality for each input tensor.

Returns:

Nothing at all

call(inputs)

Apply the composition of SharedMLP, sparse convolutions, residual operations, and upsampling to the input tensors.

Parameters:

inputs – The input tensors.

Returns:

The final tensor after all the computations.

get_config()

Return necessary data to serialize the layer

classmethod from_config(config)

Use given config data to deserialize the layer

model.deeplearn.layer.spconv3d_encoding_layer module

class model.deeplearn.layer.spconv3d_encoding_layer.SpConv3DEncodingLayer(*args, **kwargs)

Bases: Layer

Author:

Alberto M. Esmoris Pena

The sparse convolutional 3D encoding layer is a combination of other layers. For the details about the involved operations see SubmanifoldSpConv3DLayer, DownsamplingSpConv3DLayer, ShadowBatchNormalizationLayer, ShadowActivationLayer, ShadowConv1DLayer, and HierarchicalSHPreProcessorPP. For a formal overview of this layer in terms of composition see the equation below:

\[\begin{split}\left( \operatorname{DownConv} \circ \operatorname{postwrap} \circ \left[ \ldots \circ \operatorname{SpConv}_k \circ \ldots \right]_{k=1}^{n_\text{spconv}} \circ \operatorname{prewrap} \right)( \pmb{F}, \pmb{h_k}, \pmb{h^D}, \pmb{n} ) = \\ \left( \operatorname{DownConv} \circ \operatorname{ReLU} \circ \operatorname{Normalize} \circ \operatorname{SharedMLP} \circ \left[ \ldots \circ \operatorname{SpConv}_k \circ \ldots \right]_{k=1}^{n_\text{spconv}} \circ \operatorname{ReLU} \circ \operatorname{Normalize} \circ \operatorname{SharedMLP} \right)( \pmb{F}, \pmb{h_k}, \pmb{h^D}, \pmb{n} )\end{split}\]

Where \(n_{\text{spconv}}\) is the number of requested 3D sparse submanifold convolutions (SubmanifoldSpConv3DLayer), \(\pmb{F}\) are the input features, \(\pmb{h_k}\) are the keys of the submanifold sparse-to-sequential indexing map, \(\pmb{h^D}\) is the vector of sparse indices involved in the downsampling convolution, and \(\pmb{n} \in \mathbb{Z}^{3}_{>0}\) is the vector of axis-wise partitions.

The \(\operatorname{SharedMLP} \in \mathbb{R}^{m \times b}\) is computed as described below:

\[\operatorname{SharedMLP}(\pmb{F} \in \mathbb{R}^{m \times a}) = \pmb{F} \left(\pmb{A} \in \mathbb{R}^{a \times b}\right) \oplus \left(\pmb{b} \in \mathbb{R}^{b}\right)\]

Where \(\oplus\) is the broadcasting addition of a vector over a matrix, i.e., summing the vector to each row of the matrix, \(\pmb{A}\) is the kernel of the SharedMLP and \(\pmb{b}\) is the bias.

The residual block implies computing a Shared MLP or a sparse 3D submanifold convolution on the image of the pre-wrapper and then concatenating it to the output of the block of sparse convolutions, right before the post-wrapper. Alternatively, one extra Shared MLP (the post-residual Shared MLP) can be added as the last residual operation. Note that when a residual block is used, the number of features per point of the feature spaces of the residual and main flows is halved so its concatenation matches the original dimensionality expected by the post-wrap block.

Variables:

• w – The submanifold convolutional window (as a half size).

• dw – The downsampling convolutional window (as a full size).

• f – The number of submanifold convolution filters or channels.

• df – The number of downsampling convolution filters or channels.

• nfa – The number of features per point of the feature space after the pre-wrap transform.

• nfb – The number of features per point of the feature space after the sparse submanifold convolutions.

• nfc – The number of features per point of the feature space after the post-wrap.

• nfd – The number of features per point of the feature space after the sparse downsampling convolutions.

• num_spconvs – How many sparse submanifold convolutions in the sparse submanifold convolutional block, i.e., \(n_{\text{spconvs}}\).

• W_initializers – The kernel initializer for each sparse submanifold convolution.

• W_regularizers – The kernel regularizer for each sparse submanifold convolution.

• W_constraints – The kernel constraints for each sparse submanifold convolution.

• dW_initializer – The kernel initializer for the sparse downsampling convolution.

• dW_regularizer – The kernel regularizer for the sparse downsampling convolution.

• dW_constraint – The kernel constraint for the sparse downsampling convolution.

• siml – The sparse indexing map layer (SparseIndexingMapLayer) for the sparse-to-sequential index lookups.

• spconv_bn – The batch normalization layer for each sparse submanifold convolution.

• spconv_act – The activation layer for each sparse submanifold convolution.

• preact_bn – The batch normalization layer for the pre-activation.

• preact_act – The activation layer for the pre-activation.

• prewrap_bn – The batch normalization layer for the pre-wrapper.

• postwrap_bn – The batch normalization layer for the post-wrapper.

• prewrap_act – The activation layer for the pre-wrapper.

• prewrap_activation – The activation function for the pre-wrapper.

• prewrap_kernel_initializer – The kernel initializer for the pre-wrapper.

• prewrap_kernel_regularizer – The kernel regularizer for the pre-wrapper.

• prewrap_kernel_constraint – The kernel constraint for the pre-wrapper.

• postwrap_act – The activation layer for the pre-wrapper.

• postwrap_activation – The activation function for the pre-wrapper.

• postwrap_kernel_initializer – The kernel initializer for the pre-wrapper.

• postwrap_kernel_regularizer – The kernel regularizer for the pre-wrapper.

• postwrap_kernel_constraint – The kernel constraint for the pre-wrapper.

• residual_strategy – The residual strategy, if any. It can be either "ssc3d" to use a sparse submanifold convolution in the residual block, "conv1d" or "sharedmlp" to use a Shared MLP, or "null" or None to don’t use a residual block.

• res_activation – The activation function for the residual block.

• res_kernel_initializer – The kernel initializer for the residual block.

• res_kernel_regularizer – The kernel regularizer for the residual block.

• res_kernel_constraint – The kernel constraint for the residual block.

• res_spconv_bn – The batch normalization layer for the sparse convolutional residual block.

• res_spconv_act – The activation layer for the sparse convolutional residual block.

• postres_act – The activation layer for the post-residual.

• postres_bn – The batch normalization layer for the post-residual.

• postres_kernel_activation – The activation function for the post-residual SharedMLP.

• postres_kernel_regularizer – The regularizer layer for the post-residual SharedMLP.

• postres_kernel_initializer – The initializer layer for the post-residual SharedMLP.

• postres_kernel_constraint – The constraint layer for the post-residual SharedMLP.

• down_act – The activation layer just before the downsampling.

• down_bn – The batch normalization layer just before the downsampling.

__init__(w, dw, f, df, nfa, nfb, nfc, nfd, num_spconvs=1, W_initializers=[None], W_regularizers=[None], W_constraints=[None], dW_initializer=None, dW_regularizer=None, dW_constraint=None, siml=None, spconv_bn=[None], spconv_bn_momentum=0.9, spconv_act=[None], spconv_activation=<function relu>, preact_bn=None, preact_act=None, prewrap_bn=None, postwrap_bn=None, wrap_bn_momentum=0.99, prewrap_act=None, postwrap_act=None, prewrap_activation=None, prewrap_kernel_initializer='glorot_uniform', prewrap_kernel_regularizer=None, prewrap_kernel_constraint=None, postwrap_activation=None, postwrap_kernel_initializer='glorot_uniform', postwrap_kernel_regularizer=None, postwrap_kernel_constraint=None, residual_strategy=None, res_activation=None, res_kernel_initializer='glorot_uniform', res_kernel_regularizer=None, res_kernel_constraint=None, res_spconv_activation=<function relu>, res_spconv_bn_momentum=0.9, res_spconv_bn=None, res_spconv_act=None, post_residual_shared_mlp=False, postres_activation=<function relu>, postres_bn_momentum=0.99, postres_act=None, postres_bn=None, postres_kernel_activation=None, postres_kernel_initializer='glorot_uniform', postres_kernel_regularizer=None, postres_kernel_constraint=None, down_activation=<function relu>, down_act=None, down_bn_momentum=0.9, down_bn=None, submanifold_cache=None, downsampling_cache=None, built_W=False, built_dW=False, built_p=False, built_pon1=False, built_ponw=False, built_ponw2=False, built_dp=False, built_dpon1=False, built_dponw=False, built_dponw2=False, built_prewrap_kernel=False, built_prewrap_bias=False, built_postwrap_kernel=False, built_postwrap_bias=False, built_res_kernel=False, built_res_bias=False, built_postres_kernel=False, built_postres_bias=False, **kwargs)

See Layer and layer.Layer.__init__().

build(dim_in)

Build the many kernels, biases, and caches for the 3D sparse convolutional encoder.

Parameters:

dim_in – The dimensionality for each input tensor.

Returns:

Nothing at all.

call(inputs)

Apply the composition of SharedMLP, sparse convolutions, residual operations, and downsampling to the input tensors.

Parameters:

inputs – The input tensors.

Returns:

The final tensor after all the computations.

get_config()

Return necessary data to serialize the layer

classmethod from_config(config)

Use given config data to deserialize the layer

model.deeplearn.layer.strided_kpconv_layer module

class model.deeplearn.layer.strided_kpconv_layer.StridedKPConvLayer(*args, **kwargs)

Bases: KPConvLayer

Author:

Alberto M. Esmoris Pena

Strided version of the KPConvLayer layer. Instead of transforming \(R\) input points with \(D_{\mathrm{in}}\) features into \(R\) output points with \(D_{\mathrm{out}}\) features, it transforms \(R_1\) input points with \(D_{\mathrm{in}}\) feautres into \(R_2\) output points with \(D_{\mathrm{out}}\) features, where typically \(R_1 > R_2\).

See KPConvLayer.

__init__(**kwargs)

See Layer and Layer.__init__(). Also, see KPConvLayer and KPConvLayer.__init__().

build(dim_in)

See KPConvLayer and KPConvLayer.build().

call(inputs, training=False, mask=False)

Compute the strided version of KPConv. The main difference with respect to the KPConvLayer layer is that the \(\kappa\) neighbor points \(\pmb{x}_{j*} \in \mathcal{N}^{D}_{\pmb{x}_{i*}}\) are now defined such that \(\pmb{x}_{i*}\) is a point from a downsampled structure space \(\pmb{X_b} \in \mathbb{R}^{R_2 \times n_x}\), while the \(\pmb{x}_{j*}\) points belong to the structure space before downsampling \(\pmb{X_a} \in \mathbb{R}^{R_1 \times n_x}\).

Parameters:

inputs –

The input such that:

– inputs[0]

is the structure space tensor representing the geometry of the many receptive fields in the batch.

\[\mathcal{X_a} \in \mathbb{R}^{K \times R_1 \times n_x}\]

– inputs[1]

is the structure space tensor representing the geometry of the many downsampled receptive fields in the batch.

\[\mathcal{X_b} \in \mathbb{R}^{K \times R_2 \times n_x}\]

– inputs[2]

is the feature space tensor representing the features of the many receptive fields in the batch.

\[\mathcal{F} \in \mathbb{R}^{K \times R_1 \times n_f}\]

– inputs[3]

is the indexing tensor representing the neighborhoods of \(\kappa\) neighbors in the non downsampled space for each point in the downsampled.

\[\mathcal{N}^D \in \mathbb{Z}^{K \times R_2 \times \kappa}\]

Returns:

The output feature space \(\mathcal{Y} \in \mathbb{R}^{K \times R_2 \times D_{\mathrm{out}}}\).

model.deeplearn.layer.strided_light_kpconv_layer module

class model.deeplearn.layer.strided_light_kpconv_layer.StridedLightKPConvLayer(*args, **kwargs)

Bases: LightKPConvLayer

Author:

Alberto M. Esmoris Pena

Strided version of the LightKPConvLayer layer. Instead of transforming \(R\) input points with \(D_{\mathrm{in}}\) features into \(R\) output points with \(D_{\mathrm{out}}\) features, it transforms \(R_1\) input points with \(D_{\mathrm{in}}\) features into \(R_2\) output points with \(D_{\mathrm{out}}\) features, where typically \(R_1 > R_2\).

See LightKPConvLayer and StridedKPConvLayer.

__init__(**kwargs)

See Layer and Layer.__init__(). Also, see LightKPConvLayer and LightKPConvLayer.__init__().

build(dim_in)

See LightKPConvLayer and LightKPConvLayer.build().

call(inputs, training=False, mask=False)

Compute the strided version of light KPConv. The main difference with respect to the LightKPConvLayer layer is that the \(\kappa\) neighbor points \(\pmb{x}_{j*} \in \mathcal{N}^{D}_{\pmb{x}_{i*}}\) are now defined such that \(\pmb{x}_{i*}\) is a point from a downsampled structure space \(\pmb{X_{b}} \in \mathbb{R}^{R_2 \times n_x}\), while the \(\pmb{x}_{j*}\) points belong to the structure space before downsampling \(\pmb{X_a} \in \mathbb{R}^{R_1 \times n_x}\).

See LightKPConvLayer, StridedKPConvLayer, and StridedKPConvLayer.call().

Parameters:

inputs –

The input such that:

– inputs[0]

is the structure space tensor representing the geometry of the many receptive fields in the batch.

\[\mathcal{X_a} \in \mathbb{R}^{K \times R_1 \times n_x}\]

– inputs[1]

is the structure space tensor representing the geometry of the many receptive fields in the batch.

\[\mathcal{X_b} \in \mathbb{R}^{K \times R_2 \times n_x}\]

– inputs[2]

is the feature space tensor representing the features of the many receptive fields in the batch.

\[\mathcal{F} \in \mathbb{R}^{K \times R_1 \times n_f}\]

– inputs[3]

is the indexing tensor representing the neighborhoods of \(\kappa\) neighbors in the non downsampled space for each point in the downsampled.

\[\mathcal{N}^D \in \mathbb{Z}^{K \times R_2 \times \kappa}\]

Returns:

The output feature space \(\mathcal{Y} \in \mathbb{R}^{K \times R_2 \times D_{\mathrm{out}}}\).

model.deeplearn.layer.submanifold_spconv3d_layer module

class model.deeplearn.layer.submanifold_spconv3d_layer.SubmanifoldSpConv3DLayer(*args, **kwargs)

Bases: Layer

Author:

Alberto M. Esmoris Pena

A submanifold sparse 3D convolution layer consists of a dense convolution that is applied on a 3D sparse grid centering the window only on the active cells. The active cells are those cells that contain at least one point from the original input (typically, the 3D input point cloud).

A submanifold sparse 3D convolution layer is defined by a map \(h : X \subset \mathbb{Z}_{\geq 0} \rightarrow Y \subset \mathbb{Z}_{\geq 0}\). Note that both \(X\) and \(Y\) are finite subsets of the non-negative integers. Moreover, when the set \(Y\) is non-empty (i.e., there is at least one active cell) it satisfies the following:

1. \(0 \in Y\)

2. \(\forall y \in Y,\, y > 0, (y-1) \in Y\)

In other words, \(h\) maps the indices of sparse active cells to a finite set of sequential indices. These indices are used to link each active cell to its corresponding row in the matrix of features \(\pmb{F} \in \mathbb{R}^{(1+\lvert{X}\rvert) \times n_f}\), where \(n_f\) is the dimensionality of the input feature space and the \(1\) in \((1+\lvert{X}\rvert)\) accounts for the ground value (i.e., \(0\)) represented with the first row of the matrix \(\pmb{F}\).

Thus, the submanifold 3D convolution centered on the \(i\)-th cell of a sparse 3D grid with a window size \(w \in \mathbb{Z}_{>0}\) implies computing (note that for submanifold sparse convolutions \(w\) is a half size instead of a full size, thus it must be symmetrically applied to both directions of each axis):

\[\omega(i) = \left\{j_p\right\}_{p=0}^{(2w+1)^{n_x}-1} = \left\{j_p\right\}_{p=0}^{(2w+1)^3-1}\]

Where \(j_p\) is the index of a row in the matrix of input features \(\pmb{F}\) and \(n_x\) is the dimensionality of the structure space (for the case of 3D convolutions \(n_x = 3\)).

More concretely, \(j_p = h(j'_p)\) with:

\[j'_p(i) = i - w (1 + n_3 + n_3 n_2) + p \mod (2w+1) + \left( \left\lfloor\dfrac{p}{2w+1}\right\rfloor \mod (2w+1) \right)n_2 + \left( \left\lfloor\dfrac{p}{(2w+1)^2}\right\rfloor \mod (2w+1) \right) n_1 n_2\]

Where \(n_1\) is the number of partitions along the \(y\) axis and \(n_2\) is the number of partitions along the \(z\) axis. Note that these values come from the vector of axis-wise partitions \(\pmb{n} = (n_1, n_2, n_3)\).

Finally, the output matrix of features \(\pmb{G} \in \mathbb{R}^{(1 + \lvert{X}\rvert) \times n_g}\) will be computed as follows:

\[\pmb{g}_{h(i)*} = \sum_{j \in \omega(i)} \sum_{l=1}^{f}{ \pmb{f}_{j*} \pmb{W}_{l} }\]

Where \(n_g \in \mathbb{Z}_{>0}\) is the number of output features, \(f \in \mathbb{Z}_{>0}\) is the number of convolutional filters (or channels) and, for convenience, the first row of \(\pmb{G}\) is \(\pmb{g}_{0*} = \pmb{0}\) (so it represents the ground values, i.e., zeroes). Note also that \(\pmb{f}_{j*}\) refers to the \(j\)-th row if the input feature space (\(\pmb{F}\)) and \(\pmb{W}_l\) refers to the matrix of weights that represents the \(l\)-th convolutional filter.

See SubmanifoldSpConv3DCache for the cache that handles pre-computed values.

Variables:

• w (float) – The half size (i.e., number of cells along each direction of each axis) of the submanifold 3D convolutional window.

• f (int) – The number of filters/channels for the convolutions.

• nf (int) – The number of input features per cell.

• ng (int) – The number of output features per cell.

• W_initializer – The initializer for the convolutional filters.

• W_regularizer – The regularizer for the convolutional filters.

• W_constraint – The constraint for the convolutional filters.

• siml (SparseIndexingMapLayer.) – The sparse indexing map layer that is needed to translate sparse indices to sequential indices (i.e., \(h(k) = v\) map).

• wp (int) – The size (not half size) of the submanifold 3D convolutional window in terms of number of cells, i.e., \(2w+1\).

• wpsq (int) – \((2w+1)^2\)

• nw (int) – \((2w+1)^{n_x} = (2w+1)^{3}\)

• W (tf.Tensor) – The tensor of weights \(\mathcal{W} \in \mathbb{R}^{f \times n_f \times n_g}\) such that its slices are the matrices representing the trainable parameters for each convolutional filter.

• built_W (bool) – Whether the \(\mathcal{W}\) tensor of weights has been built or not. Initially it is false, but it will be updated once the layer is built.

__init__(w, f, nf, ng, built_W=False, W_initializer=None, W_regularizer=None, W_constraint=None, siml=None, built_p=False, built_pon1=False, built_ponw=False, built_ponw2=False, cache=None, **kwargs)

See Layer and layer.Layer.__init__().

build(dim_in)

Build the \(\mathcal{W} \in \mathbb{R}^{f \times n_f \times n_g}\) tensor representing the weights for each of the \(f\) convolutional filters. Also builds the caches.

See Layer and layer.Layer.build().

call(inputs, training=False, mask=False)

Compute the submaniold convolutions.

See SubmanifoldSpConv3DLayer for the maths.

Parameters:

inputs –

The feature space matrices as a 3D tensor with padding, the submanifold map \(h\) as a tensor of keys (k) with padding the matrix \(\pmb{N} \in \mathbb{Z}^{n_x \times K}\) of axis-wise partitions (where \(K\) is the batch size), and the start row indices.

Note that inputs[0] gives the feature spaces, inputs[1] the keys of the map h, inputs[2] the axis-wise partitions, and inputs[3] the start row-index for the elements in the batch so the padding can be ignored.

Returns:

The output feature space matrices as a tensor.

Return type:

tf.Tensor

static active_lookup(hk, n, w, pon1, ponw, ponw2, siml)

Assist the convolve inner function of the SubmanifoldSpConv3DLayer.call() method to compute the lookup of the active cells. See SubmanifoldSpConv3DLayer for the meaning of the variables and further details.

static spconv3d_on_elem(F, homega, W)

Assist the convolve inner function of the SubmanifoldSpConv3DLayer.call() method to compute the sparse 3D convolution. See SubmanifoldSpConv3DLayer for the meaning of the variables and further details.

get_config()

Return necessary data to serialize the layer

classmethod from_config(config)

Use given config data to deserialize the layer

class model.deeplearn.layer.submanifold_spconv3d_layer.SubmanifoldSpConv3DCache(w, built_p=False, built_pon1=False, built_ponw=False, built_ponw2=False)

Bases: object

Cache for the SubmanifoldSpConv3DLayer. The cache contains certain pre-computations that will be used when computing the layer.

Variables:

• w (float) – The half size (i.e., number of cells along each direction of each axis) of the submanifold 3D convolutional window.

• p (tf.Tensor of tf.int32) – The sequential indexing vector representing the submanifold convolutional windows.

• built_p (bool) – Whether the indexing vector representing the submanifold convolutional windows has been built or not. Initially it is false, but it will be updated once the layer is built.

• pon1 (tf.Tensor of tf.int32) – Cache for \(p \mod (2w+1)\).

• built_pon1 (bool) – Whether the cache for \(p \mod (2w+1)\) has been built or not. Initially it is false, but it will be updated once the layer is built.

• ponw (tf.Tensor of tf.int32) – Cache for \(\left(\left\lfloor\dfrac{p}{2w+1}\right\rfloor \mod (2w+1)\right)\).

• built_ponw (bool) – Whether the cache for \(\left(\left\lfloor\dfrac{p}{2w+1}\right\rfloor \mod (2w+1)\right)\) has been built or not. Initially it is false, but it will be updated once the layer is built.

• ponw2 (tf.Tensor of tf.int32) – Cache for \(\left(\left\lfloor\dfrac{p}{(2w+1)^2}\right\rfloor \mod (2w+1)\right)\).

• built_ponw2 (bool) – Whether the cache for \(\left(\left\lfloor\dfrac{p}{(2w+1)^2}\right\rfloor \mod (2w+1)\right)\) has been built or not. Initially it is false, but it will be updated once the layer is built.

__init__(w, built_p=False, built_pon1=False, built_ponw=False, built_ponw2=False)

build()

Build the cache for the SubmanifoldSpConv3DLayer.

get_config()

Return necessary data to serialize the cache

classmethod from_config(config)

Use given config data to deserialize the cache

model.deeplearn.layer.upsampling_spconv3d_layer module

class model.deeplearn.layer.upsampling_spconv3d_layer.UpsamplingSpConv3DLayer(*args, **kwargs)

Bases: Layer

Author:

Alberto M. Esmoris Pena

The UpsamplingSpConv3DLayer behaves like the SubmanifoldSpConv3DLayer but with the following changes.

1. The indices \(i\) of the active cells are considered for cells from the non-upsampled space, i.e., \(i \in h^U\).

2. The \(\omega(i)\) set is redefined as:

\[\omega(i) = \left\{j_p\right\}_{p=0}^{(w^U)^{n_x}-1} = \left\{j_p\right\}_{p=0}^{(w^U)^3-1}\]

3. The \(j_p = h(j'_p)\) index is computed considering \(h\) from the non-upsampled sparse grid. Also, the \(j'_p(i)\) function is redefined as:

\[j'_p(i) = i + p \mod w^U + \left( \left\lfloor\dfrac{p}{w^U}\right\rfloor \mod w^U \right)n_2 + \left( \left\lfloor\dfrac{p}{(w^U)^2}\right\rfloor \mod w^U \right) n_1 n_2\]

Variables:

• wU – The size (i.e., number of cells) of the upsampling 3D convolutional window.

• f (int) – The number of filters/channels for the convolutions.

• nf (int) – The number of input features per cell.

• ng (int) – The number of output features per cell.

• W_initializer – The initializer for the convolutional filters.

• W_regularizer – The regularizer for the convolutional filters.

• W_constraint – The constraint for the convolutional filters.

• siml (SparseIndexingMapLayer.) – The sparse indexing map layer that is needed to translate sparse indices to sequential indices (i.e., \(h(k) = v\) map).

• wUsq (int) – \((w^U)^2\)

• nwU – \((w^U)^{n_x} = (w^U)^{3}\)

• W (tf.Tensor) – The tensor of weights \(\mathcal{W} \in \mathbb{R}^{f \times n_f \times n_g}\) such that its slices are the matrices representing the trainable parameters for each convolutional filter.

• built_W (bool) – Whether the \(\mathcal{W}\) tensor of weights has been built or not. Initially it is false, but it will be updated once the layer is built.

__init__(wU, f, nf, ng, built_W=False, W_initializer=None, W_regularizer=None, W_constraint=None, siml=None, built_p=False, built_pon1=False, built_ponw=False, built_ponw2=False, cache=None, **kwargs)

Initialize the member attributes of the layer and the internal weights that do not depend on the input dimensionality.

Parameters:

kwargs – The key-word specification to parametrize the layer.

build(dim_in)

Build the \(\mathcal{W} \in \mathbb{R}^{f \times n_f \times n_g}\) tensor representing the weights for each of the \(f\) convolutional filters. Also builds the caches.

See Layer and layer.Layer.build().

call(inputs, training=False, mask=False)

Compute the upsampling convolutions.

See UpsamplingSpConv3DLayer for the maths.

Parameters:

inputs –

The feature space matrices as a 3D tensor with padding, the \(h^U\) vector of indices that gives the indices of the active cells in the non-upsampled sparse grid that are the centers for the upsampling convolutional windows, the matrix \(\pmb{N} \in \mathbb{Z}^{n_x \times K}\) of axis-wise partitions in the non-upsampled space (where \(K\) is the batch size), and the start row indices.

Note that inputs[0] gives the feature spaces, inputs[1] the vector of active cell indices to center the upsampling convolutional window in the non-upsampled sparse grid, inputs[2] the axis-wise partitions, inputs[3] the start row-index for the upsampled space, and inputs[4] the start row-index for the non-upsampled space (source, src). The start row-index allows to handle the padding.

Returns:

The output feature space matrices as a tensor with padding.

Return type:

tf.Tensor

static active_lookup(hU, n, pon1, ponw, ponw2, siml)

Assist the convolve inner function of the UpsamplingSpConv3DLayer.call() method to compute the lookup of the upsampling active cells. See UpsamplingSpConv3DLayer for the meaning of the variables and further details.

static up_spconv3d_on_elem(F, hUomega, W)

Assist the convolve inner function of the UpsamplingSpConv3DLayer.call() method to compute the sparse upsampling 3D convolution. See UpsamplingSpConv3DLayer for the meaning of the variables and further details.

get_config()

Return necessary data to serialize the layer

classmethod from_config(config)

Use given config data to deserialize the layer

class model.deeplearn.layer.upsampling_spconv3d_layer.UpsamplingSpConv3DCache(wU, built_p=False, built_pon1=False, built_ponw=False, built_ponw2=False)

Bases: object

Cache for the UpsamplingSpConv3DLayer. The cache contains certain pre-computations that will be used when computing the layer.

Variables:

• p (tf.Tensor of tf.int32) – The sequential indexing vector representing the upsampling convolutional windows.

• built_p (bool) – Whether the indexing vector representing the upsampling convolutional windows has been built or not. Initially it is false, but it will be updated once the layer is built.

• pon1 (tf.Tensor of tf.int32) – Cache for \(p \mod w^U\).

• built_pon1 (bool) – Whether the cache for \(p \mod w^U\) has been built or not. Initially it is false, but it will be updated once the layer is built.

• ponw (tf.Tensor of tf.int32) – Cache for \(\left(\lfloor\dfrac{p}{w^U}\rfloor \mod w^U\right)\).

• built_ponw (bool) – Whether the cache for \(\left(\lfloor\dfrac{p}{w^U}\rfloor \mod w^U\right)\) has been built or not. Initially it is false, but it will be updated once the layer is built.

• ponw2 (tf.Tensor of tf.int32) – Cache for \(\left(\lfloor\dfrac{p}{(w^U)^2}\rfloor \mod w^U\right)\).

• built_ponw2 (bool) – Whether the cache for \(\left(\lfloor\dfrac{p}{(w^U)^2}\rfloor \mod w^U\right)\) has been built or not. Initially it is false, but it will be updated once the layer is built.

__init__(wU, built_p=False, built_pon1=False, built_ponw=False, built_ponw2=False)

build()

Build the cache for the UpsamplingSpConv3DLayer.

get_config()

Return necessary data to serialize the cache

classmethod from_config(config)

Use given config data to deserialize the cache

Module contents

author:

Alberto M. Esmoris Pena

The layer package contains the logic to handle the layers of a neural network.