src.model.deeplearn.layer.kpconv_layer

Classes

KPConvLayer(*args, **kwargs)

class src.model.deeplearn.layer.kpconv_layer.KPConvLayer(*args, **kwargs)
Author:

Alberto M. Esmoris Pena

A kernel point convolution layer receives batches of \(R\) points with \(D_{\mathrm{in}}\) features each and \(\kappa\) known neighbors in the same space. These inputs are used to compute an output feature space of \(R\) points with \(D_{\mathrm{out}}\) features each. In doing so, the indexing tensor \(\mathcal{N} \in \mathbb{Z}^{K \times R \times \kappa}\) is used to link \(\kappa\) neighbors for each of the \(R\) input points in each of the \(K\) input batches.

The layer is defined by a structure space matrix \(\pmb{Q} \in \mathbb{R}^{m_q \times n_x}\) representing \(m_q\) points in a \(n_x\)-dimensional Euclidean space, and a tensor of weights \(\mathcal{W} \in \mathbb{R}^{m_q \times D_{\mathrm{in}} \times D_{\mathrm{out}}}\) who has \(m_q\) slices corresponding to a matrix of weights \(\pmb{W}_k \in \mathbb{R}^{D_{\mathrm{in}} \times D_{\mathrm{out}}}\) associated to a different kernel point each.

Further details on the KPConv operator can be read in the corresponding paper (https://arxiv.org/abs/1904.08889).

Variables:

  • sigma (float) – The influence distance governing the kernel (\(\sigma\)).

  • kernel_radius (float) – The radius of the ball where the kernel is defined.

  • num_kernel_points (int) – How many points (\(m_q\)) compose the kernel.

  • deformable (bool) – Whether the \(m_q\) kernel points must be optimized (True) or not (False, def.) by the neural network.

  • Dout (int) – The output dimensionality of the convolution.

  • built_Q (bool) – Whether the structure space matrix \(\pmb{Q} \in \mathbb{R}^{m_q \times n_x}\) is built. Initially it is false, but it will be updated once the layer is built.

  • built_W (bool) – Whether the \(m_q\) weight matrices \(\pmb{W} \in \mathbb{R}^{D_{\mathrm{in}} \times D_{\mathrm{out}}}\) corresponding to each kernel point are built or not. Initially it is false, but it will be updated once the layer is built.

  • W_initializer – The initializer for the tensor of weights \(\mathcal{W} \in \mathbb{R}^{m_q \times D_{\mathrm{in}} \times D_{\mathrm{out}}}\).

  • W_regularizer – The regularizer for the tensor of weights \(\mathcal{W} \in \mathbb{R}^{m_q \times D_{\mathrm{in}} \times D_{\mathrm{out}}}\).

  • W_constraint – The constraints for the tensor of weights \(\mathcal{W} \in \mathbb{R}^{m_q \times D_{\mathrm{in}} \times D_{\mathrm{out}}}\).

__init__(sigma=1.0, kernel_radius=1.0, num_kernel_points=19, deformable=False, Dout=320, built_Q=False, built_W=False, W_initializer=None, W_regularizer=None, W_constraint=None, **kwargs)

See Layer and layer.Layer.__init__().

build(dim_in)

Build the \(\pmb{Q} \in \mathbb{R}^{m_q \times n_x}\) matrix representing the kernel’s structure space. Typically, the initialization is delegated to a KernelPointBallInitializer object. Also, build the \(\mathcal{W} \in \mathbb{R}^{m_q \times D_{\mathrm{in}} \times D_{\mathrm{out}}}\) tensor of weights.

See Layer and layer.Layer.build().

call(inputs, training=False, mask=False)

Let \(\mathcal{Q} = (\pmb{Q}, \mathcal{W})\) represent the kernel including both the structure and weights information. Also, let \(\pmb{P} = [\pmb{X} | \pmb{F}]\) be the full matrix representation of the input receptive field including both the structure and the feature spaces.

For then, the output for each receptive field in the input batch can be computed as a matrix \(\pmb{Y} \in \mathbb{R}^{R \times D_{\mathrm{out}}}\) that arises by applying point-wise convolutions as shown below:

\[\begin{split}\pmb{Y} = \left[\begin{array}{ccc} \rule[.5ex]{3.0ex}{0.6pt} & (\pmb{P} * \mathcal{Q})(\pmb{x}_{1*}) & \rule[.5ex]{3.0ex}{0.6pt} \\ \vdots & \vdots & \vdots \\ \rule[.5ex]{3.0ex}{0.6pt} & (\pmb{P} * \mathcal{Q})(\pmb{x}_{R*}) & \rule[.5ex]{3.0ex}{0.6pt} \end{array}\right]\end{split}\]

Each point-wise convolution is defined as:

\[\left(\pmb{P} * \mathcal{Q}\right) (\pmb{x}_{i*}) = \sum_{\pmb{x}_{j*} \in \mathcal{N}_{\pmb{x}_{i*}}}{ \Biggl[{ \sum_{k=1}^{m_q} \max \; \biggl\{ 0, 1 - \dfrac{ \lVert \pmb{x}_{j*} - \pmb{x}_{i*} - \pmb{q}_{k*} \rVert }{ \sigma } \biggr\} } \pmb{W}_{k}^\intercal \Biggr] \pmb{f}_{j*} }\]

Where \(\mathcal{N}_{\pmb{x}_{i*}}\) is the neighborhood of the \(\kappa\) closest neighbors of \(\pmb{x}_{i*}\) in the input receptive field \(\pmb{X}\).

Parameters:

inputs –

The input such that:

– inputs[0]

is the structure space tensor representing the geometry of the many receptive fields in the batch.

\[\mathcal{X} \in \mathbb{R}^{K \times R \times n_x}\]
– inputs[1]

is the feature space tensor representing the features of the many receptive fields in the batch.

\[\mathcal{F} \in \mathbb{R}^{K \times R \times n_f}\]
– inputs[2]

is the indexing tensor representing the neighborhoods of \(\kappa\) neighbors for each input point, in the same space.

\[\mathcal{N} \in \mathbb{Z}^{K \times R \times \kappa}\]

Returns:

The output feature space \(\mathcal{Y} \in \mathbb{R}^{K \times R \times D_{\mathrm{out}}}\).

gather_neighborhoods(N, X, F)

Assist the KPConvLayer.call() method by gathering the data representing the structure and feature spaces of each neighborhood.

compute_output_features(X, NX, NF)

Assist the KPConvLayer.call() method by applying the point-wise convolutions to compute the output features.

build_W(Din)

Assist the KPConvLayer.build() method in building the \(\mathcal{W} \in \mathbb{R}^{K \times D_{\mathrm{in}} \times D_{\mathrm{out}}}\) tensor, i.e., the convolution weights.

Parameters:

Din (int) – The dimensionality of the input feature space \(D_{\mathrm{in}}\).

get_config()

Return necessary data to serialize the layer

classmethod from_config(config)

Use given config data to deserialize the layer

export_representation(dir_path, out_prefix=None, Wpast=None)

Export a set of files representing the state of the kernel for both the structure (Q) and the weights (W).

Parameters:

  • dir_path (str) – The directory where the representation files will be exported.

  • out_prefix (str) – The output prefix to name the output files.

  • Wpast (np.ndarray or tf.Tensor or None) – The weights of the kernel in the past.

Returns:

Nothing at all, but the representation is exported as a set of files inside the given directory.