src.model.deeplearn.arch.conv_autoenc_pwise_classif

Classes

ConvAutoencPwiseClassif(**kwargs)

class src.model.deeplearn.arch.conv_autoenc_pwise_classif.ConvAutoencPwiseClassif(**kwargs)

Author:

Alberto M. Esmoris Pena

The convolutional autoencoder architecture for point-wise classification.

Examples of convolutional autoencoders are the PointNet++ model (https://arxiv.org/abs/1706.02413) and the KPConv model (https://arxiv.org/abs/1904.08889).

__init__(**kwargs)

See architecture.Architecture.__init__().

build_input()

Build the input layer of the neural network. A convolutional autoencoder expects to receive many input tensors representing the hierarchical nature of the architecture. More concretely, for each element in the batch there must be:

1. The structure space matrices representing the points in the hierarchy of FPS receptive fields (typically, \(n_x=3\), i.e., 3D point clouds).

\[\pmb{X}_1 \in \mathbb{R}^{R_1 \times n_x}, \ldots, \pmb{X}_{d^*} \in \mathbb{R}^{R_{d^*} \times n_x}\]

2. The feature space matrix representing the points in the first receptive field of the hierarchy.

\[\pmb{F}_1 \in \mathbb{R}^{R_1 \times n_f}\]

3. The downsampling matrices after the first one (which is not used by the neural network itself but immediately before to transform the original input to the first receptive field).

\[\pmb{N}^D_2 \in \mathbb{Z}^{R_2 \times K^D_2}, \ldots, \pmb{N}^D_{d^*} \in \mathbb{Z}^{R_{d^*} \times K^D_{d^*}}\]

4. The point-wise neighborhood matrices to be used at each downsampled representation as topological information.

\[\pmb{N}_2 \in \mathbb{Z}^{R_2 \times K_2}, \ldots, \pmb{N}_{d^*} \in \mathbb{Z}^{R_{d^*} \times K_{d^*}}\]

3. The upsampling matrices after the first one (which is not used by the neural network itself but immediately after to transform the output from the first receptive field to the original space).

\[\pmb{N}^U_2 \in \mathbb{Z}^{R_2 \times K^U_2}, \ldots, \pmb{N}^U_{d^*} \in \mathbb{Z}^{R_{d^*} \times K^U_{d^*}}\]

Returns:

Built layers.

Return type:

list of tf.Tensor

build_hidden(x, **kwargs)

Build the hidden layers of the convolutional autoencoder neural network.

Parameters:

x (tf.Tensor) – The input layer for the first hidden layer.

Returns:

The last hidden layer.

Return type:

:class:.`tf.Tensor`

build_output(x, **kwargs)

Build the output layer of the convolutional autoencoder neural network for point-wise classification tasks.

See architecture.Architecture.build_output().

get_num_output_heads()

See Architecture.get_num_output_heads().

Returns:

One in the general case, two if contextual head is operating in multihead mode.

Return type:

int

build_downsampling_hierarchy()

Build the downsampling hierarchy.

Returns:

The last layer of the downsampling hierarchy.

Return type:

tf.Tensor

build_downsampling_pnet_hierarchy()

Build the downsampling hierarchy based on the PointNet operator.

build_downsampling_kpconv_hierarchy()

Build the downsampling hierarchy based on the KPConv operator.

build_downsampling_layer(Xs, x, d, i)

Build a downsampling layer in the context of the KPConv, light KPConv (supporting also StridedKPConvLayer and StridedLightKPConvLayer apart from FeaturesDownsamplingLayer), and PointTransformer models.

Parameters:

• Xs (list) – The list of receptive field-wise structure spaces.

• x (tf.Tensor) – The input features for the downsampling layer.

• d (int) – The model depth.

• i (int) – The index of the operation (because there might be different number of operations per depth level).

Returns:

The downsampling layer.

Return type:

tf.Tensor

build_downsampling_lightkpconv_hierarchy()

Build the downsampling hierarchy based on the light KPConv operator.

build_downsampling_pttransf_hierarchy()

Build the downsampling hierarchy based on the PointTransformer operator.

build_downsampling_gpttransf_hierarchy()

Build the downsampling hierarchy based on the GroupedPointTransformer operator.

build_encoding_gpttransf_block(Xs, d, i, x)

Build a block around the GroupedPointTransformerLayer to assist the building of a grouped point transformer-based encoding hierarchy (see ConvAutoencPwiseClassif.build_downsampling_gpttransf_hierarchy() ).

build_downsampling_pointmlp_hierarchy()

Build the downsampling hierarchy based on the PointMLP operator.

build_encoding_pointmlp_block(Xs, d, i, x)

Build a block around the GeometricAffineLayer and PointMLPLayer to assist the building of a PointMLP-based encoding hierarchy (see ConvAutoencPwiseClassif.build_downsampling_pointmlp_hierarchy() ).

build_downsampling_kpconvx_hierarchy()

Build the downsampling hierarchy based on the KPConvX operator.

build_encoding_kpconvx_block(Xs, d, i, x)

Build a block around the KPConvXLayer to assist the building of a KPConvX-based encoding hierarchy (see ConvAutoencPwiseClassif.build_downsampling_kpconvx_hierarchy() ).

build_downsampling_contextual_hierarchy()

Build the downsampling hierarchy based on contextual point layers.

build_encoding_contextual_block(Xs, d, i, x)

Build a block around the ContextualPointLayer to assist the building of a contextual encoding hierarchy (see ConvAutoencPwiseClassif.build_downsampling_contextual_hierarchy() ).

build_upsampling_hierarchy()

Build the upsampling hierarchy.

Returns:

The last layer of the upsampling hierarchy.

Return type:

tf.Tensor

build_upsampling_layer(Xs, x, reverse_d)

Build an upsampling layer in the context of the KPConv, light KPConv, (supporting also StridedKPConvLayer and StridedLightKPConvLayer apart from FeaturesUpsamplingLayer), and PointTransformer models.

Parameters:

• Xs (list) – The list of receptive field-wise structure spaces.

• x (tf.Tensor) – The input features for the upsampling layer.

• reverse_d (int) – The model depth in reverse order (as decoding “undoes” the encoding).

Returns:

The upsampling layer.

Return type:

tf.Tensor

build_decoding_kpconvx(Xs, N, d, x)

Build a block around the KPConvXLayer to assist the building of a PointMLP-based encoding hierarchy (see ConvAutoencPwiseClassif.build_downsampling_kpconvx_hierarchy() ).

apply_prewrap(x, depth, idx)

Wrap the input before a feature extraction layer with unary convolutions (also known as shared MLPs), hourglass, or point transformers.

Parameters:

• x – The input to be wrapped.

• depth – The depth of the feature extractor being pre-wrapped.

• idx – The index of the feature extractor being pre-wrapped.

Returns:

A tuple with the input for the next layer and the output dimensionality.

Return type:

tuple

apply_postwrap(x, depth, idx)

Wrap the output of a feature extraction block with unary convolutions (also known as shared MLPs), hourglass, or point transformers.

Parameters:

• x – The input to be wrapped.

• depth – The depth of the feature extractor being post-wrapped.

• idx – The index of the feature extractor being post-wrapped.

Returns:

The input for the next layer

unary_convolution_prewrap(wrap_spec, Din, x, depth, idx)

See ConvAutoencPwiseClassif.apply_prewrap().

unary_convolution_postwrap(wrap_spec, Dout, x, depth, idx)

See ConvAutoencPwiseClassif.unary_convolution_postwrap().

hourglass_prewrap(hourglass_spec, Din, x, depth, idx)

See ConvAutoencPwiseClassif.apply_prewrap().

hourglass_postwrap(hourglass_spec, Dout, x, depth, idx)

See ConvAutoencPwiseClassif.apply_postwrap().

point_transformer_prewrap(pttransf_spec, Din, x, d, i)

See ConvAutoencPwiseClassif.apply_prewrap().

point_transformer_postwrap(pttransf_spec, Dout, x, d, i)

See ConvAutoencPwiseClassif.unary_convolution_postwrap().

prefit_logic_callback(cache_map)

The callback implementing any necessary logic immediately before fitting a ConvAutoencPwiseClassif model.

Parameters:

cache_map – The key-word dictionary containing variables that are guaranteed to live at least during prefit, fit, and postfit.

Returns:

Nothing.

posfit_logic_callback(cache_map)

The callback implementing any necessary logic immediately after fitting a ConvAutoencPwiseClassif model.

Parameters:

cache_map – The key-word dictionary containing variables that are guaranteed to live at least during prefit, fit, and postfit.

Returns:

Nothing.