Deep learning with offline sequencer and data mining
Data
The point clouds used in this example are from the DALES dataset of 40 Aerial Laser Scanning (ALS) point clouds with \((x, y, z)\) coordinates.
The models are trained on the following 29 point clouds: 5080_54435, 5085_54320, 5095_54440, 5095_54455, 5100_54495, 5105_54405, 5105_54460, 5110_54320, 5110_54460, 5110_54475, 5110_54495, 5115_54480, 5130_54355, 5135_54495, 5140_54445, 5145_54340, 5145_54405, 5145_54460, 5145_54470, 5145_54480, 5150_54340, 5160_54330, 5165_54390, 5165_54395, 5180_54435, 5180_54485, 5185_54390, 5185_54485, and 5190_54400.
The validation of the models is carried out in the remaining 11 point clouds that have not been used during training: 5080_54400, 5080_54470, 5100_54440, 5100_54490, 5120_54445, 5135_54430, 5135_54435, 5140_54390, 5150_54325, 5155_54335, and 5175_54395.
Visualization of the raw validation point cloud 5135_54435
JSON
Data mining JSON
The JSON below was used to mine the point-wise geometric and height features.
{
"in_pcloud": [
"/oldext4/lidar_data/dales/las/dales_las/train/5080_54435.laz",
"/oldext4/lidar_data/dales/las/dales_las/train/5085_54320.laz",
"/oldext4/lidar_data/dales/las/dales_las/train/5095_54440.laz",
"/oldext4/lidar_data/dales/las/dales_las/train/5095_54455.laz",
"/oldext4/lidar_data/dales/las/dales_las/train/5100_54495.laz",
"/oldext4/lidar_data/dales/las/dales_las/train/5105_54405.laz",
"/oldext4/lidar_data/dales/las/dales_las/train/5105_54460.laz",
"/oldext4/lidar_data/dales/las/dales_las/train/5110_54320.laz",
"/oldext4/lidar_data/dales/las/dales_las/train/5110_54460.laz",
"/oldext4/lidar_data/dales/las/dales_las/train/5110_54475.laz",
"/oldext4/lidar_data/dales/las/dales_las/train/5110_54495.laz",
"/oldext4/lidar_data/dales/las/dales_las/train/5115_54480.laz",
"/oldext4/lidar_data/dales/las/dales_las/train/5130_54355.laz",
"/oldext4/lidar_data/dales/las/dales_las/train/5135_54495.laz",
"/oldext4/lidar_data/dales/las/dales_las/train/5140_54445.laz",
"/oldext4/lidar_data/dales/las/dales_las/train/5145_54340.laz",
"/oldext4/lidar_data/dales/las/dales_las/train/5145_54405.laz",
"/oldext4/lidar_data/dales/las/dales_las/train/5145_54460.laz",
"/oldext4/lidar_data/dales/las/dales_las/train/5145_54470.laz",
"/oldext4/lidar_data/dales/las/dales_las/train/5145_54480.laz",
"/oldext4/lidar_data/dales/las/dales_las/train/5150_54340.laz",
"/oldext4/lidar_data/dales/las/dales_las/train/5160_54330.laz",
"/oldext4/lidar_data/dales/las/dales_las/train/5165_54390.laz",
"/oldext4/lidar_data/dales/las/dales_las/train/5165_54395.laz",
"/oldext4/lidar_data/dales/las/dales_las/train/5180_54435.laz",
"/oldext4/lidar_data/dales/las/dales_las/train/5180_54485.laz",
"/oldext4/lidar_data/dales/las/dales_las/train/5185_54390.laz",
"/oldext4/lidar_data/dales/las/dales_las/train/5185_54485.laz",
"/oldext4/lidar_data/dales/las/dales_las/train/5190_54400.laz",
"/oldext4/lidar_data/dales/las/dales_las/test/5080_54400.laz",
"/oldext4/lidar_data/dales/las/dales_las/test/5080_54470.laz",
"/oldext4/lidar_data/dales/las/dales_las/test/5100_54440.laz",
"/oldext4/lidar_data/dales/las/dales_las/test/5100_54490.laz",
"/oldext4/lidar_data/dales/las/dales_las/test/5120_54445.laz",
"/oldext4/lidar_data/dales/las/dales_las/test/5135_54430.laz",
"/oldext4/lidar_data/dales/las/dales_las/test/5135_54435.laz",
"/oldext4/lidar_data/dales/las/dales_las/test/5140_54390.laz",
"/oldext4/lidar_data/dales/las/dales_las/test/5150_54325.laz",
"/oldext4/lidar_data/dales/las/dales_las/test/5155_54335.laz",
"/oldext4/lidar_data/dales/las/dales_las/test/5175_54395.laz"
],
"out_pcloud": [
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/dl_mined/train/5080_54435.las",
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/dl_mined/train/5085_54320.las",
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/dl_mined/train/5095_54440.las",
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/dl_mined/train/5095_54455.las",
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/dl_mined/train/5100_54495.las",
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/dl_mined/train/5105_54405.las",
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/dl_mined/train/5105_54460.las",
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/dl_mined/train/5110_54320.las",
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/dl_mined/train/5110_54460.las",
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/dl_mined/train/5110_54475.las",
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/dl_mined/train/5110_54495.las",
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/dl_mined/train/5115_54480.las",
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/dl_mined/train/5130_54355.las",
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/dl_mined/train/5135_54495.las",
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/dl_mined/train/5140_54445.las",
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/dl_mined/train/5145_54340.las",
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/dl_mined/train/5145_54405.las",
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/dl_mined/train/5145_54460.las",
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/dl_mined/train/5145_54470.las",
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/dl_mined/train/5145_54480.las",
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/dl_mined/train/5150_54340.las",
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/dl_mined/train/5160_54330.las",
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/dl_mined/train/5165_54390.las",
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/dl_mined/train/5165_54395.las",
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/dl_mined/train/5180_54435.las",
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/dl_mined/train/5180_54485.las",
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/dl_mined/train/5185_54390.las",
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/dl_mined/train/5185_54485.las",
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/dl_mined/train/5190_54400.las",
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/dl_mined/test/5080_54400.las",
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/dl_mined/test/5080_54470.las",
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/dl_mined/test/5100_54440.las",
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/dl_mined/test/5100_54490.las",
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/dl_mined/test/5120_54445.las",
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/dl_mined/test/5135_54430.las",
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/dl_mined/test/5135_54435.las",
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/dl_mined/test/5140_54390.las",
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/dl_mined/test/5150_54325.las",
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/dl_mined/test/5155_54335.las",
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/dl_mined/test/5175_54395.las"
],
"sequential_pipeline": [
{
"class_transformer": "ClassReducer",
"on_predictions": false,
"input_class_names": ["noclass", "ground", "vegetation", "car", "truck", "powerline", "fence", "pole", "building"],
"output_class_names": ["noclass", "ground", "vegetation", "vehicle", "powerline", "fence", "pole", "building"],
"class_groups": [
["noclass"],
["ground"],
["vegetation"],
["car", "truck"],
["powerline"],
["fence"],
["pole"],
["building"]
],
"report_path": null,
"plot_path": null
},
{
"miner": "GeometricFeatures",
"radius": 1.5,
"fnames": ["linearity", "planarity", "sphericity"],
"frenames": ["linearity_r1_5", "planarity_r1_5", "sphericity_r1_5"],
"nthreads": -1
},
{
"miner": "GeometricFeatures",
"radius": 3.0,
"fnames": ["linearity", "planarity", "sphericity"],
"frenames": ["linearity_r3_0", "planarity_r3_0", "sphericity_r3_0"],
"nthreads": -1
},
{
"miner": "FPSDecorated",
"fps_decorator": {
"num_points": "m/3",
"fast": 2,
"num_encoding_neighbors": 1,
"num_decoding_neighbors": 1,
"release_encoding_neighborhoods": false,
"threads": -1,
"representation_report_path": null
},
"decorated_miner": {
"miner": "HeightFeaturesPP",
"nthreads": -1,
"neighborhood": {
"type": "Cylinder",
"radius": 15.0,
"separation_factor": 0
},
"outlier_filter": "stdev",
"fnames": ["floor_distance", "ceil_distance"],
"frenames": ["floordist_r15_to30", "ceildist_r15_to30"]
}
},
{
"miner": "FPSDecorated",
"fps_decorator": {
"num_points": "m/5",
"fast": 2,
"num_encoding_neighbors": 1,
"num_decoding_neighbors": 1,
"release_encoding_neighborhoods": false,
"threads": -1,
"representation_report_path": null
},
"decorated_miner": {
"miner": "HeightFeaturesPP",
"nthreads": -1,
"neighborhood": {
"type": "Cylinder",
"radius": 30.0,
"separation_factor": 0
},
"outlier_filter": "stdev",
"fnames": ["floor_distance", "ceil_distance"],
"frenames": ["floordist_r30_to30", "ceildist_r30_to30"]
}
},
{
"feature_transformer": "MinmaxNormalizer",
"fnames": [
"floordist_r15_to30", "ceildist_r15_to30",
"floordist_r30_to30", "ceildist_r30_to30"
],
"minmax": [[0, 30], [0, 30], [0, 30], [0, 30]],
"target_range": [0, 1],
"clip": true,
"report_path": null,
"update_and_preserve": true
}
]
}
Training JSON
The JSON below was used to train the model with the offline sequencer and data mining.
{
"in_pcloud": [
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/dl_mined/train/5080_54435.laz"
],
"out_pcloud": [
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/DL_SFLNETPP_OFFLINE/T1B/*"
],
"sequential_pipeline": [
{
"train": "ConvolutionalAutoencoderPwiseClassifier",
"training_type": "base",
"fnames": ["ones", "linearity_r1_5", "planarity_r1_5", "linearity_r3_0", "planarity_r3_0", "floordist_r15_to30", "ceildist_r15_to30", "floordist_r30_to30", "ceildist_r30_to30"],
"random_seed": null,
"model_args": {
"fnames": ["ones", "linearity_r1_5", "planarity_r1_5", "linearity_r3_0", "planarity_r3_0", "floordist_r15_to30", "ceildist_r15_to30", "floordist_r30_to30", "ceildist_r30_to30"],
"num_classes": 8,
"class_names": ["noclass", "ground", "vegetation", "vehicles", "powerlines", "fences", "poles", "buildings"],
"pre_processing": {
"pre_processor": "hierarchical_fpspp",
"shuffle_training_class_distribution": true,
"training_class_distribution": [0, 2000, 2000, 2000, 2000, 2000, 2000, 2000],
"support_strategy_num_points": 10000,
"to_unit_sphere": false,
"support_strategy": "fps",
"support_strategy_fast": 4,
"receptive_field_oversampling": {
"min_points": 16,
"strategy": "nearest",
"k": 8,
"radius": 0.5
},
"center_on_pcloud": true,
"neighborhood": {
"type": "sphere",
"radius": 6.0,
"separation_factor": 0.8
},
"num_points_per_depth": [256, 128, 64, 32, 16],
"fast_flag_per_depth": [false, false, false, false, false],
"num_downsampling_neighbors": [1, 16, 16, 16, 16],
"num_pwise_neighbors": [16, 16, 16, 16, 16],
"num_upsampling_neighbors": [1, 16, 16, 16, 16],
"nthreads": -1,
"training_receptive_fields_distribution_report_path": "*/training_eval/training_receptive_fields_distribution.log",
"training_receptive_fields_distribution_plot_path": "*/training_eval/training_receptive_fields_distribution.svg",
"training_receptive_fields_dir": null,
"receptive_fields_distribution_report_path": "*/training_eval/receptive_fields_distribution.log",
"receptive_fields_distribution_plot_path": "*/training_eval/receptive_fields_distribution.svg",
"receptive_fields_dir": null,
"training_support_points_report_path": "*/training_eval/training_support_points.las",
"support_points_report_path": "*/training_eval/support_points.las"
},
"feature_extraction": {
"type": "LightKPConv",
"operations_per_depth": [2, 1, 1, 1, 1],
"feature_space_dims": [64, 64, 128, 256, 512, 1024],
"bn": true,
"bn_momentum": 0.98,
"activate": true,
"sigma": [6.0, 6.0, 7.5, 9.0, 10.5, 12.0],
"kernel_radius": [6.0, 6.0, 6.0, 6.0, 6.0, 6.0],
"num_kernel_points": [15, 15, 15, 15, 15, 15],
"deformable": [false, false, false, false, false, false],
"W_initializer": ["glorot_uniform", "glorot_uniform", "glorot_uniform", "glorot_uniform", "glorot_uniform", "glorot_uniform"],
"W_regularizer": [null, null, null, null, null, null],
"W_constraint": [null, null, null, null, null, null],
"A_trainable": [true, true, true, true, true ,true],
"A_regularizer": [null, null, null, null, null, null],
"A_constraint": [null, null, null, null, null, null],
"A_initializer": ["ones", "ones", "ones", "ones", "ones", "ones"],
"unary_convolution_wrapper": null,
"hourglass_wrapper": {
"internal_dim": [2, 2, 4, 16, 32, 64],
"parallel_internal_dim": [8, 8, 16, 32, 64, 128],
"activation": ["relu", "relu", "relu", "relu", "relu", "relu"],
"activation2": [null, null, null, null, null, null],
"regularize": [true, true, true, true, true, true],
"W1_initializer": ["glorot_uniform", "glorot_uniform", "glorot_uniform", "glorot_uniform", "glorot_uniform", "glorot_uniform"],
"W1_regularizer": [null, null, null, null, null, null],
"W1_constraint": [null, null, null, null, null, null],
"W2_initializer": ["glorot_uniform", "glorot_uniform", "glorot_uniform", "glorot_uniform", "glorot_uniform", "glorot_uniform"],
"W2_regularizer": [null, null, null, null, null, null],
"W2_constraint": [null, null, null, null, null, null],
"loss_factor": 0.1,
"subspace_factor": 0.125,
"feature_dim_divisor": 4,
"bn": false,
"bn_momentum": 0.98,
"out_bn": true,
"out_bn_momentum": 0.98,
"out_activation": "relu"
}
},
"features_alignment": null,
"downsampling_filter": "strided_lightkpconv",
"upsampling_filter": "mean",
"upsampling_bn": true,
"upsampling_momentum": 0.98,
"upsampling_hourglass": {
"activation": "relu",
"activation2": null,
"regularize": true,
"W1_initializer": "glorot_uniform",
"W1_regularizer": null,
"W1_constraint": null,
"W2_initializer": "glorot_uniform",
"W2_regularizer": null,
"W2_constraint": null,
"loss_factor": 0.1,
"subspace_factor": 0.125
},
"conv1d": false,
"conv1d_kernel_initializer": "glorot_normal",
"output_kernel_initializer": "glorot_normal",
"model_handling": {
"summary_report_path": "*/model_summary.log",
"training_history_dir": "*/training_eval/history",
"features_structuring_representation_dir": null,
"kpconv_representation_dir": "*/training_eval/kpconv_layers/",
"skpconv_representation_dir": "*/training_eval/skpconv_layers/",
"lkpconv_representation_dir": "*/training_eval/lkpconv_layers/",
"slkpconv_representation_dir": "*/training_eval/slkpconv_layers/",
"class_weight": [0.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0],
"training_epochs": 100,
"batch_size": 64,
"training_sequencer": {
"type": "DLOfflineSequencer",
"offline_storage": "/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/DL_SFLNETPP_OFFLINE/training_dataset_B.os1",
"chunk_size": 250,
"chunk_randomization": true,
"batch_randomization": true,
"disable_offline_storage_writing": false,
"offline_pcloud": [
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/dl_mined/train/5085_54320.laz",
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/dl_mined/train/5095_54440.laz",
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/dl_mined/train/5095_54455.laz",
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/dl_mined/train/5100_54495.laz",
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/dl_mined/train/5105_54405.laz",
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/dl_mined/train/5105_54460.laz",
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/dl_mined/train/5110_54320.laz",
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/dl_mined/train/5110_54460.laz",
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/dl_mined/train/5110_54475.laz",
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/dl_mined/train/5110_54495.laz",
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/dl_mined/train/5115_54480.laz",
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/dl_mined/train/5130_54355.laz",
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/dl_mined/train/5135_54495.laz",
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/dl_mined/train/5140_54445.laz",
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/dl_mined/train/5145_54340.laz",
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/dl_mined/train/5145_54405.laz",
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/dl_mined/train/5145_54460.laz",
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/dl_mined/train/5145_54470.laz",
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/dl_mined/train/5145_54480.laz",
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/dl_mined/train/5150_54340.laz",
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/dl_mined/train/5160_54330.laz",
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/dl_mined/train/5165_54390.laz",
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/dl_mined/train/5165_54395.laz",
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/dl_mined/train/5180_54435.laz",
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/dl_mined/train/5180_54485.laz",
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/dl_mined/train/5185_54390.laz",
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/dl_mined/train/5185_54485.laz",
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/dl_mined/train/5190_54400.laz"
],
"backbone": {
"type": "DLSequencer",
"random_shuffle_indices": true,
"augmentor": {
"transformations": [
{
"type": "Rotation",
"axis": [0, 0, 1],
"angle_distribution": {
"type": "uniform",
"start": -3.141592,
"end": 3.141592
}
},
{
"type": "Scale",
"scale_distribution": {
"type": "uniform",
"start": 0.99,
"end": 1.01
}
},
{
"type": "Jitter",
"noise_distribution": {
"type": "normal",
"mean": 0,
"stdev": 0.001
}
}
]
}
}
},
"prediction_reducer": {
"reduce_strategy" : {
"type": "MeanPredReduceStrategy"
},
"select_strategy": {
"type": "ArgMaxPredSelectStrategy"
}
},
"checkpoint_path": "*/checkpoint.weights.h5",
"checkpoint_monitor": "loss",
"learning_rate_on_plateau": {
"monitor": "loss",
"mode": "min",
"factor": 0.1,
"patience": 2000,
"cooldown": 5,
"min_delta": 0.01,
"min_lr": 1e-6
}
},
"compilation_args": {
"optimizer": {
"algorithm": "Adam",
"learning_rate": {
"schedule": "exponential_decay",
"schedule_args": {
"initial_learning_rate": 1e-2,
"decay_steps": 10000,
"decay_rate": 0.96,
"staircase": false
}
}
},
"loss": {
"function": "class_weighted_categorical_crossentropy"
},
"metrics": [
"categorical_accuracy"
]
},
"architecture_graph_path": "*/model_graph.png",
"architecture_graph_args": {
"show_shapes": true,
"show_dtype": true,
"show_layer_names": true,
"rankdir": "TB",
"expand_nested": true,
"dpi": 300,
"show_layer_activations": true
}
},
"autoval_metrics": ["OA", "P", "R", "F1", "IoU", "wP", "wR", "wF1", "wIoU", "MCC", "Kappa"],
"training_evaluation_metrics": ["OA", "P", "R", "F1", "IoU", "wP", "wR", "wF1", "wIoU", "MCC", "Kappa"],
"training_class_evaluation_metrics": ["P", "R", "F1", "IoU"],
"training_evaluation_report_path": "*/training_eval/evaluation.log",
"training_class_evaluation_report_path": "*/training_eval/class_evaluation.log",
"training_confusion_matrix_report_path": "*/training_eval/confusion.log",
"training_confusion_matrix_plot_path": "*/training_eval/confusion.svg",
"training_class_distribution_report_path": "*/training_eval/class_distribution.log",
"training_class_distribution_plot_path": "*/training_eval/class_distribution.svg",
"training_classified_point_cloud_path": "*/training_eval/classified_point_cloud.las",
"training_activations_path": null
},
{
"writer": "PredictivePipelineWriter",
"out_pipeline": "*/model/SFLNET.pipe",
"include_writer": false,
"include_imputer": true,
"include_feature_transformer": true,
"include_miner": true,
"include_class_transformer": false,
"include_clustering": false,
"ignore_predictions": false
}
]
}
Validation JSONs
The following JSON was used to validate the model.
{
"in_pcloud": [
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/dl_mined/train/5080_54435.laz",
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/dl_mined/test/5080_54470.laz",
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/dl_mined/test/5135_54435.laz",
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/dl_mined/test/5135_54430.laz",
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/dl_mined/test/5140_54390.laz",
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/dl_mined/test/5100_54440.laz",
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/dl_mined/test/5175_54395.laz",
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/dl_mined/test/5100_54490.laz",
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/dl_mined/test/5080_54400.laz",
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/dl_mined/test/5120_54445.laz",
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/dl_mined/test/5150_54325.laz",
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/dl_mined/test/5155_54335.laz"
],
"out_pcloud": [
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/DL_SFLNETPP_OFFLINE/T1B/pred/train/5080_54435/*",
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/DL_SFLNETPP_OFFLINE/T1B/pred/test/5080_54470/*",
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/DL_SFLNETPP_OFFLINE/T1B/pred/test/5135_54435/*",
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/DL_SFLNETPP_OFFLINE/T1B/pred/test/5135_54430/*",
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/DL_SFLNETPP_OFFLINE/T1B/pred/test/5140_54390/*",
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/DL_SFLNETPP_OFFLINE/T1B/pred/test/5100_54440/*",
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/DL_SFLNETPP_OFFLINE/T1B/pred/test/5175_54395/*",
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/DL_SFLNETPP_OFFLINE/T1B/pred/test/5100_54490/*",
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/DL_SFLNETPP_OFFLINE/T1B/pred/test/5080_54400/*",
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/DL_SFLNETPP_OFFLINE/T1B/pred/test/5120_54445/*",
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/DL_SFLNETPP_OFFLINE/T1B/pred/test/5150_54325/*",
"/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/DL_SFLNETPP_OFFLINE/T1B/pred/test/5155_54335/*"
],
"sequential_pipeline": [
{
"predict": "PredictivePipeline",
"model_path": "/oldext4/lidar_data/dales/las/dales_las/train/vl3d_framework/out/DL_SFLNETPP_OFFLINE/T1B/model/SFLNET.pipe",
"nn_path": null
},
{
"eval": "ClassificationEvaluator",
"class_names": ["noclass", "ground", "vegetation", "vehicles", "powerlines", "fences", "poles", "buildings"],
"ignore_classes": ["noclass"],
"ignore_predictions": false,
"metrics": ["OA", "P", "R", "F1", "IoU", "wP", "wR", "wF1", "wIoU", "MCC", "Kappa"],
"class_metrics": ["P", "R", "F1", "IoU"],
"confusion_matrix_normalization_strategy": "row",
"report_path": "*report/global_eval.log",
"class_report_path": "*report/class_eval.log",
"confusion_matrix_report_path" : "*/report/confusion_matrix.log",
"confusion_matrix_plot_path" : "*/report/confusion_matrix.svg",
"class_distribution_report_path": "*/report/class_distribution.log",
"class_distribution_plot_path": "*/report/class_distribution.svg"
},
{
"writer": "ClassifiedPcloudWriter",
"out_pcloud": "*predicted.las"
},
{
"eval": "ClassificationUncertaintyEvaluator",
"class_names": ["noclass", "ground", "vegetation", "vehicles", "powerlines", "fences", "poles", "buildings"],
"ignore_classes": ["noclass"],
"include_probabilities": true,
"include_weighted_entropy": false,
"include_clusters": false,
"weight_by_predictions": false,
"num_clusters": 0,
"clustering_max_iters": 0,
"clustering_batch_size": 0,
"clustering_entropy_weights": false,
"clustering_reduce_function": null,
"gaussian_kernel_points": 0,
"report_path": "*uncertainty/uncertainty.las",
"plot_path": null
}
]
}
Quantification
The table below quantifies the model trained with an offline sequencer and data mining in terms of Overall Accuracy (OA), Precision (P), Recall (R), F1-score (F1), Intersection over Union (IoU), their weighted versions (wP, wR, wF1, wIoU), Matthews Correlation Coefficient (MCC), and Cohen’s Kappa score.
Point cloud |
OA |
P |
R |
F1 |
IoU |
wP |
wR |
wF1 |
wIoU |
MCC |
Kappa |
|---|---|---|---|---|---|---|---|---|---|---|---|
5080_54400 |
96.536 |
72.783 |
73.193 |
72.81 |
64.233 |
96.597 |
96.536 |
96.54 |
93.443 |
94.739 |
94.723 |
5080_54470 |
96.244 |
78.027 |
77.272 |
77.477 |
70.402 |
96.281 |
96.244 |
96.215 |
92.829 |
93.944 |
93.902 |
5100_54440 |
95.675 |
75.525 |
73.244 |
73.733 |
65.553 |
95.831 |
95.675 |
95.667 |
91.795 |
93.205 |
93.125 |
5100_54490 |
96.561 |
77.202 |
73.593 |
75.05 |
67.003 |
96.547 |
96.561 |
96.514 |
93.454 |
94.167 |
94.134 |
5120_54445 |
96.177 |
74.971 |
79.263 |
76.848 |
69.982 |
96.277 |
96.177 |
96.187 |
92.735 |
93.774 |
93.736 |
5135_54430 |
97.569 |
72.847 |
73.935 |
73.024 |
65.142 |
97.576 |
97.569 |
97.566 |
95.332 |
96.099 |
96.094 |
5135_54435 |
97.048 |
77.538 |
75.433 |
76.261 |
69.475 |
97.068 |
97.048 |
97.043 |
94.312 |
94.625 |
94.608 |
5140_54390 |
95.064 |
75.896 |
79.22 |
77.1 |
66.61 |
95.12 |
95.064 |
95.051 |
90.723 |
91.824 |
91.792 |
5150_54325 |
95.873 |
75.784 |
72.639 |
74.017 |
66.87 |
95.903 |
95.873 |
95.824 |
92.136 |
93.428 |
93.366 |
5155_54335 |
96.922 |
58.649 |
57.303 |
57.102 |
51.729 |
96.887 |
96.922 |
96.883 |
94.103 |
94.714 |
94.705 |
5175_54395 |
96.182 |
76.246 |
76.729 |
76.437 |
68.966 |
96.203 |
96.182 |
96.17 |
92.778 |
93.893 |
93.871 |
Visualization
The figure below shows the results of the model on validation data never seen before (5135_54435).
Visualization of the references, predictions, errors, and point-wise probabilities on the validation point cloud 5315_54435 never seen before. The red color represents errors, the white color correct predictions. The probabilities are given from zero (purple) to one (yellow).
Application
This example has two main applications:
Offline sequencer example showing how to train a model with more data that can be loaded in memory at once.
Illustrate how to use data mining to improve deep learning models.