3DMASC

3DMASC is a plugin for CloudCompare developed by the LiDAR platform in Rennes. The general documentation is here. In the following section we present the work we did in Python to help you run 3DMASC or to optimize a classifier in an advanced workflow.

The plugin can be called in command line mode and there are 4 options for that:

  • 3DMASC_CLASSIFY the main option for calling 3DMASC in command line, it is mandatory in any case.

  • ONLY_FEATURES do not train the classifier, just compute the features (used when the training is to be done with Python).

  • KEEP_ATTRIBUTES at the end of the computation, the features will be stored.

  • SKIP_FEATURES do not compute the features (used when the features have already been computed).

What you can do in command line depends on what you have in store:

  • I have a valid parameter file

    • I can compute features -3DMASC_CLASSIFY -ONLY_FEATURES

  • I have a classifier as saved using 3DMASC graphical user interface (classifier.txt + classifier.yaml (yaml being saved under the hood))

    • I can compute features and apply my classifier -3DMASC_CLASSIFY

    • I can compute features, apply my classifier and save the features -3DMASC_CLASSIFY -KEEP_ATTRIBUTES

    • I can compute features and stop -3DMASC_CLASSIFY -ONLY_FEATURES

    • I can apply my classifier without computing features if they are already available -3DMASC_CLASSIFY -SKIP_FEATURES

The plugin can also be called from Python using a helper function of lidar_platform.cc which builds the command line for you and run it using subprocess.

from lidar_platform import cc

Basic actions

Classify

Once you have build your classifier with CloudCompare, it is possible to apply it to another set of data using the library.

The following Python code:

clouds = (pc1, pc2, core)  # pc1, pc2 and core are full paths to clouds
out = cc.q3dmasc(clouds, classifier)

is equivalent to this command:

CloudCompare.exe -SILENT -NO_TIMESTAMP -C_EXPORT_FMT sbf -o -GLOBAL_SHIFT AUTO pc1 -o -GLOBAL_SHIFT AUTO pc2 -o -GLOBAL_SHIFT AUTO core -3DMASC_CLASSIFY classifier.txt C3HD=2 C2HD=1 CORE=3

Be carefull to the order of the clouds in the classifier file, it may differ from the order in the parameter file. You should take care of the coherence of the order of the cloud list, either using Python or the command line when calling 3DMASC.

Compute features

If you want to only compute the features on a specific point cloud, the following Python code:

clouds = (pc1, pc2, core)  # pc1, pc2 and core are full paths to clouds
out = cc.q3dmasc(clouds, parameters, only_features=True, verbose=True)

is equivalent to this command:

CloudCompare.exe -SILENT -NO_TIMESTAMP -C_EXPORT_FMT sbf -o -GLOBAL_SHIFT AUTO pc1 -o -GLOBAL_SHIFT AUTO pc2 -o -GLOBAL_SHIFT AUTO core -3DMASC_CLASSIFY -ONLY_FEATURES 3dmasc_parameters.txt C2HD=1 C3HD=2 CORE=3

If your parameter file is named 3dmasc_parameters.txt, a file with the name 3dmasc_parameters_feature_sources.txt is created by CloudCompare, note that this file is used by other functions of the library.

Import a point cloud with features

Once the features have been calculated, and if you have chosen to save the results in sbf you can easily import the cloud with its features using the lidar_platform.sbf module. Features are stored as scalar fields by CloudCompare.

from lidar_platform import sbf
sbf_data = sbf.read(sbf_file)  # sbf_data is an object of type SbfData
xyz = sbf.pc  # coordinates of the points
scalar_fields = sbf.sf  # scalar fields as a NumPy array
scalar_fields_names = sbf.sf_names  # scalar fields names

When 3DMASC computes one feature, it may also compute intermediate scalar fields which are used to compute the feature itself. So all scalar fields are not features. For instance, when computing DimZ_C2HD_MODE_MINUS_DimZ_C3HD_MODE@1.5, 3DMASC needs to compute DimZ_C3HD_MODE@1.5 which is also stored in the scalar fields but not used as a feature unless you specified it explicitly in the 3DMASC parameter file.

This is where the file 3dmasc_parameters_feature_sources.txt created during the feature calculation is important, it let you know exactly which features were used to train the classifier. This file is used by the function classification.cc_3dmasc.load_sbf_features to load only the features related to the classifier.

Once the features have been calculated, it is possible to load them in Python, jointly with the classifier, to classify data directly in Python.

Classify in Python with OpenCV

When saving a classifier, CloudCompare saves two files: a txt file which contains a reference to the classifier itself which is in YAML (this one can be loaded in Python with the cv2 module). The yaml file can be loaded with cv2.

To run the following lines of code, you will need to compute the features on your point cloud as presented above. A file 3dmasc_parameters_feature_sources.txt is mandatory, it should be in the same directory as 3dmasc_parameters.txt.

import cv2
from lidar_platfor import classification, sbf

# Load features from sbf
features_data = classification.cc_3dmasc.load_sbf_features(
    point_cloud_WITH_FEATURES.sbf,
    3dmasc_parameters.txt)
x_test = features_data['features']

# Load a classifier and apply it
cls = cv2.ml.RTrees_load(opencv_classifier)  # load
_, y_pred = cls.predict(x_test, flags=cv2.ml.DTREES_PREDICT_MAX_VOTE)  # apply

# Read the data from the sbf file
sbf_data = sbf.read(test_with_features)

# Add an extra scalar field called class_python to the existing ones
sbf_data.add_sf("class_python", y_pred)

# Save the point cloud in sbf format
sbf.write(output_file, sbf_data.pc, sbf_data.sf, sbf_data.config)

Advanced workflow

For more advanced classification problems, a possibility is to use our Python scripts, which allow you to dive deeper into the classifier details.

To replicate classifier optimization as described in the original 3DMASC paper, you can follow the steps below, which require having the lidar_platform Python package installed and working.

First steps in Python with 3DMASC

  1. Create your parameter file as explained above.

  2. Compute the features on your training point cloud using the command line. Using the command line interface instead of the GUI will automatically generate a file with a name ending by _feature_sources.txt. This file is mandatory to use the workflow. Save the resulting point cloud (with the features) in the SBF format. Repeat this operation on your test point cloud and your core points point cloud.

  3. Import lidar_platform.classification

  4. Load the different point clouds in Python using classification.cc_3dmasc.load_sbf_features()

Training and evaluating a classifier in Python

  • use classification.cc_3dmasc.train() to train a random forest model

  • use classification.cc_3dmasc.get_acc_expe() to train a random forest model and obtain multiple metrics quantifying its performance

  • use classification.cc_3dmasc.test() to evaluate your model on a test dataset

In this module, you can also find functions to visualize the random forest feature importance, heatmaps of inter-feature linear correlation, or a SHAP summary plot.

Classifier optimization in Python

The optimization procedure detailed in the 3DMASC paper consists of iteratively pruning a set of features and scales using embedded random forest metrics and using the variation in classification performance to identify an optimal set of predictors. In the module lidar_platform.classification.feature_selection, three functions allows you to get automatically an optimized set of predictors: rf_ft_selection, get_n_optimal_sc_ft and get_best_rf_select_iter. In this module, you will also find multiple scripts to perform other operations such as selecting a given number of uncorrelated features or scales.

Shifting between GUI and Python for classifier training and application

In some cases, it can be interesting to shift from the CloudCompare GUI to Python or vice-versa. In particular, when using very large datasets, it can be useful to perform classifier training in Python rather than in the GUI. In any case, a classifier that is trained in Python can be used in CloudCompare and vice-versa.

Namely, to load in Python a classifier saved from the q3DMASC plugin, use:

import cv2
cl = cv2.ml.RTrees_create()
cl = cl.load('your_classifier.yaml')

When using Python, you can also save a trained OpenCV random forest model and use it in the CloudCompare plugin (using the save function of cv2.ml.RTrees).

Since you cannot open a .yaml file directly in the 3DMASC classify tool, you will have to indicate the path to the classifier at the top of the parameters file (when doing all the process in the plugin, after training, the classifier is saved and the parameters file is automatically updated with the path of the .yaml model, which enabled the plugin to find it. Here, it is necessary to do it manually). For that, simply add these lines at the top of the parameters file, and then use 3DMASC classify as usual.

# 3DMASC classifier file
classifier: classifier.yaml

(replace bolded text with adapted path and file name).