Evaluate Point Cloud Classification Model (3D Analyst Tools)
Summary
Evaluates the quality of one or more point cloud classification models using a well-classified point cloud as a baseline for comparing the classification results obtained from each model.
Usage
To obtain the best evaluation results, the reference point cloud must be well classified for the objects of interest that are being evaluated.
All input models being evaluated must have the same classification codes. If the reference point cloud does not have class codes whose values or meaning match the classes in the models being evaluated, use the Point Cloud Class Remap parameter to designate the reference point cloud's classification with the codes in the trained models.
The PointCNN deep learning architecture is used to create the point cloud classification model. This architecture may yield slightly different results when processing the same set of input data. You can specify the same model as an input more than once to evaluate the consistency of the classification result from the point cloud. See PointCNN: Convolution On X-Transformed Points for more information on the potential for variability in the classification results.
The evaluation process will create several output files in the Target Folder parameter value. The name of each file will begin with the text specified in the Base Name parameter value.
<base name>_ModelStatistics.csv—A table listing each model's overall accuracy, precision, recall, and F1 score.
<base name>_ClassCodeStatistics.csv—A table summarizing the evaluation results for each class code's accuracy, precision, recall, and F1 score.
<base name>_ConfusionMatrices.csv—A table summarizing how each point from each class code in the reference point cloud was classified by the input models. In addition to the number of true positives and false positives, this table also identifies the classes that the false positives were occurring in.
<base name>ConfusionMatrix<model number>.png—An image showing a chart that provides an easy to read depiction of the confusion matrix for a given model.
The Reference Surface parameter is required when the input model was trained with relative height attributes. The raster surface is used as a reference height from which relative heights are interpolated for each point. This provides additional information for the model that can be used to more readily differentiate objects. The raster surface provided for this parameter should represent the same type of data as the raster that was used in the training data that created the model. In most cases, this will be a raster created from ground classified points. A raster surface can be generated from the ground classified points in the LAS dataset by applying a ground filter and using the LAS Dataset To Raster tool. A ground surface can also be generated from a point cloud scene layer using the Point Cloud To Raster tool. Raster surfaces that are not sourced from the input point cloud can also be used, but you must ensure that the z-values in the raster correspond appropriately with the z-values in the point cloud.
When the input model was trained with points from certain classes that were excluded from the training data, use the Excluded Class Codes parameter to ensure that those points are omitted from the set of points evaluated by this model. Excluding classes that do not provide a useful context for the objectives of a given model will reduce the number of points that are evaluated, which will improve the speed of training and applying the model. For example, points representing buildings will usually have no relevance for points that represents objects such as traffic lights, power lines, or cars. Building points can also be reliably classified using the Classify LAS Building tool. If points with class 6, which represents buildings, were excluded in the training data that was used to create the model, the input point cloud must also classify building points and have them excluded in this tool.
Parameters
| Label | Explanation | Data type |
|---|---|---|
|
Input Model Definition |
The point cloud classification models and batch sizes that will be used during the evaluation process. Value table columns:
|
Value Table |
|
Reference Point Cloud |
The point cloud that will be used to evaluate the classification models. |
LAS Dataset Layer; File |
|
Target Folder |
The directory that will store the files which summarize the evaluation results. |
Folder |
|
Base Name |
The file name prefix that will be used for each of the output files summarizing the evaluation results. |
String |
|
Processing Boundary (Optional) |
The polygon feature that delineates the portions of the reference point cloud that will be used for evaluating the classification models. |
Feature Layer |
|
Point Cloud Class Remapping (Optional) |
The class codes from the reference point cloud must match the class codes in the models being evaluated. When the class codes do not match, use this parameter to associate the differing class codes in the point cloud with the classes that are supported in the models being evaluated. Value table columns:
|
Value Table |
|
Reference Surface (Optional) |
The raster surface that will be used to provide relative height values for each point in the point cloud data. Points that do not overlap with the raster will be omitted from the analysis. |
Raster Layer |
|
Excluded Class Codes (Optional) |
The class codes that will be excluded from processing. Any value in the range of 0 to 255 can be specified. |
Long |
Derived output
| Label | Explanation | Data type |
|---|---|---|
|
Output Confusion Matrices |
The CSV format table storing the confusion matrix for each class code in each input model. |
Text File |
|
Output Model Statistics |
The CSV format table that summarizes the overall statistics of the input models. |
Text File |
|
Output Class Code Statistics |
The CSV format table that summarizes the statistics for each class code in each input model. |
Text File |
Environments
Extent, Processor Type, GPU ID, Output Coordinate System, Geographic Transformations
Licensing information
- Basic: Requires 3D Analyst
- Standard: Requires 3D Analyst
- Advanced: Requires 3D Analyst