Complex Objects

In this example, we are demonstrating the detecting of the positions of multiple complexly objects in a 3D point cloud.

The Challenge

Here is a picture what we are actually dealing with. The goal is to accurately detect the position and orientations of these objects so that a robot can pick them. In other words a robot should be able to take one part after the other from this heap and put it into a machine for further processing.

As you can see, The object has a quite complex form, and it is even a bit difficult for a human to see what constitutes a single object. We have applied CANDELOR to the problem to get a feel how well it performs with that kind of complexity.

Applying CANDELOR

The workflow to get recognition results is this: First, you need some data to perform experiments and scans on. CANDELOR requires three different inputs:

  • A 3D pointcloud to actually perform recogntion on
  • a CAD model that should be detected in the point cloud
  • a configuration for detection parameters

Given a CAD model of the object that should be detected, CANDELOR can analyze the object in a preprocessing step and automatically generate parameters that are optimized for detecting the given geometry. This process takes a few seconds. The generated configuration can now be directly used as input for the detection algorithm. It is also possible to fine tune these settings by hand for even better recognition rates.

Finding Objects

Armed with this configuration, CANDELOR is loaded and ready for object localization. Each time you call CANDELOR with a point cloud, it will perform the detection and return the resulting positions and orientations of the detected objects. The results of the detection can be directly used programatically, or used for visualization.

Here is a screenshot of the point cloud we are using for this test:

CANDELOR detects multiple instances of the same object simultanuously. It is often important for prepressing steps to detect as much objects as possible, for example in this use case a robotic picking system has to grab an object. The more objects are correctly detected, the easier it is for the grasping system to select an object that is best suitable for grasping. A high detection rate is also important to prevent collisions: Without detection, it is not known how the objects are actually entangled with each other.

Here is a image of the original scan:

On a standard desktop computer (here a Intel Q9550, with 4 cores), the objects are recognized in approximately 2 seconds with a parameter set that has not been tuned by hand yet. The algorithm scales well with the number of CPUs, that means the more cores are available the faster CANDELOR will be. It is possible to specify a calculation time for the algorithm. The more time available, the more objects will be found and the higher will be the precision. This makes it possible to scale the recognition down to real time recognition, and up to detection hundreds of objects by modifying just one parameter.

Robustness & Difference Analysis

All of the objects in the above scene are occluded to varying degree. CANDELOR is robust against incomplete scans. That means it is possible to correctly detect an objects position and orientation despite missing data, to a degree. This robustness extends to inaccurate models. In this particular scans, some of the real objects are acutally manufactured slightly different: one side is mirrored. This is very hard to see without help. CANDELOR makes it possible to provide data for difference analysis:

If you look closely at the above screenshots, you see that the grey CAD model has two elevated cylindrical parts on the left side, but not on its right side. The point cloud on the other hand indicates the opposite: The yellow/greenish dots on the right side indicate that there are points from the scan at a position where the CAD model does not have any structures. The parts of the scan that are a good match with the found object are colorized in blue.

As a summary, in this example CANDELOR has been used to for a robust detection of highly complex objects in a scan of bulk material. The configuration is performed automatically, and evaluation results indicate that some parts in the bulk material are actually not the correct version of the part, but the mirrored version. This analysis can be done very efficiently with the help of CANDELOR’s algorithms.