ITECA is the developer of the WITTY3D software, which is a maintenance aid solution for industrial machines.
An AI assists in the diagnosis and resolution of faults. The breakdowns are displayed on the 3D model of the machine and animations illustrate the resolution procedure.
This software is now deployed on desktop PCs, laptops, tablets and smart-phones.
Compared to text documents and 2D diagrams, the 3D display allows technicians to more easily interpret the information displayed even if the problem does not completely disappear. Indeed, they still need to interpret and transpose the virtual information presented to them to the reality on the ground. However, any interpretation requires a cognitive effort that is a source of errors.
In order to overcome these limitations, ITECA wishes to develop WITTY3D by using mixed reality headsets such as the Microsoft Hololens.
These headsets use transparent screens to display images directly in front of the user's eyes. The perspective effect is then perfect and allows the virtual visuals to be traced to the real environment, which remains visible through the helmet's visor. These helmets provide the display hardware but it is up to the developer to provide the software tools to correctly align the visuals.
Specifically, the software must infer the position and orientation of the 3D objects of interest relative to the helmet. This problem is known in the literature as estimating the six degrees of freedom of an object (6DoF estimation).
One solution to this problem is to use neural networks to identify, localize in the image, and localize in 3D, the object of interest.
In this thesis, we considered AlexNet, VGG-19 and GoogLeNet for image classification, YOLO for image localization, and PoseNet and BB8 for 6DoF estimation.
We evaluated them on three state-of-the-art datasets for 6DoF estimation: LINEMOD, T-LESS and YCB-VIDEO.
We also built our own dataset, called NEMA.
Like LINEMOD and T-LESS we use markers to obtain quality labels. But, unlike LINEMOD and T-LESS, our dataset provides the backgrounds to erase these markers.
We also photographed the images with a finer pitch, which allows us to get more images, and very dense viewpoints.
Thus, we can train PoseNet and BB8 without problems to accuracies never reached before.