Blog

DOCTORAL THESIS: Underwater Image-Based 3D Reconstruction With Quality Estimation

By Klemen Istenic

Supervised by Dr.Rafael García Campos/ Dr.Ricardo Estrela Gracias

 

Abstract

Despite the undeniable importance of the marine ecosystem, vast areas of the seabed remain largely unexplored. Accurate and detailed 3D models of the environment yield high added value to any marine survey, as such results convey immense information easily interpretable by humans.

This thesis presents a comprehensive investigation and use of image-based 3D reconstruction techniques for enabling experts (biologists, archaeologists and geologists, among others) to carry out further in-depth investigations of the areas of interest after the missions. It focuses on the research and development of resources for accurate scaling and uncertainty estimation of 3D models for scientific purposes based on data acquired with monocular or un-synchronized camera systems in difficult-to-access GPS-denied (underwater) environments.

The developed image-based 3D reconstruction framework was built by integrating several open-source solutions and by developing additional functionalities (e.g. navigation fusion, soft enforcement of known relative constraints, underwater color correction). It enables the creation of photo-realistic textured 3D models based on optical and navigation data and is independent of a specific robot or diver-based platform, camera or mission. Such versatility enabled us to use the framework for creating 3D models in numerous collaborations with different scientists and researchers and for different purposes (accurate measurements, generation of superior omnidirectional video, augmented reality applications). Specifically, processing of data from the SUBSAINTES scientific cruise led to the creation of one of the largest collections of image-derived underwater 3D models ever made by deep-sea vehicles for geological purposes.

This dissertation also presents two novel methods for automatic scaling of Structure from Motion-based 3D models using commonly available laser scalers. The methods are able to compensate for the geometry of the scene together with the geometry of the laser scaler setup while simultaneously estimating the confidence level of each scale estimate. By removing several manual, time-consuming and error-prone steps as well as limitations of previous approaches, the proposed solutions significantly expand the applicability potential of scaling models using laser scalers. Furthermore, as both methods are not tied to data collected with specific sensors or hardware they are considered universal and can even be used with diver-based surveys as well as with a wide variety of legacy data.

By utilizing the newly proposed scaling methods, we performed one of the most in-depth scale error analysis of large-scale models of deep-sea underwater environments. Various survey and reconstruction scenarios were evaluated and used to determine the best practices of deep sea image-based 3D reconstruction. In addition, the analysis provided a precise estimates of the accuracy of the models reconstructed using the SUBSAINTES data and thus enabled the geologists from Paris Institute of Earth Physics to use these models in their respective studies. Previously performed analyses using a single model are now being performed with information from more than 30 different sites with 41 different models.

Finally, we also made a contribution by demonstrating the feasibility of mission-time 3D reconstruction and uncertainty estimation. We proposed a novel SfM-based system capable of producing a globally consistent 3D reconstruction together with an estimate of its uncertainty while the robot is still in the water or shortly after. This enables the possibility of quality-conscious data acquisitions, which consequently will not only increase the quality of the final data collected and the efficiency of the survey, but also reduces the possibility of conducting unsatisfactory optical surveys.