FFG Projektdatenbank

zurück

AI-Mars-3D

AI-Based Shape Refinement for Planetary Exploration

Projektbeschreibung

Zwei planetare Rover, Perseverance (Mars 2020 Rover) und Curiosity (MSL Rover), sind derzeit mit direkter Beteiligung österreichischer Teams auf dem Mars im Einsatz. Die Sensoren dieser Rover ermöglichen eine 3D-Sicht bis zu mehreren Dutzend Metern entlang ihrer Fahrtrouten. Allerdings nimmt die Auflösung quadratisch mit der Entfernung ab, was die Wirksamkeit geologischer Bewertungen über 30-50m hinaus einschränkt. Digitale Geländemodelle (DTMs) von Satellitensensoren wie HiRISE decken größere Oberflächenbereiche ab, bieten jedoch nur Auflösungen bis zu 1m strukturell und 25cm textuell. Eine bedeutende Herausforderung, die AI-Mars-3D addressiert, ist die Auflösungslücke zwischen diesen satellitenbasierten DTMs und 3D-Modellen, die aus Rover-Bildern abgeleitet werden.

AI-Mars-3D setzt innovative KI-basierte Methoden ein, um diese Lücke zu überbrücken und die Auflösung von 3D-Modellen sowohl aus Satelliten- als auch aus Rover-Bildern zu verbessern. Das Projekt hat zwei Hauptziele: Erstens, hochauflösende 3D-Modelle zu entwickeln und zu validieren, die die Fähigkeiten bestehender satelliten- und roverbasierter 3D-Rekonstruktionsmethoden übertreffen. Dieser Prozess beinhaltet rigorose Echtdatenstatistiken und Verfahrensverbesserungen in Zusammenarbeit mit den Urhebern der Technologie. Zweitens zielt AI-Mars-3D darauf ab, diese KI-gestützten Methoden zu validieren, um ihre Zuverlässigkeit und Objektivität bei der Unterstützung wissenschaftlicher Erkenntnisse sicherzustellen. AI-Mars-3D nutzt Nahbereichsbeobachtungen der Roverbilder als Echtdaten-Datenbank, um die Auswirkungen und Akzeptanz neuer KI-Methoden in der planetaren Erforschung zu erhöhen. Zusätzlich zum planetaren Anwendungsfall werden dieselben Methoden und Validierungsstrategien auf terrestrische Anwendungen in Bereichen wie Tunnelbau und Verkehrsüberwachung ausgedehnt, wobei industrielle Partner einbezogen werden, um deren kommerzielle Machbarkeit und Auswirkungen zu bewerten.

Abstract

Two planetary rovers, Perseverance (Mars 2020 Rover) and Curiosity (MSL Rover), are currently operational on Mars with direct involvement from Austrian teams. These rovers' sensors enable 3D vision up to several tens of meters along the rovers’ trajectory. However, their resolution decreases quadratically with distance, limiting geological assessments' effectiveness beyond 30-50m. While Digital Terrain Models (DTMs) from satellite sensors like HiRISE cover larger surface extents, they provide resolutions only down to 1m structurally and 25cm texturally. A significant challenge that AI-Mars-3D addresses is the resolution gap between these satellite-based DTMs and 3D models derived from rover imagery.

AI-Mars-3D employs innovative AI-based methods to bridge this gap, enhancing the resolution of 3D models from both satellite and rover imaging. The project has two primary objectives: firstly, to develop and validate high-resolution 3D models that surpass the capabilities of existing satellite and rover-based 3D reconstruction methods. This process involves rigorous ground truth statistics and procedure refinement in collaboration with the technology's originators. Secondly, AI-Mars-3D aims to validate these AI-enhanced methods to ensure their reliability and objectivity in supporting scientific findings. AI-Mars-3D leverages close-range rover image observations as a ground truth database, aiming to increase the impact and acceptance of novel AI methods in planetary exploration. In addition to the planetary use case, the same methods and validation strategies will be extended to terrestrial applications in fields such as tunneling and traffic monitoring, involving industrial partners to assess their commercial feasibility and impact.

Endberichtkurzfassung

The ASAP project “AI-Mars-3D” investigated and tested the use of AI methods to enhance images taken on Mars, both from rovers and orbiters, enabling scientists to perform more informed scientific analyses based on 3D models generated from these images, and to expand the scientific usability of image data from planetary surfaces.

The project pursued two complementary objectives. First, although AI-based super-resolution restoration methods have already been applied to satellite image data (specifically from the HiRISE sensor aboard the Mars Reconnaissance Orbiter), their scientific usability could not be assessed due to the lack of robust validation approaches. In the project, we therefore developed reliable validation strategies for these products. Second, we addressed the inherent resolution gap in rover-based stereoscopic reconstruction (compared to satellite data), where image quality/resolution – and thus depth accuracy – decreases quadratically with distance, leading to unreliable 3D information beyond short distances.

As part of the first objective, several iteration cycles with partner Surrey AI Imaging (SAIIL) resulted in AI-enhanced HiRISE tiles that were made available to the Mars 2020 and MSL teams. In addition, detailed validation was performed by comparing standard HiRISE products and AI-enhanced HiRISE products with full-resolution Mastcam Z data that was precisely embedded in the Martian landscape. This analysis showed that significant sedimentary layering and eroded block structures are much more clearly resolved in the SSR DTM products than in the original HiRISE reconstructions. Extensions of the PRo3D visualization tool by VRVis and 3D data fusion mechanisms by JOANNEUM RESEARCH were used for this purpose.

In the second topic area, a current approach to monoscopic depth estimation (MDE) was applied to derive pixel-by-pixel depth information from individual rover images using AI, rather than resorting to classic stereo reconstruction. Although the method showed very good results on rover image data, the estimated depths are only relatively scaled. To solve this problem, a non-uniform scaling algorithm was developed that adapts the MDE depth distributions to reference depth images from traditional stereo reconstructions. Analogous to the first objective, a robust validation strategy was developed, including supporting tools ranging from visual inspection to detailed statistical analyses of 3D surfaces and comprising a semi-automatic co-registration workflow.

The results show that AI-based methods do not outperform conventional methods at close range, but offer clear qualitative advantages beyond approximately 10–20 m, where stereo reconstruction loses quality and AI approaches become advantageous despite higher sensitivity to lighting conditions. Beyond this distance, the effective stereo range can be significantly extended, enabling geological annotations even at long range.

A central element of the project was close cooperation with industry partners from the fields of traffic monitoring (SLR Engineering), tunnel surveying (Dibit Messtechnik), and Earth observation (SAIIL). Within this framework, specific use cases for AI methods were identified and jointly investigated. The findings revealed areas where further research or validation is needed and can be incorporated into future product developments after further R&D activities.

zurück