Article published on 05/17/2024

Anomaly Detection on Satellites for Marine Environmental Protection

The oceans host a vast diversity of marine species and are a vital component of global biodiversity. Advanced technologies, such as satellite imaging and Artificial Intelligence (AI), can help scientists identify and address environmental threats to this ecosystem.

In this context, the IRMA project by IRT Saint Exupéry offers an anomaly detection solution designed for maritime imagery, intended to be deployed on Earth observation satellites. This solution allows for the rapid detection of a wide variety of threats to marine ecosystems, such as oil spills, harmful algal blooms, and sediment discharges.

The proposed approach is innovative as it requires few annotated data, can operate on embedded targets with low computational power, and is generic for all types of anomalies that differ from the normal state of water. These qualities make this method adaptable to various missions, which has earned it preselection in two competitions organized by the European Space Agency to integrate artificial intelligence onboard Φsat-2 and IMAGIN-e.

For more details about our solution, you can read our article published in the IEEE « Journal of Selected Topics in Applied Earth Observations and Remote Sensing » (JSTARS).

1. About the ESA Missions Φsat-2 and IMAGIN-e

The Φsat-2 and IMAGIN-e challenges are open competitions by the ESA promoting Edge Computing in orbit. Each challenge is a call for proposals for innovative solutions combining embedded AI and Earth observation. The proposed solutions must present social, environmental, and economic interest and be compatible with the orbits and characteristics of the missions.

Figure 1 Logos of the two missions. Images credit: ESA

The Φsat-2 mission involves the deployment of a 6U CubeSat in a sun-synchronous orbit at an altitude of 500 km. The payload consists of a sensor with a 5-meter resolution with 8 spectral bands and a CogniSat AI processor integrating an Intel Movidius Myriad2 accelerator. The satellite is scheduled to launch in early July 2024.

Figure 2 Presentation of the Φsat-2 mission: Cubesat 6U and mission areas of interest (mostly maritime). Images credit: ESA

The IMAGIN-e mission is onboard the International Space Station. Launched on March 21, 2024, the payload consists of a hyperspectral camera with a 50-meter resolution and 50 spectral bands, and 16 ARM Cortex-A72 cores integrated into a Microsoft Azure SDK.

Figure 3 IMAGIN-e mission overview. Images crédit ESA

Among hundreds of initial teams, our solution proposal is among the finalists for potential deployment on these two missions.

2. About the IRT Saint Exupéry Technical Solution

For these two missions, we propose a common solution based on an AI pipeline composed of a self-supervised encoder and data- and resource-efficient models, enabling deployment in orbit for a variety of missions.

Figure 4 Solution architecture

Based on a “normality” concept encompassing a set of images without ocean-impacting events, it offers a generic approach to detect a large number of potential threats. We have also integrated a network to specifically characterize certain anomalies (oil, harmful algae, sediments, etc.).

This solution allows for various operational use cases such as:

• Image Prioritization: Images with the highest anomaly scores are uploaded first by the satellite. This allows monitoring of vast maritime areas while enabling operators to analyze only the images containing the most crucial information.
• Alert Sending: In the event of a major incident at sea, such as an oil spill, an immediate alert is sent to facilitate a faster response by the relevant authorities, mitigating environmental, social, and economic impacts.

Figure 5 Inference results

A link to the notebook containing the codes and test images is available here:

• Notebooks for both missions,
• Dataset of representative images for the IMAGIN-e mission,
• Dataset of representative images for the ΦSat-2 mission.

The codes are distributed under GNU General Public License.

The test data is distributed under Creative Commons BY-NC-SA license.

3. About the IRMA Project

The work presented here is conducted within the framework of the IRMA project (“AI for Mission Reactivity based on Image Analysis”) by IRT Saint Exupéry. The IRMA project aims to develop AI-based technological components for mission planning and data processing, onboard or on the ground, to meet new market needs for satellite Earth observation, such as transforming raw data into interpreted information, and reactive management of satellite constellations.

IRMA also aims to demonstrate the effectiveness of these technologies through system demonstrations, explaining the project’s participation in the “OrbitalAI” challenges by the European Space Agency (ESA).

We express our gratitude to the industrial and academic partners of the project: Thales Alenia Space, Activeeon, Geo4i, JoliBrain, and the University of Côte d’Azur. We also thank the European Space Agency for allowing our team to participate in the OrbitalAI challenges.

Figure 6 Operational concept of the IRMA project