On June 2, 2022, Gaétan Bahl defended his thesis at Inria Sophia Antipolis, where it was prepared under the supervision of Florent Lafarge, within Université Côte d’Azur and STIC (Sciences & Technologies de l’Information & la Communication) doctoral school.

---

ABOUT HIS THESIS

“Deep Learning architectures for onboard satellite image analysis”

Abstract

The recent advances in high-resolution Earth observation satellites and the reduction in revisit times introduced by the creation of constellations of satellites has led to the daily creation of large amounts of image data (hundreds of TeraBytes per day). Simultaneously, the popularization of Deep Learning techniques allowed the development of architectures capable of extracting semantic content from images. While these algorithms usually require the use of powerful hardware, low-power AI inference accelerators have recently been developed and have the potential to be used in the next generations of satellites, thus opening the possibility of onboard analysis of satellite imagery.

By extracting the information of interest from satellite images directly onboard, a substantial reduction in bandwidth, storage and memory usage can be achieved. Current and future applications, such as disaster response, precision agriculture and climate monitoring, would benefit from a lower processing latency and even real-time alerts.

In this thesis, our goal is two-fold: On the one hand, we design efficient Deep Learning architectures that are able to run on low-power edge devices, such as satellites or drones, while retaining a sufficient accuracy. On the other hand, we design our algorithms while keeping in mind the importance of having a compact output that can be efficiently computed, stored, transmitted to the ground or other satellites within a constellation.

First, by using depth-wise separable convolutions and convolutional recurrent neural networks, we design efficient semantic segmentation neural networks with a low number of parameters and a low memory usage. We apply these architectures to cloud and forest segmentation in satellite images. We also specifically design an architecture for cloud segmentation on the FPGA (Field-Programmable Gate Array) of OPS-SAT, a satellite launched by ESA (European Space Agency) in 2019, and perform onboard experiments remotely.

Second, we develop an instance segmentation architecture for the regression of smooth contours based on the Fourier coefficient representation, which allows detected object shapes to be stored and transmitted efficiently. We evaluate the performance of our method on a variety of low-power computing devices.

Finally, we propose a road graph extraction architecture based on a combination of fully convolutional and graph neural networks. We show that our method is significantly faster than competing methods, while retaining a good accuracy.

Scientific publications

• Low-power Neural Networks for Semantic Segmentation of Satellite Images. Gaétan Bahl, Lionel Daniel, Matthieu Moretti, Florent Lafarge. ICCV Workshop on Low Power Computer Vision, 2019.
• In space image processing using AI embedded on system on module: example of OPS-SAT cloud segmentation. Frédéric Férésin, Erwann Kervennic, Yves Bobichon, Edgar Lemaire, Nassim Abderrahmane, Gaétan Bahl, Ingrid Grenet, Matthieu Moretti, Michael Benguigui. 2nd European Workshop on On-Board Data Processing, 2021.
• Binary Graph Neural Networks. Mehdi Bahri, Gaétan Bahl, Stephanos Zafeiriou. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021.
• Road Extraction from Overhead Images with Graph Neural Networks. Gaétan Bahl, Mehdi Bahri, Florent Lafarge. EARTHVISION CVPR Workshop 2022. arXiv preprint arXiv:2112.05215
• SCR: Smooth Contour Regression with Geometric Priors. Gaétan Bahl, Lionel Daniel, Florent Lafarge. arXiv preprint arXiv:2202.03784
• Scanner Neural Network for On-board Segmentation of Satellite Images. Gaétan Bahl, Florent Lafarge. IGARSS 2022.

ABOUT CIAR PROJECT

The CIAR project (Responsive and Autonomous Imaging Chain) studies technologies to deploy Artificial Intelligence for image processing on embedded systems (satellites, delivery drones, etc.). Three interdependent challenges are addressed: defining use cases and building relevant image database; designing AI that is both performant and well suited to embedded systems constraints; optimizing the selected algorithm and embedding them in hardware targets. For several years, the CIAR team has been collaborating with the ESA on the OPS-SAT mission. On March 22nd 2021, the team successfully uploaded a neural network on the FPGA embedded in this satellite. This illustrates the ability of IRT Saint Exupéry to tackle all these technical challenges.

JURY

Florent LAFARGE	Thesis Director	Inria, Sophia-Antipolis
Pierre ALLIEZ	Examiner	Inria, Sophia-Antipolis
Adrien GIRARD	Examiner	IRT Saint Exupéry, Toulouse
Yuliya TARABALKA	Examiner	LuxCarta
HEIPKE Christian	Reviewer	Leibniz Universität Hannover, Germany
LEFEVRE Sébastien	Reviewer	IRISA, Université Bretagne Sud, Rennes