Supporting machine learning on multisensor data from airborne remote sensing

Authors: Henning Bredel and Eike Hinderk Jürrens

The PlasticObs+ Project is part of the BMUV (Federal Ministry for the Environment, Nature Conservation, Nuclear Safety and Consumer Protection) funding initiative “AI lighthouse projects for the environment, climate, nature and resources”. It investigates machine learning methods and tools on multi-sensor data from airborne remote sensing to combat plastic waste in oceans and rivers. The project is part of the funding program “AI-Lighthouses for Environmental Protection” (funding code: 67KI21014A).

PlasticObs+ is a successor of other plastic waste related projects initiated by the Deutsche Forschungszentrum für Künstliche Intelligenz (DFKI ). They lead the project consortium, which includes Optimare Systems GmbH, Jade Hochschule and everwave. 52°North was subcontracted by DFKI.

“A truckload of waste is dumped into the sea every minute”¹

The project’s goal is to develop an integrated multi-sensoric measurement system. This system should be used for routine, quasi-synoptic acquisition and visualization of the distribution of plastic debris on the ocean surface, on shorelines or coastal strips. It uses remote sensing and artificial intelligence methods. Discovered hotspots can be re-captured on-mission with higher resolution images.

In addition to the intended real-time data acquisition of plastic objects by airborne sensors, the downstream data analysis plays a central role in the research project. The main goals are to make collected data accessible via APIs, as well as enable visualization and management of the data. In addition, tools need to be developed that integrate in to existing technologies to leverage post-processing, project-scoped batch analysis, verification, and data export.

52°North’s role as a subcontractor was to support the development of a geodata management platform that enables upload, management and data processing of the raw data. This includes developing a central geo-portal, which will provide the results as web services. Finally, the platform will be accessible to the general public, stakeholders and responsible public authorities to promote sustainable solutions to environmental pollution and targeted, effective counter strategies.

Following the requirements analysis and software launch in 2023, 52°North developed a REST API to trigger an AI inference based on a given input dataset. Together with DFKI, which trained and provided the actual AI models, 52°North made each model available via that REST API. Model metadata descriptions are used to dynamically create an OpenAPI description of the REST API.

We also developed a GeoNode app based on the GeoNode Docker Blueprint, which includes a backend and a frontend extension. The frontend extension uses the react-jsonschema-form library to auto render required input parameters for a selected AI model. Auto rendering of input forms are necessary due to the dynamically created model descriptions offered by the OpenAPI. As of GeoNode v5, the new metadata editor will use the library.

The actual requests to the REST API are proxied via the extension’s backend component. As the execution of AI inferences often leads to high resource utilization, we decided that only authorized users should be permitted to use the REST API for triggering AI inferences. For this purpose, we created a corresponding permission set via database migrations, which is enforced for all requests to the proxy REST API. As soon as an inference is triggered (asynchronously), the result is uploaded back to GeoNode and linked to the input raster.

Since we use GeoNode as the data platform, other use cases are within reach. Simple GIS features, such as layer overlay, maps, area calculation of detected debris, distance measurement, etc. are directly at hand. GeoNode offers a lot of commonly used downloadable data formats and open interfaces, making data export for external data analysis quite flexible.

If you want to learn more about PlasticObs+, visit the project Website or check out the research results at DFKI. Have a look at the geonode-blueprint-docker if you plan to spin up your own GeoNode project. Let us know if that helped you organize your GeoNode development.

¹ https://www.plasticobs.de/