Model Fencing: Towards Simpler and Safer AI

With the Copernicus programme, the European Union provides rich, continuously updated Earth Observation assets. These can be accessed via the Copernicus Services as collections of files, called granules, and alternatively through CoperniCUBE offering a high-level datacube view (Fig. 1). AI-based exploitation is of high interest, and institutes as well as industry increasingly produce models for a variety of purposes, from agriculture over environmental monitoring to disaster prevention. Clearly, there is no way around AI. 

At a closer look, however, AI is not as reliable as one would wish. Figure 2 shows a crop classification in the Netherlands, performed using an ML model from Wageningen Research. While over land the classification works well, the model over water hallucinates wildly. Actually, this is understandable: there are no crops on water, so an evaluation there is inappropriate in the first place. A simple correction would be to apply a land/water mask or polygon clipping to retain only land pixels for interpretation. For a human GIS expert, this poses no problem; for non-experts, however, it typically does; and within a longer automated workflow where no human is holding hands, this situation will likely go unnoticed. This can lead to dangerous, hard to detect misjudgments.
 

These considerations made the rasdaman team, which operates CoperniCUBE, pause for thought. Rasdaman (“raster data manager”) is a distributed datacube engine for managing and analyzing massive spatio-temporal geodata. As part of a functional extension involving AI — codename AI-Cube™ — users should be enabled to upload own models for themselves and for other users for inference anywhere in the datacubes. Initial experiments integrating ML with spatio-temporal geo datacubes showed that the now very simple application of AI models often leads to them being used outside their “comfort zone”, thereby producing incorrect results. Notably, a model’s comfort zone is not limited to its training data, it simply denotes regions in space and time where satisfying results are delivered. Of course, users cannot know about this comfort zone, so there is an inherent danger in using third-party models. Closer investigation showed how unreliable even high-quality models can be. For example, BigEarthNet, trained on Sentinel-1 and Sentinel-2 satellite data, reaches 85% accuracy and more in the best case, but can also drop to 20% accuracy in the worst case.

And there are further issues: Models published on HuggingFace, Kaggle, and other platforms are typically insufficiently documented—at least not in a machine-readable form usable by the server, so human intervention is required. This requires knowledge which often is not available, frequently combined with manual preprocessing of the input data, i.e. Python programming. This effectively constrains usage to experts only.

Bottom line, it became evident that platforms like rasdaman need to provide additional safeguards for ML.

Model Fencing: Keeping AI in its comfort zone
This is where the EU EFRE funded project FAIRgeo (Fencing AI for Enhanced Reliability in Geo Services) enters the scene. In this collaboration between Constructor University and rasdaman GmbH, both located in Bremen/Germany, two central goals are pursued: (i) simplifying use of AI on Earth observation data, and (ii) automatically restricting AI applications to those datacubes and spatio-temporal areas where reliable results can be expected — so-called “model fencing”.

FAIRgeo builds on the coverage standards from ISO/OGC/INSPIRE, in particular the Coverage Implementation Schema (CIS) and the Web Coverage Processing Service (WCPS) datacube query language. “Coverages” represent the ISO and OGC concept modeling spatio-temporal geo datacubes. The rasdaman engine implements CIS and WCPS, as well as WCS, WMS, and WMTS. With an extensibility feature, rasdaman also allows the integration of external code — such as PyTorch or TensorFlow for model inference — into WCPS. Such integration has been realized in FAIRgeo for three (intentionally rather different) types of models: BigEarthNet from TU Berlin, Prithvi from NASA/IBM, and RS-Llava. Here is an example combining the WCPS query language with the natural-language capabilities of RS-Llava:

for $c in (RGB_coverage)
return rsllava.chat($c, "Describe this scene.")

The generated answer was: “A river flows through farmland, surrounded by green fields and trees on both sides.”

The coupling of rasdaman with the ML engine is done through function rsllava.chat(). For this, the server requires additional machine-readable information extracted from existing metadata and supplemented during ingestion. The STAC Machine Learning Model (MLM) Extension has been chosen as a starting point, which gets expanded as needed. This allows the server to perform a syntactic check prior to ML execution: number of parameters, parameter types, matching datacube (e.g., Sentinel-2, but not Landsat or HLS), correct bands (e.g., near-infrared, not shortwave infrared), correct patch size (e.g., 250×250 pixels), etc. This also enables the server to automatically tile arbitrary polygonal regions (such as municipality boundaries) into bites suitable for the model. Figure 3 shows a sample call of BigEarthNet in the WCPS editor and the result.

Initial performance measurements show a surprising performance advantage: in most cases, rasdaman is significantly faster than a traditional Python implementation. Figure 4 shows the measurements for the three supported models.

Another advantage is the simplification achieved: 110 lines of Python code (that the user must write) shrink to two lines of WCPS query. Additionally, rasdaman automates further steps such as data selection, data preparation, and embedding the entire process into the workflow. The data can be local or stored elsewhere in the rasdaman federation.
With this, one subgoal of FAIRgeo has been achieved already: with AI-Cubes, the use of AI is significantly simplified and, through automated parameter checking, also safer.
In the second phase, semantic model fencing will be addressed. In this broad, essentially open-ended research field, criteria for model applicability will be classified and gradually implemented. 

Examples of such criteria include:
•    Target region: land, water, rock, snow and ice, ...
•    Target time: summer, winter, monsoon, ...
•    AI-based evaluation: An interesting study by Meyer et al. at the University of Münster trains a model to create an accuracy map of geo-based AI models. This allows the expected reliability for a concrete query to be estimated.

FAIRgeo is in contact with several relevant OGC working groups, including Data Quality, Coverages, and SensorThings. Results from the project will actively be contributed to standardization efforts to enhance coverages with additional AI-relevant metadata.

Overall, with its work on AI-Cubes FAIRgeo combines two goals: simplifying AI-supported use of Earth observation data, especially Copernicus, thereby making it accessible to broader user groups without specialized programming skills. And protecting users from inappropriate application of models outside their comfort zone, even if the model itself remains a “black box” exposing only a minimal set of details. Thus, FAIRgeo aims at making AI on Big Earth Data both simpler and safer.

Author is Peter Baumann, Constructor University & CEO rasdaman