Three gaps for foundation models to address in satellite remote sensing of inland waters

Mortimer Werther, Stefan Simis, Jan Dirk Wegner, Daniel Odermatt

Long-term Earth observation of inland waters depends on continuity between satellite missions, adequate combinations of spectral, spatial and temporal resolution, and representative calibration and validation datasets. Discontinuities and trade-offs in these components constrain the records available for studying inland waters and their change. We examine three gaps affecting inland water remote sensing: a mission gap between comparable 300 m ocean colour observations from MERIS and OLCI during 2012–2016, a coverage gap in spectral, spatial and temporal sampling, and a generalisation gap when retrieval algorithm accuracy declines beyond bio-optical properties and atmospheric conditions represented during development. Foundation models (FMs) could help address these gaps by learning reusable representations from satellite archives before adaptation to a particular retrieval task. We hypothesise that relationships learned across overlapping satellite missions can support reconstruction of what a sensor would have measured during the mission gap, that combining complementary sensors can extend usable observation coverage, and that pretraining on satellite archives, which cover more water bodies and a wider range of bio-optical properties and atmospheric conditions than available field datasets, can improve retrieval algorithm generalisation. Testing these hypotheses requires criteria that define progress. We quantify the three gaps, define a progress criterion for each and assess 100 recent machine learning studies against the criteria. For the mission gap, three studies address continuity across sensor eras, and none recovers a record from a withheld mission period. For the coverage gap, 23 studies combine complementary observations, but only three test them on withheld water bodies or periods. For the generalisation gap, only seven studies withhold whole water bodies or later periods from all model development. Across all three gaps, none of the six studies that use pretraining compares it with the same architecture trained without pretraining on independent water bodies. The assumption common to all three hypotheses, that archive pretraining adds information beyond established methods, therefore remains untested. Applying the criteria to existing generic Earth observation FMs, then to FMs pretrained on inland waters, would determine which gaps pretraining can reduce.