Multi-annual sub-pixel land cover mapping with TESSERA latent embeddings

Franz Schug , David Klehr, Jari Mahler, David Frantz

Mapping land cover in highly heterogeneous landscapes is challenging, and classifications have inherent limitations where the spatial resolution of remotely sensed data exceeds the size of small objects. Sub-pixel land cover fraction maps based on medium-resolution optical remote sensing data like Landsat or Sentinel-2 overcome this limitation but are often not consistently available across multiple years because creating temporally robust and comparable multi-class fraction models can be challenging. Latent embeddings from geospatial foundation models are promising as they represent high-dimensional, large-scale general-purpose spectral, temporal, and structural features derived from Earth Observation data that are largely independent of available high-quality observations at specific points in time. We here assessed the usability of TESSERA latent embeddings for machine-learning regression-based land cover fraction models and compared the performance to more conventional spectral-temporal metrics and spline coefficients as input features. We also assessed the suitability of these inputs to train temporally transferred and generalized fraction models. We found that TESSERA latent embeddings are a well-suitable input to land cover fraction mapping, performing slightly better than spectral-temporal-metrics-based models in many cases, both being notably outperformed by spline coefficients (MAE across classes and years 11.58 vs. 11.64 vs. 9.45). Spline coefficients also performed best in all temporally generalized and many temporally transferred models. However, models trained with TESSERA latent embeddings demonstrate the highest consistency of transferred models, highlighting their capability to handle data gaps. We suggest that TESSERA latent embeddings are a valuable input to fraction mapping where data availability across years is highly variable, but spline coefficients generally perform best when sufficient high-quality observations are available.