Labels, Thresholds and the Reference Itself: Evaluating Sentinel-1 Flood Segmentation Against Copernicus EMS Maps, with a Preregistered Blind Test

Teo Clerici Jurado 

Flood-segmentation models for Sentinel-1 synthetic aperture radar (SAR) are usually ranked by cross-validation on benchmark chips whose labels share sensors, dates and definitions with the model inputs. We measure how far such scores carry to operational reference maps, what limits them there, and whether adapting a model to the reference helps. The reference is Copernicus Emergency Management Service (CEMS) rapid mapping: the delineations of 38 flood activations from 2023 to 2026 drawn on non-Sentinel radar satellites, against which every model is applied, without training on these events, at its own calibrated threshold. A gated cross-attention SAR–optical fusion model that scores 0.608 IoU in leave-one-event-out cross-validation on Sen1Floods11 scores 0.266 on these maps, below a physics baseline (0.373); trained on Kuro Siwo's flood labels, the same architecture scores 0.407. Our v1.2 model adds a seasonal Sentinel-1 reference and is trained on 7,006 chips from 138 floods (Kuro Siwo and GEOID-Flood). It scores 0.417 IoU, against 0.405 for TerraMind-base adapted with LoRA to the same labels and tuning budget, and 0.419 against 0.321 for Copernicus's Global Flood Monitoring (GFM) ensemble on the 34 activations GFM covers, with GFM's exclusion mask scored as dry. At each activation's best threshold in hindsight, our model would reach 0.468. The reference is itself uncertain: consecutive CEMS delineations of the same area, made 6–24 h apart, agree with each other at 0.520 macro IoU over 34 activations; on the 27 activations in both sets, the model scores 0.433 and the consecutive products 0.536. Adapting to the reference does not close the remaining gap. In grouped cross-validation over the 38 activations, a threshold refitted on the other folds gains 0.002, and fine-tuning on their CEMS labels loses 0.041 (0.380 against 0.421), so an adoption rule fixed in advance adopted neither. Because this benchmark had been seen during development, all of these comparisons are exploratory. The confirmatory test is preregistered: every eligible CEMS flood activation from EMSR899 on is scored by models frozen at a git tag, and the hashes of the sealed scores are committed to a hash-chained ledger until 20 activations in 15 clusters are scored, or until the end of 2027. The frozen weights are public and drive a service that maps new floods from Sentinel-1, showing the model's map with no reference and no score. We also report two errata in our earlier results.