Spectrally structured CNN encoding for interpretable and edgeready fractional vegetation cover mapping using UAS multispectral and imaging spectroscopy

Laura N. Sotomayor , Arko Lucieer, Ryan Haynes, Darren Turner, Johan Barthelemy, Teja Kattenborn

Fractional vegetation cover (FVC) is a key indicator of semi-arid ecosystem condition, but non-photosynthetic vegetation (NPV) remains difficult to map because dry or senescent vegetation, litter, woody debris, and standing dead material can overlap spectrally with photosynthetic vegetation (PV) and bare ground (BE), especially where shadows, exposed bare earth surfaces, and mixed vegetation-soil pixels occur.
We developed an NPV-aware, spectrally structured convolutional neural network workflow for UAS-based FVC mapping using multispectral and imaging spectroscopy data. The framework used a U-Net backbone with band-aware spectral encoding, lightweight cross-channel attention, NPV-weighted optimisation, data augmentation, spectral characterisation, Monte Carlo dropout uncertainty mapping, cross-site inference, and TensorRT benchmarking.
Ablation experiments showed that NPV-weighted variants produced the largest target-class gains. M5 achieved the highest aggregate NPV F1-score (0.496; +0.148 relative to baseline M1), while M3 achieved the highest mean F1-score (0.745) and mIoU (0.629). The selected MICA 304 px M5 configuration produced stronger segmentation scores (NPV F1 = 0.725; mean F1 = 0.886), whereas the selected FX10-FX17 152 px M5 configuration provided broader VNIR-SWIR information for interpreting NPV, vegetation-substrate mixtures, dry material, and moisture-sensitive variation. These results indicate a spatial-spectral trade-off: finer spatial resolution improved segmentation detail, while broader spectral information strengthened interpretation of class ambiguity.
Reflectance separability, spectral-index maps, model-derived spectral importance, and uncertainty outputs showed that NPV remained the most uncertain class. Cross-site inference provided qualitative transfer diagnostics, and TensorRT results on Jetson AGX Orin demonstrated compatibility with embedded GPU inference. The workflow provides an interpretable, edge-compatible baseline for sensor-informed FVC mapping.