A CNN with spectral-channel adaptation for interpretable and edge-ready fractional vegetation cover mapping using UAS multispectral and imaging spectroscopy

Laura N. Sotomayor , Arko Lucieer, Ryan Haynes, Darren Turner, Johan Barthelemy, Teja Kattenborn

Non-photosynthetic vegetation (NPV) remains difficult to map in semi-arid fractional vegetation cover (FVC) because it overlaps with photosynthetic vegetation (PV), bare ground (BE), shadows, and mixed vegetation–soil surfaces.

We developed an NPV-aware convolutional neural network workflow with spectral-channel adaptation for UAS multispectral and imaging spectroscopy data. The benchmark combined controlled M1–M5 ablations with comparisons against Random Forest, plain and PCA-reduced U-Nets, and SS3FCN under shared held-out spatial folds. We also evaluated matched spatial resolution, post-inference spectral characterisation, uncertainty, zero-shot transfer, and TensorRT deployment.

NPV weighting produced the largest target-class gains. M5 achieved the highest NPV F1 (0.496; +0.148 relative to M1), whereas M3 achieved the highest mean F1 (0.745) and mIoU (0.629). In the combined 296-band FX10–FX17 benchmark, M5 achieved mean F1 of 0.705 and mIoU of 0.573; the strongest baseline values were 0.688 and 0.565, respectively. PCA underperformed the plain U-Net using the combined 296-band input despite retaining more than 99% of spectral variance. Resampling MICA to 10 cm reduced NPV F1 from 0.725 to 0.539, although it retained higher mIoU than FX10–FX17 (0.667 versus 0.573).

Post-inference analyses confirmed persistent NPV instability and spectral ambiguity. The highest zero-shot external-site NPV F1 was 0.120. On Jetson AGX Orin, mixed-precision FX10–FX17 inference achieved a 1.70-fold speed-up with 99.96% argmax agreement, although pure FP16 was unstable. The workflow provides an interpretable, edge-compatible baseline while highlighting the need for broader cross-site validation.