Tracking Progress in General-Purpose VLMs and Agents for Earth Observation

Skyler D. Liu , Sherrie Wang

The rapid advancement of vision-language models (VLMs) and agent-based workflows is expanding their potential for Earth observation (EO). Existing studies primarily compare models at a single point in time, providing limited insight into how their capabilities evolve across generations. To address this gap, we conduct a longitudinal evaluation of 48 VLMs across successive generations using a benchmark comprising 13 EO tasks, examining both overall EO performance and individual task capabilities. We investigate the effects of prompting strategies, task-specific fine-tuning, agent-based inference, and reasoning effort, as well as its association with model parameter count. Results reveal substantial but uneven progress in the EO capabilities of general-purpose VLMs. Object classification, scene understanding, and object counting improve markedly, while change detection and precise localization also improve but remain challenging. Few-shot prompting and task-specific fine-tuning provide performance gains, whereas chain-of-thought and increased reasoning effort sometimes degrade model performance. Holding the underlying model fixed, autonomous agent-based inference significantly improves GPT-6 Astra's overall EO score (+4.59 points; p=0.025), with the largest gains in object counting and change detection. These findings demonstrate that model evolution and improved execution workflows are advancing the EO capabilities of general-purpose VLMs, while fine-grained visual perception and spatial reasoning remain major challenges.