UpStream: A Modular, Metadata-First Architecture for Environmental Sensing During Intensive Observation Periods

William Mobley , Linchao Luo, Maximiliano Osorio, Vladislav Krendelev, Chou-Hsien Lin, Keuntaek Kim, Suzanne A. Pierce, Evelyn Deveraux, Chih-Shen Cheng, Pawel Misztal

Campaign-scale environmental sensing produces heterogeneous, high-frequency datasets that strain file- and portal-centric workflows and impede decision support. This paper presents UpStream, an API-first, metadata-supported, modular prototype for campaign-scale environmental sensor data. Co-designed with the SNIFFER air-quality laboratory, UpStream treats campaigns as first-class entities with explicit identity, scope, and governance, embeds metadata capture at ingestion, and provides selective, query-mediated access to high-resolution observations under concurrent use. We evaluate the architecture through three field deployments, a mobile PTR-MS laboratory and a stationary eddy-covariance flux tower, plus a remote, pull-based deployment in Alaska, showing that heterogeneous, evolving instrument outputs are absorbed without schema redesign while campaign boundaries, selective access, and metadata persist throughout the data lifecycle. We distill these results into transferable architectural principles for campaign-scale environmental sensing systems supporting decision support: modular service separation, flexible schema design, metadata-at-ingestion, query-mediated access, and federation designed in from the start.