Calling for a National Model Benchmarking Facility

This is a Preprint and has not been peer reviewed. This is version 1 of this Preprint.

Add a Comment

You must log in to post a comment.


Comments

There are no comments or no comments have been made public for this article.

Downloads

Download Preprint

Authors

Benjamin Lyle Ruddell, Martyn Clark, Jessica M Driscoll, David Gochis, Hoshin Gupta, Debbie Huntzinger, James W Kirchner, Laurel Larsen, Henry W Loescher, Yiqi Luo, Reed Maxwell, Edom Moges, William J Riley, Zexuan Xu

Abstract

The modern world uses predictive computer models for many important purposes, including weather predictions, epidemic management, flood forecasting and warnings, and economic policymaking. We need to know how much we can trust the projections of these models, not only to achieve more accurate projections for systems, but also to undertake scientific learning about systems by incrementally testing hypotheses using models. But we routinely fail to adequately benchmark the performance of our complicated models of systems due to the cost and complexity of the task and owing to social and institutional barriers. Decades of lessons learned from Model Intercomparison Projects (MIPs) and similar community modeling efforts have yielded understanding of both the challenge and the opportunity facing 21st century model benchmarking efforts. To implement this understanding at scale, we call for the establishment of a major national research facility for scientific computer model benchmarking. Such a facility would institutionalize and properly resource the technically challenging and laborious work of computer model benchmarking, thereby establishing a firm foundation for 21st century science and prediction. This facility would advance basic science, overcome many of the social barriers to benchmarking, and would improve projections and decisions.

DOI

https://doi.org/10.31223/X5195Q

Subjects

Biology, Computer Sciences, Earth Sciences, Ecology and Evolutionary Biology, Environmental Sciences, Life Sciences, Oceanography and Atmospheric Sciences and Meteorology, Other Life Sciences, Physical and Environmental Geography, Physical Sciences and Mathematics, Planetary Sciences, Systems Biology

Keywords

model diagnostics, incentives, auditing, certification, research infrastructures, major research instrumentation, systems modeling, computer modeling, Community science, error attribution, Model benchmarking, model intercomparison, data science, systems science, community science, model intercomparison, Data science, systems science, research infrastructure

Dates

Published: 2023-04-14 06:43

Last Updated: 2023-04-14 13:43

License

CC-By Attribution-NonCommercial-NoDerivatives 4.0 International