Evaluation of CMIP6 model performance against GHCND station observations across Asian cities

Benjamin Goodsell 

Models from the Coupled Model Intercomparison Project (CMIP) are commonly used to infer future atmospheric temperatures across the world, including in Asia. This region is warming at an alarming rate, and home to some of the world's most populated countries and many of its most populated, fastest-growing and most heat-vulnerable cities. CMIP models are not always checked against local city-scale conditions without correction or downscaling applied beforehand. Being almost ten years after CMIP phase 6 was released, there is an opportunity to assess the accuracy of CMIP6 data with ground observations. Here we compare 2-metre air temperatures from 11 high-resolution CMIP6 models using a Shared Socioeconomic Pathway (SSP) for scenario 5-8.5 against GHCND station observations and ERA5 reanalysis for 411 Asian cities over the period 2015 to 2024. Neither bias correction nor downscaling was applied. ERA5 agreed closely with station observations (r = 0.91, RMSE = 2.2 °C), and all CMIP6 variants correlated significantly with observations but performed noticeably worse than reanalysis data (r = 0.78–0.81, RMSE = 4.17–5.35 °C, mean bias −1.26 to +3.28 °C). HadGEM3-GC31-HH performed best most often, and CNRM-CM6-1-HR performed worst. Biases were systematic rather than seasonal, with ten of the eleven variants being warm-biased in every season, and only CMCC-CM2-VHR4 being consistently cold. Performance depended strongly on geography, with latitude being the dominant predictor (r = 0.84). Agreement was strongest in arid, cold and temperate climates (r ≈ 0.88–0.91) and weakest in tropical rainforest (r = 0.23) and monsoon (r = 0.49) climates. By region, Southeast Asian cities had the weakest agreement (mean r = 0.43) and East Asian cities the strongest (r = 0.89). The models were least reliable in tropical Southeast Asia, where heat-related mortality and urban growth are highest. Improving tropical convection and urban representation in models, and expanding observation networks, should be priorities for future model development.