• Energy Research

Abstract. This paper describes a set of benchmark tests that is designed to quantify the performance of the land surface model that is used in the UK Hadley Centre General Circulation Model (JULES: Joint UK Land Environment Simulator). The tests are designed to assess the ability of the model to reproduce the observed fluxes of water and carbon at the global and regional spatial scale, and on a seasonal basis. Five datasets are used to test the model: water and carbon dioxide fluxes from ten FLUXNET sites covering the major global biomes, atmospheric carbon dioxide concentrations at four representative stations from the global network, river flow from seven catchments, the seasonal mean NDVI over the seven catchments and the potential land cover of the globe (after the estimated anthropogenic changes have been removed). The model is run in various configurations and results are compared with the data. The results show that combined use of observations of carbon and water fluxes is essential in order to understand the causes of model errors. The benchmarking approach is suitable for application to other global models.

Flux tower observations, model spin-up and site characteristics data for Urban-PLUMBER sites associated with the manuscript: "Harmonized, gap-filled dataset from 20 urban flux tower sites" https://doi.org/10.5194/essd-14-5157-2022 Use of any data must give credit through citation of the above manuscript and other site sources as appropriate (see below). We recommend data users consult with site contributing authors and/or the coordination team in the project planning stage. Relevant site contacts are included in site metadata. Data can be downloaded from the bottom of this page. Sitename City Country Observed period References AU-Preston Melbourne Australia Aug 2003 – Nov 2004 (Coutts et al., 2007a, b) AU-SurreyHills Melbourne Australia Feb 2004 – Jul 2004 (Coutts et al., 2007a, b) CA-Sunset Vancouver Canada Jan 2012 – Dec 2016 (Christen et al., 2011; Crawford and Christen, 2015) FI-Kumpula Helsinki Finland Dec 2010 – Dec 2013 (Karsisto et al., 2016) FI-Torni Helsinki Finland Dec 2010 – Dec 2013 (Järvi et al., 2018; Nordbo et al., 2013) FR-Capitole Toulouse France Feb 2004 – Mar 2005 (Masson et al., 2008; Goret et al., 2019) GR-HECKOR Heraklion Greece Jun 2019 – Jun 2020 (Stagakis et al., 2019) JP-Yoyogi Tokyo Japan Mar 2016 – Mar 2020 (Hirano et al., 2015; Ishidoya et al., 2020) KR-Jungnang Seoul South Korea Jan 2017 – Apr 2019 (Jo et al., n.d.; Hong et al., 2020) KR-Ochang Ochang South Korea Jun 2015 – Jul 2017 (Hong et al., 2019, 2020) MX-Escandon Mexico City Mexico Jun 2011 – Sep 2012 (Velasco et al., 2011, 2014) NL-Amsterdam Amsterdam Netherlands Jan 2019 – Oct 2020 (Steeneveld et al., 2020) PL-Lipowa Łódź Poland Jan 2008 – Dec 2012 (Fortuniak et al., 2013; Pawlak et al., 2011) PL-Narutowicza Łódź Poland Jan 2008 – Dec 2012 (Fortuniak et al., 2013, 2006) SG-TelokKurau Singapore Singapore Feb 2015 – Feb 2016 (Roth et al., 2017) UK-KingsCollege London UK Apr 2012 – Jan 2014 (Bjorkegren et al., 2015; Kotthaus and Grimmond, 2014a, b) UK-Swindon Swindon UK May 2011 – Apr 2013 (Ward et al., 2013) US-Baltimore Baltimore USA Jan 2002 – Jan 2007 (Crawford et al., 2011) US-Minneapolis Minneapolis USA Jun 2006 – May 2009 (Peters et al., 2011; Menzer and McFadden, 2017) US-WestPhoenix Phoenix USA Dec 2011 – Jan 2013 (Chow, 2017; Chow et al., 2014) For further site information and timeseries plots see https://urban-plumber.github.io/sites. For processing code see https://github.com/matlipson/urban-plumber_pipeline. Data Two data archives are available on this page. The full collection includes all observed, gap-filled, spin-up and site characteristic data, in both netcdf and text form. The "obs_only" archive includes a duplicate of site observation timeseries (after quality control) in a single netcdf file. Full collection The full archive includes site folders with: index.html: A summary page with site characteristics and timeseries plots. SITENAME_sitedata_v1.csv: comma separated file for numerical site characteristics e.g. location, surface cover fraction etc. timeseries/ (following files are available as netCDF and txt) SITENAME_raw_observations_v1: site observed timeseries before project-wide quality control. SITENAME_clean_observations_v1: site observed timeseries after project-wide quality control. SITENAME_metforcing_v1: gap-filled and prepended (10yr spinup) site observation forcing dataset for model evaluation. SITENAME_era5_corrected_v1: site ERA5 surface data (1990-2020) with bias corrections as applied in the final dataset. "Obs Only" This archive contains duplicate data from the full collection (observations after QC): UP_all_clean_observations_UTC_v1.nc: in coordinated universal time (UTC) UP_all_clean_observations_localstandardtime_v1.nc: in local standard time Site references Bjorkegren, A. B., Grimmond, C. S. B., Kotthaus, S., and Malamud, B. D.: CO2 emission estimation in the urban environment: Measurement of the CO2 storage term, Atmospheric Environment, 122, 775–790, https://doi.org/10.1016/j.atmosenv.2015.10.012, 2015. Chow, W.: Eddy covariance data measured at the CAP LTER flux tower located in the west Phoenix, AZ neighborhood of Maryvale from 2011-12-16 through 2012-12-31, https://doi.org/10.6073/PASTA/FED17D67583EDA16C439216CA40B0669, 2017. Chow, W. T. L., Volo, T. J., Vivoni, E. R., Jenerette, G. D., and Ruddell, B. L.: Seasonal dynamics of a suburban energy balance in Phoenix, Arizona, International Journal of Climatology, 34, 3863–3880, https://doi.org/10.1002/joc.3947, 2014. Christen, A., Coops, N. C., Crawford, B. R., Kellett, R., Liss, K. N., Olchovski, I., Tooke, T. R., van der Laan, M., and Voogt, J. A.: Validation of modeled carbon-dioxide emissions from an urban neighborhood with direct eddy-covariance measurements, Atmospheric Environment, 45, 6057–6069, https://doi.org/10.1016/j.atmosenv.2011.07.040, 2011. Coutts, A. M., Beringer, J., and Tapper, N. J.: Characteristics influencing the variability of urban CO2 fluxes in Melbourne, Australia, Atmospheric Environment, 41, 51–62, https://doi.org/10.1016/j.atmosenv.2006.08.030, 2007a. Coutts, A. M., Beringer, J., and Tapper, N. J.: Impact of Increasing Urban Density on Local Climate: Spatial and Temporal Variations in the Surface Energy Balance in Melbourne, Australia, J. Appl. Meteor. Climatol., 46, 477–493, https://doi.org/10.1175/JAM2462.1, 2007b. Crawford, B. and Christen, A.: Spatial source attribution of measured urban eddy covariance CO2 fluxes, Theor Appl Climatol, 119, 733–755, https://doi.org/10.1007/s00704-014-1124-0, 2015. Crawford, B., Grimmond, C. S. B., and Christen, A.: Five years of carbon dioxide fluxes measurements in a highly vegetated suburban area, Atmospheric Environment, 45, 896–905, https://doi.org/10.1016/j.atmosenv.2010.11.017, 2011. Fortuniak, K., Kłysik, K., and Siedlecki, M.: New measurements of the energy balance components in Łódź, in: Preprints, sixth International Conference on Urban Climate: 12-16 June, 2006, Göteborg, Sweden, Sixth International Conference On Urban Climate, Göteborg, Sweden, 64–67, 2006. Fortuniak, K., Pawlak, W., and Siedlecki, M.: Integral Turbulence Statistics Over a Central European City Centre, Boundary Layer Meteorology; Dordrecht, 146, 257–276, https://doi.org/10.1007/s10546-012-9762-1, 2013. Goret, M., Masson, V., Schoetter, R., and Moine, M.-P.: Inclusion of CO2 flux modelling in an urban canopy layer model and an evaluation over an old European city centre, Atmospheric Environment: X, 3, 100042, https://doi.org/10.1016/j.aeaoa.2019.100042, 2019. Hirano, T., Sugawara, H., Murayama, S., and Kondo, H.: Diurnal Variation of CO2 Flux in an Urban Area of Tokyo, Sola, 11, 100–103, https://doi.org/10.2151/sola.2015-024, 2015. Hong, J., Lee, K., and Hong, J.-W.: Observational data of Ochang and Jungnang in Korea, 2020. Hong, J.-W., Hong, J., Chun, J., Lee, Y. H., Chang, L.-S., Lee, J.-B., Yi, K., Park, Y.-S., Byun, Y.-H., and Joo, S.: Comparative assessment of net CO2 exchange across an urbanization gradient in Korea based on eddy covariance measurements, Carbon Balance and Management, 14, 13, https://doi.org/10.1186/s13021-019-0128-6, 2019. Ishidoya, S., Sugawara, H., Terao, Y., Kaneyasu, N., Aoki, N., Tsuboi, K., and Kondo, H.: O2 : CO2 exchange ratio for net turbulent flux observed in an urban area of Tokyo, Japan, and its application to an evaluation of anthropogenic CO2 emissions, Atmospheric Chemistry and Physics, 20, 5293–5308, https://doi.org/10.5194/acp-20-5293-2020, 2020. Järvi, L., Rannik, Ü., Kokkonen, T. V., Kurppa, M., Karppinen, A., Kouznetsov, R. D., Rantala, P., Vesala, T., and Wood, C. R.: Uncertainty of eddy covariance flux measurements over an urban area based on two towers, Atmospheric Measurement Techniques, 11, 5421–5438, https://doi.org/10.5194/amt-11-5421-2018, 2018. Jo, S., Hong, J.-W., and Hong, J.: The observational flux measurement data of suburban and low-residential areas in Korea (in preparation), n.d. Karsisto, P., Fortelius, C., Demuzere, M., Grimmond, C. S. B., W., O. K., Kouznetsov, R., Masson, V., and Järvi, L.: Seasonal surface urban energy balance and wintertime stability simulated using three land‐surface models in the high‐latitude city Helsinki, Q.J.R. Meteorol. Soc., 142, 401–417, https://doi.org/10.1002/qj.2659, 2016. Kotthaus, S. and Grimmond, C. S. B.: Energy exchange in a dense urban environment – Part I: Temporal variability of long-term observations in central London, Urban Climate, 10, Part 2, 261–280, https://doi.org/10.1016/j.uclim.2013.10.002, 2014a. Kotthaus, S. and Grimmond, C. S. B.: Energy exchange in a dense urban environment – Part II: Impact of spatial heterogeneity of the surface, Urban Climate, 10, Part 2, 281–307, https://doi.org/10.1016/j.uclim.2013.10.001, 2014b. Masson, V., Gomes, L., Pigeon, G., Liousse, C., Pont, V., Lagouarde, J.-P., Voogt, J., Salmond, J., Oke, T. R., Hidalgo, J., Legain, D., Garrouste, O., Lac, C., Connan, O., Briottet, X., Lachérade, S., and Tulet, P.: The Canopy and Aerosol Particles Interactions in TOulouse Urban Layer (CAPITOUL) experiment, Meteorol Atmos Phys, 102, 135, https://doi.org/10.1007/s00703-008-0289-4, 2008. Menzer, O. and McFadden, J. P.: Statistical partitioning of a three-year time series of direct urban net CO2 flux measurements into biogenic and anthropogenic components, Atmospheric Environment, 170, 319–333, https://doi.org/10.1016/j.atmosenv.2017.09.049, 2017. Nordbo, A., Järvi, L., Haapanala, S., Moilanen, J., and Vesala, T.: Intra-City Variation in Urban Morphology and Turbulence Structure in Helsinki, Finland, Boundary-Layer Meteorol, 146, 469–496, https://doi.org/10.1007/s10546-012-9773-y, 2013. Pawlak, W., Fortuniak, K., and Siedlecki, M.: Carbon dioxide flux in the centre of Łódź, Poland—analysis of a 2-year eddy covariance measurement data set, International Journal of Climatology, 31, 232–243, https://doi.org/10.1002/joc.2247, 2011. Peters, E. B., Hiller, R. V., and McFadden, J. P.: Seasonal contributions of vegetation types to suburban evapotranspiration, Journal of Geophysical Research: Biogeosciences, 116, https://doi.org/10.1029/2010JG001463, 2011. Roth, M., Jansson, C., and Velasco, E.: Multi-year energy balance and carbon dioxide fluxes over a residential neighbourhood in a tropical city, Int. J. Climatol., 37, 2679–2698, https://doi.org/10.1002/joc.4873, 2017. Stagakis, S., Chrysoulakis, N., Spyridakis, N., Feigenwinter, C., and Vogt, R.: Eddy Covariance measurements and source partitioning of CO2 emissions in an urban environment: Application for Heraklion, Greece, Atmospheric Environment, 201, 278–292, https://doi.org/10.1016/j.atmosenv.2019.01.009, 2019. Steeneveld, G.-J., Horst, S. van der, and Heusinkveld, B.: Observing the surface radiation and energy balance, carbon dioxide and methane fluxes over the city centre of Amsterdam, Copernicus Meetings, https://doi.org/10.5194/egusphere-egu2020-1547, 2020. Velasco, E., Pressley, S., Grivicke, R., Allwine, E., Molina, L. T., and Lamb, B.: Energy balance in urban Mexico City: observation and parameterization during the MILAGRO/MCMA-2006 field campaign, Theor Appl Climatol, 103, 501–517, https://doi.org/10.1007/s00704-010-0314-7, 2011. Velasco, E., Roth, M., Tan, S. H., Quak, M., Nabarro, S. D. A., and Norford, L.: The role of vegetation in the CO2 flux from a tropical urban neighbourhood, Atmospheric Chemistry and Physics, 13, 10185–10202, https://doi.org/10.5194/acp-13-10185-2013, 2013. Velasco, E., Perrusquia, R., Jiménez, E., Hernández, F., Camacho, P., Rodríguez, S., Retama, A., and Molina, L. T.: Sources and sinks of carbon dioxide in a neighborhood of Mexico City, Atmospheric Environment, 97, 226–238, https://doi.org/10.1016/j.atmosenv.2014.08.018, 2014. Ward, H. C., Evans, J. G., and Grimmond, C. S. B.: Multi-season eddy covariance observations of energy, water and carbon fluxes over a suburban area in Swindon, UK, Atmospheric Chemistry and Physics, 13, 4645–4666, https://doi.org/10.5194/acp-13-4645-2013, 2013.

Abstract The Protocol for the Analysis of Land Surface Models (PALS) Land Surface Model Benchmarking Evaluation Project (PLUMBER) illustrated the value of prescribing a priori performance targets in model intercomparisons. It showed that the performance of turbulent energy flux predictions from different land surface models, at a broad range of flux tower sites using common evaluation metrics, was on average worse than relatively simple empirical models. For sensible heat fluxes, all land surface models were outperformed by a linear regression against downward shortwave radiation. For latent heat flux, all land surface models were outperformed by a regression against downward shortwave radiation, surface air temperature, and relative humidity. These results are explored here in greater detail and possible causes are investigated. It is examined whether particular metrics or sites unduly influence the collated results, whether results change according to time-scale aggregation, and whether a lack of energy conservation in flux tower data gives the empirical models an unfair advantage in the intercomparison. It is demonstrated that energy conservation in the observational data is not responsible for these results. It is also shown that the partitioning between sensible and latent heat fluxes in LSMs, rather than the calculation of available energy, is the cause of the original findings. Finally, evidence is presented that suggests that the nature of this partitioning problem is likely shared among all contributing LSMs. While a single candidate explanation for why land surface models perform poorly relative to empirical benchmarks in PLUMBER could not be found, multiple possible explanations are excluded and guidance is provided on where future research should focus.

------------------------------------------------------------------------------------------------------------------------------------------- This version has been superseded. The latest version is at https://doi.org/10.5281/zenodo.5517550 ------------------------------------------------------------------------------------------------------------------------------------------- Eddy covariance flux tower datasets of all Urban-PLUMBER sites, associated with the manuscript: "Harmonized, gap-filled dataset from 20 urban flux tower sites" Use of any data must give credit through citation of the above manuscript and other sources as appropriate. We recommend data users consult with site contributing authors and/or the coordination team in the project planning stage. Relevant contacts are included in timeseries metadata. For site information and timeseries plots see https://urban-plumber.github.io/sites. For processing code see https://github.com/matlipson/urban-plumber_pipeline. Within each site folder: - `index.html`: A summary page with site characteristics and timeseries plots. - `SITENAME_sitedata_vX.csv`: comma seperated file for numerical site characteristics e.g. location, surface cover fraction etc. - `timeseries/` (following files available as netCDF and txt) - `SITENAME_raw_observations_vX`: site observed timeseries before project-wide quality control. - `SITENAME_clean_observations_vX`: site observed timeseries after project-wide quality control. - `SITENAME_metforcing_vX`: site observed timeseries after project-wide quality control and gap filling. - `SITENAME_era5_corrected_vX`: site ERA5 surface data (1990-2020) with bias corrections as applied in the final dataset. - `log_processing_SITENAME_vX.txt`: a log of the print statements through running the create_dataset_SITENAME scripts. Authors Mathew Lipson, Sue Grimmond, Martin Best, Andreas Christen, Andrew Coutts, Ben Crawford, Bert Heusinkveld, Erik Velasco, Helen Claire Ward, Hirofumi Sugawara, Je-Woo Hong, Jinkyu Hong, Jonathan Evans, Joseph McFadden, Keunmin Lee, Krzysztof Fortuniak, Leena Järvi, Matthias Roth, Nektarios Chrysoulakis, Nigel Tapper, Oliver Michels, Simone Kotthaus, Stevan Earl, Sungsoo Jo, Valéry Masson, Winston Chow, Wlodzimierz Pawlak, Yeon-Hee Kim. Corresponding author: Mathew Lipson <m.lipson@unsw.edu.au>

Abstract A large number of urban surface energy balance models now exist with different assumptions about the important features of the surface and exchange processes that need to be incorporated. To date, no comparison of these models has been conducted; in contrast, models for natural surfaces have been compared extensively as part of the Project for Intercomparison of Land-surface Parameterization Schemes. Here, the methods and first results from an extensive international comparison of 33 models are presented. The aim of the comparison overall is to understand the complexity required to model energy and water exchanges in urban areas. The degree of complexity included in the models is outlined and impacts on model performance are discussed. During the comparison there have been significant developments in the models with resulting improvements in performance (root-mean-square error falling by up to two-thirds). Evaluation is based on a dataset containing net all-wave radiation, sensible heat, and latent heat flux observations for an industrial area in Vancouver, British Columbia, Canada. The aim of the comparison is twofold: to identify those modeling approaches that minimize the errors in the simulated fluxes of the urban energy balance and to determine the degree of model complexity required for accurate simulations. There is evidence that some classes of models perform better for individual fluxes but no model performs best or worst for all fluxes. In general, the simpler models perform as well as the more complex models based on all statistical measures. Generally the schemes have best overall capability to model net all-wave radiation and least capability to model latent heat flux.