U.S. Department of Commerce | National Oceanic & Atmospheric Administration | NOAA Research

What is HIWPP Doing About Verification?

Initial HIWPP verification work has focused on building a common verification system for the evaluation of model forecasts within the project. HIWPP scientists are using this system to produce verification statistics, initially for a set of individual (i.e. deterministic) global high-resolution hydrostatic models in both retrospective and real-time research modes. The initial verification system includes components from both NCEP/EMC and ESRL/GSD, who have independently developed verification tools that share a number of common attributes. The Naval Research Laboratory also has expertise in model forecast verification and is participating in a collaboration to define the most important features in these currently available tools and combine them into a new unified verification platform.

Sample of various verification tools

Sample of various verification tools

The new system, engineered with best practices, will include a database for storage of a variety of verification data and an interactive web-based interface, accessible by all project members. A range of verification capabilities already exist, spanning traditional large-scale metrics (anomaly correlation coefficient, upper-air [rawinsonde variables]) to metrics reflecting smaller-scale atmospheric processes (surface [temperature, dewpoint, winds] and precipitation. As a reference benchmark, the team has ported and adapted existing tools and applied them to HIWPP deterministic models. From this baseline, the verification team is working to establish a common framework for the comprehensive verification system, with provision for the addition of newer verification capabilities, including evaluation of ensembles of models, calculation and display of multi-dimensional verification "scorecard" metrics, and display of innovative verification diagnostic fields. The team is also investigating new data sets and methodologies for evaluation of precipitation forecasts.

Although the verification system is being built to serve HIWPP’s needs, a design goal is to provide a platform that can serve broader NOAA model development needs and accommodate future enhancements. Upon successful demonstration of the enhanced system, it will become a candidate for transition into operations.