The first week of April concluded the end of the 2021 Warn-on-Forecast Testbed Experiment as part of the NOAA Hazardous Weather Testbed. In this experiment, a total of 16 forecasters from nine southern region National Weather Service Forecast Offices (WFOs), the NOAA NWS Storm Prediction Center (SPC), and the NOAA NWS Weather Prediction Center (WPC) came together over four weeks to explore the use of Warn-on-Forecast System (WoFS) guidance in the watch-to-warning time frame.

Like many other scientific activities, this experiment was delayed and then moved virtually due to the ongoing COVID-19 pandemic. Despite the many challenges this unique situation presented, our research team is pleased to report the experiment was very successful. This success is attributable to the significant efforts of numerous University of Oklahoma Cooperative Institute for Mesoscale Meteorological Studies (CIMMS) and NOAA National Severe Storms Laboratory scientists, as well as participants and collaborators in the NWS. 

Together, Pat Skinner (with OU CIMMS/NSSL), Patrick Burke (with NSSL), Burkely Gallo (with OU CIMMS/SPC), and I (Katie Wilson with OU CIMMS/NSSL) designed and executed this experiment to examine how forecasters envision WoFS guidance fitting into both their existing and visionary forecast processes, and to explore the ways that WoFS guidance can be used most effectively given national center and local office forecasting responsibilities.

A Collaborative Undertaking

During the experiment, Skinner delivered an overview presentation to build familiarity with WoFS guidance prior to participants’ completion of two case studies. These cases formed the first of two major activities during the experiment, which was for participants to immerse themselves in simulated real-time events and use WoFS guidance to make forecast and communication decisions. 

To prepare for the case study activity, each week Burke provided participants with a hands-on AWIPS-2 demo. Furthermore, his work to design AWIPS-2 perspectives and procedures, which was conducted jointly with Gallo, enabled a simulation setup that was more familiar to participants, especially to those from national centers who do not use AWIPS-2 in the way local office forecasters do.

The preparation of these case studies was a major task undertaken by Jonny Madden (OU CIMMS/NSSL), Justin Monroe (OU CIMMS/NSSL), Jorge Guerra (OU CIMMS/NSSL), and Dale Morris (OU CIMMS/NWS Warning Decision Training Division).

The case studies presented two notable firsts:

  • Running AWIPS-2 in-the-cloud such that participants could complete the case studies from their own homes, and;
  • Presenting WoFS guidance in AWIPS-2, including the development of a tool to visualize paintball plots.

Madden, Monroe, Guerra, and Morris worked together to accomplish numerous tasks, including: aggregating and processing a full suite of observational and model datasets for both cases, setting up the WES- 2 Bridge and AWIPS-2 interfaces, and collaborating with federal partners to get datasets onto the cloud framework. Much of what was accomplished for the case study portion of this experiment has laid the AWIPS-2 in-the-cloud groundwork for future virtual experiments.

The second major activity during the experiment was focus groups. Together, Wilson and Gallo led three semi-structured discussions each week to explore forecasters’ visions for how WoFS will impact the current and future forecast process. Additionally, the presence of both national center and local office forecasters meant that much was learned about each others’ workflows, how one another makes decisions, and where there is an opportunity to strengthen collaboration. In a post-experiment questionnaire, participants rated the focus groups as a highly effective activity for sharing thoughts and ideas, and was the most enjoyed activity of the week.

In addition to the efforts of scientists at OU CIMMS and NOAA NSSL, we were grateful for input from our collaborators at NWS Southern Region, including Chad Gravelle (SR HQ), Todd Lindley (OUN SOO), Stephen Bieda (AMA SOO), and Randy Bowers (OUN). Gravelle and Lindleyalso joined the experiment for multiple weeks, and Randy created two excellent weather briefing videos to prepare forecasters for the case studies. A big thank you also goes to our pilot participants, Laren Reynolds (El Paso, Texas) and Joseph Merchant (Lubbock, Texas), for volunteering their time to fulfill an important support role throughout the whole experiment. This support role emerged following findings from the pilot week, and made for a much stronger experiment.

We are extremely appreciative to the 16 NWS forecasters who participated in this experiment. We realize the stressful conditions many people continue to live and work with, and have done so over the past year. We also realize the disappointment from not being able to attend this experiment in the NOAA Hazardous Weather Testbed, in Norman, Oklahoma, as originally planned. However, participants showed up to our virtual experiment with enthusiasm and made meaningful contributions to the experiment. We collected an enormous amount of data, and we can’t wait to analyze it and share what was learned.

For questions on this or other WoFS-related research please contact WoFS Program Lead at NOAA NSSL, Patrick Burke, nssl.outreach@noaa.gov.

Dr. Katie Wilson is an OU CIMMS research scientist working at the NOAA National Severe Storms Laboratory. She has pursued a career path which incorporates knowledge and research methods from multiple disciplines, including meteorology, human factors, and related social science fields. After completing her Ph.D. in 2017, Wilson joined the Warn-on-Forecast team to lead efforts in the testing and evaluation of WoFS guidance with end users.