Developing Emergency Risk Communication Measures to Determine Impact of Messaging During Public Health Responses: Preliminary Results and Next Steps

Theoretical Background and research questions/hypothesis:

During an emergency, public health officials communicate with the general public, at-risk or affected populations, clinicians, partner organizations, and across levels of government to deliver information to protect health. The Centers for Disease Control and Prevention (CDC) is responsible for responding to large-scale public health threats that span all hazards. Although the field of emergency risk communication (ERC) is mature, evaluating the impact of ERC remains a challenge. For example, ERC evaluation remains difficult due to data collection challenges, an emphasis on post-crisis evaluation, different constructs, questions, and scales, and limited resources. Moreover, available surveys are often proprietary and lack methodological rigor. Further, existing surveys typically evaluate the ability of an organization to accomplish goals specified in an emergency communication or crisis management plan, such as reviewing if all identified stakeholder groups were notified, if important information was processed appropriately, or if inappropriate information was released. In short, existing measurement tools fall short of fully evaluating key impacts of ERC. To help address this gap and to fulfill the need for evaluative assessments, we developed instruments to measure constructs related to understanding the effectiveness of CDC ERC messages.

Methods:

We designed four survey instruments to measure ERC: a process instrument to assess message development and dissemination, and three outcome instruments assessing the impacts of ERC messaging to 1) state, local, tribal, and territorial health departments, 2) healthcare associations, and 3) healthcare providers. We conducted research to assess the content validity and importance of proposed measures in these instruments (expert panel), if the instrument is appropriate and practical for continued use (feasibility testing), and ensure respondent understanding of instrument is sufficient (cognitive testing).

Results:

We will share results from an expert panel, feasibility, and cognitive testing. Beginning with the expert panel, leading risk and crisis communication experts rated and prioritized proposed constructs and measures. Following the panel, we revised the instruments to reflect experts’ opinions and new recommendations. Next, feasibility testing with each audience was conducted to assess whether data for the ERC constructs are sufficiently accessible to those queried, estimate how much effort it will require (i.e., time and materials) to collect the data, and identify potential barriers to data collection and reporting. Then, we conducted cognitive testing to examine the way respondents understand, mentally process, and respond to the instruments. In addition, an overview of the forthcoming quantitative pilot test will be provided.

Conclusions:

Validated ERC measures are necessary to evaluate key impacts of ERC implementation during a public health emergency. Assessment instruments have the potential to improve the execution of ERC, and consequently the ability of communicators to facilitate the uptake of health protection information.

Implications for research and/or practice:

Public health communicators can learn where ERC is working and, more importantly, not working during an emergency and make necessary adjustments. We encourage session participants to give feedback on the feasibility and usefulness of these instruments and how they could potentially utilize them to evaluate the impact of ERC messages during a public health emergency.