Using Time-Series Data to Measure Campaign Effectiveness: An Example from the truth® FinishIt Campaign

Theoretical Background and research questions/hypothesis:

Counter-tobacco mass media campaigns have been hailed as one of the most effective population-level interventions at preventing tobacco use. One such campaign, truth®, launched the FinishIt campaign in 2014, delivering messages through various media platforms aimed at promoting anti-tobacco norms and anti-tobacco industry attitudes. Evaluation studies have found the campaign to be successful at changing smoking attitudes, intentions, and behaviors. However, as the mass media landscape has been fractured across television, digital, and social media platforms over the past two decades, traditional exogenous measures of campaign exposure (e.g., television gross rating points) have been less able to capture true variability in exposure. Endogenous measures, such as self-reported ad recall, are subject to recall biases, some of which may be related to the outcome of interest (e.g., smokers may be more likely to recall an anti-smoking ad). This study addresses these limitations by aggregating self-reported ad recall across participants within a particular week in order to model the variability in visibility and salience of the campaign over time across the population, while eliminating recall biases at the individual level. The objective of this study is to present an alternative method of measuring campaign effectiveness by assessing the impact of exposure to the truth FinishIt campaign on smoking intentions and behaviors, using an aggregated self-reported measure of exposure that varies over time.

Methods:

The sample comes from a cross-sectional, continuous tracking survey of participants from a nationally-representative online panel. Daily online surveys are conducted among U.S. youth and young adults (ages 15-24), totaling approximately n=240 per week. Surveys assess demographics, tobacco-related intentions and behaviors, and truth ad exposure. All participants within a given survey week were assigned an exposure variable equal to the average of reported exposure across the participants for the week. Logistic regression models assessed the relationship between the aggregated ad exposure measure and intentions to smoke and current smoking, controlling for demographics. Robust standard errors were estimated because participants were clustered within week, leading to non-independence in responses.

Results:

For every 5-percentage point increase in weekly ad exposure, we found 0.98 times lower odds of intentions to smoke and current smoking. This is roughly equivalent to a 2% decrease in the odds of intentions to smoke or current smoking for every 5-percentage point increase in weekly ad exposure.

Conclusions:

The basic premise of campaign evaluation is to test whether those with more exposure to the campaign are more or less likely to engage in behaviors that are aligned with campaign goals. Findings from this study are consistent with prior research finding exposure to the FinishIt campaign to be associated with less tobacco use.

Implications for research and/or practice:

This study presents an alternate method for measuring campaign effectiveness that addresses several limitations of more traditional approaches. This method reduces biases related to self-reported measures of exposure, as well as the limitations associated with capturing exposure through traditional GRP measures. These methods could be implemented by evaluators of campaigns, beyond tobacco prevention, to more rigorously assess campaign impact.