What’s the evidence?

By Emily Groot

There has been a big push in public health to use the best available evidence to inform policy development.  When it comes to radical public health policy change, however, developing the evidence base for policy can be difficult.

In part, this is because randomized controlled trials (RCTs) are often considered the gold standard in health research.  When it comes to interventions that address the determinants of health, RCTs are often not feasible because the root causes of illness and well-being are interconnected and cannot be reduced to component parts (Norman, 2009).  Natural experiments are hard to come by.

Even when feasible, RCTs may not be preferred way to study systems change because the careful selection of the study population means that it is difficult to generalize the results to the broader population (Rockers et al., 2012).  As well, RCTs usually attempt to control the impact of historical, cultural, and political factors that are essential to understanding how an intervention will fare in a particular context (Best et al., 2009; Rockers et al., 2012).

Most RCTs only address single interventions, while in real life there are multiple policies already in effect.  These policies may interact in non-linear ways that are difficult to predict from the available evidence (e.g., a change to agricultural subsidies and a change to social assistance funding may both decrease the risk of obesity, but when both policies are implemented simultaneously, the total impact may be much greater than the sum of the impacts of both programs) (Peters & Bennett, 2012; Sanderson, 2009).

In essence, interventions addressing the determinants of health are often so context-dependent it is difficult to predict their effectiveness in a particular context.  That is not to say it is impossible.  What is necessary to change health systems to better address the determinants of health is a range of different types of evidence, including:

  • Information about overcoming local barriers to implementation;
  • Evidence from non-research environments;
  • Description of the unintended consequences or political ramifications of the intervention;
  • Evidence of the impact of the intervention on equity;
  • Assessments of the intervention’s technical feasibility; and
  • Assessment of the intervention’s acceptability to policymakers and stakeholders (Bosch-Capblanch et al., 2012; Peters & Bennett, 2012).

This can also include evidence of effectiveness from RCTs, when feasible.


Best, A., Terpstra, J. L., Moor, G., Riley, B., Norman, C. D., & Glasgow, R. E. (2009). Building

knowledge integration systems for evidence-informed decisions. Journal of Health Organisation and Management, 23(6), 627-641.

Bosch-Capblanch, X., Lavis, J. N., Lewin, S., Atun, R., Røttingen, J.-A., Dröschel, D., Beck, L., et al. (2012). Guidance for evidence-informed policies about health systems: Rationale for and challenges of guidance development. PLoS Medicine, 9(3), e1001185.

Norman, C. D. (2009). Health promotion as a systems science and practice. Journal of Evaluation in Clinical Practice, 15(5), 868-872.

Peters, D. H., & Bennett, S. (2012). Better guidance is welcome, but without blinders. PLoS Medicine, 9(3), e1001188.

Rockers, P. C., Feigl, A. B., Røttingen, J.-A., Fretheim, A., de Ferranti, D., Lavis, J. N., Melberg, H. O., et al. (2012). Study-design selection criteria in systematic reviews of effectiveness of health systems interventions and reforms: A meta-review. Health Policy, 104(3), 206-214.

Sanderson, I. (2009). Intelligent policy making for a complex world: Pragmatism, evidence and learningPolitical Studies, 57(4), 699-719.


Parts of this post were originally written for a paper submitted for CHL6020Y Y at the University of Toronto.

Tagged , , , ,

What are your thoughts?

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: