Impact evaluation of public health strategies in low and middle-income settings. Measuring implementation strength: why and how?

Weak health systems are a common feature of low and middle income settings, and the 'know-do� gap between proven health interventions and routine practice is particularly wide. Evaluation of programmes seeking to narrow the gap is needed, to fuel both course-correction and impact evaluation.

For both implementers and evaluaters, questions of interest include: which packages of individual evidence-based interventions can be delivered at scale? How to optimise these packages in a new setting? To what extent to these result in public health gains?

Large-scale public health programmes are often set in a rapidly-changing context. Randomised trials are often unfeasible or answer other questions rather than those given above. A plausibility design based on a comparison of changes over time in both intervention and comparison areas may be able to answer the questions, but untouched comparison areas rarely exist. One alternative is to measure the strength or intensity with which packages of interventions are delivered, with a view to exploring the association between implementation strength and public health gains.

Advantages and disadvantages of using implementation strength within large-scale programme evaluations will be discussed. Drawing from three published examples in low- and middle-income settings, we show how implementation strength has been used in health programme evaluation, and discuss when the approach might be counter-productive.