Measuring Implementation Strength Literature Review: Possibilities for maternal and newborn health programmes

Measuring the strength of programme implementation is an emerging evaluation approach to help to understand: a) the predictability of outcomes and impact of proven interventions, b) why some programmes are more successful than others, and c) how changes in outcomes can be attributed to a particular programme. However, there is a lack of consensus about what are the most efficient approaches, including statistical techniques to generate implementation strength scores for a given health intervention or programme.

In order to address this knowledge gap, we carried out a systematic literature review with the following aims: a) what conceptual frameworks have been used to identify components of implementation strength scores in the scientific literature and b) what types of epidemiological and statistical techniques have been employed to measure implementation strength scores in programmatic settings

The review included evidence from peer reviewed journal databases and seminal grey literature. Data was extracted independently by three reviewers into a structured form. 2,297 titles and abstracts were examined. After studying the full texts of 184 documents, 26 studies were selected for the review. The studies included were from mental health, chronic care, primary care, public service, health promotion, public health, and education disciplines.

The review found that a range of scaling and scoring systems are used to measure quantity and quality of implementation. Commonly used components of implementation strength score were organizational structure e.g., leadership, human resources, information systems, and processes of services delivery, e.g. activities, types, availability, and quality of services. However, there was no uniform approach in defining and measuring implementation. Finally, the review suggests a lack of rigorous evidence for measuring large-scale implementation of complex interventions in low income countries.