ReliefWeb

By Abebual Demilew, Mesay Girma, Elizabeth McElwee, Saugato Datta, Jeremy Barofsky, and Tolera Disasa

Executive summary

Despite substantial economic and development progress, Ethiopia’s full immunisation rates remain low, with only 39 per cent of children aged 12–13 months and 22 per cent of those in the three lowest wealth quintiles fully immunised in 2016. In this study, we sought to explore two questions: (1) how to engage communities by using behaviourally informed feedback to mobilise healthcare workers around improving immunisation rates; and (2) how to create a positive reinforcement loop for those improvements using nonmonetary rewards.

The key outcomes of interest for immunisation rates were children receiving a full dose of the DTP/PCV vaccine, full immunisation coverage, and incomplete immunisation. This information was collected for each vaccine through household surveys conducted before the intervention was implemented, and upon completion of the intervention. For health extension worker (HEW) service provision, the outcomes of interest were an increase in home visits from baseline to follow-up and an increase in HEW self-reported efficacy, which were measured in baseline and follow-up surveys with the HEWs.

Our impact evaluation design was a randomised controlled trial, which used random assignment to control and treatment groups at the health post level. The main intervention was a tracking poster, called the ‘Protected Children’ poster, which used a stamp system as a simple and salient way for HEWs to track immunisation achievement and dropouts.

The theory of change for this intervention postulates that public feedback via the posters will lead to additional HEW outreach to families with children in need of immunisation. This outreach will then lead to parents taking their children to immunisation clinics at health posts or other centres for immunisation. Improved community immunisation rates will lead to recognition for HEWs, which will start a positive reinforcement cycle for them, leading to improvements in self-efficacy and increased engagement with the community. The increase in immunised children will lead to lower overall morbidity among children.

Our study area consisted of 90 health posts from Marie Stopes International Ethiopia field offices in Aris and East Shewa zones in 12 woredas (districts) of the Oromia region. Of these health posts, 45 were randomly assigned to the treatment group, with the remaining 45 assigned to the control group, which only received the Ministry of Health’s standard training. Marie Stopes field staff were responsible for recruiting, training and monitoring the HEWs. Participants were not blinded to their treatment assignment as they had to undergo additional training.

We observe that there is no statistically significant treatment effect on children receiving a full dose of DPT/PCV. The mean difference in full dose DPT/PCV between treatment and control health posts implies a non-significant reduction of 1.13 percentage points (p-value = 0.506) from the behavioural intervention. We found a 7.7 per cent increase in HEWs’ household visits (p-value = 0.2) as a result of the intervention. Although only marginally significant across some models, these results indicate that treatment may have induced changes to HEWs’ behaviour and effort but had no effect on immunisation rates.

Although we find some suggestive evidence that HEWs’ behaviour changed because of the intervention, we do not observe changes in immunisation. One potential reason is that the data were collected during a time when overall immunisation rates increased rapidly. Given other health system changes and economic growth that occurred in Ethiopia during data collection, we were not able to differentiate the additional impact of treatment from the underlying upward trend.

Also of note, the time between baseline and follow-up was substantially extended compared to our pre-analysis plan because of civil unrest and political instability in Ethiopia during data collection. These results indicate that even when evidence has been developed indicating that behavioural interventions are effective, they must be tested in a wide range of contexts to build evidence on what circumstances maximise the likelihood of impact.