ReliefWeb

EXECUTIVE SUMMARY

This executive summary for the Kalabo Drought Response Operation evaluation report highlights the purpose of the evaluation, key questions, research methodology, evaluation findings, conclusions, and recommendations.

Purpose/Objective:

The overall purpose was to undertake an evaluation for the drought response operation in Kalabo District thereby determining the relevance and effectiveness of the Red Cross intervention, the Movement cooperation within the operation and identification of lessons learned and recommendations to inform future similar operations. In addition to the final evaluation assessed the outcomes of the Cash Transfer, Agricultural input support and WASH programs against the planned project results.

Methodology:

The assignment was undertaken in 3 phases. Phase I focused on inception (developing shared understanding between IFRC and consultant on the assignment). This involved desk review of relevant documents and initial brief systematic consultations with IFRC staff before proceeding with the rest of the assignment. Phase II involved data collection, detailed literature review, stakeholder consultations, household Survey and Key informant interviews. Phase III focused on data analysis, report compilation and presentation. This phase focused on consolidation of information collected, categorization of the information, data entry and analysis and report writing paying adequate attention to the assignment terms of reference and the emerging issues captured in the inception report.

Key Challenges and Limitations:

The evaluation team recognizes the following limitations:

▪ Timelines: The period allocated was short considering scope defined by IFRC to be covered in the evaluation. The evaluation team managed to address this challenge through support from the project team who provided enumerators and handled study logistics in timely fashion.

▪ Potential enumerator bias: Some of the enumerators used in the household and community data collection process were ZRCS volunteers that had been part of the operation delivery.

Key Findings:

Relevance: The operation was found relevant as there was evidence that it fulfilled the set development and emergency goals. However, observation was made on how the design and execution would have been optimised to better achieve results. The operation results chain was found to be valid, and the project made significant efforts to address strategic requirements of the beneficiaries, partners, national government although there were minor gaps especially at community leadership level.

Effectiveness: The operation was effective since the evaluation findings demonstrate that activities were useful on the lives of the target people. Most of the project objectives were over - achieved although the planned target for household access to clean drinking water was significantly low. People in Karabo had meaningful participation and engagement with the operation and they did take ownership of the project. Furthermore, the operation introduced technology and equipment which is being managed well by the beneficiaries with the support of key stakeholders like government ministries.

Impact: The operation has had positive net social, economic and environmental changes. The overarching target humanitarian impact to assist drought affected households was achieved. Among others the intervention contributed significantly to addressing emergency food security and livelihoods needs for target beneficiaries, however, food and nutrition security is yet to be fully realised. Agricultural production had improved by 48.1% compared to the drought year after households received agricultural inputs and the related climate smart agriculture training compared to the drought year. However, the level of production was lower than previous years where there were better rains. Reliance on rain fed agriculture production is a key challenge to crop production hence the need to intensify investment in irrigation farming systems. WASH impact indicators were not considered at project design and at baseline. However, from the key outputs reported number of safe and clean water sources increased in Kalabo which has a contribution to increased percentage of households with access to improved, safe and clean water sources and quantity of water used per capita per day. Awareness raising on good hygiene practices were also undertaken and stakeholders both at district and community levels confirmed reduced prevalence of water borne diseases within households and schools in the district.

Sustainability: The operation managed to achieve short-term improvements with potential long-term sustainable results. The emergency cash transfers were meant to be a short-term relief intervention as such they were phased out within the agreed period. Evidence from the consultations and the case studies suggest there are long-term benefits that will continue among beneficiaries like cell phone connectivity, those that started IGAs and those that invested in housing or purchase of assets like land. Within the agricultural productivity farmers gained access to seed for some crops which are recyclable and gained skills and knowledge related to climate smart farming which will continue to assist farmers increase productivity. Similar WASH infrastructure if well maintained will continue to provide water to the community for years and hygiene education will assist with medium to long term behaviour changes. Although the project did not develop a dedicated exit strategy at inception it managed to have some aspects related to exit strategies streamlined within the execution of the interventions. There is also evidence of institutional collaboration that will ensure continuity and sustainability of some of the initiatives. Solar based water and irrigation systems are great demonstration that project was sensitive to environmental sustainability.

Coherence and Coordination: The operation demonstrated adequate coherence as its focus aligned well with ZRCS, IFRC and Zambia Government policy objectives. The interventions in the Kalabo Operation supported the 2015 National Disaster Management Policy, 2030 IFRC’s Strategic goals and IFRC Southern Africa Disaster Risk Management Strategy among others. Movement coordination and broader project level of cooperation at multiple levels worked well. There was adequate cooperation between and among international, national, and community actors for the project.

Efficiency and Value for money (vfm): A comprehensive criterion of efficiency and vfm was not applied fully in this assessment due to data limitations. More focus was therefore put into assessing the timeliness of delivery of interventions under a crisis. The evaluation revealed that the outputs were completed in specified time and within allocated budget regardless of the delays to declare the drought emergency that subsequently delayed the IFRC Appeal processes. Cost efficiency for rural humanitarian interventions is not easy to calculate in part due to lack of cost-estimation methods, limited scope for comparison to national and global benchmarks and limited capabilities related to sensitivity analyses. However, based on the information provided, cost per beneficiary can be estimated at USD 189 per household or USD 37 per person. The operation objectives were achieved on time and both recruitment and procurement processes were competitive and due diligence was taken as observed in the selection and contracting of the individuals and the service providers.

Mainstreaming of Accountability, gender and inclusion: There was no dedicated project gender mainstreaming strategy, but gender was streamlined across all interventions. For instance, initial consultations and the project baseline had some form of gender analysis although not too elaborate. Community based targeting considered gender and inclusion dimensions in prioritising poor female headed households, lactating females, child headed households, households with orphans and the elderly.

In terms of accountability and transparency, IFRC and ZRCS accountability systems were applied for Kalabo operation including the financial systems at all levels throughout the entire project cycle. Project financial documents including cash transfer files are held in a transparent manner and files are accessible on demand. There are further internal ZRCS and IFRC systems to prevent, measure, and report fraud and corruption not unique to the Kalabo Operation but institutional in nature. The Kalabo Operation also had adequate internal monitoring and control mechanisms including a logical framework which had a slight mix up in the intervention pathway although a minor reorganization or re-arrangement of the pathways would have still contributed to the same broader outcome.

There was adequate monitoring of implementation and processes including the targeting, registration, and distribution processes. A project baseline was carried out to set the out key indicators for monitoring progress, however it was not as comprehensive to cover all key required indicators. The project did not develop a dedicated Monitoring and Evaluation (M and E) plan which normally assist staff and partners with adherence to set M and E commitments. Post distribution monitoring exercises were undertaken for the cash transfer programme.

Main Conclusions:

• Lives saved, livelihoods safe guarded, dignity preserved, and gender equality improved: The interventions supported by the Kalabo Drought Operation contributed to restore and strengthen livelihoods of the drought affected households lifting majority out of hunger, risk of diseases and death, restoring their asset base and livelihoods in the process. The Cash transfers assisted the targeted beneficiaries to meet their basic needs such as food, shelter and generate an income for other household needs. This intervention preserved people’s dignity as it gave them greater choice for which to buy unlike if they were provided maize. Cash transfer was appropriate in the Kalabo case as the supply of food to markets and shops was sustained but affected communities had lost the means to buy it. Beyond the cash transfers the households also increased food production from own gardens due to agricultural inputs and training provided through the intervention. Clean and safe water sources have been increased for household and agricultural purposes plus hygiene awareness and education has been provided.

• Advocate for reforms in drought declaration processes and complementarity with social cash transfers and emergency cash transfers: Currently in Zambia and other SADC countries technical evidence regarding drought emergencies is ignored or downplayed by political interference consequently this delays drought response fundraising and programming. On the other hand, IFRC based on the Kalabo experience IFRC can focus on raising stakeholder awareness to reduce existing misconceptions related to social cash transfers with emergency cash transfers in term s of design, execution, and complementarity.

• The future is cash-plus: Cash-plus approaches are proving to provide better impact in emergency droughts responses, but the other interventions such as WASH, support agricultural productivity, health services among others may not be measured together with the cash component.

• Beyond drought response towards adaptive and resilient livelihoods: There is need to consider social protection and resilient livelihoods in Kalabo District. Social protection as the appropriate intervention in the longer term for current beneficiaries mostly those who were beneficiaries of the cash component of the drought response and for the other high potential farmers to consider adaptive and transformative capacity strengthening towards resilient livelihoods. More women compared to men were the targeted beneficiaries in their role as household managers and mothers and were empowered to fulfil these roles more effectively through the intervention. While no significant long-term change in women’s roles or behaviours can be inferred at this point intra-household effects on gender such as reductions in domestic violence and better relations between spouses and other household members were reported.

• Technology in emergency drought response: Embracing integration of technology within drought emergency response mechanisms beyond food trucks harness innovative technologies such as mobile-money based cash transfers, solar water pumping systems and multi purposes water utilization for domestic and productive use.

Main Recommendations:

• Hybrid approach to cash transfers: mobile money is not always reliable in remote areas due to coverage limitations. Therefore, there is need for hybrid solutions to cash disbursements involving multiple mobile money service providers, using cash in transit facilities or linking beneficiaries to the existing banking systems.
• Collaborative assessment key to better project design: It is important to undertake a thorough assessment of the preferred and practical mechanisms for disaster-affected people before setting up any mechanism.
• You win the leaders you win the community: Traditional leaders need to be sensitized on who the project is targeting and should be involved in the validation of the beneficiary list.
• Improve Monitoring and Evaluation for Impact Analysis: A dedicated project M and E framework and plan is a prerequisite for all interventions of this nature. The ZRCS project log frame and baseline were not comprehensive. The impact pathways were mixed up and the baseline only focused on beneficiary households without considering surveying a control population. Similarly, the post distribution monitoring was mostly focused on checking if the money was received and did not investigate matters such as dietary intake and diversity.
• Peer to peer learning for enhanced productivity: For livelihood interventions such as input support efforts, ZCRS should consider organizing farmers to work in groups to boost production and better manage and training beneficiaries in good agricultural practices. Demonstration farms or plots should be used to teach various agricultural techniques and technologies, showcase new or improved crops to encourage farmers to adopt and apply these in their own fields.
• Plan project activities with consideration on Access to Kalabo: Expected seasonal changes between project inception and closeout and their possible impact on the timeliness and deliverables need to be anticipated and controlled at planning stage. There were incidences of inputs being delivered late or boreholes being sunk late due to seasonal changes that resulted into impassable roads.
• Support to traders during cash transfer programmes: There is need to consider supporting traders with once-off grants that will help them to restock their shops with food stuffs and essential items to meet the high demand triggered by excess cash during future cash transfer programmes. Grants to traders would not only prevent/ reduce food shortages but also prevent inflation resulting from excess cash and high volume of activity on local markets. This would in turn keep prices stable.
• Follow up on investment made: Since significant resources in terms of funds, human resources and time were made available to ensure that water supply for domestic and irrigation system are installed and functional it will be critical that ZRCS make a deliberate follow up plan on these investments linked to development programming in the area to ensure sustainability of the investment is made. As part of this follow up process a detailed market assessment would be recommended to understand market systems in the area.