Adventures in Tracking Orphan and Vulnerable Children (OVC) Outcomes: PEPFAR Monitoring, Evaluation, and Reporting (MER) Essential Survey Indicators

Session Number: 2794
Track: International and Cross Cultural Evaluation
Session Type: Panel
Session Chair: Christine Fu [Senior Evaluation Officer - UNHCR]
Presenter 1: Susan K Settergren [Technical Director - Palladium]
Presenter 2: Walter Obiero
Presenter 3: Jenifer Chapman [Palladium]
Presenter 4: Lisa Parker
Time: Nov 09, 2017 (08:00 AM - 09:00 AM)
Room: Washington 3

Abstract 1 Title: What are the OVC Essential Survey Indicators Telling us? Insights From Three MER Surveys in Kenya
Presentation Abstract 1:

In 2016, the USAID- and PEPFAR-funded MEASURE Evaluation conducted surveys to collect the first round of orphans and vulnerable children (OVC) monitoring, evaluation, and reporting (MER) essential survey indicators (ESI) for three PEPFAR-funded projects in western Kenya. The projects delivered similar packages of OVC services and had been providing services to OVC populations in the region for at least five years. As of early 2016, beneficiary populations ranged in size from 5,000 to more than 180,000 OVC. Survey design and implementation adhered to MER ESI guidelines. Indicator results were calculated as specified and shared with the OVC projects and stakeholders for validation and interpretation. We will present outcomes of this review process, highlighting insights, lessons, and challenges in interpretation. We will discuss strategies to improve interpretation and optimize use of survey results, including additional analysis and data collection during the next survey round.    


Abstract 2 Title: Lessons Learned from the Implementation of the PEPFAR Orphans and Vulnerable Children Monitoring, Evaluation, and Reporting Essential Survey Indicators in Nigeria
Presentation Abstract 2:

The USAID- and PEPFAR-funded MEASURE Evaluation, in partnership with a local organization, conducted a cross-sectional household survey among currently enrolled beneficiaries of five PEPFAR- funded orphans and vulnerable children (OVC) projects in Nigeria. These projects were estimated to serve more than 400,000 beneficiaries. A total of 216 clusters were randomly selected using equal probability selection procedures, from which a total sample of 3,100 caregivers were drawn. The survey covered the nine OVC monitoring, evaluation, and reporting (MER) essential survey indicators (ESI) and four Nigeria-specific outcome indicators for OVC. Implementation was hampered by such challenges as out-of-date beneficiary databases, and the lack of a conceptual framework linking project activities to the indicators. Based on lessons learned, this presentation will describe the issues, our solutions, and implications for data interpretation and global standardization.


Abstract 3 Title: Collecting the OVC PEPFAR Monitoring, Evaluation, and Reporting Essential Survey Indicators in Mozambique: Design Challenges and Solutions
Presentation Abstract 3:

Under the USAID/Mozambique-funded COVida project, Palladium is collecting PEPFAR’s monitoring, evaluation, and reporting (MER) orphan and vulnerable children (OVC) essential survey indicators (ESIs). COVida is a five-year national project aiming to reach 300,000 OVC in Year 1. To accomplish this, COVida adopted an innovative care model: beneficiaries may transition in as little as three months (if needs are met), while others may remain enrolled until needs are met. If Palladium had adopted the standard MER ESI approach, they would have been unable to interpret changes detected, because populations at baseline and endline would have differed vastly. Thus, Palladium adapted the MER ESI approach to fit the real-world situation. The design considerations encountered and Palladium’s approach to resolving them has influenced the PEPFAR MER ESI guidance: an example of reciprocal learning. This presentation will describe the issues, the solutions, and the implications for data interpretation and global standardization.


Abstract 4 Title: Development and Real-time Adaptation of the Orphan and Vulnerable Children Monitoring, Evaluation, and Reporting Essential Survey Indicator Guidance

Presentation Abstract 4:

As part of its monitoring, evaluation, and reporting (MER) guidance, the U.S. President’s Emergency Plan for AIDS Relief (PEPFAR) launched a set of outcome indicators for orphans and vulnerable children (OVC) programs in 2014. These outcome indicators were designated as “essential survey indicators,” and are a reporting requirement. The purpose of collecting data on these indicators is to obtain a snapshot of PEPFAR-funded project outcomes at a point in time and to assess changes in outcomes among OVC project beneficiaries over time. These outcome data will support evidence-informed strategic portfolio development, programming, and resource allocation decisions. This presentation includes an overview of the stakeholder-driven process of selecting the indicators and developing the guidance. It will describe the recent evolution of PEPFAR’s OVC programming and how those changes required adaptation of implementation guidance. We will describe how lessons learned were incorporated in revised guidance and detail how this is being disseminated.


Theme: Learning to Enhance Evaluation
Audience Level: All Audiences

Session Abstract (150 words): 

This session covers the USAID- and PEPFAR-funded MEASURE Evaluation’s experience in supporting guidance on outcomes monitoring for PEPFAR’s orphans and vulnerable children (OVC) programs. The session will present an overview of the stakeholder-driven process of selecting the indicators and developing the guidance. It will describe the recent evolution of PEPFAR’s OVC programming and how those changes required adaptation of implementation guidance. We will highlight the real-time learning that has taken place in the design, sampling, implementation, analysis, reporting, and use of the Monitoring, Evaluation, and Reporting (MER) essential survey indicators (ESI). Panelists from MEASURE Evaluation and Palladium from the COVida project will share their experiences designing and implementing MER surveys in Kenya, Nigeria, Namibia, and Mozambique. We will then describe how we incorporated these lessons learned in revised guidance and detail how we are disseminating this information. This session will highlight the importance of adaptability in today’s complex and resource-constrained evaluation environment.