Innovative Evaluation Designs in International Contexts: An Overview of Three Case Studies

Session Number: 2513
Track: International and Cross Cultural Evaluation
Session Type: Panel
Tags: effectiveness
Session Chair: Shawna Hoffman [Monitoring and Evaluation Specialist - The Rockefeller Foundation]
Presenter 1: Kristin Lindell [Monitoring, Evaluation, Research and Learning Specialist - USAID LEARN/Dexis Consulting Group]
Presenter 2: Ja-Eun Lee [M&E associate - One Acre Fund]
Presenter 3: Rogonga Augustine [M&E Program Manager]
Time: Oct 27, 2016 (03:00 PM - 04:30 PM)
Room: L508

Abstract 1 Title: Learning for Monitoring and Evaluation
Presentation Abstract 1:

On the USAID Learning and Knowledge Management contract (LEARN), we believe that learning is the primary purpose of monitoring and evaluation systems. In this panel, we'll talk about how we've set our contract up to track progress toward our goals in a way that fuels continuous learning and improvement. 

Abstract 2 Title: Qualitative Evaluation Design within Cultural and Organizational Constraints: A Rwandan Case Study

Presentation Abstract 2:

In qualitative research where the interaction between an interviewer and an interviewee is an important determinant of quality, mitigating potential obstacles from cultural and linguistic differences in evaluation design may be a common challenge among evaluators. Moreover, budget and timeline constraints add another layer of complexity. One Acre Fund’s (1AF) longitudinal study assessing the improvements in quality of life among small-holder farmers in Rwanda can be a good example of an evaluator’s endeavor to address these two issues through evaluation design. 1AF is an International NGO working in East Africa that supports smallholder farmers through training and providing agricultural inputs on credit with favorable terms. This paper illustrates how the evaluator navigated the complexities of different cultures and languages, and time and budget constraints. We explore the trade-offs between various strategies, and the unique role of ethnicity and culture in Rwanda in shaping interview and focus group content.

Abstract 3 Title: Measuring the Impact of a Single-Sector Model Versus an Integrated Approach in Kenya

Presentation Abstract 3:

The social venture Nuru International seeks to eradicate extreme poverty through an integrated, holistic approach. By targeting smallholder farmers in remote rural areas, Nuru addresses some of the key stressors of extreme poverty through the implementation of four impact programs: Agriculture, Financial Inclusion, Healthcare and Education. To support Nuru in making data driven decisions in its Kenya project, the Monitoring and Evaluation (M&E) team designed a quasi-experimental evaluation to examine the effectiveness of the integrated model versus a single sector or vertical approach. This presentation will review this innovative study design in hopes of contributing to the broader conversations around measuring the impact of integrated programming.

Audience Level: Intermediate

Session Abstract: 

For many organizations working in an international context, measuring social impact can prove to be a burdensome endeavor. Oftentimes, the costs and time commitments involved with implementing a robust study causes some stakeholders to forgo comprehensive measurement systems altogether; however, various organizations have developed strategies to overcome these obstacles. To this extent, the discussion starts with a presentation of One Acre Fund’s qualitative evaluation design, which assesses quality of life improvements due to its agriculture intervention. It then segues into a review of Nuru International Kenya’s quasi-experimental evaluation, which was designed to analyze the effectiveness of the organization’s integrated model versus a single-sector model. Lastly, it closes with a presentation on USAID Learning and Knowledge Management (LEARN) contract's internal monitoring, evaluation, and learning system. Ultimately, the panel aims to provide evaluators with several examples of cost-effective and innovative evaluation designs that fuel learning and adaptive management.