Evaluation 2015: Exemplary Evaluations in a Multicultural World

View Printable Version

Update on Exemplary Complexity-Aware Monitoring and Evaluation Trials in the United States Agency for International Development (USAID)

Session Number: 1676
Track: Presidential Strand
Session Type: Panel
Tags: systems-oriented evaluation, USAID
Session Chair: Patricia L Rader [USAID]
Discussant: Bob Williams [All over the place]
Presenter 1: Travis Mayo [USAID]
Presenter 2: Ricardo Wilson-Grau [Ricardo Wilson-Grau Consultoria em Gestão Empresarial Ltda]
Presenter 3: Lorina McAdam [Mercy Corps Iraq]
Presenter 4: Heather Britt [USAID/PPL/MECap]
Time: Nov 13, 2015 (05:30 PM - 06:15 PM)
Room: Plaza A

Abstract 1 Title: Overcoming Common Challenges to Applying Complexity-Aware Approaches
Presentation Abstract 1: Those who recognize the value of complexity-aware approaches often face challenges when adopting them in the field. Foremost among these challenges are the difficulties in prioritizing monitoring needs related to complexity and selecting an appropriate approach to meet the need. Dissatisfaction with results can often be traced back to decisions made in the early stages of design. USAID is piloting an approach to support practitioners through the early stages of designing a complexity-aware M&E approach in order to better prepare trial partners for working with external experts. M&E innovators also face challenges when adopting complexity-aware approaches to their project and context. Lack of intentional learning can undermine attempts at organization-wide change. Rather than rely on individual trail-blazers, USAID advocates for rewarding M&E innovators with well-planned technical assistance. In return trial partners contribute to a learning agenda that will ultimately inform guidance about applying complexity-aware approaches in USAID programming more broadly.
Abstract 2 Title: External Expert Report on a Field-Based Trial of Outcome Harvesting as a Complexity-Aware M&E Approach
Presentation Abstract 2: This presentation explains how the Outcome Harvesting six-step approach was customized to the specific needs of USAID’s ¡Honduras Convive! program, whose goal is to contribute to reducing homicides and other violent crime as part of U.S. Government support to Honduras to stem rising criminality and violence. The presenter explains how the primary intended users and the external facilitator of the M&E process ensured that the Outcome Harvesting process was designed to meet their needs and then re-designed as the process generated findings and encountered and resolved obstacles. The challenges of collecting and substantiating data about trust between communities and government institutions working as active partners through small-scale, quick-impact, strategically-targeted initiatives to prevent crime and violence are also described. Equally important are the difficulties of analyzing and interpreting the data in order to answer useful, actionable questions for the ¡Honduras Convive! team.
Abstract 3 Title: Implementing Partner Report on a Field-Based Trial of Complexity-Aware Monitoring Approaches
Presentation Abstract 3: Implementing partners are critical to real and lasting improvements in monitoring and evaluation practice. What are implementing partners’ perspectives on, and experiences with, the complexity-aware monitoring initiative to-date? The third presentation describes the trial of Outcome Harvesting in the ¡Honduras Convive! program from the perspective of Creative Associates, USAID’s implementing partner. The presenter discusses the extent to which the Outcome Harvesting trial was effective in informing adaptive management in the field, and outlines the challenges of working across geographical and organizational boundaries to support trials. She outlines key enabling factors supporting M&E innovation in her organization and the unique contributions implementing partners such as Creative Associates bring to this donor-supported initiative. She also discusses the feasibility of expanding complexity-aware M&E approaches more broadly in her organization’s programming.
Abstract 4 Title: Action Research Findings on Applying Complexity-Aware Monitoring and Evaluation Approaches
Presentation Abstract 4: The fourth presentation outlines key learnings from several field-based trials. This is a real-time, front-lines report on what helps and hinders application of systems concepts and approaches in a large, multi-lateral donor. USAID has instituted a learning agenda focused on answering core questions about whether and how the complexity-aware approaches add value, and what is required at both the project and organizational level to use them. The study also includes questions about each of the approaches being trialed. Results from the study are being used to develop field-friendly resources for applying these approaches and using data to inform adaptive management. With two trials nearing their conclusion, and more underway, the presentation will highlight key findings from the action-research study.
Audience Level: Advanced

Session Abstract: 

This panel provides an update on USAID’s initiative to trial complexity-aware monitoring and evaluation approaches in Agency-funded programs.  Most monitoring and evaluation (M&E) approaches measure the predicted – desired results, planned implementation strategies and forecasted pathways of change – using indicators expected to provide useful information over the life of the project.  Complexity-aware M&E complements performance monitoring by tracking the unpredictable, and informing adaptive management in dynamic contexts.  Presentations will describe USAID’s approach to supporting trials and answer questions about the value-added provided by the approaches.  Have the diagnostic tools, learning circles, and other forms of technical support translated into practical solutions to M&E challenges in the field?  Presenters from the donor, trial partners and external experts share their views.  



For questions or concerns about your event registration, please contact registration@eval.org or 202-367-1173.

For questions about your account, membership status, or help logging in, please contact info@eval.org.



Cancellation Policy: Refunds less a $50 fee will be granted for requests received in writing prior to 11:59 PM EDT October 6, 2015. Email cancellation requests to registration@eval.org. Fax request to (202) 367-2173. All refunds are processed after the meeting. After October 6, 2015 all sales are final.