Evaluation 2017: From Learning to Action

View Printable Version

"Assessing M&E practice, evidence and Return on Investment (ROI) of interventions in complex emergency settings".

Session Number: 3168
Track: Disaster & Emergency Management Evaluation
Session Type: Panel
Tags: Actionable Results, advancing evidence, after-action report, Aid dependency, aid effectiveness, aid organizations, Analysis/interpretation:mixed methods, behavior change, benefit-cost analysis, Budget Data in Evaluation, Causality, client relationships, community context, Cost Benefits, Cost-effectiveness analysis, credibility and use, Criteria and Standards, development cooperation, disasters, displaced populations, efficiency, emergency preparedness, emergency response, Evaluation criteria, evaluation guidelines, evaluation practices, evaluation stakeholders, evaluation tools, evidence, evidence synthesis, food security, Foreign assistance, fragile and conflict environments, generalizability, health and social services, Humanitarian Evaluation, Humanitarian Response, Impact Assessment, lessons from the field, meta-evaluation, mortality and morbidity, nutrition, OECD-DAC, post-conflict, public health, Resilience, shelter, Social science methods, stakeholder involvement approaches, sustainability, systematic review, validity, Valuation, Vulnerable populations
Session Chair: Michael Zeleke [Knowledge Management and Learning Advisor - Macfadden and associates / USAID/OFFA ]
Presenter 1: steven hansch, MPH [Director - IBTCI]
Presenter 2: Dale Miller Hill [Senior Consultant - Independent]
Time: Nov 09, 2017 (01:15 PM - 02:00 PM)
Room: Marriott Balcony B

Abstract 1 Title: Donor Guidelines for Emergency Relief Evaluations: How They Treat Return on Investment and Value for Money
Presentation Abstract 1:

Steven Hansch will compare the evolution of donor approaches to monitoring and evaluation of relief, including the gradually emerging attention to Return on Investment (ROI) and Value for Money (VfM). RoI is more often referenced in the United States and VfM in the United Kingdom and Canada.  This session will articulate how these tools can be readily applied in crisis situations to determine the effectiveness of health, nutrition, shelter and food aid programs. Ideally, the analysis of the “bang for the buck” would be predicated both on distinct theories of change of projects as well as their ability to scale up.  The speaker will also draw from real world cases, e.g. post-earthquake Haiti, to critique the reluctance of operational and donor agencies to learn how best to use these tools, and the implications of the wide range of calculated ROIs seen across interventions and settings when they were applied.


Abstract 2 Title: Assessing Strength of Evidence on Effectiveness of Humanitarian Interventions: Donors’ and Stakeholders Perspectives; Rigor and Actionability.
Presentation Abstract 2:

Dale Hill, Independent Consultant, will discuss the need for evidence on the effectiveness of humanitarian interventions. The most popular criterion for judging evidence has been based on rigor of methods. Ms. Hill will present a wider set of perspectives. First she will assess how well the United Kingdom Department for International Development‘s guidance-“Assessing the Strength of the Evidence” (2014)- applies to the humanitarian sector. Second, she will discuss the importance of stakeholders’ perspectives in judging quality of evidence—it must be actionable and inform the decisions they wish to make. Ms. Hill will present cases to illustrate how these different perspectives can be applied in humanitarian situations. She may also comment on  the previous speaker's presentation,  noting the limitations of considering cost-effectiveness alone, and the  importance of considering criteria used in behavior theory such as acceptability, practicability, affordability, safety/side-effects, and equity, all criteria on which stakeholders need to be consulted.   


Theme: Learning What Works and Why
Audience Level: Intermediate

Session Abstract (150 words): 

The session will explain guidelines that have emerged over time to judge the effectiveness of humanitarian action, and the degree to which they have proved successful in application on the ground. The first presenter will focus on guidelines on cost-effectiveness, which, from one donor have favored “Return on Investment” methods, and for others, “Value for Money”. The presenter will talk about the differences, how they should be applied, and some case evidence on the still wide range of analysis they bring to the still tricky subject of effectiveness of humanitarian aid. The second presenter will discuss recent guidelines issued by the United Kingdom’s Department for International Development (DFID) to judge quality of evidence, and their relevance in particular cases, adding the importance of considering also stakeholder perspectives which require actionability, and a contribution to the decisions they wish the study or evaluation to inform. 



For questions or concerns about your event registration, please contact registration@eval.org or 202-367-1173.

For questions about your account, membership status, or help logging in, please contact info@eval.org.



Cancellation Policy: Refunds less a $50 fee will be granted for requests received in writing prior to 11:59 PM EDT October 16, 2017. Email cancellation requests to registration@eval.org. Fax request to (202) 367-2173. All refunds are processed after the meeting. After October 16, 2017 all sales are final. For Evaluation 2017, international attendees and presenters who encounter complications due to the international travel environment will have up to 30 days after the event to request a refund and submit appropriate documentation. No administrative fee will apply for the international requests. The $50 fee will be waived for registrants who planned to travel into the US and experienced international travel issues.