Multi-dimensionality of measurement in international evaluation: Challenges and opportunities

Session Number: ICCE3
Track: International and Cross Cultural Evaluation
Session Type: TIG Multipaper
Session Chair: Sarah McLaughlin [Deputy Director of Learning and Evaluation - Alliance for Peacebuilding]
Presenter 1: Hubert Paulmer [Partner - Goss Gilroy Inc]
Presenter 2: Vijayalakshmi Vadivelu, Dr., Evaluation Advisor [Evaluation Advisor - United Nations Development Programme]
Presenter 3: Kate Goddard Rohrbaugh [Program Analyst (Evaluation) - Peace Corps, OSIRP]
Presenter 4: Andrea Brown Murga [Evaluation Officer - Institute of International Education]
Presentation 3 Additional Author: Kate Goddard Rohrbaugh [Program Analyst (Evaluation) - Peace Corps, OSIRP]
Time: Oct 27, 2016 (04:45 PM - 06:15 PM)
Room: International North A

Abstract 1 Title: Retrospective "Outcome Harvesting": A Case from East Africa - Experience and Lessons
Presentation Abstract 1:

Outcome measurement has become one of the most important aspects in the international development initiatives. But what do we do when we have to design an evaluation to measure outcomes, nine months after the project completion and it is too early to see the impact of the intervention? Typically, the traditional measurement will assess the achievement of the intended outcomes, the behavioural changes. However, outcomes can be positive or negative, intended or unintended, direct or indirect, but the connection between the intervention and the outcomes has to be plausible. This session draws on experiences from designing and conducting the evaluation, assessing outcomes, of a health project in Tanzania using an adapted “Outcome Harvesting” approach. The session presents highlights, lessons learned, and challenges from the experience. Furthermore, the session also presents an overview of the Outcome Harvesting concepts and approach.

 


Abstract 2 Title: Multi-dimensional approaches to measuring poverty: Strengths and challenges
Presentation Abstract 2:

Drawing from an in-depth analysis of 30 countries, carried out as part of the evaluation of the contribution of global Human Development Reports to the public policy process, this paper discusses issues pertaining to the application of capability approach and framework to poverty measurement. The paper examines how the capability concept has been applied to measuring poverty and inequality, and how the framework has informed policy processes. First, the paper provides a brief review of the use of capability concept in poverty measurement and public policy, its distinctness from other approaches that has long dominated development discourse, as well as the complementary areas.  Second, the paper analyses empirical applications of the concept, at global and national levels, particularly in poverty measurements. Third, makes a critical assessment of the multidimensional poverty measurements (MPI), their strengths and limitations. The paper concludes by providing insights and lessons for poverty measurements; and the importance of contextual specificities that make the indices more relevant for policy making.Drawing from an in-depth analysis of 30 countries, carried out as part of the evaluation of the contribution of global Human Development Reports to the public policy process, this paper discusses issues pertaining to the application of capability approach and framework to poverty measurement. The paper examines how the capability concept has been applied to measuring poverty and inequality, and how the framework has informed policy processes. First, the paper provides a brief review of the use of capability concept in poverty measurement and public policy, its distinctness from other approaches that has long dominated development discourse, as well as the complementary areas.  Second, the paper analyses empirical applications of the concept, at global and national levels, particularly in poverty measurements. Third, makes a critical assessment of the multidimensional poverty measurements (MPI), their strengths and limitations. The paper concludes by providing insights and lessons for poverty measurements; and the importance of contextual specificities that make the indices more relevant for policy making. 


Abstract 3 Title: Measure Locally, Evaluate Globally: Establishing Internal Reliability in Quantifying Peace Corps Outcomes
Presentation Abstract 3:

Between 2008 and 2012, the Peace Corps’ Office of Strategic Information, Research, and Planning coordinated studies in 24 countries to better understand the impact of its volunteer programs. While the resulting data and findings were initially disseminated through discrete country reports, the data have since been combined to generate a cross-sectional global review of the Peace Corps’ impact. This session will address the challenges of merging and analyzing data from hundreds of interviews that used a largely consistent protocol, but which were captured in 26 different languages over a five-year period. The session will also address the solutions that were developed to maintain validity and generate a rigorous and fair analysis of the Peace Corps’ impact.


Abstract 4 Title: Evaluating Multilateral Research Partnerships: An Evaluation of the Global Innovation Initiative
Presentation Abstract 4:

This paper will discuss the evaluation of a multilateral research partnership program, the Global Innovation Initiative (GII). Sponsored by the U.S. and British governments and implemented by the Institute of International Education (IIE) and the British Council, GII seeks to strengthen global multilateral collaboration through grants to university consortia that focus on STEM-related issues of global significance. IIE is conducting a mixed-methods evaluation of GII that involves group-level observation and analysis and relies on a modified version of the Kirkpatrick model. The presenter will provide an overview of midterm evaluation findings and reflect upon the pros and cons of key aspects of the evaluation design, namely: (1) group-level observation and analysis; (2) measuring outcomes and impacts unique to international partnerships (such as intercultural competence); and (3) the utility of the Kirkpatrick model as a framework for measuring gains in institutional capacity.

 


Audience Level: None

Session Abstract:  Multi-dimensionality of measurement in international evaluation: Challenges and opportunities