Evaluation 2017: From Learning to Action

View Printable Version

Integrating Economic Evaluation with Other Ways of Evaluating Impact, Value and Equity

Session Number: 2686
Track: Costs, Effectiveness, Benefits, and Economics
Session Type: Panel
Session Chair: Ronald Scott Visscher [Evaluator, Systems Developer and PhD Student - RSVPro and Western Michigan University]
Presenter 1: Hubert Paulmer [Partner - Goss Gilroy Inc]
Presenter 2: Julian C King [Director, Julian King & Associates Limited - Kinnect Group]
Presenter 3: Nicky Pouw [Associate Professor - University of Amsterdam]
Presenter 4: Ronald Scott Visscher [Evaluator, Systems Developer and PhD Student - RSVPro and Western Michigan University]
Time: Nov 08, 2017 (06:15 PM - 07:15 PM)
Room: Wilson A

Abstract 1 Title: Value for Money: lessons and perspectives
Presentation Abstract 1:

It is argued that value for money (VfM) can be useful and relevant in the development cooperation context, as long as it is applied pragmatically with an understanding of the limitations of the concept. VfM is gaining more prominence and is being championed  in different ways by various development partners (funding agencies).  What does VfM mean? How can VfM add value or not to the evaluations that we conduct? Is it a tool, a method, or a way of thinking? The paper explores the various definitions on VfM and diverse perspectives of development practitioners/policy makers to provide an understanding of the concept. So does the beneficiaries’ VfM opinion count and how? The paper looks at the “4E” approach to analyzing VfM and shares lessons from practical experience evaluating projects in the international development context. What are/have been the challenges in assessing VfM?


Abstract 2 Title: Using rubrics to integrate economic evaluation with other evidence
Presentation Abstract 2:

In the March, 2017 issue of the American Journal of Evaluation, Julian King argues that evaluators need to "reach across disciplinary boundaries and make better use of economic methods". However, he argues, while economic analysis can enhance evaluation it is "usually insufficient to fully answer the evaluative question" (for example, because it privileges economic efficiency and quantitative valuing) and that "a stronger approach would involve explicit evaluative reasoning, supported by judicious use of economic and other methods". 

In this presentation, Julian will provide an example of what this approach looks like in practice. Drawing on recent evaluations of international development programs, a framework will be presented that builds on the "4Es" approach to assessing value for money (DFID, 2011), using criteria and standards (e.g., rubrics) to guide evaluative reasoning in a mixed methods evaluation of value for money, drawing together multiple strands of evidence including narrative, measurement and economic analysis. 


Abstract 3 Title: Participatory Assessment of Development; lessons learned from a new evaluation methodology in Ghana and Burkina Faso
Presentation Abstract 3:

In this presentation, the key premises and field-testing findings of a new Participatory Assessment of Development (PADev) evaluation approach will be presented. PADev was co-designed with Dutch NGOs and Northern and Southern research institutes over a period of four years in the context of rural development in Ghana and Burkina Faso. Although participatory approaches in development evaluations have become widely accepted since the 1990s, the PADev approach is different by taking the principles of holism and local knowledge as starting points for its methodological elaboration and case selections. The PADev approach complements econometric approaches by assessing the differentiated impacts of development interventions through inter-subjectivity. If PADev is taken-up by a multitude of stakeholders, including the intended beneficiaries of development interventions and development stakeholders, it can contribute to a process of local history writing, knowledge sharing, capacity development and agenda setting, and providing input into community action plans and the strategies of CBOs and NGOs.


Abstract 4 Title: Coevaluation by interdependent ‘partners’ to overcome ‘gaps’ in service of greater need and greater good
Presentation Abstract 4:

Bidirectional coevaluation among interdependent subjects, or development ‘partners’, can inspire real mutual understanding, problem-solving, coordination and equity. Partners may include local or remote (up and down-stream) beneficiaries, policy-makers, development practitioners and others, represented by self or others as individuals and groups. Mobile communicators access uniform, yet flexibly adaptable, evaluation frameworks through which data flows to and from secure yet freely available ‘personal domains’ in the cloud. This facilitates fitting design and participative inclusion of appropriate perspectives, methods, criteria, metrics, standards and rubrics, enabling empowered bidirectional coevaluation of unique individual needs, values, capabilities and actions of potential and actual partners. Ex-anti estimates and ex-post conclusions regarding value, impact and equity are vetted throughout iterative phases of development and agreement. Thus, intentions are transparently discerned, shared, affirmed, enacted and monitored in real-time. Meanwhile, ‘learning’, ‘application’, ‘use’ and ‘fulfillment’ ‘gaps’ may be conscientiously overcome, resulting in coordinated need satisfaction and economic resource stewardship.


Theme: Learning What Works and Why
Audience Level: All Audiences

Session Abstract (150 words): 

Despite expectations that evaluations include conclusions about comparative value, most do not. This session demonstrates how to systematically integrate economic evaluation with other evaluation methods to synthesize valid conclusions about comparative value, impact and equity. First, program-centric economic and rubric methods are presented, and second, community-wide inter-subject participative methods. We then conclude by showing how such top-down and bottom-up methods combine to bring valid determination of value-for-money (VFM), impact and equity within reach. This enhances cross-cultural understanding and development harmonization while empowering holistic well-being of all stakeholders, including beneficiaries with greatest needs.

We will provide new insights for seasoned professionals, while reducing the mystery of determining value-for-money (VFM), impact and equity for aspiring evaluators. Together these complementary presentations teach how to integrate necessary evaluation components (perspectives, frameworks, methods, criteria, metrics, standards, and rubrics) in a cohesive and context-sensitive way using a paradigmatic example about enhancing health, education, and productive capacity.



For questions or concerns about your event registration, please contact registration@eval.org or 202-367-1173.

For questions about your account, membership status, or help logging in, please contact info@eval.org.



Cancellation Policy: Refunds less a $50 fee will be granted for requests received in writing prior to 11:59 PM EDT October 16, 2017. Email cancellation requests to registration@eval.org. Fax request to (202) 367-2173. All refunds are processed after the meeting. After October 16, 2017 all sales are final. For Evaluation 2017, international attendees and presenters who encounter complications due to the international travel environment will have up to 30 days after the event to request a refund and submit appropriate documentation. No administrative fee will apply for the international requests. The $50 fee will be waived for registrants who planned to travel into the US and experienced international travel issues.