Evaluation 2015: Exemplary Evaluations in a Multicultural World

View Printable Version

Working with Assumptions for Theories of Change

Session Number: 1548
Track: Program Theory and Theory-Driven Evaluation
Session Type: Panel
Tags: Assumptions, complexity, Theories of change
Session Chair: Nathan Morrow [Tulane University]
Discussant: Donna M Mertens [Independent Consultant]
Presenter 1: Apollo Nkwake [Tulane University]
Presenter 2: Thomas Archibald [Virginia Tech]
Presenter 3: Madhabi Chatterji [Columbia University]
Presenter 4: Nathan Morrow [Tulane University]
Presentation 2 Additional Author: Guy O'Grady Sharrock [Senior Advisor for Learning - Catholic Relief Services]
Time: Nov 13, 2015 (08:00 AM - 09:30 AM)
Room: Dusable

Abstract 1 Title: Clarifying concepts and categories of assumptions for use in evaluation
Presentation Abstract 1: This presentation discusses key concepts and assumption categories, synthesizes the development of the concept of assumptions in various lines of philosophical inquiry to introduce a typology of assumptions. The critical role and potential contribution of the articulation of diagnostic, prescriptive, causal, external and other assumptions in program design and evaluation are discussed.
Abstract 2 Title: Every Practitioner a “Knowledge Worker”: Promoting Evaluative Thinking to Enhance Capacity for Working with Assumptions
Presentation Abstract 2: The presentation introduces the identification of critical micro-assumptions as the building blocks of evaluative thinking in participatory program management and capacity development activities. The presentation examines the notion of “evaluative thinking” as it is applied in evaluation capacity building (ECB) initiatives designed to help program administrators and implementers reveal, question, and rethink the assumptions that pervade their work. Specifically, the presentation is focused on efforts undertaken by Catholic Relief Services (CRS)—a large international non-governmental organization (INGO) that implements social transformation projects worldwide—in collaboration with Virginia Tech to promote evaluative thinking in furtherance of the agency’s strategic agenda for monitoring, evaluation, accountability, and learning (MEAL). Evaluative thinking (ET) is an emergent concept in the field of evaluation that has recently garnered increased attention as various scholars work to better define and explicate the construct.
Abstract 3 Title: Modeling and Examining Assumptions Underlying Complex Social Interventions: Techniques and Tools for Improving Evaluation Designs
Presentation Abstract 3: The presenter applies assumptions as a unifying concept for selection and sequencing of methods in a complex programing environment. As a member of an international team, the author is presently evaluating the effects of a comprehensive mental health program for displaced and vulnerable refugees in Amman, Jordan. The program has multiple components delivered by a team of professionals who are a part of the International Medical Corps (IMC). The team’s charge is to examine mental health outcomes in refugees served, as compared to the “treatment as usual” protocol provided through government hospitals. In evaluation design terms, IMC’s program is a classic example of a complex social intervention. Outside the challenges of an unstable target population, numerous assumptions and complex systems of variables define the program and its context.
Abstract 4 Title: Building better boxes; an emergent typology of assumption aware tools to improve theory-based approaches to program design and evaluation
Presentation Abstract 4: Assumption aware tools for more appropriate and effective program theory development have not been systematically compared or organized into useful descriptive categories based on intended use. This paper reviews twelve alternative approaches with associated tools that work with assumptions to strengthen the causal linkages and theory for program design and evaluation. Evaluators interested in the potential benefits of understanding how programs work with different stakeholders in different contexts have often characterized evaluations that focus only on measuring achievement of predetermined objectives as ‘black box’ evaluations
Audience Level: None

Session Abstract: 

Assumptions are inherent in every aspect of program evaluation, yet there is a broad and uncoordinated literature on how to work with assumptions. The panel will systematically capture the state of practice, highlight commonalities linking existing and emerging approaches, as well as an emergent typology for working with assumptions in program designs, monitoring and evaluation. Presenters will also discuss the potential role of improved articulation of assumptions in improving evaluation practice:

  • how a focus on assumptions is a foundational yet so far poorly developed area of evaluation theory and practice,

  • the emergent approaches to working with assumptions, their challenges and potential,

  • a typology that links program theory with epistemological, ontological and methodological assumptions, a framework for determining when to use which assumption analysis tools, practical and applied, as well as

Next major steps in assumptions and theory driven approaches, and connections among the approaches and point out future directions of work in assumptions and program evaluation.

 



For questions or concerns about your event registration, please contact registration@eval.org or 202-367-1173.

For questions about your account, membership status, or help logging in, please contact info@eval.org.



Cancellation Policy: Refunds less a $50 fee will be granted for requests received in writing prior to 11:59 PM EDT October 6, 2015. Email cancellation requests to registration@eval.org. Fax request to (202) 367-2173. All refunds are processed after the meeting. After October 6, 2015 all sales are final.