Evaluation 2018: Speaking Truth to Power

View Printable Version

Negotiating Power and Equity Systems Change Evaluations

Session Number: 2669
Track: Systems in Evaluation
Session Type: Multipaper
Tags: community-based evaluation, equity, systems change
Session Chair: Pennie Foster-Fishman [Professor - Michigan State University]
Presenter 1: Pennie Foster-Fishman [Professor - Michigan State University]
Presenter 2: Lucy O Aymor
Presenter 3: Erin Watson [Michigan State University]
Presenter 4: Abby Wattenberg [Michigan State University]
Presentation 4 Additional Author: Erin Watson [Michigan State University]
Presentation 4 Additional Author: Pennie Foster-Fishman [Professor - Michigan State University]
Session Facilitator: Pennie Foster-Fishman [Professor - Michigan State University]
First Author or Discussion Group Leader: Pennie Foster-Fishman [Professor - Michigan State University]
Second Author or Discussion Group Leader: Lucy O Aymor
Third Author or Discussion Group Leader: Erin Watson [Michigan State University]
Fourth Author or Discussion Group Leader: Abby Wattenberg [Michigan State University]
Time: Nov 02, 2018 (11:30 AM - 12:15 PM)
Room: Hilton - Veterans Meeting Room C

Other Authors: Corbin Stanley, Rome Meeks, Tyler Virden
Abstract 1 Title: Re-engineering Our Evaluation of Systems Change through a Developmental Equity Lens
Presentation Abstract 1:

Traditional frameworks for evaluating transformative change provide sets of competencies and components that indicate various forms of systems change or ‘needle moving’ in community conditions. Although some frameworks distinguish between ‘early’ and ‘intermediate’ indicators, a coherent framework that considers the developmental process of change and how that process reveals distinct competencies, conditions, and potential power issues does not yet exist. Additionally, while current frameworks broadly aim to reduce inequities in community conditions, explicit equity language and outcomes are largely missing, whereby reinforcing the status quo. In this paper, we present our developmental framework for transformative community change and how this process emerges across critical facets of collective work. Responding to Wolff et al. (2017), we also make the case for putting the ‘equity’ back into our considerations of inequity as a core developmental dimension, and consider the tensions between calling out and embedding equity within systems change evaluation frameworks.

Presentation 1 Other Authors: Lisa Szymecko
Abstract 2 Title: The Struggle is Real: Revealing Hidden Power and Priorities in Systems Change Evaluations
Presentation Abstract 2:

While systems change evaluation can shed light on changes made to complex systems conditions, processes, and community outcomes, this paper argues that the process of evaluation can also provide evaluators with insights into the hidden power dynamics present within community collaborative partnerships. In this paper, we present data gathered from a statewide initiative, which illuminates the competing priorities at work within local collaborative partnerships. The data shows differences in stakeholder priorities for their collective work horizontally, across sector, and vertically, according to role within the collaborative, revealing previously invisible power differentials in stakeholder relationships. We discuss the implications of these findings for community collaboratives, and how this data was used by members of the collaborative to improve its effectiveness. We also consider the broader contributions of these insights to current theories of evaluation and learning.

Presentation 2 Other Authors: Corbin Standley
Tyler Virden
Abstract 3 Title: Title: Using system scanning methods to reveal and legitimize perspectives across power differentials
Presentation Abstract 3:

A system scan is a practical method evaluators and community stakeholders can use to engage stakeholders across multiple perspectives in understanding system conditions (e.g., mindsets; policies and regulations; human and system resources; relationships; and decision-making structures) and dynamics (e.g., feedback loops) affecting their efforts and change goals. The system scanning method is particularily well designed to help evaluators reveal and legitimize differences in perspectives across stakeholders with varying levels of power in the system. For example, the scanning method engages stakeholders in positions of power in critiquing their current boundaries around relevant perspectives for understanding targeted problems, and includes sense-making techniques that help stakeholders see patterns across different perspective groups while equalizing the legitimacy of all perspectives. This presentation will provide a description of how evaluators can use these methods and techniques to reveal and understand critical perspectives across power differentials, examples of real world application, and lessons learned.

Presentation 3 Other Authors: Rome Meeks
Abstract 4 Title: Navigating common evaluation dilemmas related to truth, power, and equity
Presentation Abstract 4:

Community systems are structured to reinforce differences in power and advantage across stakeholders. Because change efforts and programs are situated within this community system context, evaluators are often faced with dilemmas in how to traverse complex power dynamics within each phase of their evaluation efforts. This presentation describes approaches evaluators can use to navigate common dilemmas related to speaking truth to power including: determining whose truth deserves our loyalty, raising the critical consciousness of system power brokers whose current truth is ultimately contributing to targeted challenges and problems, and engaging in conversations and collaborative settings in ways that do not reinforce existing power relations and inequities. The presentation will include examples of how evaluators have used these approaches within different evaluation contexts and lessons learned.

Audience Level: Intermediate

Session Abstract (150 words): 

Evaluators are often faced with difficult decisions about how to navigate complex power dynamics as they seek to understand and inform systems change efforts. To aid evaluators in traversing this landscape, there is a need for additional evaluation frameworks and methods that take into account issues of power, equity, perspective, and multiple boundaries around truth. This multi-paper session proposes several tools grounded within systems science to aid this investigation including: a developmental systems framework to understand the process of equitable change and power; an approach for revealing and understanding hidden power dynamics through indicator prioritization processes;  system scanning methods to reveal and legitimize differences in perspectives across stakeholders with varying levels of power; and approaches for navigating the real dilemmas of power and truth inherent within every evaluation. Presenters will describe the tools and their application in diverse communities aiming to create more effective and equitable community systems. 

For questions or concerns about your event registration, please contact registration@eval.org or 202-367-1173.

For questions about your account, membership status, or help logging in, please contact info@eval.org.

Cancellation Policy: Refunds less a $50 fee will be granted for requests received in writing prior to 11:59 PM EDT October 8, 2018. Email cancellation requests to registration@eval.org. All refunds are processed after the meeting. After October 8, 2018 all sales are final. For Evaluation 2018, international attendees and presenters who encounter complications due to the international travel environment will have up to 30 days after the event to request a refund and submit appropriate documentation. No administrative fee will apply for the international requests.