Evaluation 2018: Speaking Truth to Power

View Printable Version

Systems Approaches to Speaking Truth to Power

Session Number: SE1
Track: Systems in Evaluation
Session Type: TIG Multipaper
Session Chair: Elizabeth McGee [Consultant - Independant]
Presenter 1: Elizabeth McGee [Consultant - Independant]
Presenter 2: Roxanne Adele Moore [Research Engineer - Georgia Institute of Technology]
Presenter 3: Eileen Eisen-Cohen
Presenter 4: Ronald J. Iarussi [Superintendent - Mahoning County Educational Service Center]
Presentation 2 Additional Author: Tom McKlin [Director - The Findings Group]
Presentation 3 Additional Author: Breanne Lott, MPH [PhD Student - University of Arizona]
Presentation 3 Additional Author: Mac McCullough [Assistant Professor - Arizona State University]
Presentation 4 Additional Author: Karen H Larwin, PhD [Associate Professor - Youngstown State University]
Presentation 4 Additional Author: David Arthur Larwin [Professor - Kent State University at Salem & REM Consulting]
Time: Nov 02, 2018 (05:45 PM - 06:45 PM)
Room: Hilton - Center St. Meeting Room C

Abstract 1 Title: Bringing An Equity Lens Into Systems Change Evaluative Processes
Presentation Abstract 1:

If systems change initiatives are to be effective, explicit attention must be given to the culture, policies, and processes, which reinforce patterns of inequity (Reynolds and Williams, 2012). Without focused consideration given to equity, efforts to align and coordinate resources or change systems can unintentionally reinforce institutional patterns that endorse inequalities for vulnerable community members. Systems change evaluators must cultivate specific strategies that remain mindful of the underlying advantages that certain groups have, as well as the disadvantages other groups encounter. In this roundtable, two systems evaluators will discuss the importance of equity in systems change evaluations and the challenges inherent this work. Concrete, equity-focused strategies that can be introduced throughout systems change evaluative processes will be offered. Roundtable attendees will discuss challenges and collectively brainstorm strategies to address equity in systems change evaluative work.


Presentation 1 Other Authors: Cristina Whyte (cristina.whyte@gmail.com)
Abstract 2 Title: System Dynamics Models and Evaluation: Enabling Sound Decision-Making
Presentation Abstract 2:

Models have long been a part of the evaluation landscape, including logic models, social networks, and occasionally agent-based models. In this paper, we discuss system dynamics models, a way of representing complex, non-linear systems and feedback loops, as a method for facilitating common understanding and decision-making across stakeholders. Specifically, we present system dynamics models that were developed and utilized during the design and implementation of a school intervention in a high school computer science course.  Development of the models led to some common language among the intervention team, including terms like ‘virtuous and vicious cycles’, and enabled discussion about the possible pathways for breaking the vicious cycles.  As illustrated through qualitative analysis of interview data from the intervention team, the models and the process of developing the models affected the curriculum development, interface, teacher materials, teacher professional development, and understanding of the attributes likely to affect intervention success and sustainability. 


Presentation 2 Other Authors: Taneisha Lee, Michael Helms
Abstract 3 Title: The Aftermath of Silo-Busting: Measuring Collaboration in Your Organization
Presentation Abstract 3:

As public health departments heighten focus on population-focused services, both external and internal collaboration become more necessary.  Population-focused services require higher levels of coordination and interdependence than individually-focused services, suggesting an increased importance for internal collaboration for policy and systems work.  Evaluating collaboration through the use of the Relational Coordination survey tool; an instrument that has been previously validated for measuring collaboration and integration and has been linked to better performance outcomes in other health care settings, can identify areas of opportunity for public health leaders to intervene.  Participants will learn why and how to measure collaboration based on Maricopa County Department of Public Health’s experience using this tool as part of a mixed-methods evaluation and developing actionable strategies to facilitate intentional collaboration within this large health department.


Abstract 4 Title: The Evaluation FITS Model for Global Evaluation: Empowerment and Truth-seeking for Educational Service Agencies.
Presentation Abstract 4:

Educational Service Centers (ESC) in Ohio operate according to a “Best Practices” model of education.  Their role is to provide professional development and technical assistance to the local school districts in the “hope” that these supports will empower the school districts to provide the best education possible for all children.  Evaluation FITS is a comprehensive evaluation model that was developed for a local ESC that employs more than 1700 people who provide educational support to more than 100,000 students.  The Evaluation FITS model was developed and employed to understand the fidelity, impact, transferability, and sustainability of all professional development, technical assistance, and service delivery, from the local agency personnel level to the student level. The Evaluation FITs model development, administration, and results will be presented with the goal of aiding other organizations and agencies in their efforts to ensure their program and process are meeting the needs of their clientele.  


Audience Level: Intermediate

Session Abstract (150 words): 

Systems Approaches to Speaking Truth to Power



For questions or concerns about your event registration, please contact registration@eval.org or 202-367-1173.

For questions about your account, membership status, or help logging in, please contact info@eval.org.



Cancellation Policy: Refunds less a $50 fee will be granted for requests received in writing prior to 11:59 PM EDT October 8, 2018. Email cancellation requests to registration@eval.org. All refunds are processed after the meeting. After October 8, 2018 all sales are final. For Evaluation 2018, international attendees and presenters who encounter complications due to the international travel environment will have up to 30 days after the event to request a refund and submit appropriate documentation. No administrative fee will apply for the international requests.