Evaluation 2018: Speaking Truth to Power

View Printable Version

Moving the Needle on Program Quality: Positioning External Evaluation to Support Continuous Quality Improvement in Programs for Children and Youth

Session Number: 2339
Track: Youth Focused Evaluation
Session Type: Panel
Tags: Afterschool Programs, continuous quality improvement, organizational learning, participatory evaluation, youth participatory evaluation, youth voice
Session Chair: Tiffany D Berry [Research Associate Professor - Claremont Graduate University]
Discussant: Abraham Wandersman [Professor - University of South Carolina]
Presenter 1: Jessica Manta-Meyer [Director - Public Profit]
Presenter 2: Silvana McCormick [Executive Director - Redwood Consulting Collective ]
Presenter 3: Michelle A Sloper [Senior Evaluation Fellow - Claremont Evaluation Center]
Presentation 1 Additional Author: Silvana McCormick [Executive Director - Redwood Consulting Collective ]
Presentation 1 Additional Author: Corey Newhouse [Founder and Principal - Public Profit]
Presentation 2 Additional Author: Tiffany D Berry [Research Associate Professor - Claremont Graduate University]
Presentation 3 Additional Author: Tiffany D Berry [Research Associate Professor - Claremont Graduate University]
Time: Nov 01, 2018 (03:45 PM - 04:45 PM)
Room: CC - 26B

Abstract 1 Title: Continuous Quality Improvement in Youth-Serving Programs: A Framework and Recommendations for Evaluators
Presentation Abstract 1:

Continuous Quality Improvement (CQI) is a mechanism for the ongoing and iterative use of inquiry, experimentation, and data-driven reflection to improve practice. CQI has been found to be an impactful strategy for closing the theory-to-practice gap in many fields and is particularly relevant as an evaluation approach for youth-serving organizations in that it draws on some of the same principles as Positive Youth Development (i.e., prioritizing youth voice, collaboration, active engagement with ideas, leaning from mistakes). This presentation introduces attendees to the concept of CQI and its critical elements, discusses the strengths of a CQI approach, and outlines how evaluation practice can be adapted to operate within a broader CQI framework. This presentation will also explore how CQI can be used as a mechanism to meaningfully engage youth in the evaluation process. This presentation draws from ideas the authors developed for a CQI Guidebook for California’s publicly funded afterschool programs.


Abstract 2 Title: Lessons from the Field: An Examination of the Organizational Characteristics that Drive CQI in Afterschool Programs
Presentation Abstract 2:

This presentation describes empirical findings on the evaluation practices and organizational characteristics that drive high-impact Continuous Quality Improvement (CQI). These findings are from a qualitative study conducted on exemplar afterschool programs as part of a larger statewide study that was implemented in partnership with the California Department of Education’s Expanded Learning Division. The presentation will describe the crosscutting themes that emerged from those afterschool programs that were engaged in comprehensive evaluation of program quality and that reported high levels of use of evaluation for program improvement. It will discuss the facilitative attitudes, relationships, and practices (planning, evaluation, and program practices) that are being used in the afterschool field to move the needle on program quality. The presentation will also describe how external evaluation is positioned in these agencies to support CQI and how this is reflected in evaluator roles and evaluation priorities, designs, and activities.


Abstract 3 Title: CQI in Context: Promoting Quality Afterschool Programs in the Los Angeles Unified School District
Presentation Abstract 3:

LAUSD Beyond the Bell operates structured afterschool programs in over 600 schools serving over 100,000 K-12 students daily. Since 2014, our team has contracted with Beyond the Bell to provide technical assistance, build evaluation infrastructure, offer professional development, and develop of a strategic quality improvement system (Berry et al., 2016). After creating a comprehensive CQI system aligned with organizational priorities and quality standards, the evaluation team was challenged to convince staff to value and understand the process and ensure CQI tools and systems were being used (and used appropriately). This presentation will share critical considerations for evaluators when building CQI systems within complex youth-serving organizations, including investing in considerable relationship-building and information-gathering early in the process, building a theory-driven conceptual model to frame CQI processes, finding the appropriate balance between building new systems and capitalizing on existing practices, and ensuring systems are user-friendly and efficient.


Audience Level: All Audiences

Session Abstract (150 words): 

High quality programs – those that connect youth to caring adults, provide structured, purposeful activities, and prioritize youth voice – lead to positive outcomes for youth. Continuous Quality Improvement (CQI) is a key mechanism that enables programs to be high quality. This panel explores how external evaluation can be positioned and leveraged to support CQI efforts in youth-serving and youth leadership programs and discusses how evaluation can best contribute to exemplary programming for our youth. Our first presentation will provide an introduction to CQI, discuss why it’s important for fostering youth voice, and the role of external evaluation. Next, we present empirical findings on the evaluation practices and organizational characteristics that drive high-impact CQI from a qualitative study of exemplar afterschool programs. Finally, we provide an illustrative case study of a partnership between external evaluators and a large urban school district to build and implement a CQI system for afterschool programs.



For questions or concerns about your event registration, please contact registration@eval.org or 202-367-1173.

For questions about your account, membership status, or help logging in, please contact info@eval.org.



Cancellation Policy: Refunds less a $50 fee will be granted for requests received in writing prior to 11:59 PM EDT October 8, 2018. Email cancellation requests to registration@eval.org. All refunds are processed after the meeting. After October 8, 2018 all sales are final. For Evaluation 2018, international attendees and presenters who encounter complications due to the international travel environment will have up to 30 days after the event to request a refund and submit appropriate documentation. No administrative fee will apply for the international requests.