Evaluation 2019: Paths to the Future of Evaluation: Contribution, Leadership, and Renewal

View Printable Version

Advancing Equitable Development for Youth: Making New Tools for Both Program and Evaluation

Session Number: 1731
Track: Youth Focused Evaluation
Session Type: Multipaper
Tags: Evaluation capacity building, Youth Development Programs
Session Chair: Yetunde Zannou [Evaluation Project Manager - Southern Methodist University]
Presenter 1: Bonny Gildin [Vice President Afterschool Development Research & Policy - All Stars Project Inc.]
Presenter 2: Yetunde Zannou [Evaluation Project Manager - Southern Methodist University]
Presenter 3: Yusuf Kara [Evaluation Coordinator - Southern Methodist University]
Time: Nov 14, 2019 (08:00 AM - 09:00 AM)
Room: Hilton Marquette VI

Abstract 1 Title: Measuring Success in New Ways: The All Stars Project Model and Engaging Evaluation
Presentation Abstract 1:

In recent years, approaches to equity and social and economic mobility that are qualitative and in some cases non-systematized have emerged. The non-profit All Stars Project (ASP) was early in focusing on the subjective, immobilizing effect of poverty on young people and communities, i.e., locking them into narrow and limited identities. ASP successfully experimented with the tools of theatrical and improvisational performance to support youth in continuously creating multiple and layered identities and building rich development and learning environments for themselves and their communities. ASP developed a portfolio of “development through performance” programs, launched and has sustained them over 38 years in six US cities, demonstrating and refining this approach to promoting equitable development and access to opportunity. This paper will describe key programmatic features of ASP’s approach and the decision to partner with a university evaluation partner to engage the unique challenges to measuring resulting youth and community transformation.

Abstract 2 Title: Benefits of a Phased Approach: Design and Initial Outcomes of a Four-Year Evaluation of a Performance-based Youth Development Program
Presentation Abstract 2:

This paper describes a four-year evaluation partnership with the All Stars Project (ASP), a nonprofit organization that has successfully attracted inner-city youth to its varied performance-based programs for over three decades. Historically, ASP described its impact through the stories of youth who participated in the program and shared their success stories publicly. The All Stars sought evaluation in order to reach a broader audience, contribute to and advance the field of after school development, and to document the effects they observed in countless poor youths since the 1980s. This paper outlines the four phases used to understand the intervention in Year 1 through collecting initial outcome data in Year 4. In addition, the author discusses how the findings reflect key aspects of the equity-focused youth intervention and directions for further research.

Abstract 3 Title: A Higher-Order Factor Model for Measuring Youth Development
Presentation Abstract 3:

This study describes how a tool to measure the construct “development” and its latent components were developed and validated using exploratory and confirmatory factor analyses (EFA and CFA). Sixty-three pilot items were written and formatted to have five-point Likert type response categories (“strongly disagree” to “strongly agree”). Participants included N=516 youth aged 16-21 from all five national sites. Data were randomly divided to two halves for EFA and CFA. EFA was conducted to select the most effective items among others for each component of development. A higher-order CFA was then used to validate the hypothesized two-layer factor model of development, defined by selected items. CFA results suggested a good model fit (RMSEA=.052, SRMR=.066, CFI=.939, TLI=.932), which provides evidence for construct validity. The final form consisted of 27 items that measure eight latent components of development: personal responsibility, vocational competence, confidence, improvisation, appreciation, giving, interpersonal competence, and open worldview.

Audience Level: All Audiences

Session Abstract (150 words): 

Program evaluations are used for many reasons. Beyond being required for funding accountability, evaluations support continuous program improvement, facilitate organizational learning, and contribute to various fields through impact studies. This session combines three perspectives to describe how a well-established, privately funded organization sought out evaluation to document evidence in support of their youth development model. During the 4-year partnership, the evaluation was used to increase organizational clarity about what they do, create organizational systems to maintain program fidelity, increase evaluative thinking and capacity, and generate momentum to lead in the emerging after school youth development space. This session includes three linked perspectives: (1) a national director provides context for the youth program model; (2) the evaluator describes the evaluation design, process, and initial results; and (3) the analyst explains how pilot test data were used to statistically validate tools and the program model. We discuss implications for evaluating similar programs.

For questions or concerns about your event registration, please contact registration@eval.org or 202-367-1173.

For questions about your account, membership status, or help logging in, please contact info@eval.org.

Cancellation Policy: Refunds less a $50 fee will be granted for requests received in writing prior to 11:59 PM EDT October 11, 2019. Email cancellation requests to registration@eval.org. All refunds are processed after the meeting. After October 11, 2019 all sales are final. For Evaluation 2019, international attendees and presenters who encounter complications due to the international travel environment will have up to 30 days after the event to request a refund and submit appropriate documentation. No administrative fee will apply for approved international requests.