Evaluation 2019: Paths to the Future of Evaluation: Contribution, Leadership, and Renewal

View Printable Version

Evaluating Arts Education: The art of mixing methods for measuring creativity

Session Number: 1846
Track: Arts, Culture, and Museums
Session Type: Panel
Tags: arts education, Mixed-Methods Evaluation, validity
Session Chair: Patricia Moore Shaffer [Deputy Director, Research & Analysis - National Endowment for the Arts]
Discussant: Patricia Moore Shaffer [Deputy Director, Research & Analysis - National Endowment for the Arts]
Presenter 1: Laura Pryor [Social Policy Research Associates]
Presenter 3: Rachel Estrella [Senior Associate - Social Policy Research Associates]
Time: Nov 16, 2019 (09:15 AM - 10:00 AM)
Room: Hilton Symphony II

Abstract 1 Title: Practices and Considerations for Quantifying the Arts
Presentation Abstract 1:

Arts-based education initiatives often have nuanced outcomes that are challenging to accurately quantify. Furthermore, in the era of standardized testing, researchers and evaluators are often asked to link arts education programs to test scores. Yet, given the limited quantitative research in arts education, creating and utilizing valid and meaningful quantitative measures into an arts-based evaluation can contribute a unique perspective to our understanding of the role of the arts in education. This presentation discusses how an evaluation of the Poetry Out Loud program attempted to quantify arts education outcomes via a quasi-experimental design incorporating survey and administrative data. In addition to outlining the study’s design, this presentation discusses challenges with creating valid measures and reflects on the limitations of the approach. The presentation concludes by raising considerations and future opportunities for ensuring that quantitative studies of arts education meaningfully capture student outcomes.

Abstract 2 Title: Honoring Nuance in Arts Education Evaluation
Presentation Abstract 2:

The goals of arts education programs in schools may include, but are not limited to, academic outcomes. Participation in these programs has also been shown to improve students’ social emotional outcomes (e.g. self-concept, self-confidence, positive behavioral change) and deepen their appreciation for the arts. Strong qualitative methods in the evaluation of arts education can capture these more nuanced outcomes and provide insight into why or how arts programs work. This presentation discusses how SPR structured its evaluation of Poetry Out Loud, incorporating student surveys and in-depth site visits to capture student and teacher voice, and demonstrate how program implementation influenced social emotional outcomes and poetry appreciation. These findings depict the potential of well-implemented arts programming on schools, teachers, and students, and provide essential context for quantitative findings.

Presentation 2 Other Authors: Jennifer Hogg
Abstract 3 Title: Validity, Credibility, and Relevance: Addressing the Tensions in Arts Education Evaluation
Presentation Abstract 3:

How we measure the value of arts education has long been a source of tension in the field. Some believe that focusing on connections between arts involvement and academic achievement in core subjects (e.g. math and language arts) validates the importance of arts education. Others feel strongly that measuring the power of the arts primarily by examining its influence on other subjects limits our understanding of its power. This is further exacerbated by the push to apply methods deemed as the “gold standard” and the conflation of those methods with notions of “validity” and “credibility”. This presentation encourages participants to think about the purpose of evaluation, to deconstruct how we think about “evidence” when considering evidence-based research in arts education and who decides what is “credible”, and the importance of asking those questions to ensure that evaluations produce meaningful results and are relevant and useful for programs and practitioners.

Audience Level: All Audiences

Session Abstract (150 words): 

The conference theme states that the role of evaluation is to provide “trusted, credible, evidence-based, and balanced conclusions about the quality, importance and value of what is relevant in our society.”  While evaluating arts education programs is no exception to this standard, this context requires evaluators to grapple with designing evaluations that include multifaceted outcomes (i.e. creativity, self-expression) alongside standardized test scores. This panel explores this challenge through discussing the methodological approach to an evaluation of the National Endowment for the Arts’ Poetry Out Loud program and reflecting on the role of evaluation in arts education. The first two panelists discuss how the qualitative and quantitative components of the evaluation attempted to balance the need for external rigor via standardized tests scores with capturing multifaceted outcomes. The final presentation discusses questions related to credible and valid findings in an arts education context and the role of stakeholders in shaping validity.

For questions or concerns about your event registration, please contact registration@eval.org or 202-367-1173.

For questions about your account, membership status, or help logging in, please contact info@eval.org.

Cancellation Policy: Refunds less a $50 fee will be granted for requests received in writing prior to 11:59 PM EDT October 11, 2019. Email cancellation requests to registration@eval.org. All refunds are processed after the meeting. After October 11, 2019 all sales are final. For Evaluation 2019, international attendees and presenters who encounter complications due to the international travel environment will have up to 30 days after the event to request a refund and submit appropriate documentation. No administrative fee will apply for approved international requests.