Evaluation 2015: Exemplary Evaluations in a Multicultural World

View Printable Version

The Science (and Technology, Engineering, and Mathematics) of STEM Evaluation Methods

Session Number: PreK11
Track: PreK-12 Educational Evaluation
Session Type: Multipaper
Session Chair: Sheila Arens [McREL International]
Discussant: Anane Olatunji [Align Education, LLC]
Presenter 1: Ted D. Serrant [Houston Independent School District]
Presenter 2: Cathy Malerba [Round Rock ISD]
Presenter 3: Steven Ziebarth [Western Michigan University]
Presenter 4: Ann Cisney-Booth [Orange County Public Schools]
Presentation 2 Additional Author: Megan Olshavsky [Coordinator of Research & Evaluation - Round Rock ISD]
Presentation 4 Additional Author: Jennifer Sasser, Strategic Planning and Improvement [Director - Orange County Public Schools]
Presentation 4 Additional Author: Rob Bixler [Executive Area Director - Orange County Public Schools]
Time: Nov 12, 2015 (08:00 AM - 09:30 AM)
Room: Burnham

Abstract 1 Title: The Evaluation of the Summer Science Institute (SSI) as Best Professional Development Practice in a PreK-12 Urban School District
Presentation Abstract 1: The Summer Science Institute (SSI) is a professional development program that offers current and effective teaching strategies, assessment, and relevant science concepts that are aligned with the State Essential Knowledge and Skills (TEKS) and the State Assessment recommended learning objectives (BCM, 2013). This program targets the prekindergarten to eighth-grade teachers who taught science in self-contained classes, science labs, and regular education classes in an urban school district.
The evaluation uses a combination of qualitative and quantitative methods of inquiry using data and information from previous evaluations, repeated measures of teacher content knowledge, experiences, and perceptions, and students' science performance. It includes the input of key program administrators and managers in the program design and evaluation and their administrative responses to the report recommendations.
The SSI has had statistically significant positive effects on teacher content knowledge and student science performance for the past four years, 2011 - 2014.
Abstract 2 Title: Yes You Can: Applying the Principals of Developmental Evaluation to a District Technology Initiative
Presentation Abstract 2: <p class="Normal1">School districts are rapidly adopting new technologies in the classroom; however, there is much to learn about the strengths and weakness of various platforms and how they should be used to support learning. Developmental Evaluation (DE) is an appropriate fit for contexts of innovation, exploration, and complexity; however, it is unfamiliar to many K-12 stakeholders. This paper describes the evaluation of a district technology initiative in which all aspects of evaluation planning, methods, and activities were guided by the principals of DE. It is valuable to K-12 evaluators because it demonstrates that DE can be successful in a school district environment if stakeholders and participants are directly instructed about the approach and the importance of their role in the evaluation. The presentation highlights some of the results of the evaluation regarding device utility, student learning and engagement, teacher professional development, and creating a community of technology innovators within the district.
Abstract 3 Title: A Mixed-Methods Evaluation of a K-12 Mathematics Course Designed for Non-STEM Intending High School Seniors
Presentation Abstract 3: This session presents summative findings from a goal-oriented curriculum evaluation using mixed methods to measure the effectiveness of a senior high school course designed to prepare non-STEM students for college mathematics and statistics. The curriculum development and evaluation were supported by NSF-funding directed to develop curriculum materials for high school seniors planning to pursue majors in college that do not require calculus. This population represents a majority of college students. This four-year project consisted of a year of recruiting six national field-test schools with various curriculum backgrounds, a year of intense on-site data collection using both qualitative and quantitative methods, and two years of data analysis that produced a published set of materials and a final evaluation report to NSF in November 2014. Evaluation data included student achievement measures, surveys of student and teacher beliefs, classroom observations, student interviews, and post-high school performance in college mathematics classes.
Abstract 4 Title: Measuring Success: Evaluating 1-to-1 Technology Initiatives
Presentation Abstract 4: The use of technology in school systems is increasing at a rapid pace. "Today's education system faces irrelevance unless we bridge the gap between how students live and how they learn" (Partnership for 21st Century Skills, n.d., p. 4). Digital natives, students who have grown up immersed in technology, are said to have skills and interests that can have significant implications for education (Bennett, Maton, & Kervin, 2008). However, just adding technology is not considered a recipe for 1-to-1 success (THE Journal, 2014). In this presentation, the evaluators will discuss how to measure success and the complexity of evaluating 1-to-1 digital curriculum initiatives.
Audience Level: None

Session Abstract: 

The Science (and Technology, Engineering, and Mathematics) of STEM Evaluation Methods

For questions or concerns about your event registration, please contact registration@eval.org or 202-367-1173.

For questions about your account, membership status, or help logging in, please contact info@eval.org.

Cancellation Policy: Refunds less a $50 fee will be granted for requests received in writing prior to 11:59 PM EDT October 6, 2015. Email cancellation requests to registration@eval.org. Fax request to (202) 367-2173. All refunds are processed after the meeting. After October 6, 2015 all sales are final.