Evaluation 2015: Exemplary Evaluations in a Multicultural World

View Printable Version

Evaluating Distance Education in the US and in Brazil: Approaches to Understanding What’s Happening on the Ground

Session Number: 1977
Track: Distance Education & Other Educational Technologies
Session Type: Multipaper
Tags: "hard to reach" populations, capacity building, Cross-cultural evaluation
Session Chair: Talbot Bielefeldt [Clearwater Program Evaluation]
Discussant: Donna M Mertens [Independent Consultant]
Presenter 1: Ana Lucia Lima [Paulo Montenegro Institute]
Presenter 2: Tara Shepperson [Eastern Kentucky University]
Presenter 3: Kas Aruskevich, PhD [Evaluation Research Associates LLC]
Presentation 1 Additional Author: Vilma Guimarães [Fundação Roberto Marinho]
Presentation 1 Additional Author: Tereza Farias [Roberto Marinho Foundation]
Time: Nov 13, 2015 (07:00 AM - 07:45 AM)
Room: Skyway 265

Abstract 1 Title: A new index to assess the effectiveness of a youth/adult distance education literacy program in remote areas of the Brazilian Amazon region
Presentation Abstract 1: This paper discusses the use of an alternative to high stakes standardized tests to assess cognitive abilities of adults and indigenous students. The Functional Literacy Index – INAF, developed by two civil society organizations, is based on daily tasks and contexts. The authors will present the challenges for implementing the index in a longitudinal study to assess the development of reading, writing and mathematical skills of 15 to 65 years old students participating in an educational program in the Brazilian Amazon (Igarité – EJA Mediado). The program was created by the Roberto Marinho Foundation and is implemented by the Department of Education of the State of Amazonas, Brazil. It uses the Telessala methodology, which provides teachers training and support through videos and other didactic materials. In this program, teachers and students in isolated communities interact in real-time with experienced teachers located at a Media Center, through a complex satellite-dish network.
Abstract 2 Title: Empowering Rural Teachers through Problem-to-Solution Modeling Instruction
Presentation Abstract 2: As part of an online graduate course, teachers in largely rural Central Appalachia developed problem-to-solution models based on actual educational issues present in their school. Logic model diagrams, coursework, rank order surveys, and open-ended course assessments were analyzed using qualitative analysis software, a constant-comparative method, and SPSS for correlation and analysis of variance. Course evaluators found that of the 300 teachers who over three years completed the course, a minority sought to address problems resulting from social ills (hunger, health care). Many sought to improve academic underperformance blamed on a lack of resources and devaluing of formal schooling. Others focused on changing practices that they felt reflected outmoded learning and discipline methods. Implications are that teachers found power in implementing small remedies to large societal problems and poor school customs, and problem-based learning can be translated to online delivery, and authentic learning can assist larger social change.
Abstract 3 Title: Techniques and Technology to Evaluate at a Distance: From Urban to Rural Alaska
Presentation Abstract 3: You may have heard of it. Perhaps you practice it. Helicopter evaluation is a somewhat disparaging term used to describe an evaluation conducted by someone who lives outside of the community, flies in for a short site visit, gathers information, then leaves. Conducting this type of evaluation is intensified when there are cultural considerations and complex collaborations where culture and mainstream intersect.

Techniques and Technology to Evaluate at a Distance: From Urban to Rural Alaska is built on the presenter’s dissertation research on indigenous evaluation in four post-settler states and the past seven years of full time practice in rural primarily Alaska Native villages. While there is no cookie cutter approach to evaluation in these remote communities, this presentation will describe the eclectic set of evaluation approaches, techniques to gain credibility and validity, and technologies to collect information from a wide client base that Distance Evaluators need in their toolbox.
Audience Level: Intermediate

Session Abstract: 

This session discusses the methodologies and findings of evaluations in two distance learning programs. One program targets synchronously delivered literacy training for youths and adults in the Amazon rain forest. The second paper presents information from a master’s course delivered asynchronously online to teachers in the rural and poor region of Central Appalachia. Both program evaluations utilized quantitative and qualitative approaches, and found that local contexts impacted learning outcomes. Findings suggest that partnerships with local agencies and schools, coursework relevant to local circumstances, and use of mixed methods approaches, provide more responsive and comprehensive information than a single approach. Mixed methods evaluations result in deeper understanding, better designed program changes, and ultimately educational programming that specifically suits the local context and ultimately increases the chance for positive social change. 


For questions or concerns about your event registration, please contact registration@eval.org or 202-367-1173.

For questions about your account, membership status, or help logging in, please contact info@eval.org.

Cancellation Policy: Refunds less a $50 fee will be granted for requests received in writing prior to 11:59 PM EDT October 6, 2015. Email cancellation requests to registration@eval.org. Fax request to (202) 367-2173. All refunds are processed after the meeting. After October 6, 2015 all sales are final.