Session Number: 1633
Track: AEA Sponsored Sessions
Session Type: Multipaper
Session Chair: Jean A. King [Professor - University of Minnesota]
Presenter 1: Jean A. King [Professor - University of Minnesota]
Presenter 2: Sandra Ayoo [Graduate Student - University of Minnesota]
Presenter 3: John LaVelle [Assistant Professor - University of Minnesota]
Presenter 4: Hazel Louise Symonette [Program Development & Assessment Specialist, Emerita - University of Wisconsin-Madison]
Presenter 5: Susan Tucker [Principal - Evaluation and Development Associates LLC]
Presentation 1 Additional Author: Laurie Stevahn [Professor of Educational Leadership - Seattle University]
Presentation 2 Additional Author: Gail Vallance Barrington, PhD, FCMC, CE [President - Barrington Research Group, Inc.]
Presentation 2 Additional Author: Donna Podems [University of Johannesburg]
Presentation 3 Additional Author: Nicole Galport [Research & Evaluation Associate - Claremont Graduate University; Cobblestone Applied Research & Evaluation, Inc.]
Presentation 3 Additional Author: Anna Rodell [Founder, evaluation consultant - Collective Progress]
Presentation 4 Additional Author: Eric Barela [Measurement & Evaluation Senior Manager - Salesforce.org]
Presentation 4 Additional Author: Robin Lin Miller [Professor - Michigan State University]
Presentation 5 Additional Author: Dale E Berger [Emeritus Professor of Psychology - Claremont Graduate University]
Presentation 5 Additional Author: Michelle Gensinger [University of Minnesota - Twin Cities]
Presentation 5 Additional Author: Elizabeth Wilcox
Time: Nov 09, 2017 (01:15 PM - 02:00 PM)
Room: Wilson C
Abstract 1 Title: Overview: The Draft AEA Evaluator Competencies and Their Origin
Presentation Abstract 1:
This introductory presentation will have two sections. First, it will provide a brief overview of the multi-step process that led to the final draft of the AEA Evaluator Competencies. The process included several activities in the past two years: (1) reviewing existing sets of competencies for program, policy, and personnel evaluation to identify and frame foundational competencies; (2) presenting a poster at AEA’s 2015 conference and sponsoring listening sessions at AEA’s 2015 and 2016 conferences; (3) analyzing and integrating feedback at a day-long meeting of the Task Force; (4) conducting member engagement in several forms; and (5) twice revising the competencies based on the feedback received. Second, to make sure everyone is on the same page, it will present the current version of the proposed competencies in five domains: professional practice, methodology, context, project planning and management, and interpersonal.
Abstract 2 Title: Grounding the AEA Competencies in the Broader Field of Professionalizing
Presentation Abstract 2:
Members of the national and international evaluation communities have long debated topics related to evaluator competencies and their relationship to the real and perceived moved towards “professionalizing” evaluation, although these debates have not always been fully grounded in empirical literature. The sociology of professions provides organizing frameworks for grounding the AEA Evaluator Competencies in the broader context of professionalization and sociological sciences. This presentation will draw from the process model of professionalization (Forsyth & Danisiwicz, 1985) to illustrate the role of evaluator competencies in professionalization systems. The presenters will contextualize the reasonable benefits as well as limitations of the professionalization processes. Additionally, the presenters will speak to some of the challenges of professionalization, such as perceived elitism, the exclusion of “would-be” evaluators, and the implicit consolidation of influence and social capital.
Abstract 3 Title: The Role of Competencies as Pathways into the Evaluation “Profession”
Presentation Abstract 3:
The education and socialization of evaluators into evaluation practice and scholarship have been a recurring theme in the evaluation community, often condensed into questions of where, how, why, and for what purposes are evaluators educated. As the field of evaluation has grown in recent decades, scholars and practitioners are beginning to explore more fully the various ways in which evaluators can (1) learn the fundamentals of evaluation practice, (2) network into communities of practice, and (3) maintain current or develop new skill sets. The presenters will describe the explicit and implicit ways evaluator competencies frame educational opportunities available to current or future evaluators, such as university systems, professional development opportunities, participation in VOPEs, self-guided learning systems, etc. Last, the presenters will describe ways the competencies can be used to enhance performance and the competitive edge for evaluators in a range of settings.
Abstract 4 Title: How the AEA Competencies Integrate Cultural Competence and Address Issues of Social Justice, Power, and Privilege
Presentation Abstract 4:
The global nature of evaluation practice and AEA’s core values of inclusivity and the public good provided a lens through which many members vetted each draft of the competencies throughout our engagement process. Members called attention to the need for evaluators to possess skills in addressing power and privilege/oppression and work to secure socially just outcomes. These considerations proved challenging to address because some evaluators prioritize and embrace these issues while others question their relevance and role in practice. Of course, such considerations have greatest relevance when evaluations involve humans rather than only inanimate objects. We sought to consider how evaluators appropriately and effectively engage their understandings of the roles of cultural dynamics, context, and power in shaping evaluation’s processes, products, and consequences. Given their centrality to evaluation practice, this presentation will describe how we explicitly attended to issues of culture, power, and privilege/oppression in each competency domain.
Abstract 5 Title: Attention to Detail: Assessing and Documenting Evidence of Evaluator Competence
Presentation Abstract 5:
Evaluation practitioners and scholars have long debated the merits and limitations of AEA adopting a formal set of competencies, and recurring challenges have risen with fundamental questions about measurement. Members of the evaluation community have reasonably asked, “How can the competencies be operationally defined and measured” and “How can evidence of mastery be documented?” The speakers will first outline possible processes for defining and measuring the newly-refined competency statements, followed by lessons learned from existing documentation processes such as the Canadian Evaluation Society portfolio and the European Evaluation Society’s Voluntary Evaluator Peer Review system. Specific attention will be paid to how a documentation system might be implemented in the AEA-specific context with a focus on accessibility and utility.
Theme: Learning to Enhance Evaluation
Audience Level: All Audiences
Session Abstract (150 words):
AEA’s Board approved the development of a set of evaluator competencies in 2015 as the next step in AEA’s development. Task Force members reviewed existing sets of competencies to identify and frame foundational competencies necessary to the diverse evaluation practice of AEA members. Following feedback from a session at AEA’s 2015 conference, the Task Force revised the competencies and conducted member engagement in several forms. In 2016 the Task Force revised the draft competencies, which were then critiqued at a listening session. Following additional feedback, the competencies were finalized and sent with a survey to the AEA membership. This session will present the contextual grounding for AEA’s draft competencies, discussing issues underlying their creation: the status of evaluation professionalization internationally; the role of competencies for entry into the field; how the competencies address cultural competence, social justice, power, and privilege; and the challenge of measuring evaluator competence.