Evaluation 2016: Evaluation + Design

View Printable Version

Winning the Telephone Game: Communication Techniques in Program Evaluation

Session Number: 1427
Track: Collaborative, Participatory & Empowerment Evaluation
Session Type: Panel
Tags: communication, empowerment evaluation, evaluation use, innternal evaluation, organisational learning
Session Chair: Stanley Thomas Capela, MA [VP for Quality Management & Corporate Compliance Officer - HeartShare Human Services of New York]
Discussant: Stacie R Powers [Senior Associate - Philliber Research]
Presenter 1: Rakesh Mohan [Director, Office of Performance Evaluations - Idaho State Legislature]
Presenter 2: Jennifer Dewey, WRMA [Senior Research Associate - WRMA]
Presenter 3: Stanley Thomas Capela, MA [VP for Quality Management & Corporate Compliance Officer - HeartShare Human Services of New York]
Presenter 4: Ben Kerman [The Atlantic Philanthropies]
Time: Oct 29, 2016 (08:00 AM - 09:30 AM)
Room: A703

Abstract 1 Title: My Daily Sojourns
Presentation Abstract 1: Because I work in a public policy environment, I start thinking about evaluation use even before we take the first step in conducting an evaluation. I visit our state capitol during legislative sessions daily. I call these visits “my daily sojourns.” These visits allow me to run into policymakers, program officials, lobbyists, the press, and other stakeholders. Through these visits, I inform them about our work and at the same time I become informed about the political context in which we work. Over the years, I have found that face-to-face communication like this is the best way to build relationships and trusts that are indispensable when conducting evaluations and promoting their use. Effective communication is critical to evaluation design, as well as to the use of evaluation in designing policies and programs.
Abstract 2 Title: Enhancing a State’s Commitment to Improve the Lives of Children
Presentation Abstract 2: The purposes of communicating and reporting in evaluation are to 1) convey information, 2) facilitate understanding and create meaning, and 3) support decision-making (Torres, Preskill, & Piontek, 2005). How this happens depends on a variety of factors, one of the most important being your audience, or more likely, audiences. Based on principles of participatory action research and the PII Approach to Evaluation (2015), evaluators worked with a state-level partnership to assess a culturally responsive child welfare practice model designed to increase permanent living situations and reduce disparities in outcomes for children and families. Organizational learning was promoted throughout the process as evaluators worked with partnership leaders and members in co-designing and implementing plans for primary data collection and co-developing plans for secondary data analysis. In addition to local stakeholders, plans were also written and presented for approval by funders, a state-level IRB, and the Office of Management and Budget.
Abstract 3 Title: Turning Communication into Collaboration: The Development of an Outcomes-Based Management Training Program
Presentation Abstract 3: When an organization has a diverse client base, it is crucial that front-line staff develop an understanding of how to evaluate the needs of the varied individuals they serve. The presenter’s New York City based agency has implemented a Management Training Program (MTP) in which emerging leaders in the field are taught to identify systems to strengthen program performance and bring their knowledge back to their programs to benefit their supervisory, communication, and leadership skills. Utilizing a mixed method approach that includes a set of outcome measures, the group has developed an evaluation design that pinpoints the program’s strengths, assesses impact on program performance, and identifies ways to adapt to changing workforce conditions over a period of several years. The presenter will discuss how employing a wide spectrum of service modalities in the development and implementation of the MTP has enhanced communication and collaboration between line workers and senior management.
Abstract 4 Title: Communication and Context: Lessons Learned from an evaluation of School Discipline Reform advocacy
Presentation Abstract 4: Communication’s content and process shape evaluation both formally and informally. Context is communicated as well, providing important clues for stakeholder engagement. The current paper borrows from the literature on organizational communication to describe how Atlantic Philanthropies' external communication needs helped shape evaluation design and products. We adapted the "cultural web" model to ask these questions about evaluation stakeholders: (1) What are the important stories stakeholders like to tell that define this Initiative? (2) How do the people involved in this Initiative interact with each other? (3) Who are the leaders in this Initiative, both formal and informal, and whose contributions are most valued by others? (4) What are the control systems that may influence how information is reported about this Initiative (i.e., existing media coverage); and (5) What are the power structures that help identify the people who have the greatest amount of influence on how findings will be utilized?
Audience Level: All Audiences

Session Abstract: 

Very often evaluators fail to realize the importance
of communication techniques in the design of program evaluation. This panel
will offer four case studies on how the evaluator utilized a variety of
communication techniques to engage the stakeholder in the design of the
evaluation process. Further, these communication strategies empowered the
stakeholder to play not only an active role in the evaluation process but also
led to outcomes that strengthened program performance. Participants will walk
away with a series of communication tools that will help to strengthen their
program evaluation designs.

For questions or concerns about your event registration, please contact registration@eval.org or 202-367-1173.

For questions about your account, membership status, or help logging in, please contact info@eval.org.

Cancellation Policy: Refunds less a $50 fee will be granted for requests received in writing prior to 11:59 PM EDT October 3, 2016. Email cancellation requests to registration@eval.org. Fax request to (202) 367-2173. All refunds are processed after the meeting. After October 3, 2016 all sales are final.