Evaluation 2016: Evaluation + Design

View Printable Version

The power of communicating evaluation conclusions through rubrics

Session Number: 1980
Track: Data Visualization and Reporting
Session Type: Panel
Tags: Data Visualization, Michael Scriven, participatory evaluation, Rubrics Methodology
Session Chair: Thomaz Kauark Chianca [Managing Partner - COMEA Relevant Evaluations]
Discussant: David M Fetterman [President - Fetterman & Associates]
Presenter 1: Nan Wehipeihana, Independent Evaluator [Kinnect Group]
Presenter 2: Mônica Pinto [Roberto Marinho Foundation]
Presenter 3: E Jane Davidson [President - Real Evaluation]
Presentation 1 Additional Author: Julian King [Director, Julian King & Associates Limited - Kinnect Group]
Time: Oct 28, 2016 (08:00 AM - 09:30 AM)
Room: Atrium Ballroom C

Abstract 1 Title: Communicating evaluation conclusions to different audiences through rubrics in New Zealand
Presentation Abstract 1:

Using a range of examples from small and larger evaluations, in NGO and government settings, this presentation will demonstrate how effectively evaluative conclusions can be communicated to different audiences when rubrics form the basis of evaluation design. From the use of simple tables to more sophisticated infographics, when evaluation conclusions are grounded in the use of rubrics, they can be artfully designed to ensure maximum stakeholder understanding and usability, even when complex data analysis and synthesis underpin the conclusions reached. This presentation will unpack the steps in rubric design and implementation that lead to the ability to communicate evaluative conclusions in ways that make sense to clients and communities.

Abstract 2 Title: Rubrics as a potent strategy to communicate evaluation conclusions in a Brazilian foundation: A clients’ perspective.
Presentation Abstract 2:

This presentation will focus on a client’s perspective on how the rubrics methodology helped devise strategies to communicate evaluative conclusions. Monica Pinto, Manager for Institutional Development for the Roberto Marinho Foundation, will explore positive aspects as well as challenges they faced when applying the rubrics methodology in evaluations of three of their major programs. Combining and presenting complex, extensive qualitative and quantitative data in a simple way was one of the methodology’s most powerful feature. Finding a balance between being synthetic in communicating conclusions without falling into the trap of not presenting enough convincing evidence was always a challenge. At the same time, there was some noise when providing such straightforward feedback to program staff, especially when the news were not so good. Clear and trustworthy conclusions are essential to help programs to produce meaningful social transformations; the rubrics methodology can undoubtedly contribute to that end.

Presentation 2 Other Authors: - Rosalina Soares; Roberto Marinho Foundation; rosalina.soares@frm.org.br
- Geraldine de Araujo e Silva; Roberto Marinho Foundation; geraldine.silva@frm.org.br
Abstract 3 Title: Hot Tips for Rubric Design: Powerful depths under the methodological hood.
Presentation Abstract 3:

The more simply a rubric captures complex ideas, the more deep thinking and careful design lies underneath. In this presentation, Jane will share some of the lesser-known tips for designing rubrics that are conceptually and methodologically sound, as well as great data visualization ideas for validly presenting evaluative conclusions drawn using rubric methodology. With awesome rubric design and dataviz, we can achieve breathtaking clarity without boiling rich outcomes down to something overly simplistic.

Audience Level: All Audiences

Session Abstract: 

How to deliver well-reasoned answers to real evaluative questions in a simple but not simplistic way that generates valid, credible insights stakeholders can use to take action? Evaluation rubrics is probably one of the most powerful methodologies to address those issues. It is common to evaluations to provide findings with abundant descriptive data on several indicators, leaving stakeholders without a clear sense of whether their program was a great success or a complete failure. In contrast, rubrics have as one of its most important features, the power to combine complex quantitative and qualitative data, using a consistent analytical framework, that leads to clear evaluative conclusions about the quality and importance of whatever is being evaluated. This session brings together some of the most experienced evaluators in using rubrics from the US, New Zealand and Brazil to discuss positive aspects and challenges to communicate evaluative conclusions using the methodology.

For questions or concerns about your event registration, please contact registration@eval.org or 202-367-1173.

For questions about your account, membership status, or help logging in, please contact info@eval.org.

Cancellation Policy: Refunds less a $50 fee will be granted for requests received in writing prior to 11:59 PM EDT October 3, 2016. Email cancellation requests to registration@eval.org. Fax request to (202) 367-2173. All refunds are processed after the meeting. After October 3, 2016 all sales are final.