Evaluation 2018: Speaking Truth to Power

View Printable Version

Lone wolf evaluators in nonprofits: Creating and refining evaluation processes

Session Number: 2567
Track: Internal Evaluation
Session Type: Panel
Tags: internal evaluation, Nonprofit Evaluations
Session Chair: Lisa Raphael [Director of Applied Research and Program Evaluation - Communities in Schools of Central Texas]
Presenter 1: Lisa Raphael [Director of Applied Research and Program Evaluation - Communities in Schools of Central Texas]
Presenter 2: Ann Gillard [Director of Research and Evaluation - Hole In The Wall Gang Camp]
Presenter 3: Jillian Papa, MPH [Director of Research and Evaluation - Action for Healthy Kids]
Presenter 4: Hamida Jinnah [Assistant Research Faculty - University of Georgia]
Time: Nov 02, 2018 (04:30 PM - 05:30 PM)
Room: CC - 26A

Abstract 1 Title: New conversations for stakeholder engagement and agency learning

Presentation Abstract 1:

Communities in Schools (CIS) of Central TX surrounds students with a community of support, empowering them to stay in school and achieve in life. Performance monitoring data, required by funders, is used to assess staff and agency performance. The agency created my position in 2017, hoping to look more deeply at their data, using it to inform programmatic changes. While agency leaders are expected to engage program staff in data-driven conversations about performance, program staff are consumed with the realities of their job, some questioning the value of the data. There was a need to have new conversations about the data. As a lone wolf evaluator, I have worked on infusing evaluation into existing agency conversations, and changing current conversations about data. Strategies include building relationships early on, embedding evaluation into current processes, and engaging stakeholders at multiple levels in reflective data learning conversations.

Abstract 2 Title: Strategies for the alpha and omega phases of evaluation

Presentation Abstract 2:

Hired into a new position at The Hole in the Wall Gang Camp five years ago, I assumed there were goals for the many different year-round programs for children with serious illnesses and their families. While the programs did have some goals, they were not necessarily explicit or commonly discussed. Never had it been clearer that a good first step of evaluation is to create a theory of change. Three years later, theories of change exist for all programs, making evaluation much easier and more concrete and accessible to non-evaluators. My colleagues feel good about the work they do, have a shared understanding, and now identify areas in which they aim to excel. On the other side of the evaluation cycle, I developed strategies for integrating myself into existing meetings to share results and discuss implications, which has provided me a strong opportunity to ensure that evaluation informs practice.

Abstract 3 Title: Starting from scratch: Building a culture of evaluation from the ground up

Presentation Abstract 3:

Action for Healthy Kids (AFHK) mobilizes school professionals, families and communities to take actions that lead to  healthy eating, physical activity and healthier schools where kids thrive.  Upon entering a new strategic planning period, AFHK decided it was time to bring evaluation in-house.  This was both an opportunity and a challenge as the first evaluator within a rapidly growing nonprofit organization. Trying my best to navigate unchartered territory at AFHK as a lone wolf, I created a more structured and comprehensive plan for evaluation, which included developing a framework and theory of change model, an annual monitoring plan, and a monthly and quarterly metric dashboard.  These changes resulted in more streamlined and effective processes for results management, an increased understanding of the importance of evaluation, and an evaluative approach to program areas not historically evaluated. Implementing these new processes also unexpectedly facilitated exploration and curiosity for evaluation and data.

Abstract 4 Title: Moving from Outputs to Outcomes - A Shift in Thinking about the Value of Data

Presentation Abstract 4:

Institute on Human Development and Disability (IHDD) is a federally funded University Center for Excellence in Developmental Disabilities (UCEDD) that works with people who have disabilities and others. We provide training, engage in research, and implement outreach projects. As the sole evaluator, I have incorporated strategies such as prioritizing evaluation during the program development phase, making sure each program has a defined and articulated theory of change or logic model, developing a data tracking or monitoring system, and collecting data on outcomes, in addition to outputs. Infusing myself into discussions during the project conceptualization stage has helped project managers think more strategically about their work, and build an agency culture of evaluation. My biggest challenges have included shifting the focus on ‘outputs’ to ‘outcomes,' as well as encouraging staff to use data collected for reporting and fund raising purposes for learning and project improvement.

Abstract 5 Title:

Presentation Abstract 5:


Audience Level: All Audiences

Session Abstract (150 words): 

According to a 2016 survey, only eight percent of nonprofit organizations employ evaluation staff (Innovation Network, 2016). This session will introduce several evaluators who are “lone wolves” in their respective nonprofit organizations. The presenters will share the steps and strategies they took at various points to advance evaluation in their organizations. Many of the presenters took on the lone wolf mantle by working in a new position for the organization and were charged with building the infrastructure from the ground up. Others walked the fine lines between using evaluation for program improvement and using evaluation to bring in more funding. All have navigated the unknown territories of organizational cultures, ethics, and commitment to quality with many lessons learned.This panel will convene to discuss how they navigate systems of power and knowledge, and discuss why they speak truth to those who are powerful - often their very own colleagues.

For questions or concerns about your event registration, please contact registration@eval.org or 202-367-1173.

For questions about your account, membership status, or help logging in, please contact info@eval.org.

Cancellation Policy: Refunds less a $50 fee will be granted for requests received in writing prior to 11:59 PM EDT October 8, 2018. Email cancellation requests to registration@eval.org. All refunds are processed after the meeting. After October 8, 2018 all sales are final. For Evaluation 2018, international attendees and presenters who encounter complications due to the international travel environment will have up to 30 days after the event to request a refund and submit appropriate documentation. No administrative fee will apply for the international requests.