Session Number: 1309
Track: Organizational Learning & Evaluation Capacity Building
Session Type: Panel
Tags: capacity building, evaluation capacity development, evidence-based practices, government evaluation, grantmaking, Organizational change, quasi-experimental design
Session Chair: Mary Hyde [Corporation for National and Community Service]
Presenter 1: Lily Zandniapour [Research and Evaluation Manager - Corporation for National and Community Service]
Presenter 2: Janet D Griffith [Senior Fellow]
Presenter 3: Lance D Potter [Evaluation Strategy LLC]
Time: Oct 27, 2016 (08:00 AM - 09:30 AM)
Abstract 1 Title: Overview of the Social Innovation Fund and the National Assessment
Presentation Abstract 1:
Presenter will provide a brief background about the Social Innovation Fund and its key features to provide context for the presentation and outline the various domains of interest in terms of targets of organizational change.
She will then discuss the key driving questions for the assessment:
1. Does participation in the SIF increase the capacity of organizations (for example, skills, attitudes, and behaviors) in the use of evidence-based grantmaking strategies?
2. Does participation in the SIF increase the willingness and ability of organizations to build the evidence base for high-impact models?
3. Does participation in the SIF increase organizational knowledge of how to scale effective program models?
4. Does participation in the SIF facilitate collaborative approaches to addressing local community needs?
She will then introduce ICF team, the independent evaluator that was engaged in designing the SIF National Assessment and implementing it.
Abstract 2 Title: The Design and Implementation of the SIF National Assessment: Insights and Lessons Learned
Presentation Abstract 2:
This presenter will provide context by discussing the methodological problem that evaluators face in assessing the effectiveness of organizational change initiatives. She will then describe the strategy, design elements, and methodologies used (e.g. surveys, interviews, document reviews) for conducting the SIF National Assessment, which can serve as a case study of research on change in organizational practice. She will explain the many challenges that this assessment posed such as:
1. Timing of data collection – retrospective
2. Period of change in field – needed to rule out secular trend and other threats to validity
3. Need to validate self-report of change
4. Who speaks to organizational capacity within the agency
She will then outline solutions developed by the ICF team to address those challenges, highlight key domains of capacity measured, and effect sizes. The presenter will finally describe key takeaways from this effort and implications for future work on similar evaluation efforts.
Abstract 3 Title: The SIF National Assessment: Lessons learned from the SIF National Assessment and the experience of New Profit, a grantee of the Social Innovation Fund participating in the assessment
Presentation Abstract 3:
New Profit is one of SIF intermediary organizational partners from among the program’s first cohort of grantees. The presenter will discuss New Profit’s experience in participating in SIF and the National Assessment and will share findings and lessons learned from this experience that has informed the work of New Profit. He will elaborate on the extent to which the key measurement and findings from this effort align with the experiences of New Profit with its own cohort of subgrantees. The presenter will talk about baseline capacity of participating organizations, indicators of readiness especially relating to conducting and using rigorous evaluations, and how capacities and practices have changed over the course of the program.
Finally, he will share his perspective on the implications of this work for the policymakers, as well as philanthropic and nonprofit community. His presentation will set the stage for an exchange on this topic with session participants.
Audience Level: All Audiences
This session will highlight key design elements used for evaluating the effectiveness of the Social Innovation Fund (SIF), a high-profile, federal tiered-evidence program and share lessons learned from the implementation of the assessment. The SIF is a national program that is administered by the Corporation for National and Community Service. It was launched in 2009 as a model for funding what works. The program aims to bring about change in organizational practices and policies of high-performing nonprofits. This session focuses on the design and methodological decisions that were made to address the challenges that this complex, multi-component program posed while ensuring that the evaluation is rigorous and provides credible evidence to answer the assessment’s key driving questions. With presentations setting the stage, session participants will engage in a conversation about the applicability of the features of this evaluation for informing the evaluations of similar programs or strategies in the future.