Evaluation 2016: Evaluation + Design

View Printable Version

Critical Meta-Evaluation of Social Return on Investment (SROI): Evaluator, Economist, and Accountant Perspectives on Evaluation Design, Part 1

Session Number: 2903
Track: Presidential Strand
Session Type: Panel
Tags: accounting, cost-inclusive evaluation, economics, meta-evaluation, SROI Social Return On Investment
Session Chair: Brian T Yates, Ph.D. [Professor - American University]
Discussant: Jonathan Morell [Director of Evaluation - Syntek ]
Presenter 1: Brian T Yates, Ph.D. [Professor - American University]
Presenter 2: Joe Cordes [Professor - Dept of Economics, George Washington University]
Presenter 3: Robert Louis Fischer [Research Professor - Case Western Reserve University]
Presenter 4: Kathryn (Kate) Cooney [Lecturer in Social Enterprise and Management - Yale University]
Presenter 5: Brian T Yates, Ph.D. [Professor - American University]
Presentation 1 Additional Author: Mita Marra [Associate Professor of Public Policy and Public Administration - University of Salerno, Italy]
Presentation 3 Additional Author: Elizabeth R Anthony [Research assistant professor - Case Western Reserve University]
Presentation 3 Additional Author: David Crampton [Case Western Reserve University]
Presentation 3 Additional Author: Claudia Coulton [Case Western Reserve University]
Presentation 5 Additional Author: Mita Marra [Associate Professor of Public Policy and Public Administration - University of Salerno, Italy]
Time: Oct 28, 2016 (04:30 PM - 05:15 PM)
Room: A704

Abstract 1 Title: Introduction to SROI: Methods, Terms, Current Use
Presentation Abstract 1:

Social Return On Investment (SROI) is a form of evaluation that offers answers to these questions of evaluation design, intent, and utilization, via mechanisms such as Pay for Success (PFS) and Social Impact Bonds (SIBs). SROI attempts to use carefully sourced information on resources input, activities enacted, processes inspired, and outcomes attained to formatively evaluate societal enterprises. Moreover, SROI is designed to stimulate funding by private as well as public entities of innovation in, and wider provision of, programs for remediating or preventing suffering and fulfilling human potential. Using the tools of PFS and SIBs, SROI promises a new era in evaluation which guides funders to help us all “… to do well by doing good.”

This form of evaluation is not an esoteric, academic exercise practiced by a few policy wonks. It is being used increasingly in many areas of our world to advocate for, and make, major funding decisions.

Abstract 2 Title: Using Cost-Benefit Analysis and Social Return on Investment to Evaluate the Impact of Social Enterprise: Promises, Implementation, and Limitations
Presentation Abstract 2:

A recent approach for assessing the performance of social enterprises has been to calculate the “Social Return on Investment” (SROI) to account for the economic and social return attributable to the activities of traditional nonprofits, double-bottom line social enterprises, and for-benefit corporations. When the mission of an organization can be readily translated into objectives of public interest, then the “public policy” accounting framework of cost-benefit analysis can be fairly transferred to aid evaluating funding priorities and/or performance. But is SROI simply cost-benefit analysis by a different name, or are the two approaches similar, yet also different in some important dimensions? Methodological foundations of CBA and SROI, and their implementation in practice, are examined, followed by discussion of  when CBA and/or SROI approaches are a useful lens for setting priorities and/or evaluating performance, as well as important limitations of such methods.

Abstract 3 Title: Evaluating Social Impact Financing in Human Services
Presentation Abstract 3:

Local governments often lack the resources to invest in innovative models to address key community needs. So-called “Social Impact Bonds” offer a mechanism for attracting private sector funding for initiatives that are targeted to complex and high-cost social conditions. In 2013, Cuyahoga County, Ohio became the first U.S. county to launch a social impact bond under the name “Pay For Success.” This initiative involves an intensive approach to keep homeless families together and to reduce foster care costs. This paper provides an overview of how social impact financing works and what the experience with them has been thus far. The presentation describes the development and evaluation of the initiative in Cuyahoga County focused on homeless families with children involved in child welfare. The paper explores the implications of these funding models for the evaluation of nonprofits that carry out innovative services.

Abstract 4 Title: Legitimation Dynamics: How SROI Could Mobilize Resources for New Constituencies
Presentation Abstract 4:

The “Social Return on Investment” (SROI) is an evaluation tool that exists at the crossroads of categorization and legitimation dynamics. On the one hand, much of the writing on SROI calculation in the US and UK, for example, is dominated by methodological explanations of how to construct the equivalence values and guidelines for standardization. However, the overall project itself is heavily involved in what Lamont (2012) refers to as “contestation and negotiation of value.” Thus, focusing on the categorization dynamics, this presentation reviews a set of methodological and ethical tensions surrounding the SROI, using specific examples, drawn on the author’s current research and other published works using SROI. Focusing on the legitimation dynamics, the presenter examines how the SROI can be quite effective in building a rhetorical argument for creating visibilities that can potentially mobilize resources to new constituencies.

Abstract 5 Title: Drawing Conclusions About SROI: Problems, Solutions … and Is SROI Itself a Good Investment?
Presentation Abstract 5:

The conclusion of this panel on Social Return On Investment (SROI) starts by listing the more significant problems of SROI identified in the preceding articles, along with potential solutions offered for most. An additional, emerging concern about SROI is that we do not yet know if it adds sufficient benefit to justify its cost. Two frameworks for this proposed metaevaluation of SROI are suggested, the first comparing benefits to costs summatively, the second itemizing costs and benefits according to how they contribute to and are caused by different activities of SROI so that outcomes of SROI could be maximized within resource constraints on SROI, or costs of SROI could be minimized for achieving threshold returns on investment in SROI.


Audience Level: All Audiences
Learning Outcomes: 1. What SROI is: methods, terms, current uses
2. SROI's major problems, and possible solutions
3. Meta-analytic approaches to evaluation, focusing on cost-benefit and resource → activity → process → outcome analysis of SROI
Facilitation Experience: I have conducted half- and full-day workshops for AEA and other professional organizations since the late 1970s. In the past year, among other presentations, I conducted a half-day workshop for the Office of the Inspector General of the US Environmental Protect Agency.

Session Abstract: 


Social return on investment (SROI) is a form of evaluation that promises to stimulate funding from both public and private sources for programs that prevent or remediate human suffering and fulfill human potential. By connecting societal resources with evidence-based programs, SROI attempts to guide private investors and corporations as well as governments “… to do well (in investments) by doing good (in aiding fellow human beings).” SROI can be critiqued as being a new version of existing forms of cost-inclusive evaluation, i.e., cost-benefit analysis, with more stakeholders. Evaluators, advocates, providers, and consumers who conceptualize human services as entitlements may fear that funding for critical services will end unless those services are found to return not only more than they consume, but more than competing programs. This panel orients participants to these issues, starting with definitions, then examples, and finally issues of primary concern to evaluators and program planners.


For questions or concerns about your event registration, please contact registration@eval.org or 202-367-1173.

For questions about your account, membership status, or help logging in, please contact info@eval.org.

Cancellation Policy: Refunds less a $50 fee will be granted for requests received in writing prior to 11:59 PM EDT October 3, 2016. Email cancellation requests to registration@eval.org. Fax request to (202) 367-2173. All refunds are processed after the meeting. After October 3, 2016 all sales are final.