Evaluation 2018: Speaking Truth to Power

View Printable Version

Systems Approaches for Organizational Development

Session Number: 1906
Track: Systems in Evaluation
Session Type: Panel
Tags: evaluation capacity development, organizational capacity building, systems approaches, Systems Thinking
Session Chair: Heather Britt [Senior Monitoring and Evaluation Specialist - Social Solutions International]
Discussant: Francis Higdon
Presenter 1: Heather Britt [Senior Monitoring and Evaluation Specialist - Social Solutions International]
Presenter 2: Kristin Saarlas [Senior M&E Advisor - USAID]
Presenter 3: Kristin Saarlas [Senior M&E Advisor - USAID]
Presenter 4: Tonya Caprarola Giannoni [Chief of Party - Social Solutions International]
Time: Nov 01, 2018 (08:00 AM - 09:00 AM)
Room: Hilton - Veterans Meeting Room A

Abstract 1 Title: Systems Thinking for Organizational Development in Complex Adaptive Systems
Presentation Abstract 1:

This presentation provides an overview of MECap and explains how systems thinking equips MECap to contribute to MEL capacity outcomes in a complex adaptive system. Organizational typologies such as “bureaucracy” suggest static and hierarchical structures within an organization such as USAID.  In contrast, the concept of interrelationships sheds light on the fluid nature of exchanges between USAID’s offices, and acknowledges multiple sources of power within USAID. MECap’s effectiveness depends on its ability to meet the unique capacity needs of approximately 59 offices.  Each office represents a distinct perspective on the Agency, its mission, the role of MEL. MECap must understand those perspectives and “speak” the language of its diverse clients. Outcome spread and sustainability depends on effective knowledge sharing across the organization and integration of new practices into ongoing work processes. For each fellowship and clinic, MECap considers and sets boundaries, expanding and contracting the network to achieve optimum results.


Abstract 2 Title: Combining Outcome Harvesting and Social Network Analysis to Strengthen a Global Fellowship Program
Presentation Abstract 2:

MECap’s original design and current implementation is based on the assumption that collaboration supports outcomes in fellowship placements.  This presentation explains how MECap uses outcome harvesting (OH) with social network analysis (SNA) to examine that assumption and support the success of the Fellows Program.  Fellows’ outcomes (planned and emerging) are collected through in-person gatherings and monthly reports.  MECap core staff engage with Fellows to finalize outcome statements.  Core staff verify outcomes and conduct preliminary analysis. A streamlined version of SNA captures collaboration within the MECap network. MECap is using a basic network analysis approach with low sample-sizes and open-source software to obtain actionable answers on the nature, frequency, and content of collaboration. Data from SNA is analyzed with findings from OH on Fellows’ outcomes. Together, SNA and OH explain whether and how collaboration supports fellows’ effectiveness. Findings inform interventions to optimize collaboration for strengthening organizational MEL capacity.


Abstract 3 Title: Success Case Method for Actionable Evaluation Capacity Building
Presentation Abstract 3:

Because of the diversity of workplace settings across USAID, and their influence on trainees’ application of learning, evaluation approaches that exclude or downplay contextual factors are of limited use to MECap.  In contrast, the Success Case Method (SCM) captures both training outcomes, and the influence of the workplace setting on those outcomes.  Following a clinic, participants are surveyed to determine whether they have applied knowledge and skills from the clinic to make a significance difference in their office. A small sample of respondents reporting success and a small sample reporting non-success are interviewed to collect data on how knowledge and skills are being used, the impact of the use of learning, and workplace factors that enabled or inhibited results. Findings enable managers to deliver workshop content applicable in diverse and changing workplace settings across USAID. 


Abstract 4 Title: Systems Approaches inform Program Management
Presentation Abstract 4:

MECap's Chief of Party, explains how all three systems approaches provide information needed to ensure delivery of high-quality fellowships and clinics, and contribute to USAID’s MEL capacity. MECap uses systems think to manage adaptively and achieve results in a diverse and continually evolving context. OH, SNA and SCM provide actionable data to inform decision-making while accounting for several types of complexity in MECap’s operational context, including diverse workplace settings and perspectives across the Agency, fluid interrelationships within USAID, and shifting boundaries for each task. Analysis of findings from both clinics and fellowships may suggest common enabling factors for achieving MEL capacity in offices.  Finally, all three approaches help assess MECap’s progress towards objectives and will contribute to the final evaluation at the close of the task order.


Audience Level: All Audiences

Session Abstract (150 words): 

The Expanding Monitoring and Evaluation Capacities (MECap) task order collaborates with USAID offices to foster good practices in planning, design, monitoring, evaluation, and learning. MECap’s effectiveness requires meeting the unique capacity needs of more than 59 offices. The spread and sustainability of its outcomes requires effective knowledge sharing and integration into ongoing work processes. First, the session explains how systems thinking equips MECap to contribute to MEL capacity in a complex adaptive system. Second, the session describes how outcome harvesting and social network analysis together explain how collaboration influences the outcomes of two MECap components – fellowships and clinics. The third presentation explains how the success case method ensures that clinics provide content that can be applied in diverse workplaces settings and contribute to improved MEL capacity.  Finally, the session describes how findings from all three approaches provide actionable information for managing MECap and inform USAID about progress expanding M&E capacities.



For questions or concerns about your event registration, please contact registration@eval.org or 202-367-1173.

For questions about your account, membership status, or help logging in, please contact info@eval.org.



Cancellation Policy: Refunds less a $50 fee will be granted for requests received in writing prior to 11:59 PM EDT October 8, 2018. Email cancellation requests to registration@eval.org. All refunds are processed after the meeting. After October 8, 2018 all sales are final. For Evaluation 2018, international attendees and presenters who encounter complications due to the international travel environment will have up to 30 days after the event to request a refund and submit appropriate documentation. No administrative fee will apply for the international requests.