Designing for Meaningful Evaluation: Examples from CDC’s DELTA FOCUS Program

Session Number: 1256
Track: Cluster, Multi-site and Multi-level Evaluation
Session Type: Panel
Tags: CDC, cluster, multi-site and multi-level eval, Data Management
Session Chair: Theresa L Armstead [Behavioral Scientist - Centers for Disease Control and Prevention]
Presenter 1: Lianne Fuino Estefan [Behavioral Scientist - CDC]
Presenter 2: Cynthia Roberts [Empowerment Evaluator - Rhode Island Coalition Against Domestic Violence]
Presenter 3: Moira S. Rivera, Dr. [Senior Evaluation Associate - Contracting Resources Group]
Presentation 1 Additional Author: Megan Kearns [Behavioral Scientist - Centers for Disease Control and Prevention]
Presentation 1 Additional Author: Theresa L Armstead [Behavioral Scientist - Centers for Disease Control and Prevention]
Presentation 3 Additional Author: Rasha Mariam El-Beshti [Program Evaluator - Contracting Resources Group, Inc. ]
Time: Oct 27, 2016 (02:00 PM - 02:45 PM)
Room: A705

Abstract 1 Title: An Adaptive Evaluation Framework: The DELTA FOCUS Evaluation Design
Presentation Abstract 1:

CDC’s DELTA FOCUS program uses an adaptive evaluation framework to address the challenges of multi-site, multi-level evaluations in support of primary prevention for intimate partner violence. The evaluation design balances flexibility and standardization to document and describe the multiple programs and strategies implemented by grantee coalitions at the state-level and the coalitions they fund at the local-level. This is accomplished in part by the development and use of a data management system that was originally designed for performance monitoring and is being leveraged for program evaluation. While it presents some challenges the data management system offers several advantages including electronically provided information on evaluation progress which facilitates analyses that would be impracticable using the paper-based information systems of the past. This presentation describes the overall evaluation design of the CDC DELTA FOCUS Program, context for state-specific evaluation designs, and use of an innovative method to account for multi-site evaluation challenges.

Presentation 1 Other Authors: Kirsten Rambo, Centers for Disease Control and Prevention; Jenny Dills, Centers for Disease Control and Prevention
Abstract 2 Title: Designing for Meaningful Evaluation: A State-wide DELTA FOCUS Example
Presentation Abstract 2:

The Rhode Island Coalition Against Domestic Violence is evaluating three overarching strategies in intimate partner violence prevention: systems change, engaging men, and structural determinants of health. Given that complexity and innovation are common characteristics across these strategies, embracing a Developmental Evaluation approach has helped us meaningfully evolve the strategies through various stages of development and fruition. The purpose of this presentation will be to (1) describe key components of Rhode Island’s evaluation of statewide prevention strategies, (2) describe contextual factors which contributed to and continue to shape how design decisions are made, (3) describe data collection methods (4) share preliminary findings, and (5) share lessons learned from our own experience of aiming to authentically meet the expectations of a national evaluation framework while remaining focused on meaningfully evaluating our local and emergent prevention strategies.

Abstract 3 Title: A Prevention Strategy Database and Case Studies: Evaluation Solutions for Multi-Site Community Coalitions
Presentation Abstract 3:

The DELTA FOCUS Project-wide evaluation includes an assessment of the impact of each individual prevention strategy on intimate partner violence (IPV) and IPV’s risk and protective factors. Because each Coalition executes multiple strategies, the Project-wide evaluation incorporates a system – The Prevention Strategy Database (PSD) – that houses information related to strategies’ characteristics, implementation, process and outcome evaluation methods, and evaluation findings. The PSD enables the DELTA FOCUS Team to monitor each strategy, compile findings across sites for strategies with similar characteristics, and promote promising strategies. The purpose of this presentation will be to describe (1) the PSD and its development, (2) the data collected through the PSD, (3) how the PSD was used to identify a promising prevention strategy and develop the strategy into a case study, (4) the strengths and limitations of the PSD approach, and (5) lessons-learned from using the PSD to monitor multi-site, multi-level coalitions’ implementation and effectiveness.

Audience Level: All Audiences

Session Abstract: 

Evaluating community coalitions’ impact on health outcomes is a common challenge for evaluation professionals. This challenge is magnified for programs that involve multiple coalitions at different sites that share a common goal, but choose initiatives based on community needs and capacities. The Centers for Disease Control and Prevention’s DELTA FOCUS Program is an example of such a program. DELTA FOCUS funds 26 state and community coalitions that implement intimate partner violence (IPV) prevention strategies. Each coalition executes prevention strategies based on state and local priorities, while maintaining alignment with the broader Program goals. This presentation will describe the DELTA FOCUS Project-wide evaluation approach, a design that balances flexibility and standardization. The panel will present an overview of the evaluation design, discuss how integrating these methods addresses issues with the evaluation of multi-site community coalitions, and outline a detailed example of an embedded evaluation design – including evaluation methods and findings.