Evaluation 2017: From Learning to Action

View Printable Version

Improving outcome measurement

Session Number: 3233
Track: Quantitative Methods: Theory and Design
Session Type: Multipaper
Tags: analytics, CFA, child welfare, foster youth, home visiting, Metrics
Session Chair: Mende F Davis [Research Assistant Professor - University of Arizona]
Discussant: Vajeera Dorabawila [Assistant Director/Research Scientist - NYS Children and Family Services]
Presenter 1: William Edward Pate, II [Principal - Business]
Presenter 2: Mende F Davis [Research Assistant Professor - University of Arizona]
Presentation 1 Additional Author: Anne J Atkinson [President - PolicyWorks, Ltd., Petersburg, VA]
Presentation 2 Additional Author: Craig Lecroy
Time: Nov 10, 2017 (05:30 PM - 06:15 PM)
Room: Delaware A

Abstract 1 Title: Creating a New Metric to Assess Placement Stability in Foster Care
Presentation Abstract 1:

This paper presents the development and application of a new measure for assessing foster care placement stability, an important performance indicator in child welfare. Although counts of placement moves are commonly used in both child welfare practice and research they do not adequately control for length of time in foster care. Using child welfare administrative data, a new metric -- rate of placement change per month -- was developed. A brief review of related child welfare literature and overview of the administrative dataset provide the framework within which this metric was created and used. Sub-study findings related to this metric will be briefly presented and discussed.

Abstract 2 Title: Examining the dimensionality of outcomes
Presentation Abstract 2:

Home visitation programs for new mothers are designed to reduce child abuse and neglect.  Previous studies of home visitation interventions have often examined a narrow range of outcomes. Often, the outcomes reported across clinical studies are disparate and do not overlap, making study comparisons difficult.  We report on a study that examines the impact of a widely implemented home visitation program (Healthy Families) on a broad array of outcomes for participants. We examined the underlying latent structure of the study measures using confirmatory factor analysis. In other research areas, we found that a single latent factor provided the best fit for an array of multiple outcomes.  However, this was not the case for the home visitation outcomes, which were best described by multiple factors. Understanding the diversity in the client population and the factorial structure of the outcome measures has the potential to improve the selection of measures and data analyses. 

Theme: Learning What Works and Why
Audience Level: All Audiences

Session Abstract (150 words): 

Improving outcome measures is a strong approach to strengthen child welfare evaluations.   This session will discuss measurement issues and results from two evaluation studies; the development of a new measure to reflect foster child placement stability, and an examination of the dimensionality of multiple outcomes that are frequently used to assess home visitation interventions. The results of both evaluations may be informative for study design and the selection of outcome measures.




For questions or concerns about your event registration, please contact registration@eval.org or 202-367-1173.

For questions about your account, membership status, or help logging in, please contact info@eval.org.

Cancellation Policy: Refunds less a $50 fee will be granted for requests received in writing prior to 11:59 PM EDT October 16, 2017. Email cancellation requests to registration@eval.org. Fax request to (202) 367-2173. All refunds are processed after the meeting. After October 16, 2017 all sales are final. For Evaluation 2017, international attendees and presenters who encounter complications due to the international travel environment will have up to 30 days after the event to request a refund and submit appropriate documentation. No administrative fee will apply for the international requests. The $50 fee will be waived for registrants who planned to travel into the US and experienced international travel issues.