Evaluation 2017: From Learning to Action

View Printable Version

Impact evaluations of TAACCCT grants: quasi-experimental approaches, challenges and lessons learned

Session Number: 1081
Track: Government Evaluation
Session Type: Panel
Tags: propensity score matching, quasi-experimental design, TAACCCT
Session Chair: Ashweeta Patnaik [Research Associate - Ray Marshall Center, The University of Texas at Austin]
Discussant: Heath Prince, Director and Research Scientist [Director and Research Scientist - University of Texas, Austin; LBJ School of Public Affairs; Ray Marshall Center]
Presenter 1: Ashweeta Patnaik [Research Associate - Ray Marshall Center, The University of Texas at Austin]
Presenter 2: Heath Prince, Director and Research Scientist [Director and Research Scientist - University of Texas, Austin; LBJ School of Public Affairs; Ray Marshall Center]
Time: Nov 09, 2017 (02:15 PM - 03:00 PM)
Room: Wilson B

Abstract 1 Title: Using propensity score matching to study the impact of TAACCCT programs
Presentation Abstract 1:

The Retraining the Gulf Coast Workforce through Information Technology Pathways Consortium project was a four-year project funded by the Department of Labor’s TAACCCT grants program.  The grant was awarded in September 2012 to a consortium of nine community colleges across the states of Louisiana and Mississippi to implement the grant through March of 2016.  The program’s objective was to capitalize on the region’s growing IT sector and its increased demand for skilled labor by training almost individuals with basic skills needs for jobs. The Ray Marshall Center conducted the impact evaluation.The evaluation included a quasi-experimental impact analysis to estimate program impacts on key education and employment outcomes. Propensity score matching was used to construct a comparison group and address the key issue of the counterfactual: what would have happened absent the intervention? In this presentation, the lead evaluator share her team's methodology, findings and lessons learned.


Abstract 2 Title: Dealing with data access challenges in TAACCCT evaluations
Presentation Abstract 2:

Training for Regional Energy in North Dakota (TREND) was a consortium of five community colleges that collaborated to address labor shortages in the state’s energy sector, funded by the Department of Labor’s TAACCCT grants program.  Leaders from these institutions equipped workers with post-secondary certificates and credentials in order to meet employers’ needs for a skilled workforce, as well as to help ensure that workers can access and succeed in this rapidly expanding sector. The Ray Marshall Center conducted the impact evaluation of the TREND program. Although a rigorous evaluation using a quasi-experimental methodology had been originally planned, evaluators were forced to scale back their approach and step down in rigor due to lack of data. In this presentation, the lead evaluator shares the challenges faced by the evaluation team and their approach to addressing these challenges.


Theme: Learning to Enhance Evaluation
Audience Level: All Audiences

Session Abstract (150 words): 

The Ray Marshall Center (RMC) at The University of Texas has served as the external evaluator for several TAACCCT grants, including large multi-state consortium grants that included multiple institutions across states. RMC's impact evaluations included an outcomes analysis as well as a quasi-experimental impact analysis to estimate the TAACCCT program's impacts on key education and employment outcomes. In this panel, researchers from RC share their experience of conducting such impact evaluations. They share findings from recently concluded evaluations, focusing on their methodology,  findings, challenges faced, approaches to address these challenges and lessons learned. The evaluators also share how these lessons learned are now being applied to currently ongoing evaluations.



For questions or concerns about your event registration, please contact registration@eval.org or 202-367-1173.

For questions about your account, membership status, or help logging in, please contact info@eval.org.



Cancellation Policy: Refunds less a $50 fee will be granted for requests received in writing prior to 11:59 PM EDT October 16, 2017. Email cancellation requests to registration@eval.org. Fax request to (202) 367-2173. All refunds are processed after the meeting. After October 16, 2017 all sales are final. For Evaluation 2017, international attendees and presenters who encounter complications due to the international travel environment will have up to 30 days after the event to request a refund and submit appropriate documentation. No administrative fee will apply for the international requests. The $50 fee will be waived for registrants who planned to travel into the US and experienced international travel issues.