(08) Big data and evaluation

Session Number: 08
Track: Professional Development Workshops
Session Type: Professional Development Workshops
Workshop Lead Presenter: Michael Bamberger, Dr. [Independent consultant - independent consultant]
Other Workshop Presenter 2: Peter York [Principal Associate - Community Science]
Other Workshop Presenter 3: Kerry Bruce [Vice President, Performance Evaluation - Social Impact]
Time: Nov 07, 2017 (09:00 AM - 04:00 PM)
Room: Tues 08

Audience Level: Beginner
Learning Outcomes (PD Workshops): • Tools and techniques for the collection and analysis of big data
• Potential benefits and challenges associated with using big data
• The four main kinds of data analytics that can be used with big data
• Practical guidelines for integrating big data into the design and implementation of monitoring and evaluation in both industrial and developing countries
Facilitation Experience (PD Workshops): Michael Bamberger has worked in international development evaluation for more than 40 years, and has been involved in the evaluation of programs in more than 30 countries. He has worked with a number of NGOs and was a senior sociologist at the World Bank. Since retirement he has consulted on evaluation issues with 10 UN agencies as well as with multilateral development banks, foundations, developing country governments and NGOs. Over the past 5 years he has worked on the integration of new information technologies into the evaluation of development programs, with particular emphasis on the social, political and ethical dimensions. Recent publications include: "Integrating big data into the monitoring and evaluation of development programs" Commissioned by UN Global Pulse 2016; (with Linda Raftree 2014) "Emerging opportunities: Monitoring and Evaluation in a Tech-Enabled World"; (with Jos Vaessen and Estelle Raimondo (2016) "Dealing with complexity in development evaluation"; (with Linda Raftree and Veronica Olazabal 2016) "The role of new information and communication technologies in equity-focused evaluation: opportunities and challenges."

Session Abstract (150 words):  The purpose of this workshop is to provide an introduction to the tools and techniques for the collection and analysis of big data, and to use case studies to illustrate the multiple ways in which big data and data analytics can strengthen program evaluation in both industrial and developing countries. The workshop seeks to offer a balanced view of both the potential benefits as well as the political, ethical, methodological and logistical challenges.  It will also address the question of what are the conditions required for the successful application of big data to evaluation, and to what extent do these conditions exist in developing countries?  After presenting an overview of the main kinds of data collection instruments (the analysis of secondary data, the use of twitter and other social media, electronic data from smart phones, and satellite images); the workshop will present the three main methods used to analyze these data (descriptive, predictive, and prescriptive analytics). These include both sophisticated techniques, such as predictive modelling and machine learning, as well as tools for combining multiple sources of information into an integrated database with a common metric that permits the identification of new patterns and insights. Big data and smart data analytics are playing an increasingly important role in all areas of research, policy, and program design, delivery and evaluation.  While the advances have been most rapid in industrial economies, applications are spreading rapidly in developing countries. Media coverage portrays big data as everything from the start of a higher level of social development to insidious tools promoting exploitation by hidden commercial empires and political control by powerful elites.  Most evaluators are still trying to understand and assess the exciting potential and the troubling downsides. Many evaluators also wonder whether these sophisticated new technologies will ever have much impact in communities and countries with limited digital infrastructure. Case studies from the US and developing countries will illustrate how big data is already being used for the evaluation of social programs through real-time diagnostic studies (description), predicting which groups are likely to succeed and fail and identifying individuals or groups at risk so that remedial actions can be taken (prediction), and proposing guidelines to improve the design and implementation of future programs (prescription). The workshop will provide practical guidelines for integrating these potentially powerful tools and techniques into the design and implementation of the kinds of evaluations with which participants are familiar. It will also use principles of adult learning. The focus will be problem-based and collaborative (rather than didactic), encouraging participants to share their professional experiences and knowledge.  It will also show how big data can help participants’ achieve their own work objectives.  The workshop will be structured around the discussion of real-life cases providing hands-on experience. Participants will receive a report published earlier this year by UN Global Pulse and the Rockefeller Foundation on “Integrating big data into the monitoring and evaluation of development programs.”