CITP Lunch Seminar: Matthew Salganik - Measuring the Predictability of Life Outcomes

Researchers have long theorized about the processes through which childhood experiences shape life outcomes. However, statistical models in the social science often have poor predictive performance. Despite this track record, policy makers are increasingly considering using complex predictive models for high-stakes decisions in settings such as criminal justice and child protective services.

In this talk, we present results from the Fragile Families Challenge, a scientific mass collaboration designed to assess the limits of predictability of life outcomes and improve our understanding of these limits. Using data from the Fragile Fragile Families and Child Wellbeing Study, a high-quality, birth cohort study that has followed about 5,000 mainly disadvantaged families for the past 15 years, 457 researchers built predictive models of six life outcomes, such as a child’s grades in school or whether the family would be evicted from their home. Research participants in the Challenge could use any theoretical, statistical, or machine learning approach they wished and could draw on the more than 12,000 features that had been measured about the child, parents, and family since the birth of the child. All predictions were evaluated on held-out data. Our empirical results have implications for social science theory, data, and methods and for algorithmic decision-making in high-stakes social settings.