Why Benchmarks Aren't Enough: Eve Fleisig on Sociolinguistics & AI Evaluation | NeurIPS 2025
Автор: SAIL Media
Загружено: 2025-12-26
Просмотров: 4
Is Natural Language Processing (NLP) dead, or has it just evolved? At NeurIPS 2025, Jessica Dai sits down with Eve Fleisig from @BerkeleyEECS to discuss the shifting culture of machine learning and why "loving words" still matters in the age of LLMs.
In this interview, Eve breaks down the critical difference between benchmarks (fixed proxies for optimization) and true evaluations (measuring real-world, downstream impacts). She explores how sociolinguistics can help us detect subtle harms like dialect discrimination—where models discriminate based on how you speak rather than just who you are—and why we need to move beyond optimizing for a single number.
Timestamps:
0:00 - The NeurIPS experience & where did the "NLP" go?
1:32 - The challenge of evaluation when humans disagree
2:17 - Bringing linguistics back: Syntax vs. Sociolinguistics
3:06 - The hidden harm of Dialect Discrimination
4:45 - The optimization trap: Are our proxies good enough?
5:40 - The crucial distinction: Benchmarks vs. Evaluations
6:50 - The future: Measuring long-term downstream impacts
#NeurIPS2025 #AI #NLP #MachineLearning #Sociolinguistics #EthicalAI #LLM
Доступные форматы для скачивания:
Скачать видео mp4
-
Информация по загрузке:
