LabelBench: A Comprehensive Framework for Benchmarking Adaptive Label-Efficient Learning

Speaker: Jifan Zhang (https://jifanz.github.io/) from UW-Madison
Time: Oct 27, 2023, 12:30 PM – 1:30 PM CT
Location: Engineering Centers Building (1550 Engineering Drive) M1059 (M floor)
Abstract: Labeled data are critical to modern machine learning applications, but obtaining labels can be expensive. To mitigate this cost, machine learning methods, such as transfer learning, semi-supervised learning and active learning, aim to be label-efficient: achieving high predictive performance from relatively few labeled examples. While obtaining the best label-efficiency in practice often requires combinations of these techniques, existing benchmark and evaluation frameworks do not capture a concerted combination of all such techniques. This paper addresses this deficiency by introducing LabelBench, a new computationally-efficient framework for joint evaluation of multiple label-efficient learning techniques. As an application of LabelBench, we introduce a novel benchmark of state-of-the-art active learning methods in combination with semi-supervised learning for fine-tuning pretrained vision transformers. Our benchmark demonstrates better label-efficiencies than previously reported in active learning. LabelBench’s modular codebase is open-sourced for the broader community to contribute label-efficient learning methods and benchmarks. The repository can be found at: https://github.com/EfficientTraining/....
Bio: Jifan is a PhD student at UW-Madison working with Prof. Robert Nowak. His work focuses on label-efficient learning and its modern application to large-scale deep learning systems. He is also generally interested in human-in-the-loop learning and cross-disciplinary research that apply these methods in the real world.