Achieving Data-Efficient Neural Networks with Hybrid Concept-based Models: Tobias Opsahl (UiO)

Tobias Opsahl, a master's student in data science at Institute of Mathematics, University of Oslo, gave a presentation titled "Achieving Data-efficient Neural Networks with Hybrid Concept-based Models" (15th Feb. 2024)

Abstract:

Most datasets used for machine learning consist of a single label per data point, which is used to optimise the model. However, in cases where more information than just the class label is available, would it be possible to train models more efficiently? We introduce two novel neural network architectures that train with both class labels and additional information in the dataset, referred to as concepts. We call these models hybrid concept- based models, since they use both concept predictions and information not interfering with the concepts to predict the class. In order to thoroughly explore their performance, we introduce ConceptShapes, an open and flexible class of datasets with concept labels. Through various experiments, we show that the hybrid concept-based models outperform standard computer vision models and previously proposed concept-based models with respect to performance, especially in sparse data settings. We also introduce an algorithm for performing adversarial concept attacks, where an image is perturbed in a way that does not change a concept-based model’s concept predictions, but changes the class prediction. We argue that this puts the interpretable qualities promised from previously proposed concept-based models into question.