BRAIN SIGNALS TO ACTION: Monitoring and Explaining User Cognitive Load with Foundation Models

Host: Dimitra Emmanouilidou, Microsoft Research
Speaker: Deeksha Moodasarige Shama

Passive monitoring of cognitive load can enable personalized user experiences and even accelerate human learning by leveraging closed-loop adaptive training systems. Electroencephalography (EEG) provides a cost-effective, non-invasive window into brain activity, yet conventional methods struggle with cross-subject variability. Leveraging the power of large pretrained brain foundation models (BFMs), we demonstrate that fine-tuning even a small subset of BFM layers enables cognitive load predictions that reasonably improve on traditional approaches. Despite their scale, BFMs support an end-to-end pipeline that executes in under one second, a fitting requirement for real and practical scenarios. Additional explainability tests via partition SHAP analysis reveal that predictive performance hinges on signals from prefrontal, frontal, and posterior regions linked to working memory and spatial-visual processing. Longitudinal trends show increasing reliance on prefrontal and frontal regions, correlating with blink patterns and indicating learning progression. These findings position BFMs as an effective and informative solution for automatic cognitive load monitoring, with future work focused on mitigating effects of multimodal input integration.