1W-MINDS, Oct. 9: Anna Veselovska (Technical University of Munich), Gradient Descent and...
Автор: Mark Iwen
Загружено: 2025-10-09
Просмотров: 82
Gradient Descent and Implicit Bias in Tensor Factorizations
Why does gradient descent, when run on highly over-parameterized models, prefer simple solutions? For matrices, this implicit bias toward low-rank structure is well established, but extending such results to tensors is much harder. In this talk, I will present our recent work that establishes implicit regularization in tensor factorizations under gradient descent. We focus on the tubal tensor product and the associated notion of tubal rank, motivated by applications to image data. Our results show that, in overparameterized settings, small random initialization plays a key role: it steers gradient descent toward solutions of low tubal rank. Alongside the theory, I will present simulations that illustrate how these dynamics shape the optimization trajectory. This work bridges a gap between the matrix and tensor cases and connects implicit regularization to a broader class of learning problems.
Joint work with Santhosh Karnik, Mark Iwen, and Felix Krahmer.
Доступные форматы для скачивания:
Скачать видео mp4
-
Информация по загрузке:
