Tomasz Piotrowski - Fixed Points of Nonnegative Neural Networks | ML in PL 2024

We use fixed point theory to analyze nonnegative neural networks, which we define as neural networks that map nonnegative vectors to nonnegative vectors. We first show that nonnegative neural networks with nonnegative weights and biases can be recognized as monotonic and (weakly) scalable mappings within the framework of nonlinear Perron-Frobenius theory. This fact enables us to provide conditions for the existence of fixed points of nonnegative neural networks having inputs and outputs of the same dimension, and these conditions are weaker than those recently obtained using arguments in convex analysis. Furthermore, we prove that the shape of the fixed point set of nonnegative neural networks with nonnegative weights and biases is an interval, which under mild conditions degenerates to a point. These results are then used to obtain the existence of fixed points of more general nonnegative neural networks. From a practical perspective, our results contribute to the understanding of the behavior of autoencoders, and we also offer valuable mathematical machinery for future developments in deep equilibrium models.

Tomasz Piotrowski received the M.Sc. degree in Mathematics from Silesian University of Technology, Poland, in 2004, the M.Sc. degree in Information Processing & Neural Networks from King’s College London, UK, in 2005, the Ph.D. degree from Tokyo Institute of Technology, Japan, in 2008, and the D.Sc. degree from Systems Research Institute, Polish Academy of Sciences, in 2021. From 2009 to 2010 he worked in industry as a data analyst at Comarch SA. In 2011, he joined Nicolaus Copernicus University (NCU) in Toruń, Poland as an Assistant Professor. Since 2022, he is an Associate Professor at NCU. He is involved in brain research, signal processing, and mathematical foundations of deep learning.

This talk was one of the Contributed Talks at the ML in PL Conference 2024.

ML in PL Conference 2024 website: https://conference2024.mlinpl.org
ML in PL Association Website: https://mlinpl.org