LEMAS Seminar by Professor Rasoul Etesami (UIUC) on Learning in Stochastic Games

Title: Learning Stationary Nash Equilibrium Policies in n-Player Stochastic Games with Independent Chains

Abstract: Motivated by applications such as bandwidth allocation in wireless communication networks and energy management in smart grids, we consider a subclass of n-player stochastic games, in which players have their own internal state spaces while they are coupled through their payoff functions. It is assumed that players' internal chains are driven by independent transition probabilities. Moreover, players receive only realizations of their payoffs and cannot observe each other's states or actions. For this class of stochastic games, we first show that finding a stationary Nash equilibrium (NE) policy is intractable without additional assumptions on the reward functions. Nevertheless, for general reward functions, we develop polynomial-time learning algorithms based on dual averaging that converge to the set of ε-NE policies in terms of the averaged Nikaido-Isoda distance, either almost surely or with high probability. We further extend our results to the settings with unknown transition probabilities. In particular, under additional assumptions on the reward functions or the equilibrium structure, we show that our algorithms indeed converge to an ε-NE policy with high probability in polynomial time. Finally, we demonstrate the effectiveness of the proposed algorithms through numerical experiments on energy management in smart grids.

Bio: Rasoul Etesami is an Associate Professor in the Department of Industrial and Systems Engineering at the University of Illinois Urbana-Champaign (UIUC), where he is also affiliated with the Department of Electrical and Computer Engineering and the Coordinated Science Laboratory. Prior to joining the faculty at UIUC, he was a Postdoctoral Research Fellow in the Department of Electrical Engineering at Princeton University and WINLAB. He received his Ph.D. in Electrical and Computer Engineering from UIUC in 2015. His research interests include the analysis of complex socioeconomic and decision-making systems using tools from control theory, game theory, optimization, and learning theory. He is the recipient of the Best CSL Ph.D. Thesis Award at UIUC in 2016, the Springer Outstanding Ph.D. Thesis Award in 2017, the NSF CAREER Award in 2020, the US Air Force Young Investigator Award in 2023, the SIAM Journal on Control and Optimization Best Paper Award in 2025, and multiple best teaching and reviewer awards. He has served as General Chair for conferences such as the Annual Allerton Conference and the C3.ai DTI Workshop, and as an Associate Editor for journals including IET Smart Grid and Dynamic Games and Applications