Do LLM Agents Cooperate? Insights from GovSim and SanctSim

Speaker: ​Zhijing Jin (Max Planck Institute for Intelligent Systems, University of Toronto)

Time: 16:00 - 17:00 UTC on Thursday 11th September 2025

Abstract: As AI increasingly operates in multi-agent environments, understanding how LLM agents handle social dilemmas is critical for safe deployment. In this talk, Zhijing Jin presents findings from her two studies exploring LLM cooperation in social simulations.

First, GovSim work examines common-pool resource management, where agents must balance immediate extraction with long-term sustainability. Most LLMs fail to sustain cooperation, except for the most powerful models.

Next, SanctSim work studies public goods games with institutional mechanisms, revealing four behavioural patterns: cooperators, oscillators, deteriorators, and rigid strategists. Counterintuitively, reasoning-optimised LLMs often underperform compared to traditional LLMs in sustaining cooperation.

These findings show that enhanced reasoning doesn't guarantee cooperative behaviour – cooperation depends on communication strategies and moral reasoning. The insights have important implications for designing collaborative multi-agent AI systems.

Bio: Zhijing Jin (she/her) is an incoming Assistant Professor at the University of Toronto, and currently a postdoc at Max Planck Institute in Germany. She is an ELLIS advisor and will also be a faculty member at the Vector Institute. Her research focuses on causal reasoning with LLMs and AI safety in multi-agent LLMs. She has received three Rising Star awards, two Best Paper awards at NeurIPS 2024 Workshops, two PhD Fellowships, and a postdoc fellowship. More information can be found on her personal website: https://zhijing-jin.com/home/.