Less is More: Tiny Recursive Models

The paper introduces the Tiny Recursion Model (TRM), a new approach to recursive reasoning that surpasses the Hierarchical Reasoning Model (HRM) and many Large Language Models (LLMs) on complex puzzle tasks like Sudoku and ARC-AGI. TRM significantly simplifies the previous HRM by using only a single, small two-layer network (7M parameters) instead of two larger networks (27M parameters), achieving higher generalisation while reducing complexity. The authors present TRM as a more straightforward and theoretically sound method, bypassing HRM's reliance on complex biological arguments and potentially inapplicable fixed-point theorems. Crucially, TRM's enhanced performance on tasks requiring intricate problem-solving suggests that small, recursively trained networks can be highly parameter-efficient for difficult problems with limited training data.