TUMIX: Multi-Agent Tool-Use Mixture for LLM Reasoning

The paper introduces Tool-Use Mixture (TUMIX), a novel ensemble framework designed to enhance the reasoning capabilities of Large Language Models (LLMs) by optimising their use of external tools like Code Interpreters and Search. TUMIX achieves this by running multiple, diverse agents in parallel, each employing distinct tool-use strategies and iteratively sharing and refining their answers. Experiments, primarily using Gemini-2.5-Pro and Gemini-2.5-Flash across challenging benchmarks such as HLE and GPQA, demonstrate that TUMIX significantly outperforms state-of-the-art test-time scaling methods, delivering an average accuracy improvement of up to 3.55% over the best baselines. Key findings include that agent diversity and quality are more crucial than scale alone, and that using an LLM-as-Judge to adaptively terminate the refinement process can preserve performance while reducing inference costs by nearly 50%. The research also shows that LLMs can be successfully employed to automatically generate more effective and diverse agents.