Evaluating SageMath-Augmented LLM Agents for Advanced Mathematics Research

Pavel Snopov and German Magai have introduced a novel framework for evaluating SageMath-augmented LLM agents in computational and experimental mathematics. This research, submitted on July 7, 2026, explores the integration of Computer Algebra Systems (CAS) within agentic workflows, aiming to enhance problem-solving capabilities in mathematics.

Integration of SageMath in AI Workflows

The study emphasizes the significant yet underexplored role of CAS in enhancing LLM workflows. By proposing a ReAct-style agentic setup, the authors combine LLM reasoning with verifiable feedback from SageMath, facilitating a more robust approach to tackling complex mathematical problems.

Utilizing Context7 for up-to-date documentation, the framework evaluates the performance of various models against the RealMath benchmark. The results indicate a substantial performance improvement when SageMath is integrated, showcasing its potential to assist mathematicians in computational exploration.

Performance Evaluation and Benchmark Improvements

The research introduces refinements to the RealMath benchmark, including a multi-step post-processing procedure and a multi-stage validation pipeline. These enhancements significantly improve the quality and reliability of the problem set extracted during experiments.

The findings reveal that models benefitting from SageMath access show an average performance gain of 9.7 percentage points, with variations ranging from 1.5 to 27.8 percentage points. Notably, Qwen 3.7-Max exhibited the highest benefits from SageMath, while GPT-5.5 achieved a solve rate of 75.2% and minimized token usage among tool-enabled configurations.

Implications for Future Mathematical Research

The implications of this research suggest that CAS-augmented agents could significantly enhance the capabilities of mathematicians, particularly in computational exploration. The authors assert that this work is a progressive step towards automated conjecture discovery, indicating a promising future for AI in the field of mathematics.

The full project repository is available online, providing access to further details and experimental results.