Reinforcement Learning with Verifiable Rewards Enhances Tool-Use Agents in Atlassian Workflows

On July 1, 2026, researchers Karthikeya Aditya Vissa and colleagues published a paper titled Beyond Next-Token Prediction: An RLVR Proof of Concept for Tool-Use Agents on Atlassian Workflows. The study explores the limitations of large language models in executing specific API tasks within niche enterprise SaaS workflows, proposing a solution through Reinforcement Learning with Verifiable Rewards (RLVR).

Understanding the Challenges of API Integration

Large language models (LLMs) are primarily designed for next-token prediction, which can lead to significant issues when applied to specific APIs. In environments like Jira and Confluence, this can result in failures such as missing required fields or hallucinated tools. These silent failures undermine the effectiveness of LLMs in real-world applications.

The research highlights the need for a tailored approach to enhance the performance of these models when interacting with APIs, particularly in enterprise settings. The authors argue that the existing objectives of LLMs do not align with the requirements of precise API interactions.

RLVR: A Novel Approach for Improvement

The authors developed a proof-of-concept using RLVR, which applies reinforcement learning directly in the target environment. This method evaluates the model's performance based on tool-call traces rather than relying on human feedback or external judges. The study involved creating five synthetic environments that closely emulate the Jira REST v3 and Confluence v2 APIs.

Initial findings indicate that RLVR significantly improves the performance of tool-use agents. For instance, the average reward for scenarios with non-degenerate rewards increased from a baseline of 0.35–0.92 to 0.95–1.00. Notably, the most substantial improvement was observed in the Confluence page creation task, where the reward jumped from 0.35 to 1.00.

Limitations and Future Directions

While the results are promising, the study acknowledges two critical limitations. First, the process of hand-crafting verifiable rewards is not scalable beyond the limited endpoints tested in this research. Second, one scenario, ticket-transition, has a reward structure that saturates, indicating that the current model has already maximized its performance in that area.

This research marks an important step toward developing optimized small models for niche enterprise APIs. Future work will need to address the scalability of reward crafting and explore additional scenarios to validate the RLVR approach further.