World Feedback for Clinical Agents: Diagnosing Reinforcement Learning in FHIR Environments

On July 1, 2026, researchers Ananya Mantravadi, Harshit Rajgarhia, Prasanna Desikan, and Abhishek Mukherji published a paper titled World Feedback for Clinical Agents: Diagnosing RL in FHIR Environments. This study explores the application of reinforcement learning (RL) in clinical tasks within FHIR environments, highlighting the challenges and potential solutions.

Understanding Reinforcement Learning in Clinical Protocols

Reinforcement learning is increasingly recognized for its potential in clinical protocol-execution tasks such as checking lab values and placing structured FHIR orders. The authors argue that these tasks are natural candidates for RL due to their ability to leverage world feedback. The study emphasizes the need for a robust feedback channel and sufficient base capabilities for effective RL implementation.

The research audits MedAgentBench versions 1 and 2, revealing a concerning 41.7% silent-finish ceiling, indicating that inaction becomes the dominant strategy for RL. To address this, the authors developed MedAgentBench-v3 (MAB-v3), which features 508 tasks and improves the ceiling to 8.9%.

Identifying Barriers to Effective Learning

The study highlights two significant barriers encountered during the training of Qwen3-8B: the capability ceiling and the format-knowledge barrier. Specifically, 10 out of 20 task types demonstrated 0% base performance, resulting in zero gradient. Furthermore, 3 out of 20 types necessitated exact clinical codes that are not easily discoverable through exploration.

In terms of performance, pure RL achieved a 18.2% pass@1 rate, while rule-based supervised fine-tuning (SFT) reached 34.1%. The 15.9 percentage point gap in performance can be attributed entirely to the aforementioned barriers, underscoring the challenges faced by RL in clinical settings.

Strategies for Improvement in RL Applications

The authors propose a decision/format-knowledge/lookup taxonomy to predict RL learnability and suggest remedies for the identified barriers. They advocate for SFT to inject necessary codes and RL to effectively learn conditionals, which could facilitate better performance in clinical applications.

• 41.7% silent-finish ceiling in MedAgentBench v1/v2
• Development of MedAgentBench-v3 with 508 tasks
• Pure RL performance: 18.2% pass@1
• Rule-based SFT performance: 34.1% pass@1

This research sheds light on the intricate relationship between reinforcement learning and clinical processes, offering pathways to enhance learning capabilities in FHIR environments.