Behavior Leverage Imbalance in Multi-Teacher On-Policy Distillation Enhances Tool-Calling Accuracy

Behavior leverage imbalance in multi-teacher on-policy distillation is a pivotal strategy for training agentic language models. A recent study by Jiabin Shen, Guang Chen, and Chengjun Mao, published on July 8, 2026, explores how this method can refine the model's handling of tool calls and direct responses.

Understanding Multi-Teacher On-Policy Distillation

Multi-teacher on-policy distillation involves training a student model using several teacher models, each specializing in different aspects of task performance. One teacher might focus on making tool calls while another emphasizes providing direct answers.

This division of labor allows the student model to learn from a diverse set of behaviors, which is essential for improving its overall performance. However, the study reveals that aggregate loss metrics do not fully capture the effectiveness of this approach.

Behavior Leverage Imbalance Explained

The researchers identified a phenomenon termed behavior leverage imbalance, where local token-level signals at specific positions can disproportionately influence the model's generation mode. For instance, specific tokens like <tool_call> and function names can lead to over-calling, where the model excessively relies on tool calls instead of providing direct answers.

To tackle this issue, the authors propose a novel method called Soft Clamp, which dynamically calibrates token-level divergence. This method compresses extreme divergences while maintaining essential gradients, ultimately improving the model's decision-making process.

Results and Implications of Soft Clamp

In their experiments on the APIGen-MT dataset, the implementation of Soft Clamp successfully reduced the over-calling rate from 13.7% to 9.0% compared to vanilla generalized knowledge distillation (GKD), all while preserving decision accuracy. Additionally, Soft Clamp demonstrated effectiveness in mitigating tool-call loops and repeated calls among various GKD variants.

These findings suggest that monitoring the effectiveness of teacher signals should not only focus on their aggregate size but also on their specific impacts on the model's behavior. This nuanced understanding of multi-teacher OPD can lead to more effective training strategies for agentic language models.