On June 29, 2026, researchers Yongbin Kim, Yashar Talebirad, and Osmar R. Zaiane introduced HASTE, a hierarchical multi-agent system designed to improve machine learning (ML) engineering. The system aims to prevent ML agents from redundantly rediscovering known techniques during competitions, thereby enhancing efficiency and skill transfer.
Understanding HASTE's Hierarchical Structure
The HASTE system organizes knowledge into three tiers: global, domain, and competition-specific. Each tier corresponds to a specific level of agents, allowing for a more structured approach to learning and skill application. An orchestrator coordinates the interactions between these domain specialists, leveraging LLM-driven abstraction to promote knowledge transfer across tiers.
By effectively categorizing skills, HASTE enables agents to start with a limited but strategic skill set, which expands as they progress through competitions. This design aims to minimize redundant learning and maximize the utilization of previously acquired knowledge.
Performance Metrics and Findings
A controlled ablation study demonstrated the effectiveness of tiered skill loading. While maintaining a constant inventory of 159 skills across eight competitions, HASTE achieved a remarkable 100% medal rate. In contrast, flat loading methods resulted in only a 62.5% medal rate, equivalent to that of using no skills at all, but with double the output token consumption.
- Medal Rate: 100% with tiered loading
- Medal Rate: 62.5% with flat loading
- Output Tokens: 2x more with flat loading
Cold Start vs. Warm Start in Competitions
HASTE's approach includes both cold-start and warm-start runs. In cold-start scenarios, the system begins with no accumulated skills, while warm-start runs leverage previously learned skills. This method of skill reloading not only reduces the number of required refinement iterations by 52% but also increases the retention of proposed changes from 42% to 85% as more skills become available.
The results suggest that optimizing knowledge organization can significantly enhance the performance of ML engineering agents, potentially compensating for limitations in model strength and compute resources.
🤖 This article was rewritten by Feed and Figures' editorial AI from a report originally published by arXiv AI. Facts and quotes are preserved from the original; the rewrite focuses on clarity and structure. For the unedited original, see the source link below.
