On July 2, 2026, researchers Juarez Monteiro and Nathan Gavenski, along with their colleagues, published a paper titled ASK in the Dark: Uncertainty-Gated LLM Assistance under Partial Observability. This study addresses the challenges faced by reinforcement learning agents when operating under conditions of incomplete information and proposes a novel framework for integrating small language models (SLMs) to enhance decision-making.
Understanding Uncertainty-Gated LLMs
Reinforcement learning agents often struggle with partial observability, as they must make decisions based on limited data. The authors highlight that traditional uncertainty-gated approaches have not effectively utilized SLMs, achieving an overwrite rate near zero, meaning the SLM rarely contributes independently. This issue arises from the use of an egocentric prompt, which fails to provide adequate context for meaningful reasoning.
Monteiro and his team argue that the problem lies in the context provided to the SLM rather than its capacity. They emphasize that enhancing the context can significantly improve the performance of these agents in real-world applications.
Introducing the ASK+ Framework
The researchers propose the ASK+ framework, which incorporates trajectory-aware context by supplying the SLM with a partially revealed map, visited positions, and action history. This structured approach transforms the SLM from a passive redundancy check into an active consultant that can correct the agent's policy when necessary.
The use of predictive entropy signals for selective querying is also a key feature of ASK+. This method measures action uncertainty rather than state uncertainty, thus maintaining its effectiveness in partially observable Markov decision processes (POMDPs). The results indicate that this approach is viable even in environments that are not fully observable.
