On October 10, 2023, researchers presented a groundbreaking study on in-process retrieval as extended working memory for language agents. This innovative approach challenges traditional memory storage methods by integrating memory directly into the agent's operational loop, significantly enhancing performance.
Redefining Memory in Language Agents
Language agents typically operate within a cycle of observing, reasoning, and acting. Traditionally, memory retrieval occurs outside this loop, which can lead to delays. The new study proposes an in-process memory store that allows agents to read and write memory at each step. This method reduces latency drastically, with in-process retrieval being approximately 100 microseconds, compared to the 110 milliseconds typical of cloud-based solutions.
The researchers argue that memory latency is determined by the location of the memory store rather than the retrieval pattern itself. By ensuring that the memory is readily available, the concept aligns with the extended-mind thesis, suggesting that fast and constant access to memory transforms it into extended working memory.
Significant Performance Improvements
The research demonstrates notable improvements in recall rates across four different models of GPT-5. With in-loop memory, recall jumped from 0 out of 5 to between 3.6 and 4.8 out of 5. This enhancement is attributed to the reduction of redundant actions as latency decreases. The study recorded that at the in-process speed, the redundant actions were 0.0 of 12, compared to 7.2 of 12 at the longer cloud response times.
