Researchers from the field of computer science have made significant strides in scaling point-in-time language models, as detailed in a recent paper. On April 24, 2026, authors Bryan Kelly, Semyon Malamud, Johannes Schwab, and Teng Andrea Xu released their findings, which address the issue of lookahead bias in large language models trained on unrestricted internet data.
Understanding Lookahead Bias in Language Models
Large language models often incorporate future information, leading to lookahead bias that undermines the integrity of backtests and causal inference, particularly in finance and social sciences. This research introduces point-in-time language models that exclusively utilize text available up to specific calendar dates, thus eliminating this leakage by design.
Despite their advantages, traditional point-in-time models have struggled to match the performance of their unrestricted counterparts. The authors demonstrate that this performance gap can be significantly reduced through increased scale, utilizing up to 4 billion parameters and training on 1 trillion chronologically filtered tokens from FineWeb.
Advancements in Model Training and Performance
The study outlines a sequence of monthly model checkpoints spanning from 2013 to 2024. The results show that these models approach the performance levels of leading open-weight models, such as Gemma-3-4B and LLaMA-7B, which were trained on unrestricted data. However, a performance gap remains on several tasks, indicating ongoing challenges in achieving parity.
