Language model perplexity (PPL) has traditionally served as a proxy for automatic speech recognition (ASR) word error rate (WER). However, a recent paper by Mohammad Zeineldeen and colleagues, submitted on July 6, 2026, revisits this assumption in the context of modern end-to-end ASR systems that incorporate significant internal language modeling capabilities.
Understanding the PPL-WER Relationship
The paper explores whether external language models still enhance the performance of current end-to-end ASR systems. It questions the linearity of the PPL-WER relationship in log-log space, which previous studies had established. The authors present findings that indicate a complex interplay between the internal language models of ASR systems and external models.
Through their analysis, they reveal that the internal language model (ILM) subtraction alters the PPL-WER relationship. This suggests that the decoder's internal language model must be considered when interpreting the quality impact of external language models.
Key Findings on Encoder Context Length
The research delves into how encoder context length affects the PPL-WER relationship. By varying the context length, they were able to observe different outcomes regarding ASR performance. This aspect is particularly crucial as it highlights the adaptability of modern ASR systems and their reliance on neural and large language models (LLMs).
