Index SLM Technical Report Unveils Advanced Language Models Developed by Bilibili

On July 10, 2026, researchers Lusheng Zhang and a team of eight others from Bilibili released the Index-1.9B Technical Report, detailing a series of innovative open small language models. This report outlines four distinct models: Index-1.9B-Base, Index-1.9B-Pure, Index-1.9B-Chat, and Index-1.9B-Character, each designed to enhance language processing capabilities.

Overview of Index-1.9B Models

The series includes:

• Index-1.9B-Base: A foundation model with 1.9 billion non-embedding parameters, pre-trained on 2.8 trillion tokens primarily in Chinese and English.
• Index-1.9B-Pure: A variant that filters out instruction-like data from the training corpus.
• Index-1.9B-Chat: A model aligned from the base with supervised fine-tuning and optimization for user preferences.
• Index-1.9B-Character: An augmented chat model that incorporates retrieval-augmented generation for enhanced customization.

These models are built using a Warmup-Stable-Decay learning-rate schedule, which significantly increases data quality during the decay phase, alongside a Norm-Head output layer that stabilizes training with large learning rates.

Performance Metrics and Benchmarking

The Index-1.9B-Base model has achieved an impressive average score of 64.92 across various benchmarks, making it competitive with larger open models. The report details controlled studies on several factors, including:

• Model depth
• Learning-rate magnitude and scheduling
• Data quality interactions
• Effects of instruction data during pre-training

Notably, the researchers documented an unexplained surge in benchmark performance during the constant-learning-rate phase, indicating potential areas for further investigation.

Accessing the Models and Future Research

The Index-1.9B series, along with evaluation code, is publicly available at the designated URL. This open access aims to foster further research and development in the field of language processing.