Black-Box Inference Technique Reveals LLM Architectural Details Despite API Restrictions

Researchers Christopher Ellis, Shreyas Chaudhari, Mei-Yu Wang, Leighton Barnes, Giulia Fanti, and José M. F. Moura have developed a new method called NightVision to infer the architectural properties of large language models (LLMs) despite restrictive API access. The study, titled Black-Box Inference of LLM Architectural Properties with Restrictive API Access, was submitted on July 1, 2026, and highlights the challenges faced by developers when limited API information is provided.

NightVision: A New Approach to LLM Analysis

NightVision employs a novel prompting technique to extract log probabilities for specific output tokens, allowing researchers to estimate crucial architectural parameters. Despite the limitations imposed by LLM providers—who often restrict API access to a single logit for each token—NightVision successfully estimates the hidden dimension, depth, and parameter count of various LLMs. This method underscores the inadequacy of current restrictions in fully concealing model architectures.

Through empirical evaluations of 32 open-source LLMs, the researchers recovered the hidden dimension with an average relative error of 23%. For models exceeding three billion parameters, estimates of depth and parameter count were within 53%. These findings raise questions about the effectiveness of current API restrictions.

Implications of Limited API Access

The research indicates that as LLM providers tighten API access, significant architectural details remain retrievable through innovative techniques like NightVision. This revelation poses challenges for both developers and users, as the ability to analyze LLM architectures directly impacts the understanding of their capabilities and limitations.

NightVision's reliance on end-to-end time to first token (TTFT) measurements combined with spectral analysis enables a comprehensive evaluation of LLMs, revealing that even with restricted access, essential model characteristics can still be inferred. This could lead to a reevaluation of how LLM providers manage API access and data sharing.

Future Directions for LLM Research

The findings suggest that the ongoing evolution of LLM architectures necessitates further investigation into the implications of restrictive API access. As LLMs continue to grow in complexity and application, understanding their architectural properties becomes increasingly critical.

Future research could focus on enhancing the NightVision technique, improving the accuracy of architectural inference, and exploring how these insights can inform model development and deployment strategies across various applications.

• NightVision estimates hidden dimensions with 23% average error.
• Depth and parameter count estimates within 53% for large models.
• Empirical evaluation conducted on 32 open-source LLMs.