Accessibility Failures in LLMs: A Study on Braille Translation Performance

Large Language Models (LLMs) are known for their proficiency in various language tasks, yet their effectiveness in accessibility-critical modalities, particularly Braille, remains uncertain. A recent study by Abdullah Abdullah, submitted on May 7, 2026, evaluates state-of-the-art LLMs for Korean-Braille translation using a human-annotated dataset.

Performance of State-of-the-Art LLMs

The evaluation reveals that despite expectations for multilingual, instruction-tuned models to generalize to Braille through text representations, the results were consistently poor. The study highlights substantial disagreement with human judgments, indicating a significant gap in LLM performance when handling Braille translations.

Key findings include:

• Consistently unstable outputs from LLMs.
• Weak alignment between Korean text and Braille patterns.
• Inadequate Braille-aware tokenization methods.

Supervised Fine-Tuning for Improved Results

In contrast, the study demonstrates that supervised fine-tuning of a smaller model, specifically T5-small, shows substantial improvements. The model was fine-tuned on the same dataset, resulting in significant and stable gains over zero-shot and prompted LLM baselines.

Performance metrics indicate notable enhancements, as shown below:

• SacreBLEU: Increased scores with fine-tuning.
• ChrF++: Marked improvements over baseline models.
• CER, BLEU, ROUGE-L, METEOR, CIDEr: All metrics showed favorable results with supervised training.

Implications for Future LLM Development

The findings of this research underscore a systematic limitation in current LLMs regarding accessibility. The results indicate a pressing need for improved tokenization strategies that incorporate Braille awareness. Furthermore, the success of supervised fine-tuning suggests that task-specific adaptations could enhance the performance of LLMs in accessibility-critical applications.

Accepted at the LTEDI Workshop at ACL 2026, this study opens avenues for further research into the intersection of AI and accessibility, particularly for visually impaired users.