Healthier LLMs: Enhancing Public Health Question Answering with RAG Techniques

On July 7, 2026, researchers including Felix Feldman and Joshua Harris published a paper titled Healthier LLMs: Retrieval-Augmented Generation for Public Health Question Answering. The study outlines how Retrieval-Augmented Generation (RAG) can significantly improve the reliability of large language models (LLMs) in answering public health queries.

Understanding Retrieval-Augmented Generation in Public Health

LLMs have shown potential in various medical question-answering benchmarks, but their application in public health has been limited due to issues like hallucinations and the fast-paced changes in official health guidance. The implementation of RAG addresses these challenges by grounding responses in a well-maintained corpus of data.

The research extends the PubHealthBench, a benchmark consisting of 7,929 questions drawn from UK Government public health directives, into a retrieval-augmented framework. The study critically evaluates various retrieval configurations, demonstrating that hybrid retrieval methods consistently enhance recall and ranking quality.

Comparing Retrieval Methods for Optimal Performance

In the study, dense, sparse, and hybrid retrieval methods were compared across different embedding models and corpus variations. The findings indicate that hybrid retrieval not only improves recall but also optimizes ranking performance, especially when considering factors like chunk length and topic relevance.

By providing context through retrieval, the accuracy of multiple-choice answers across various LLMs was significantly enhanced. Notably, smaller open-weight models were able to match or surpass the performance of larger models that did not utilize retrieval, with improvements largely attributed to the quality of retrieval and the selection of context.

Evaluating LLMs with a New Rubric

To assess the effectiveness of free-form answering by LLMs, the researchers introduced a rubric-based evaluation method, termed LLM-as-a-judge. This rubric focuses on four key dimensions: faithfulness, completeness, clarity, and factual consistency. Validation against dual human annotations revealed that agreement was strongest for faithfulness and completeness.

However, the evaluation highlighted challenges in reliably reproducing clarity and factual consistency at scale, suggesting a cautious approach when interpreting these dimensions. The overall results underscore retrieval as a pivotal factor for reliable public health question answering and offer actionable insights for developing and assessing RAG systems.