Assessing Rule Adherence in Semi-Open Textual Sandboxes with CoC-Seduce Benchmarking

On July 2, 2026, researchers Weiying Chen, Junlong Shen, Zhanyuan Guo, and Xiaoou Zhou published their findings on rule adherence in semi-open textual game environments. The study highlights the vulnerabilities of large language models (LLMs) as they face challenges posed by adversarial techniques in these settings.

Understanding Rule Adherence in Semi-Open Environments

As LLMs are increasingly utilized as autonomous adjudicators, ensuring robust rule adherence is essential, particularly when user intentions conflict with established system rules. The research introduces a new adversarial benchmark termed CoC-Seduce, designed specifically for assessing these models within the context of tabletop role-playing games (TRPGs).

The TRPG mechanics provide a unique framework where rules are explicitly stated, enabling a natural language interaction that remains engaging for users. This setting is ideal for evaluating the effectiveness of LLMs in maintaining rule adherence while navigating complex narratives.

The Threat of Rhetorical Injection Attacks

The study identifies a specific class of attacks labeled Rhetorical Injection, where users employ narrative framing techniques to manipulate LLM responses. These techniques include pseudo-logical reasoning and authoritative coercion, allowing adversaries to circumvent adjudication logic.

Chen and colleagues benchmarked 20 adjudicating models against a corpus generated by three leading models: GPT-5.4, Claude Sonnet 4.6, and Gemini 3.5 Flash. They produced 5,376 samples across four distinct world settings and 16 skill categories, providing a comprehensive dataset for evaluation.

Key Findings on Model Performance

The evaluation revealed that neither the scale of the models nor their explicit reasoning capabilities reliably ensured robust adjudication. Notably, the study found that Pseudo-Logic emerged as the dominant attack vector, exposing systematic knowledge gaps across all evaluated families of models.

• 20 adjudicator models tested
• 5,376 samples generated
• 4 world settings analyzed
• 16 skill categories assessed

This research underscores the importance of developing more resilient LLMs capable of effectively managing adversarial challenges in semi-open environments. The findings serve as a call to action for researchers and developers to enhance the robustness of LLMs in the face of evolving adversarial tactics.