Skip to content

For Your Eyes Only: Evaluating Coordination Between Isolated Language Model Instances

Sep 2026 · 0 citations · 36 references
Computer Science

TL;DR

For Your Eyes Only, a cooperative signalling game designed to evaluate can a model embed a signal in natural language that an independent instance of the same model can detect, without any shared memory or coordination-specific training is introduced.

Abstract

As model-generated content is increasingly consumed by other model instances in automated workflows, a practically important question arises: can a model embed a signal in natural language that an independent instance of the same model can detect, relying only on shared pre-training and task instructions, without any shared memory or coordination-specific training? We introduce For Your Eyes Only, a cooperative signalling game designed to evaluate this directly. A Sender produces free-form descriptions for two words, one of which is a hidden target; an isolated Receiver must identify it. We evaluate seven contemporary models from four architectural families on 300 word pairs from established psycholinguistic corpora, using the Double-Pass Success Rate to control for output biases. We find that most models struggle to maintain coordination once they are required to avoid detectable signals, while one frontier model retains near-perfect performance even after such filtering. We further show that models can direct this capability toward deliberate misdirection, and that coordination is consistently weaker across architectures than within them.

View source

Similar papers

Language Models Act on Hidden Valence

It is demonstrated that valence-related activation patterns leave hidden traces that predictably govern later choices, even when all visible tokens are identical across conditions, and whether these traces are accompanied by any subjective experience relevant to model welfare remains unclear.

Cameron Berg, Caspar Kaiser · 2 citations · ⚡1

A mechanistic study of language model introspection

Across three model families, two small groups of attention heads are identified with distinct roles in introspective reporting supporting introspective detection and localization of large language models.

Jia-Hong Zou, Xiang-Kun Sun, Ling-Kai Kong et al. · 1 citation
Preprint Aug 2026

Skill Issue: Are Skills Language-Invariant in LLMs?

This work quantifies cross-lingual skill inconsistency orthogonally from knowledge and general benchmark performance via multilingual self-play, and shows that skill discrepancies are a measurable major roadblock in the development of truly multilingual models.

Bobby Cheng, Adam Gaber, Zhengzhe Liu et al. · 1 citation

Chronologic: Measuring Language Models' Ability to Represent the Past

Language models are appealing tools for research on the past. But to trust the evidence a model provides, researchers need to know whether its responses fit the period represented. Validation is challenging, because this is not a task living people ordinarily perform, and because many questions have multiple correct an...

T. Underwood, Zi-Liang Qiu, Sarah Griebel et al. · 1 citation
#artificial intelligence Preprint Sep 2026

Communication between Frozen Large Language Models via Prompt Optimization in a Referential Game

Communication between two frozen large language models from different providers, with different tokenizers, accessed through their API endpoints is studied, finding that successful place value communication in some runs is rare.

V. Anand, Muthu Kumar Chandrasekaran, Shiva Chaitanya · 0 citations

Related blog posts

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.