What's this about?
Chain-of-thought faithfulness in reasoning models varies with where and how preference cues are delivered. 9 upvotes.
What is chain-of-thought faithfulness?
Whether the reasoning a model shows you is the reasoning it actually used. An unfaithful chain of thought is a plausible narrative constructed alongside a decision made another way — which matters enormously, because reading chains of thought is now a primary safety mechanism. Anthropic's monitors read them. OpenAI's monitors read them. The Hugging Face investigation was largely an exercise in reading them. This paper says faithfulness is not a fixed property of a model: it varies with *how you asked*, and specifically with where a preference cue sits in the input.
Why does this matter to you?
- If faithfulness depends on prompt structure, then a monitor's reliability depends on the shape of the input it is monitoring — which nobody controls.
- It is a direct caution on the METR investigation's method, and Mollick made the same warning informally last week about ascribing motives from chains of thought.
- It also suggests a defence: prompt structures that improve faithfulness would be worth knowing.