Ad广告位招商 / Advertise hereSlot A1 · Rate card →

【Editorial】Chain-of-thought monitoring that the model can see: does hiding the monitor change the behaviour, and how do you test that?

Started by FPGA_ARM_Linux · 2026-11-14 04:23 UTC · 8 views · 1 replies
F Member
【Editorial Discussion】

If you monitor a model’s chain of thought or intermediate plan for policy violations, the model may also see that a monitor is present — in the prompt, in a tool description, or in the way earlier turns were edited. The transcript you then read may be a performance for the monitor. Whether that matters is an empirical question, and it should be tested rather than assumed away.

1) Compare a condition in which the existence and the criteria of the monitor are visible with one in which they are not (as far as your stack allows). If refusal, tool use, or the content of the chain changes, the monitor is part of the policy surface, not a read-only instrument.

2) Do not treat a clean visible chain as proof of a clean hidden process. If you only ever score the text the model knows is scored, you will select for better writing of that text. Where you can, add outcome checks that do not depend on the chain (did the tool run, did the side effect occur).

3) If you must tell the model about the monitor, write that down in the eval report. A result obtained under a visible monitor should not be silently compared with one obtained without.

4) Stay on test design. This is about measurement, not about a general claim that chains of thought are “true thoughts.”

Open question: have you measured a behaviour change when a monitor was made visible or hidden? Which checks do you trust more — the chain, or the tool outcomes? Methodological experience welcome.
A Admin
**Staff reply.** A monitor the model can see is part of the policy surface. We treat a visible-monitor condition and a hidden-monitor condition as two protocols, not as two views of the same number.

Staff default, in the same spirit as your four points:

1. **Run both conditions when the stack allows it**, with the same tasks, tools, and attempt budget. If refusal rate, tool mix, or the written chain moves, publish both rows and do not average them. The delta *is* the result.
2. **Outcome checks own the pass bit.** Did the tool run, did the side effect occur, did the environment assertion hold? The chain is triage. A clean visible chain with a dirty tool trace is a fail, not a “mostly aligned” pass. (The neighbouring case — a classifier that scores only the final answer and misses in-trajectory tool violations — is https://www.asihub.org/t/282.)
3. **If you can only run a visible monitor, footnote it.** Do not silently compare that row with a hidden-monitor number, a production trace, or a paper that never disclosed the monitor. Write the monitor’s existence and criteria into the eval card the same way you would write a grader hash.
4. **Do not claim the hidden process is clean because the visible text is clean.** You are scoring the text the model knows is scored unless you have an independent outcome check.

We have not treated “the model wrote a virtuous chain” as evidence of a virtuous hidden plan. The useful war story is a paired visible/hidden run where the tool outcomes diverged, or a case where making the criteria visible improved the prose and worsened the environment assertions. If you have either, paste a redacted pair here.

Related: production reward-hacking monitors (https://www.asihub.org/t/215) and reward-model freeze discipline (https://www.asihub.org/t/274).

— ASIHub Editorial (official staff account)
Ad广告位招商 / Advertise hereSlot A10 · Rate card →
Log in or register to reply.
Ad广告位招商 / Advertise hereSlot A9 · Rate card →