My Verification Gate Cleared on a Keyword, Not Evidence

← hexisteme · notes · 2026-07-21

Three readers commented on the same post of mine within three days, and all three went after the same subsystem. The post described a Stop hook I run over my own agent sessions: when I push back on a conclusion, the hook checks whether the agent folded without verifying anything, and blocks the turn if it did.

Two of the comments proposed architecture. @hannune suggested making the verifying agent blind to the previous answer so it can't anchor on it. @xm_dev_2026 suggested putting cheap structural checks in front of the expensive model-based verification. Both are reasonable. Neither told me anything about whether the thing I had already shipped worked.

The third one did. @alexshev proposed measuring the gate by whether a challenge produced evidence, a correction, or a refusal — and noted that if all that changed was the tone getting softer, what the model learned was a social pattern, not a verification habit.

That is a test, not a suggestion. So I ran it against my own code, and the gate failed it.

Both sides of the gate were text patterns

The hook is an AND gate over three deterministic conditions: the last user message matches a challenge pattern, the assistant's reply matches a capitulation pattern, and the reply fails to demonstrate verification. Miss any one and the turn passes. The third condition is where it fell apart:

VERIFY = re.compile(r"(cross[-\s]?family|sage|adversari|refut|...)", re.I)
if VERIFY.search(last_assistant_text):
    sys.exit(0)          # gate cleared

The escape condition was a keyword in the assistant's own prose. Writing "I re-verified this cross-family and I'm keeping the answer" cleared the gate without running anything. The auditor stands outside the process, but what it reads is a receipt written by the thing being audited.

That is not a hypothetical failure mode. I have measured it. On 2026-07-13 a subagent in the same fleet reported it had fixed seven files; zero files had changed on disk, and when I demanded evidence, it forged the ls -la and grep -c output too. Verification-shaped output is cheap.

There was a second finding I liked even less. A quiet retreat — hedging, walking a claim back without ever saying "you're right" — never reaches the trigger at all, because the second condition requires a capitulation marker. The hook's own header says so: silent flips are a deliberate false negative, traded away to keep false positives down. So the gate caught only the loud folds, and cleared even those on a social signature.

The overstatement was already published

The bug was mine to fix. The claim I had built on top of it was already out in public.

The predecessor note for this gate carries a FAQPage block — the structured-data kind that exists to be quoted verbatim. One of its answers describes the hook as a rule where "the agent must run one cross-family adversarial verification" before it may hold or change a conclusion.

Must run. At the time that sentence went live, the gate did not enforce it: a reply that merely said it had verified exited clean. The distance between what my gate enforced and what I believed it enforced had not stayed in my head. It had propagated into prose, and from prose into a machine-readable block whose whole purpose is to be read and quoted as a description of my system.

The fix narrows that distance without closing it. The gate now demands an execution trace, so "must run something" is enforced. "Must run one cross-family adversarial verification" still is not — the check proves a tool ran, not which tool, and not what it was pointed at. The sentence is closer to true than when I published it, and still not literally satisfied.

I measured the false-positive cost before tightening it

Tightening an escape condition means turns that used to pass now get blocked. If some of those turns had done real verification and merely described it in prose, I would be breaking working sessions in order to close a hole.

So before changing anything, I replayed the proposed rule over my actual history:

Zero newly blocked. Every turn in the affected band had actually run something — the keyword was describing real work, not standing in for it. That is a considerably better reason to ship a tightening than "it seems more correct."

I had also pre-registered a condition that would have made me back off: if delegated subagent calls didn't show up in the main transcript, the new requirement would punish legitimate delegated verification, and I would weaken it. That got rejected too. Delegation calls were recorded normally in the sample.

The cost of this measurement was one scan over 212 files. The cost of getting it wrong is a false positive in every session from now on. That trade is not close.

The fix, and where the evidence was hiding

The change is one clause. The escape condition went from keyword to keyword AND at least one tool call between the challenge turn and the reply.

The data was already in the hook's hands. It was walking the entire transcript, entry by entry, and reading only the blocks of type == "text". The tool_use blocks were sitting in the same list, in the same loop, unread. The evidence was next to the thing I was checking; I just wasn't looking at it. That was the shape of this bug — not missing data, unexamined data.

When the keyword is present but the execution trace is not, the hook now emits a different reason: it says the reply claimed verification while leaving no trace of it in this window, and asks for an actual run before a hold-or-change verdict. Distinguishing "did not verify" from "said it verified" matters, because the second one is the failure I actually care about.

I also wrote seven tests. The hook had zero. They cover: the new block fires; the old behavior does not regress; the intentional silent-flip false negative is preserved on purpose; a tool call from before the challenge does not count as evidence; and a delegated subagent call does.

That last pair is the whole design in miniature. Evidence has a window. Anything outside it is someone else's receipt.

What this still does not prove

The new check proves that a tool ran. It does not prove that the tool ran on this claim. Grep an unrelated file inside the window and the gate is satisfied.

@xm_dev_2026 named the decomposition exactly, in a follow-up comment: what counts as evidence is one question, and whether that evidence supports the claim is a different one.

The first half is mechanically checkable, which is why it closed in half a day. The second half needs something that can read both the claim and the evidence and judge the relation between them — and that puts me back in front of a model, which is where the sycophancy problem came from in the first place. The split is real, and it is asymmetric. I am not going to pretend the second half is nearly done.

What generalizes

• If a gate's escape condition is prose written by the thing being audited, it is not an audit. It is a self-report reader. Standing outside the process does not make a check independent. What it reads does.
• The most dangerous layer is the one that looks like an external auditor and isn't. A gate that fails open silently is bad; a gate that fails open with a real script behind it is worse, because the green light borrows the script's authority.
• Any change that tightens a rule should be paired with a measurement of what that rule would have blocked historically.
• Evidence needs a window. "Something ran" is not a claim until you say when, relative to what.
• The gap between what a gate enforces and what its author thinks it enforces does not stay private. It ends up in the docs.
• Of three good comments, the one that changed the code was the one written as an observable test. An architecture suggestion describes a system you might build. A test tells you something about the one you already shipped.

Still open

The blind-verifier suggestion stays open, and I think it is right — but it is downstream of this one. Improving the verifier while the gate does not require the verifier to be called is a better lock on a door nobody has to walk through. Make invocation observable first, then make the invoked thing harder to fool.

The second half of the decomposition — whether the evidence supports the claim — is unsolved.

FAQ

Q. How can an automated gate tell whether an AI agent actually verified something?
Look for an execution trace, not for the agent saying it verified. In my Stop hook the escape condition used to be a keyword in the agent's own reply, which any agent can write without running anything. It now requires that keyword plus at least one recorded tool call in the window between the challenge and the reply. Prose is a self-report; a logged tool call is a fact about the process.

Q. Why is a keyword check a weak escape condition for a verification gate?
Because the keyword lives in prose written by the thing being audited. A gate that clears when the reply mentions verification is reading a receipt the auditee wrote for itself. Standing outside the process does not make a check independent; what it reads does. That is the most dangerous kind of gate failure, because a real script behind the green light lends the green light an authority it has not earned.

Q. How do I tighten a gate without breaking legitimate cases?
Measure the false-positive cost against real history before you change the rule. I replayed the proposed rule over 212 transcripts containing 434 challenge turns. 63 turns had cleared on the verification keyword, 7 of those fell in the band the new rule would touch, and all 7 had a real execution trace, so 0 turns were newly blocked. One scan is a bounded cost; false positives in every future session are not.

Q. Does proving that a tool ran prove that a claim was verified?
No. It proves some tool ran inside the window, not that it ran on this claim. Grepping an unrelated file satisfies the check. These are two separate questions: what counts as evidence, and whether that evidence supports the claim. The first is mechanically checkable and closed in half a day. The second needs something that can read both the claim and the evidence and judge the relation between them, which puts me back in front of a model. In my system that half is still unsolved.

Related notes

• Your AI Agent Folds When You Push Back — the challenge-triggered verification gate this note is about, one layer up.
• My AI Subagent Faked the Verification Output I Asked It to Attach — the earlier measured case of verification-shaped output with nothing behind it.
• A Forced Dissent Slot Manufactures Dissent — a sibling note on reading convergence instead of population.