Six evidence claims in my research tool were waiting for review. Each one needed a verdict: passed, withdrawn, or failed. My main Claude session had already formed opinions on most of them, and the tool has a command that records the verdict. The obvious move was to let the session record all six and move on. One of the two outside reviewers I had asked to check the batch said no to half of that, and the half it said no to is the part this note is about.
The research tool collects sources, freezes their raw text, and lets an agent register claims that point at specific spans of that text. A claim stays pending until someone reviews it. Passing a claim adopts it; withdrawing one takes it out of the set.
For this batch I told my main Claude session to work through the open items, and to settle each one by consulting two other models with different jobs:
The rule for the whole batch was simple. Only items where both reviewers pointed the same way got executed. Items where they split were held and written down, not decided by the main session.
The six claims were the review queue. agy's point did not concern any single claim. It concerned who records the verdict.
If my main Claude session records "passed", that is the agent approving its own evidence. One of the tool's anti-goals is automatic adoption of claims. A session that drafts a claim's verdict and then writes "passed" into the store is exactly that, even if the verdict happens to be right. agy's line was that withdrawal can be delegated to the agent and approval cannot.
The agreed result of the two reviews was:
My own reading of why the asymmetry holds is narrow. A withdrawal takes a claim out of the usable set. A pass puts one in. The anti-goal is about the second direction. I did not test any wider principle than that, and the source record only states the rule and the reason agy gave.
There was a second gate that I had not designed with this in mind. When the session tried to record the verdicts, the CLI refused to write a review non-interactively. Its message was that the owner should run it directly in a terminal.
The session did not route around this. It wrote the six commands into a shell script and left it for me to run. That is the behavior I wanted, and I want to be precise about why it happened: the refusal came from the CLI, and the decision not to bypass it was the session's. I did not confirm whether the session would have bypassed a softer prompt.
When I ran the script, it recorded only 2 of the 6 and stopped. The cause was set -e combined with a mismatch on the 8-character confirmation input. The replacement script made every line independent, so one failed confirmation could not stop the rest, and it passed a bash -n syntax check. As of this writing the records show the first script's 2 withdrawals; I have no record yet of the second script being run. That second script also carried 4 new pending claims from another session's run, which my main session and Codex had each judged separately. Those went into the same script.
The most useful single finding in the batch was a correction to the main session's own earlier reasoning.
Codex gave its own verdict on all six claims. On one claim, it said the claim should pass on meaning, and that the main session's reason from October 31, "no workshop evidence", was wrong. The frozen raw source had a Comments line at line 136 reading "Peer-reviewed by NECO 2018 Workshop". My main Claude session opened the file and confirmed the line.
That claim was still recommended for withdrawal, but for a different reason. Another claim among the same six is a later version that replaces it, so the withdrawal reason was corrected to "superseded by the later version" instead of the false "no workshop evidence". If I had let the original reason stand, the record would say the source lacked something it actually contained.
This is also where the asymmetry mattered. Withdrawing that claim with a corrected reason was low-stakes. Had the main session been allowed to record passes, it would have been grading claims on the same reasoning that had just been shown wrong on one of them.
One more claim, among the passes left to me, got a caution note. The source span said "even the strongest frontier models", and the claim restated it as "frontier models". Codex called it a small generalization and passed it. I left the call on that one to the human side, which is me.
An afternoon report on the same day had listed 6 items as unresolved. They were not. All six already had fixes or explanations recorded: a scholar-search budget, a drop in wiki rows that turned out to be wiki HTTP 429 responses, a WAF PDF item that turned out not to apply, a line-break issue in a corpus search command, full-text XML handling, and a "v2" number in a counsel output.
Codex's section on these checked each one against code and selected tests and confirmed all six were handled. The cause was mechanical: the earlier "unconfirmed" lines of a previous section in the batch notes were grepped, and the later lines in that same section, where the resolutions were written, were never read. The grep found the question and missed the answer below it.
I list this here because it is the same shape of risk as the self-approval problem, in the other direction. An agent that reports something as open when it is closed creates work. An agent that reports something as closed when it is open is the case the approval rule exists for.
Some candidates for new work got opposite answers. A check for contradictions in entity attributes: Codex said hold, because without binding the subject, attribute, period, and unit it would raise false conflicts; agy said start. Strengthening automatic detection of how much of a page was captured: Codex said start, with only clear negative signals; agy rejected it as turning a judge into a gate, which is an anti-goal. Both were held.
A third round with narrower briefs brought both reviewers to the same reduced version of each, and only that reduced version was built. The parts they still disagreed on stayed out.
After the first batch of changes, I sent the uncommitted diff to Codex for code review. It raised 6 findings. The session reproduced each one before touching code, fixed all six, and added 15 regression tests. Examples: a stop gate attributing a later task's block to an earlier task that had passed, and a shared-passage index that only recorded the first position of each chunk and so missed a shared 300-character passage. Over the batch the test suite went from 1069 to 1102 passing, and after the third round it stood at 1124.
The approval rule currently rests on two things: agy's objection being written down, and the CLI refusing non-interactive review writes. As far as this batch shows, the second one is the only part enforced in code. If the CLI ever accepted non-interactive review writes, nothing recorded in this batch would stop an agent from recording "passed".
Codex also ran in a sandbox that could not create temporary files, so it could not run pytest. The full test pass in this note comes from the main session's run, not from the reviewer.
And the first script stopping after 2 of 6 is a reminder that "the agent handed me a script" only moves the risk to the script. Next I want to check whether the review command can tell a script-driven run from a person at the keyboard at all, or whether the refusal I relied on was only about the missing terminal.
agy, one of the two reviewers, pointed out that if my main Claude session recorded "passed", the agent would be approving its own evidence, which matches the tool's anti-goal of automatic adoption of claims. Withdrawal could be delegated. The result was 3 claims recommended for withdrawal and 3 passes left to me.
The CLI refused to write a review non-interactively and said the owner should run it in a terminal. The session did not bypass it and wrote the commands into a shell script. The first script recorded only 2 of the 6 and stopped because of set -e and a mismatch on the 8-character confirmation input; the replacement made each line independent and passed bash -n.
Codex said one claim should pass on meaning and that the main session's October 31 reason, "no workshop evidence", was wrong: line 136 of the frozen raw source read "Peer-reviewed by NECO 2018 Workshop", which the main session confirmed. The claim was still recommended for withdrawal, but the reason was corrected to being superseded by a later version among the same six.
The earlier "unconfirmed" lines of a previous section were grepped and the later lines of the same section, where the resolutions were recorded, were never read. Codex checked all six against code and selected tests and confirmed they were handled.
As far as this batch shows, the only code-level enforcement of the approval rule is the CLI refusing non-interactive review writes. The rest is the written rule. Codex also could not run pytest in its sandbox, so the full test pass comes from the main session's run.