I run a Stop hook called stop_decision_ownership_check.sh — a shell script that runs at the end of every agent turn in my coding sessions — whose job is to catch a specific failure: the agent doing analysis it is fully capable of doing itself, then handing the conclusion back to me as a question instead of just answering it. It fires on an AND of two conditions: the agent's last message matches a pattern for that kind of hand-back, and the ten transcript entries before it show the agent actually pulling data first — a file read, a grep, a shell command. The second condition exists because the pattern match alone can't tell a real question from a punt; the same sentence is a punt if it followed genuine digging, and legitimate if it didn't. It's one of seven hooks built the same way over my sessions, and until recently I had no idea how often any of them actually fired relative to how often they ran, because the code writes nothing down except on the one outcome that blocks the turn.
A reader, @xm_dev_2026, left a comment that put the situation in sharp terms: the cheapest next step is to add a timestamp to the fingerprint file the hook already writes when it blocks. A week of that would date the fires; it would still not provide a trigger rate without the missing denominator. The real question was whether I was willing to act on a week of data from a gate that had already been running uninstrumented for months. As it turned out, the retained transcripts could answer the denominator question without that wait.
The decision-ownership hook sits on every Stop event and has five possible outcomes, and exactly one of them touches disk. If the agent's last message has no text, it exits quietly. If the ten prior entries show no data-gathering tool call, it exits quietly. If the hand-back pattern doesn't match, it exits quietly. If it matches but an identical response was already recorded, a deduplication check catches it and it exits quietly too. Only if none of that stops it does the hook block the turn — and only then does it append one line, a hash of the response text, to ~/.claude/decision-ownership.warned.
The dedup check runs before that append, which means a second or third instance of the exact same blocked text produces no new line. So the warned log's row count was never a count of how many times the hook fired. It was a count of how many distinct pieces of text it had ever fired on. And the one number that would make either count mean something — how many times the hook ran at all — appeared nowhere, in this hook or the six others built the same way: zero counters across all seven, five of them keeping nothing but a fingerprint set like this one.
The timestamp suggestion deserves to be taken seriously on its own terms, because it isn't wrong. It's aimed at the side of the hook that already had a signal. A timestamp on the warned log tells you when a block happened. It says nothing about the four quiet exits standing in front of it, because the warned log only exists on the path that fires. The denominator, the pattern-match rate, how much filtering the tool-evidence condition is actually doing — every number that would answer "how often does this hook do something" lives on the entry path, and the entry path writes nothing, timestamped or otherwise. The comment carried its own diagnosis of why: the exit that fires gets instrumented, the four that don't, don't. That line predicts the limit of the fix it's attached to.
Instead of changing the hook or waiting a week, I reproduced its exact exit logic — same AND-gate, same pattern, same ten-entry window, same dedup check — in a standalone, re-runnable script (hook_denominator_audit.py), and replayed it over transcripts that already existed: 5,537 of them, archived from past sessions across projects. Nothing about this needed the hook to have been instrumented differently while those sessions were running, because the hook's decision at any point is a pure function of the transcript around it. If you can recompute the function, you can recompute the decision after the fact, over however much history you kept.
Replayed against that corpus, the script found 116,022 points where the hook's Stop logic would have evaluated — a denominator that had existed nowhere. Every one of the 116,022 sorted into exactly one of the hook's five outcomes:
| Outcome | Count | Share |
|---|---|---|
| No last-assistant text | 15,871 | 13.68% |
| No data tool in prior 10 entries | 23,040 | 19.86% |
| Pattern didn't match | 76,999 | 66.37% |
| Matched, but deduped | 17 | 0.015% |
| Fired | 95 | 0.082% |
Ninety-five fires out of 116,022 points would be a rate of 0.082%, about one candidate Stop point in every 1,221 — a replayed figure, not an observed log, for reasons I come back to below. That number alone was what the timestamp suggestion was reaching for, and the replay reached it without a single new session having to run first.
The corpus gave more than the top-line rate, though, because it let me pull the AND-gate's two conditions apart instead of only seeing their combined output. Across the same 116,022 points, the hand-back pattern matched on its own 316 times — 0.272% of all Stop events, before the tool-evidence condition gets a vote. Of those 316, the tool-evidence condition — the one I'd always treated as a plausibility check riding along on top of the pattern match — suppressed 204 of them, 64.6%. Most of what decides whether this hook fires turned out not to be the pattern at all. It's the second condition, the one I added specifically to compensate for the regex being context-blind. Of the 112 that survived both conditions, the dedup check silently passed 17, 15.2%, before a single new line ever reached the log. That 15.2% used to be a sentence I could only phrase as "the warned log undercounts, by some amount I don't know." It's a number now.
None of this needed a new write anywhere. The transcripts were already an append-only record of exactly the material the hook's logic runs over: the last message, the ten entries before it, the response text a dedup check would hash. The hook's rules are deterministic — given the same transcript, they sort a Stop point into the same one of five outcomes every time. A deterministic function over retained input doesn't have to be watched running forward. It can produce a counterfactual replay across retained history, provided that the retained record reproduces the gate's inputs and evaluation points closely enough.
That's the part of this that isn't specific to Stop hooks or coding agents. Any gate that reaches its decision through a fixed, reproducible procedure, and reads from inputs retained somewhere — logs, transcripts, request bodies, event streams — can be replayed instead of re-instrumented. A rate limiter's rule can be replayed over old request logs to see how often it would have tripped before anyone thought to count it. A fraud rule can be replayed over historical transactions. A CI gate's pass condition can be replayed over old build logs. The instinct in all of these cases is to add a counter and wait, because that's the right move when the logic isn't fully known yet, or the past wasn't fully kept. When both of those are actually true, waiting is the expensive option, not the safe one.
A recovered denominator is an easy thing to overtrust, so I want to be specific about where this one stops being solid.
The replay approximates a Stop point rather than reproducing it exactly — it treats any assistant entry with no tool call after it as a turn boundary, which is the right rule most of the time and not guaranteed to match what the live hook does under every interrupt or edge case. And a substantial share of the 5,537 transcripts in the corpus predate the hook's existence and span projects it was never watching in the first place. Both facts point the same way: what I have is not a log of 95 real blocks. It's a counterfactual — what the current logic would have decided, replayed over sessions it never actually ran against. The hook's real, live warned log held 22 lines at the time of this audit. Ninety-five and 22 aren't in tension. They answer different questions over different populations, and treating them as the same number would be the actual mistake here.
There's a second limit, and it has nothing to do with corpus scope — it would hold even against a perfect live log going back to day one. Whether 316 matches out of 116,022 points means "this behavior is genuinely rare" or "the pattern is too narrow to see most of it" isn't something any counter sitting on the fire path can ever answer. A false negative — a real hand-back the pattern fails to catch — never reaches the code that counts anything, by definition. It doesn't fire, it doesn't dedup, it doesn't land in any of the five buckets I can measure, because as far as the hook is concerned, nothing happened.
Rate is free now. Recall isn't. Knowing how often the gate fires relative to how often it runs is a replay away. Knowing how often it should have fired and didn't needs something a counter can never supply: a sample labeled by something other than the gate itself, which still means a person reading transcripts by hand and deciding, case by case, whether a hand-back was really there. I got the first number without spending that cost. I still owe the second one.
What this leaves me with is a question to ask before opening an editor to add a counter anywhere: is the decision this gate makes actually deterministic, and do I already have the inputs it decided over, retained somewhere? One caution on the second half, learned by extending this replay past a single gate: retained has to mean the whole history the condition read, not just its present value. A gate whose condition queries a live table — one that stores only what is true now — fails this test even though its logic is deterministic and the table is sitting right there, because the population it was eligible to fire on cannot be reconstructed from a snapshot. If both answers are yes, the fastest route to a trigger rate isn't a new write. It's writing the replay — the same logic, pointed backward — and reading what was already there. New instrumentation only earns its keep when one of those answers is no: the logic depends on something that isn't logged, or the history that would make a replay worth running was never kept. Neither was true here. The week the timestamp suggestion would have cost bought nothing the archive didn't already have.
Q. What was the actual problem with the Stop hook?
It only wrote to disk on one outcome, blocking a turn, out of five possible outcomes. Every other exit — no assistant text, no data-tool evidence, no pattern match, or a deduplicated repeat — left no trace, so there was no denominator: no way to know how many times the hook had run in total, or how much of its filtering came from the pattern match versus the tool-evidence condition.
Q. What did the reader suggest, and why wasn't it enough on its own?
Add a timestamp to the fingerprint file the hook already writes when it blocks a turn. A week of that would date the fires; it would still not provide a trigger rate without the missing denominator. The fingerprint file only exists on the path that fires, so it says nothing about the four silent exits, all of which live on the entry path where nothing gets written.
Q. How did you get the numbers without adding new instrumentation?
I reproduced the hook's exact exit logic in a standalone script and replayed it over 5,537 archived transcripts that already contained everything the hook's deterministic logic needs in order to decide. The replay produced 116,022 candidate stop points and sorted every one into the hook's five outcomes, including 95 fires — a rate of 0.082%, about one candidate stop point in every 1,221.
Q. Is 95 fires an actual log of what the hook did live?
No, it's a counterfactual. The replay approximates stop points rather than reproducing them exactly, and most of the corpus predates the hook and spans projects it never watched. It shows what the current logic would have decided, not a record of what happened live — the hook's real, live warned log held 22 lines at the time of this audit, which answers a different question over a different population, not a contradiction of the 95.
Q. Can this replay method tell you if the gate is missing real cases?
No, and no fire-path counter ever can. A false negative, a real case the pattern fails to catch, never touches the code that counts anything, so no amount of replay or logging on that path can distinguish genuine rarity from a pattern that's simply too narrow. That requires a labeled sample someone reads by hand, independent of the gate itself.