I run a video pipeline where every shot carries a contract: a must_have list, a must_not_have list, and a serves_line field — the narration line the shot exists to support. One rule in that pipeline, which I call the enactment rule, checks each contract for self-contradiction: if must_not_have forbids the very thing serves_line requires, the shot can't honor its own contract and support its sentence at the same time, so the rule fails it hard at author time.
It's a good rule to want. It had never once run.
This adds another entry to the same line of investigation: Grep won't find your dead gates. A fill-rate query will. found rules nobody was calling — the wiring itself was absent; The Guard Passed on an Empty Table found a correctly-called rule whose target population was empty at runtime, so it passed on nothing left to check; and The check that cannot fire found a rule that ran against real data and silently disabled itself because its threshold was an absolute constant that didn't match the input's scale. This one sits a layer deeper than all three: the callers existed, the rule fired on every invocation, and what was actually missing was a producer for the one field the rule needed to read.
serves_line is a derived field — by definition, it's the narration of the beat the shot is attached to. I checked that derivation against reality before touching anything else: every stored value that existed matched its beat's narration exactly, 6 of 6, zero drift. Wherever the field existed, it was right.
But almost nowhere did it exist. A repo-wide search for anything that writes serves_line returned zero producers. The archetype templates that generate contracts don't set it. So every contract created through the supported authoring path had serves_line = None, and the rule — which needs that field as its input — silently evaluated nothing. A published episode had all five of its shots sitting in exactly that state.
The rule wasn't failing. It wasn't passing either. It was unmeasured, and unmeasured looks exactly like clean from the outside.
The obvious move is to derive the field at author time from the beat it's attached to. The derivation is provably right, the plumbing is small, and it turns a dormant rule live. I wired it. Tests passed.
Then, before trusting that, I ran it retroactively against every existing contract the rule could now evaluate.
| outcome | count |
|---|---|
| hard failure | 6 |
| exempted (declared metaphor) | 2 |
| true positive | 0 |
Eight firings. Zero of them were catching a real contradiction.
serves_line is narration — audio. must_not_have governs pixels — what's allowed to appear on screen. Those are orthogonal channels. A voiceover saying "one million dollars" while the frame shows no digits at all is not a contradiction; it's the ordinary craft of the format. The rule was firing on the co-occurrence of a spoken quantity and a visual numeral ban, and calling that a conflict.
The clearest proof is the best-authored contract in the repository. Its must_have reads, literally, "blank faces free of pips and numerals" — the dice are deliberately blank while the narration speaks the numbers aloud. The author had even declared that relationship as metaphorical, specifically so the tooling would know not to flag it. The rule flagged it anyway.
Had I shipped the derivation as-is, six correct contracts would have hard-failed at author time. Three of those six belong to an episode that had already shipped.
I reverted the wiring. What stays is a comment at the rule itself, recording the measurement — eight firings, zero true positives — where the next person who goes to wire a producer for serves_line will read it before they do.
I kept one separate piece in place: something that reports the unmeasured state by name. It doesn't enforce anything. It just refuses to let "no exemptions recorded" be confused with "never checked" — which is the exact confusion that let this rule sit dormant and clean-looking for as long as it did.
must_have fighting must_not_have — sits in two lists that do govern the same medium. A rule aimed at the wrong pair produces false positives at exactly the rate the wrong pair happens to co-occur.This rhymes with something I wrote up before: The Known-Good Sample Was Not Known-Good. There, a threshold was derived from a control sample nobody had actually verified belonged to its class. Here, a rule's entire positive class had never been observed at all. Different mechanism, same artifact: a number or a rule that feels validated because something was measured, when the thing measured was never the thing being claimed.
Ask whether it has a demonstrated way to fail, and check what its inputs actually are at runtime. In this case, every contract on the supported authoring path had serves_line = None — the rule was structurally incapable of evaluating anything. A rule can have zero failures on record because it has zero real evaluations on record, and those look identical from a dashboard.
Because a rule nobody has ever watched fire is a rule whose behavior on real, non-null input has never been checked either. Deriving the field and wiring it in passed every test — then a retroactive dry run against every existing contract the rule could now evaluate returned 8 firings and 0 true positives. The plumbing was correct and the rule underneath it was wrong; only running it against real data surfaced that.
It compared two fields that describe different media. serves_line is narration — audio — and must_not_have governs what may appear on screen — pixels. A voiceover can say a number while the frame shows none; that's not a contradiction, it's ordinary craft. The rule was treating co-occurrence of a spoken quantity and a visual ban as a logical conflict.
It would have. Six correct contracts would have hard-failed at author time had the wiring stayed in, and three of those six belonged to an episode that had already shipped. The retroactive dry run caught that before it reached production.
For every rule, ask who produces each input it needs, not just who calls the rule. Callers existing is not enough — if one of the fields a rule reads has no writer anywhere in the codebase, the rule can run on every invocation and still evaluate nothing. And before wiring a missing producer, verify the rule's positive class first: getting that order backwards starts rejecting correct inputs the day the plumbing lands.