Unit Tests Passed. The Feature Never Ran. Three Times in One Session.

← hexisteme · notes · 2026-07-19

Three features shipped in one session, each with dedicated unit tests, all green, and none of the three ever executed in the live app. The first had eight passing tests and was broken by a hand-written merge that dropped three new fields plus two call sites that never passed them. The second had a 692 MB data pipeline, a generated survival table and fifteen passing tests, and existed only inside preview blocks. The third was reported as unwired by the worker that built it, in a line that got buried. The shared root cause is the default value that quietly removes the compiler's call-site audit, and the fixes are structural rather than more tests.

Round one: eight passing tests, zero live executions

The first feature was a handoff row: buttons that take you from the app to a phone dialer or a maps app. The URL construction lived in a pure function, HandoffRow.urls(), written that way on purpose, with a comment in the source declaring it testable. Eight dedicated tests, all passing, covering whether coordinates are present, whether the target app is installed, percent-encoding of the business name, and the precedence rule for using a place identifier over a name search.

In production those arguments were never populated. The wiring broke in two places.

First, a merge function. The live app composes two review sources, and that merge is hand-written: it copies fields one explicit line at a time. Three new fields had been added upstream (a place identifier, a phone number, a maps URI). None were added to the copy list. It compiled fine, because dropping a field during an explicit field-by-field copy is not a type error. It is just not copying.

Second, the call sites. Four view call sites construct this row. Two of them, the two on the live path, didn't pass the new arguments at all.

So the phone button never rendered, in any state, ever, and the maps deep link always degraded to a plain name search. No crash, no error, tests green. The tests didn't find it; an adversarial code review did, run with three lenses, two of which flagged it independently.

Round two: the same failure, at much larger scale

Same session. I added a new axis to the verdict: how long the restaurant has been operating, benchmarked against how long comparable businesses survive. The deliverables:

Zero of the five production call sites passed the tenure: argument. Grepping for the type constructor found every construction site in the app inside a #Preview block, three of them. The feature existed exclusively in the preview canvas.

The most instructive call site was the map-pin lookup path. That function already held both values the new axis needed, the license date and the category, in local variables a few lines above the call. It just didn't pass them.

So the pipeline, the table, the types, the view and fifteen green tests all existed, and the feature rendered zero pixels on a real device. This happened after I had diagnosed round one and written a dedicated test file for it.

Round three: the worker said so out loud

Same session, same workflow. I wired restaurant photos in from a places API. The verdict object, the thing that carries the analysis from the logic layer to the view layer, didn't carry the photo fields, so the hero image fell back to a stock illustration on every live path.

The detail that stings: the subagent that did this work wrote "backend not wired" in its own completion report. It was honest. It said the thing. That report was one of several returning at once, and the line got buried.

Three for three, on a pattern I already knew.

Why unit tests cannot catch this, in principle

A pure-function test looks like this:

let destinations = HandoffRow.urls(placeName: "...", googlePlaceId: "ChIJabc", ...)
XCTAssertEqual(destinations.google.absoluteString, "...query_place_id=ChIJabc")

The proposition it verifies: does the function handle the argument correctly? The proposition it does not verify: does that argument ever arrive?

The second is out of reach for as long as the test supplies the argument itself. That isn't a gap in the suite, it is what isolation means. More tests, or better ones, hit the same wall, because every one constructs the input by hand, which is exactly the step production was skipping. Coverage doesn't rescue it either: coverage counts lines that executed, and the failure here is a call site that does execute and passes nothing.

So the category matters more than the three bugs. Unit tests validate components in isolation, which makes every state where components are not connected to each other invisible to them. Here that was unpassed arguments and a lossy merge. Elsewhere it is the handler written but never registered, the route that exists but was never mounted, the feature flag defaulting to off, the DI binding that never made it into the container. Different surface, same blind spot. All of them go green.

The real culprit is the default value

All three had the same thing at the root: to add a new field without breaking existing call sites, I gave it a default.

var googlePlaceId: String? = nil
static func read(from: FetchedSignals, tenure: TenureRecord? = nil) -> TransparencyRead

That single = nil is the whole story. Without it, the compiler lists every call site needing an update, immediately, as errors. With it, the compiler is satisfied and the failure relocates to runtime, where it manifests as an absence, and absences don't throw.

The uncomfortable part: the default almost always looks like the correct call at the moment you make it. It is the textbook incremental migration, and often the only way to add a field without breaking a file somebody else is editing right now. You reach for it because it is the professional move, and it quietly trades a compile-time guarantee for a runtime nothing.

Why AI-assisted development widens the gap

This predates LLMs, but delegating implementation to a code generator makes it structurally more likely.

Scoping. A generator builds the component you asked for. "Add a tenure axis" yields a pipeline, types, a view and tests, all self-contained, all good. Connecting it to the five places that should call it was never in the request, and a component that compiles with passing tests looks finished from the inside. The generator produces the thing; it does not produce the thing's callers.

File ownership. Parallel delegation only works if you split files between workers, or they collide on edits. So the instruction becomes "don't touch files you don't own", which, followed faithfully, becomes "give the new parameter a default so files you don't own keep compiling". Wiring is precisely the work that crosses ownership boundaries, which is the work nobody was assigned.

Every worker did its job correctly. Nobody lied. Round three's worker reported the gap explicitly. And the feature was dead. That is a structural outcome, not a diligence problem, and it recurs until you change the structure.

Fix one: make the omission inexpressible

Stop hand-passing the data. Instead of threading new values through view properties at each call site, load them onto an object that already flows to the consumer, here the verdict itself. It travels with the verdict, so a new call site cannot fail to bring it.

// Before: hand-passed at every call site -> 2 of 4 forgot
TransparencyReadView(read: read, googlePlaceId: ..., phoneNumber: ..., googleMapsUri: ...)

// After: the verdict carries it -> the call site has nothing to forget
TransparencyReadView(read: read)

The principle isn't "remember to pass it". It is make forgetting inexpressible. A rule that depends on remembering had already failed three times inside this one story.

Fix two: a wiring test beside the unit tests

A separate file whose only job is asserting the assembly path: not the functions, the connections between them. Three assertion points:

It is a separate file because it answers a different question and would otherwise get deleted as redundant, so a comment at the top explains why it exists, for the next person doing a cleanup pass.

What actually caught all three

Adversarial code review, every time. Three reviewers with different lenses (regression risk, contract integrity, honesty of reporting) run independently, converging on the same finding. One line in the review instruction did most of the work:

Trace new features to their call sites and verify they actually receive values on the live path.

In round one that line wasn't there, and the reviewer saw eight passing tests and rated the item low severity, which was a reasonable read of the evidence in front of it. The instruction is what changes the evidence a reviewer goes and collects. You don't get call-site tracing by hoping the reviewer is thorough; you get it by asking for it by name.

The portable checklist

Language-agnostic. Applies anywhere default arguments and optional fields exist.

  1. Giving a new field a default opts you out of the compiler's call-site audit. Budget for paying that cost elsewhere.
  2. Partial merges (merge, combine, reduce, any hand-written field-by-field copy) get no compiler help when a field is added. Add a field, grep the merges first.
  3. Keep a wiring pass-through test beside the unit tests, in its own file, with its reason for existing written at the top.
  4. Before trusting a new test, check that it can fail: delete the thing it protects and confirm it goes red.
  5. Put "trace to the call site" in the review instruction. Don't rely on reviewer diligence for something you can ask for.
  6. Splitting file ownership across parallel workers structurally produces incomplete wiring. Schedule a dedicated wiring pass at the end, or move the data onto a domain object so there is nothing left to wire.

Honest limits

One project, one developer, one session. A case study, not a statistic. The review caught all three this time, which is not evidence it catches everything, since there is no way to count what it missed.

"Put the data on the domain object" isn't universally right; the counter-cost is that the object grows. It was cheap here because that object was already a display-oriented container carrying a dozen derived values, and never gets persisted. If it were a persisted model, adding fields for view convenience would be a schema decision, and I'd have chosen differently.

Two conditions would tell me I'm wrong. If a wiring gap of the same shape recurs despite the pass-through test, the answer isn't more test placement, it is applying the structural fix more broadly. And if six months pass with no recurrence, that only counts if new fields with defaults were actually added in that window. No incidents can mean no attempts.

FAQ

Q. How can unit tests pass while the feature never runs in production?
Because a unit test supplies the arguments itself. It verifies that the function handles an argument correctly, which is a different claim from the argument actually arriving on a live code path. In this case a pure URL-building function had eight passing tests covering coordinate branches, app-installed branches, percent-encoding and identifier precedence, and in production the new fields were dropped by a hand-written merge function and omitted by two of the four view call sites. None of that is visible to a test that constructs its own input.

Q. Why does code coverage not detect a wiring gap?
Coverage counts lines that executed. A wiring gap is a call site that does execute and simply passes nothing, so there is no uncovered line to point at. In the second incident the only places constructing the new type were three preview blocks, the test suite exercised the type thoroughly with fifteen passing tests, and the feature still rendered zero pixels on a real device.

Q. How does giving a new field a default value cause this?
A default value removes the compiler's call-site audit. Without a default, adding a field makes every call site that needs updating fail to compile, immediately and exhaustively. With a default, the compiler is satisfied and the failure relocates to runtime, where it manifests as an absence, and absences do not throw. The uncomfortable part is that the default almost always looks like the correct call at the moment you make it: it is the textbook incremental migration, and often the only way to add a field without breaking a file somebody else is editing right now.

Q. Why does AI-assisted or parallel-agent development widen this gap?
Two reasons. Scoping: a code generator builds the component you asked for, complete with its own tests, but connecting it to the call sites was never part of the request. The generator produces the thing, not the thing's callers. File ownership: parallel delegation only works if you split files between workers, so the instruction becomes "don't touch files you don't own", which in practice becomes "give the new parameter a default so files you don't own keep compiling". Wiring is precisely the work that crosses ownership boundaries, which is the work nobody was assigned.

Q. What is a wiring pass-through test and how is it different from a unit test?
It is a separate test file whose only job is asserting the assembly path rather than the functions. Three assertion points cover it: the merge boundary, checking that a partial merge preserves newly added fields regardless of completion order; the assembly boundary, reproducing the live assembly expression exactly and asserting the value reaches the final object; and orthogonality, asserting the newly carried value does not change the verdict it travels with. It lives in its own file with a comment explaining why, because next to the unit tests it looks redundant and would otherwise get deleted.

Related notes

← hexisteme · notes · CC-BY 4.0