Part of how I run background coding tasks is by shelling out to a subscription-gated CLI from a different vendor than my main assistant, one task at a time. No daemon, no shared server — each worker starts, does one job, and exits, and I read back whatever it produced. That pattern itself is fine.
What I got wrong for a while was how I decided whether a worker had actually done anything.
On 2026-08-07, one of these workers — OpenAI's Codex CLI, invoked non-interactively — hit its own usage cap mid-task. The real failure message was:
ERROR: You've hit your usage limit
printed to stderr. Meanwhile:
If a dispatcher only checks $?, this looks identical to a worker that quietly finished a trivial task and had nothing to say. There's no crash, no nonzero status, no exception to catch anywhere in the normal control flow. The failure is completely real; it's just filed under the wrong file descriptor, and the exit code actively lies about it.
A tool that fails loudly — nonzero exit, a stack trace, a panic: line — is annoying but honest: an if $? -ne 0 branch catches it whether or not you anticipated the specific failure mode. This is a different shape of problem. The interface contract the orchestration is trusting — exit code as the success/failure signal — stays green, while the actual work product (stdout) is empty. Anyone who wires "exit code equals 0" to "mark the task done, move on" will silently record a quota death as a completed job.
timeoutIt gets worse on macOS specifically: there's no timeout binary by default, so wrapping a worker call in timeout ... on a machine where it doesn't resolve hands back a shell "command not found" as exit 0 — a second, independent path to the same false-success signal.
I'd already tripped over a version of this once before, with a different vendor's CLI, and filed it away as "that tool is just weird about how it reports quota." Watching the identical shape — real error on stderr, empty stdout, exit 0 — show up in a completely separate CLI from a completely different vendor changed the diagnosis: this isn't a bug in one wrapper, it's how subscription-gated command-line tools tend to communicate "you're out of quota." They treat it as a billing condition rather than a program error, so the message goes to stderr and the process exits cleanly rather than breaking a caller's shell pipeline with a nonzero status.
Once that's the assumption, you stop trusting exit codes from any subscription CLI by default.
The rule I apply to every worker dispatch now is boring and mechanical, which is the point:
timeout exists on every machine it runs on, that assumption is itself a failure mode to check for — not just the worker's behavior.None of this requires knowing anything vendor-specific ahead of time. It requires treating "exit 0" as one weak signal among several, not the whole verdict.
This isn't specific to LLM CLIs — any pipeline-friendly tool tends to swallow certain failure classes into stderr-plus-exit-0 rather than a hard nonzero exit, because tool authors don't want a quota, rate-limit, or auth condition to break a caller's pipeline. The fix generalizes too: treat the exit code of anything you didn't write as a hint, and check the actual output before marking a step done.
I've also hit the mirror-image bug elsewhere — a wrapper failing while the underlying capability still works (a false negative); this one is the opposite: dead on arrival, but the harness says yes (a false positive).
The same distrust-the-green-light instinct applies at the HTTP layer — I've separately seen a 200 OK with an error payload inside get cached as if it were good data; same root cause, different transport.
Q. Why does a quota-exhausted CLI exit with code 0 instead of a nonzero code?
Because many subscription-gated command-line tools treat hitting a usage limit as a billing condition rather than a program error, so they route the real message to stderr and let the process exit cleanly rather than breaking a caller's shell pipeline with a nonzero status.
Q. Is this specific to one vendor's CLI?
No. I saw the identical shape — real error on stderr, empty stdout, exit 0 — in two unrelated CLIs from two different vendors, which is what turned it from "one weird tool" into an assumption I now hold about any subscription CLI I shell out to.
Q. Why does macOS make this worse?
macOS does not ship the GNU timeout binary, so a dispatch script that wraps a worker call in timeout without checking that it resolves can have the shell fail to find the command and still hand back exit code 0, producing a second, independent path to a false success signal.
Q. What should an orchestrator check instead of the exit code?
Whether stdout is non-empty, and whether the expected artifact (a file, a result payload) actually exists with the content it should have. The exit code should be treated as a weak, advisory signal, not the verdict.
Q. Is this the same bug as "wrapper failure is not capability failure"?
Yes, the same family of bug — trusting a signal layer instead of the real output — but pointed in the opposite direction: that case is a false negative (a wrapper failing while the capability still works); this one is a false positive (dead on arrival, but the harness says yes).