Why I rejected an event bus for my solo agent fleet: state is truth, events are rumors

For a small, high-churn fleet of agents, cron jobs, and MCP servers you don't fully control, prefer pull (scan on-disk state) over push (an event bus): a poller is self-healing and needs zero instrumentation, while an event bus goes silently blind the moment a component fails to emit, crashes before emitting, or is a third-party tool you can't modify at all. I designed the inbox as a computed view over existing state and rejected the event bus on purpose. Here is the reasoning, because it generalizes past my setup.

← hexisteme · notes · June 20, 2026 · 8 min read

I run a personal fleet on one machine — a handful of small agents, a pile of cron jobs and LaunchAgents, and several MCP servers, some of them third-party. I wanted a single inbox that answers "what needs my attention right now?": new outputs I haven't seen, decisions only a human can close, dependencies that broke.

The obvious architecture is an event bus. Every component emits events — job.finished, output.created, decision.pending — to an append-only log; the inbox reads the log; closing an item writes a close-event that advances the next step. It's clean on a whiteboard. I rejected it for four reasons, and chose a pull design instead.

The four reasons I killed the event bus

1. The instrumentation tax is a project killer

Push means every producer must emit. In a fleet that grows every week, that's a standing tax on each new agent, each new cron line, and — fatally — each third-party MCP server you cannot modify. You can't add an emit call to a server someone else wrote. So the moment you add a component and forget to instrument it (or can't), it becomes invisible in the inbox. An observability layer whose blind spots grow with your system is worse than useless: it's a "single source of truth" that quietly lies.

The trap: the components you least control — third-party tools, things that crash early — are exactly the ones an event bus can't see. Push optimizes for the easy case (code you own) and fails the hard case (code you don't).

2. State is truth, events are rumors

An event is a claim that depends on the claimant surviving and remembering to speak. If an agent crashes before it emits done, an event-based monitor shows nothing — the failure is invisible. State is the evidence left behind regardless: a stale output file, a log that stopped growing, a health check that fails, a process that isn't there. A pull monitor reads that evidence and reflects the crash without the agent's cooperation. This makes pull self-healing — it converges on reality every cycle — while push is only as honest as its least-reliable emitter.

# pull: derive status from evidence the component already leaves behind
status = {
  "alive":   process_is_running(job),         # ps / launchctl print — external
  "fresh":   newest_output_mtime(job) > expected,   # ⚠️ the job's OWN output dir — see correction
  "healthy": health_check(dependency),        # poll endpoint — external
}
# no emit() anywhere; a component that never reports is still seen
Correction (2026-07-14) — this section is right about the conclusion and wrong about the reason. @anp2network pointed out in the comments that push-vs-pull is the wrong axis; the useful axis is self-report versus external observation. An event bus is self-report over a channel; a poller reading files the producer wrote is self-report over a filesystem. Both are only as honest as the writer. Look at the fresh line above — it globs the job's own output directory. I argued that a bus is only as honest as its least-reliable emitter, then shipped a poller only as honest as its least-honest writer.

The failure mode this hides is liveness without progress, and I had already lived it without recognizing it. My dev.to publishing job ran on schedule for four straight days: the process was alive, its log grew every morning, every freshness signal was green — and it published nothing, because the queue upstream had quietly emptied. The monitor was reading the job's own log to decide whether the job was working. I found it by looking at the published count, not at the monitor.

What survives is narrower than what I wrote. File writes and process liveness are side effects of execution rather than voluntary emissions, so a poller can observe a component that never agreed to be observed — including one that is crashing, wedged, or written by someone else. A bus cannot force a liar to emit. That asymmetry is real, and it is why I still run pull. But it is an argument for external observation; pull is merely the transport that makes external observation cheap.

The monitor now scores freshness on the effect a job produces — published-article count, rows appended to a ledger, a marker written only after a remote page was fetched back and verified — rather than on artifacts the job authors about itself. A job whose effect stalls while its log keeps growing now reads "stalled: dev.to publishes flat for 96 hours (process is running)." The old code returned green for that exact scenario. One caveat, since the rule can be over-applied: "every job must move something it does not own" would make terminal sinks unmonitorable — a cache warmer or a log rotator legitimately has no external artifact to move. The rule I settled on is score the effect, and define the effect from the job's purpose, not from ownership; where a job genuinely has no external effect, the monitor labels its card self-reported rather than pretending otherwise.

3. Orchestration schizophrenia

A closed-loop bus — where "close this item" emits an event that advances the next step — turns the inbox into a workflow engine. I already have an orchestration hub that decomposes goals into steps. Building a second one inside the monitor duplicates that responsibility and doubles the debugging surface: now a stuck task could be the hub's fault or the inbox's. A monitor should report state, not drive it. Keeping those two jobs in two systems is what keeps either one debuggable.

4. The bus itself becomes an unreviewed dump

An append-only event log is not free infrastructure. It accrues schema drift (the shape of output.created changes and old readers break), duplicate events, events nobody ever closes, and unbounded growth that demands compaction. You've added a database with none of a database's guarantees — and it needs its own monitoring. Pull has no such artifact: there's nothing to compact because there's nothing stored but the state that already exists on disk.

What pull looks like instead

The inbox is a computed view over state that already exists — no new store, no emit calls:

Attention typeDerived from (pull)
New / unread outputper-job output glob mtime vs a read-timestamp record
Pending decisionexisting on-disk sources — an attention scan's output, an undecided ledger entry, an expired evaluation date
Broken dependencyhealth-check failure, propagated to anything that declares a dependency on it

Priority is deterministic, not a model's guess: BLOCKED → STALE → NEW, where each is a hard fact (a failed health check, a file newer than its read-timestamp, a schedule past due). An LLM-generated priority number would be unfalsifiable and would drift; a deterministic trigger is reproducible and debuggable. The model stays out of the ranking entirely.

Freshness is "unread," not "old"

Pull also fixes a subtle metric. The intuitive freshness signal is elapsed time — "this ran 3 days ago." But age isn't the problem; unread output is. A report that ran an hour ago and that you haven't opened is more demanding of attention than one from last week you already read. So freshness is computed as a join: does an output exist whose mtime is newer than the last time you opened it? Clicking a card records a read-timestamp; unread items rise to the top; read ones sink. This is only cheap because the design already scans state — freshness falls out of the same glob, where in a push system it would be yet another event to emit and reconcile.

The boundary that keeps it honest

Choosing pull also forces a discipline: the monitor must not mutate fleet state. "Closing" an inbox item means acknowledge and deep-link to the real place the work is closed — it does not reach in and change a job, a ledger, or an agent's state. The moment a monitor starts writing back, it's an orchestrator again, and reasons 1–3 return. Read the world; link to the controls; never become the controls.

When push is right

None of this says event buses are bad — it says they fit a different shape. If you own every producer, can instrument all of them, and need high-throughput, low-latency fan-out, push is the right tool. The pull argument wins specifically for a small, heterogeneous, high-churn fleet with components you don't control, where the cost that dominates is instrumentation and the failure that hurts most is the silent one. Match the architecture to which cost is fatal: throughput, or blind spots.

FAQ

Q. Should I use an event bus or polling for a small multi-agent system?
Polling, if the fleet is high-churn and includes components you can't instrument. An event bus is blind to anything that doesn't emit; a poller can observe a component that never agreed to be observed. But do not mistake polling for external observation: if your poller reads files the job itself wrote, that is still self-report — just over a filesystem instead of a channel, and only as honest as the writer. Score the effect the job produces, not the artifacts it authors about itself. Use a bus when you own and can instrument every producer and need high throughput.

Q. What does "state is truth, events are rumors" mean — and is it right?
The slogan says on-disk state is evidence left behind regardless of whether anyone remembered to report it, while an event only exists if the emitter survived long enough to send it. That half holds. But the slogan is incomplete, and taken alone it misleads: a file the job itself wrote is still self-report, merely over a filesystem instead of a channel. It is only as honest as the writer. The slogan is true exactly when the state was authored by someone other than the job being judged — a process table, a health endpoint, a downstream artifact, a remote page you fetch back. Read it as "prefer evidence over claims, and check who wrote the evidence."

Q. Why does my monitor show green while a job does nothing for days?
Because you are scoring the job's report, not its effect. This is liveness without progress: a job wedged in a retry loop keeps appending to its own log, its mtime advances every cycle, the process is alive, and every freshness check passes — while nothing useful happens. My publishing job did exactly this for four days; the queue upstream had emptied and the monitor was reading the job's own log to decide whether the job was working. The fix is to score an effect the job cannot fake by merely running: a downstream artifact count, rows appended to a ledger someone else owns, a remote page fetched back and verified. Where a job genuinely has no external effect (a cache warmer, a log rotator), label the card self-reported rather than pretending the signal means more than it does.

Q. Is push or pull better for monitoring agents I don't control?
Pull. You can't add emit calls to a third-party MCP server, so push can't see it. File writes and process liveness are side effects of execution rather than voluntary emissions, so a poller works on uncooperative components — a bus cannot force a liar to emit. The real axis is self-report versus external observation; pull is simply the transport that makes external observation cheap.

Q. How do I monitor third-party MCP servers that don't emit events?
Observe their external state: poll a health endpoint, check the process is alive, watch files they touch, and propagate status to dependents. The component never has to cooperate.

Q. How should a monitoring inbox decide priority without an LLM?
Deterministic state triggers — BLOCKED, then STALE, then NEW — each a fact derived from disk (health check, mtime vs read-timestamp, schedule vs log). An LLM priority score is unfalsifiable and drifts.

Related notes

← hexisteme · notes · CC-BY 4.0