The Guardrail Has to Be Code: How a Runaway Local LLM Corrupted APFS and Bricked a Mac Mini

I run a small fleet of local agents on a Mac Mini M4 with 24 GB of unified memory. One afternoon it went down hard — no wake from sleep, no boot chime past the logo, and in recovery mode an APFS volume that refused to mount. The unsettling part: I hadn't asked it to do anything heavy. Something had quietly started a large language model in the background, on a disk already nearly full, and by the time I noticed, the filesystem was past saving. Recovery took more than a day. This is the postmortem — and the fix, which turned out to be a single False and a checklist, not a note to myself.

← hexisteme · notes · 2026-07-10

The timeline: from "a little slow" to "won't boot"

It didn't look like a memory problem — just the machine getting sludgy: beachballs, slow app switching, the stuff you blame on too many tabs. Then the display went black, gone. A forced restart got the Apple logo and a progress bar that stalled and stayed. In recovery mode the shape came clear: Disk Utility saw the container, the data volume wouldn't mount, and the repair pass — fsck_apfs — hit metadata it couldn't reconcile, a volume whose own consistency layer had lost consistency. Repair couldn't finish, and the fix cost the better part of a day between attempts and a restore.

The root-cause chain: disk pressure is what turns OOM into corruption

Reconstructed after the fact, the chain had six links, none exotic:

• The disk was already low on free space — the precondition that makes the rest dangerous, fine until the OS needs room and there isn't any.
• A background server loaded large models — a 14B and an 8B, several gigabytes of resident memory with no window open.
• Resident memory exceeded physical RAM. macOS doesn't refuse this; it manages it.
• macOS reached for swap — compress, then page the overflow to swapfiles on the boot volume. The escape valve needs disk room.
• The swap writes failed — no room to grow the swapfiles, so page-outs returned I/O errors.
• The APFS metadata didn't survive it — left inconsistent, and an inconsistent APFS volume won't mount.

I'll flag that last link, the one I can't fully prove. APFS is copy-on-write and crash-consistent; a full disk alone should give clean ENOSPC errors, not corruption. The honest reading isn't "disk-full corrupts APFS"; it's that a nearly-full disk under sustained failing swap writes is a rare corner where the filesystem's guarantees get tested at the worst moment, and mine didn't hold. The rule doesn't need the exact trigger: never let the OS be forced to swap onto a disk with no room. You defend that corner by never entering it.

Why a note in the config was never going to hold

The detail that decided the fix: I didn't start that server. A global config had a line of guidance that local model delegation was preferred for background chores; some path read that as license and auto-started it. I was never in the loop. That's why the fix couldn't be another note. A soft default gets flipped by things that don't read it — an environment variable exported in another shell, an auto-start script, an agent following the global rule over the local exception. If a code path can physically brick the machine, the guardrail can't be a sentence asking it not to. It has to be code that can't.

The fix: make the dangerous path refuse to run

So I replaced guidance with refusal, at every layer that could start the server — defense in depth, independent blocks, each alone enough to stop the process. Across the two repositories that could spawn the model:

• One constant as the source of truth — a single _MLX_ENABLED = False every layer reads.
• The router's health check is wired to fail. _check_local_server_health() returns False unconditionally, so decide_router() has no branch that can select local.
• The executor escalates first — its entry point returns {"escalate": true} and exits before importing mlx_lm or opening a socket.
• The spawner is stubbed, original preserved — the command that used to nohup mlx_lm.server ... now refuses; the real one is renamed with a _LEGACY_ prefix, kept out of the live call graph. The dormant server manager and batch engine got the same treatment.
• A companion constant in the other repo, so neither codebase can start what the other one blocks.

_MLX_ENABLED = False              # one source of truth; every layer reads it

def _check_local_server_health() -> bool:
    return False                  # router has no branch that selects local

if __name__ == "__main__":        # executor: escalate & exit before the
    print(json.dumps({"escalate": True}))   # heavy import or any socket
    sys.exit(0)

def cmd_start(*args):             # CLI start refuses...
    raise SystemExit("local model server disabled — see revival checklist")

def _LEGACY_cmd_start(*args):     # ...original kept intact, out of the call graph
    ...                           # `nohup mlx_lm.server ...`

Every block is a positive refusal, not a missing feature: it runs, and says no. A missing feature invites re-adding; a refusal is a decision you must consciously reverse — and because the _LEGACY_ originals are right there, reversing it is a rename, not a reconstruction.

The other half of a hard block: a revival checklist

A hard block with no documented way back is its own trap — it calcifies into dead weight, or gets ripped out in a hurry without restoring what made it necessary. So it ships with its inverse, an explicit ordered checklist to re-enable it, deliberately:

1. Verify disk headroom first — df -h / must show 20 GB+ free before anything competes for memory.
2. Re-check iogpu.wired_limit_mb — so a model can't wire away the memory the OS needs.
3. Rename the _LEGACY_ functions back, restoring the original call graph.
4. Restore the second repository in lockstep — its constant, its server manager, its batch engine.
5. Re-enter with the smallest model — a 3B 4-bit model first, confirm the machine stays healthy, then scale up.

The checklist turns "turn the LLM back on" from an impulse into a gated act: the block stops the machine bricking today, the checklist stops a careless un-block tomorrow.

The rule the whole thing taught: disk free space is a memory safety net

Step back from the specific server and the failure has one transferable lesson, about the hardware more than the software. On Apple Silicon, memory is unified: CPU and GPU share one physical pool, no separate VRAM. That pool is 24 GB, and everything draws from it — the OS, every app, any ML workload. A model's weights come out of the same 24 GB the OS is trying to live in. (The iogpu.wired_limit_mb sysctl caps how much of the pool can be wired for GPU/ML use — set it too high and a model can starve the OS.)

And macOS backs memory pressure by swapping to the boot volume — so free disk space isn't just for files; it is the runway the memory system needs when the pool overflows. When that runway is gone, "out of memory" becomes a filesystem-integrity problem. Treat free disk space as reserved memory: below a comfortable buffer — 20 GB is my floor — you are risking not a full disk but the swap subsystem, and through it the filesystem. That produced three standing rules, all aimed at never reaching overcommit:

• One heavy job at a time. On 24 GB of shared memory, two 14 GB-class workloads don't coexist, and neither does one heavy ML task alongside Photoshop, a video editor, and a VM. Light work is free; simultaneous heavyweights are banned.
• A memory preflight before generative work. Sum the genuinely available memory — free + inactive + speculative + purgeable pages, which vm_stat reports — and if it's under roughly 8 GB, close apps first.
• Low-memory mode by default, and 20 GB+ of disk free as a standing condition — the buffer only helps if it's there when the pressure hits.

The recurrence that proved the rule

About two and a half months later, I got to run the experiment again by accident: a heavy image-generation workflow at 13.7 GB, plus Photoshop, a video editor, and a VM all open. The sum blew past 24 GB and the machine blacked out — same overcommit, same trigger as before.

But this time it came back: a blackout, a hard restart, a clean boot, no corruption. The one variable that had changed was the one the postmortem told me to watch — the disk. It had 55 GB free and a healthy swap buffer, so the overflow writes succeeded. The machine fell over instead of falling apart, and I lost minutes instead of a day.

Same trigger, opposite outcomes; the dimension separating "brick" from "annoyance" was free disk space. So the rules got sharper — "one heavy job" now names the combination that broke it, and disk headroom went from hygiene to load-bearing invariant. The code block keeps that server from sneaking back on; the disk rule makes sure the next overcommit — and there's always a next — costs a reboot, not a filesystem.

FAQ

Q. Can a local LLM running out of memory corrupt your filesystem?
Not directly, but it can create the conditions that do. On a unified-memory Mac, an oversized model exhausts the shared RAM pool; macOS responds by swapping to the boot disk; if that disk is nearly full, the swap writes fail, and sustained I/O errors during that failure can leave the volume's APFS metadata inconsistent enough that the machine won't boot. The LLM never touches the filesystem itself; it just forces the OS into the corner where the corruption happens. A disk with room to swap turns the same event into a slow machine instead of a dead one.

Q. Why is free disk space a memory safety net on Apple Silicon Macs?
Because macOS backs memory pressure with swapfiles on the boot volume. Unified memory means the OS, apps, and any ML workload all draw from one physical pool; when it overflows, the kernel compresses and then swaps to disk. Free disk space is the runway for that swap. If it's gone, the memory system has nowhere to overflow to, and running out of memory escalates from a slow machine to I/O failures on the system volume. Keeping roughly 20 GB+ free keeps the swap buffer healthy, which is why a full disk is a memory problem, not just a storage one.

Q. Why replace a do-not-run note with code that refuses to start?
Because a note is a soft default, and defaults get flipped by another session's environment export, an auto-start script, or simply not being read. A note assumes something reads and heeds it every time; a hard block assumes nothing. Turning the dangerous path into code that actively refuses — a False constant, stubbed spawners, a health check that always fails, a start command that declines — removes the failure mode instead of documenting it. If a path can physically brick the machine, it needs a block, not a warning.

Q. What is a code-level hard block, and how do you make it reversible?
It's defense in depth: every layer that could start the risky process is made to refuse. A single _ENABLED = False constant as the source of truth, server-spawn functions stubbed out, a health check hard-wired to False so the router never selects the local path, an executor that escalates and exits before importing the heavy library, and a start command that refuses. You keep it reversible by preserving the originals under a _LEGACY_ prefix, so revival is a rename rather than a rewrite, and by pairing it with an explicit ordered checklist of preconditions and a staged re-entry that starts with the smallest model.

Q. How do you safely run heavy GPU or ML workloads on a 24 GB unified-memory Mac?
One heavy job at a time, and never let two 14 GB-class workloads — or a heavy ML task plus Photoshop, a video editor, and a VM — overlap, because they contend for the same pool. Run a memory preflight before generative work (sum free, inactive, speculative, and purgeable pages from vm_stat; if it's under about 8 GB, close apps first), keep the model in a low-memory mode, and hold 20 GB+ of disk free so the swap buffer stays healthy. The goal is never to reach the memory-overcommit state that forces the OS to swap onto a disk with no room.

Related notes

• Claude Code Stop Hooks as Behavioral Gates
• macOS: nslookup works but curl and Python "Could not resolve host" — the mDNSResponder zombie
• Claude Code "400: no low surrogate in string" on every turn: repairing a permanently broken session transcript