I Bought a GPU to Run My Own Agents. Then I Red-Teamed the Plan.

The views expressed here are my own and do not represent those of any current or former employer.

I hit the wall the boring way, with a rate limit.

Not a bill. A limit. That distinction is the whole post, so let me be precise about it. I am on the maxed-out flat-rate plan, Anthropic’s Max 20x at $200 a month, and the thing I run out of is not money, it is capacity. The plan is a fixed-size bucket that refills on a clock. When I am deep in a multi-agent run, with a planner here, two executors there, and a reviewer gating the merge, the bucket empties faster than it refills, and the work stops. Not because I cannot afford the next token. Because the plan will not give it to me yet.

This is a different problem from the one most cost articles solve. I am not per-token-cost-bound, where the answer is “find cheaper tokens.” I am capacity-bound, where the answer is “find another bucket.” So on June 25th I am splitting the spend: $100 of Claude and $100 of Codex instead of $200 of Claude. Same money, two flat-rate pools that throttle on independent clocks, plus the agent diversity of two different model families. When one bucket drains, the other is still full. You stop watching one meter empty and start draining two in parallel.

A workstation is the third bucket, and it is a strange one. So I did the thing a recovering infrastructure engineer does when the rent gets annoying. I priced the mortgage.

The mortgage, in this case, is a workstation with an RTX PRO 6000 (96GB) and a 5090 (32GB) in the same chassis. The plan was simple. When both flat-rate pools are throttled, route the overflow to a box whose marginal cost, once it is powered on, is essentially electricity. The thing to understand about that box is that I do not price it against my flat plan, where the marginal task is already paid for. I price it against the per-token API, the OpenRouter or Cursor rate I would otherwise reach for when both subscriptions are tapped out. Against that number, plus the privacy and the part people underrate, which is the raw speed, it wins. And it is fast. On the 5090 I measured open coder models running at over 200 tokens per second. That is not a feeling, it is a number off a meter. Tokens fall out of it.

This post is the story of designing the system that would sit on top of that box, and then taking it apart on purpose before I built it. The taking-apart is the useful half.

The 32GB lesson

Before the 96GB card showed up, I had already been running agents locally on the 5090 with SGLang, and it mostly worked. Three Qwen coder agents at once, pulling real weight on real tasks, the first time local agents felt less like a demo and more like coworkers. And then, every single time, compaction hit like clockwork. At 32GB I capped the serving window at roughly 32K per agent, and the reserve floor plus my skills and scaffolding left almost no room before the agent read a single file. The agents were fast and capable, but claustrophobic: they would get rolling, fill the window, compact, lose the thread, and I would be back to babysitting.

The 96GB card fixes that arithmetic, and the VRAM math, the model-by-model KV cost, and the prefix-caching trick that makes it pay off are their own story, which I wrote up separately. But sitting with that 32K failure taught me the more important thing, and it had nothing to do with VRAM. I did not want to babysit. I wanted to point a fast local model at a pile of work and walk away.

That is a system design problem, not a hardware problem. And it is where I had already been making the same mistake for a year.

The packet is the worldview

Here is the thing the 32K box finally beat into me. I had assumed the gap between a frontier model and a fast local one was raw intelligence, and that no plumbing could close it. That is true for open-ended work, where a frontier model’s worldview, its vast implicit prior about how code is shaped and where the bodies are buried, is exactly what does the wandering. But most of my backlog is not open-ended. It is bounded: rename this, add that field with this validation, port this pattern from there to here. A bounded task does not need a worldview, it needs a specification, the same spec-driven idea I chased across a run of posts, pointed now at a cheap local model instead of a frontier one. Hand it a tight packet and it does not have to explore or reconstruct the worldview the frontier model gets for free. The packet is the worldview, narrowed to one task, and it is also what buys the capacity back, because a model that does not explore burns far fewer tokens to finish. How a packet decides which tier a task can run on is its own post; here it is enough that a good packet is what makes the cheap bucket useful at all.

The layer that keeps getting eaten

I build agent tooling. I built a thing called fakoli-flow, a proper intent-driven orchestration pipeline: brainstorm, plan, execute, verify, finish, with wave-based dispatch and a mandatory critic gate between every code-writing wave. I was proud of it. It was good.

Then Anthropic shipped Dynamic Workflows in Claude Code, with subagents that fan work out in parallel, and OpenAI shipped its own. It was not just fakoli-flow. I had also hand-built fakoli-crew, a roster of specialist agents, and the same fate found it: the wave dispatch became a platform feature, the loop-and-heartbeat pattern got baked into the runtimes I lean on (OpenClaw among them), and Codex grew its own automation. The orchestration I was proud of got absorbed, for free, improving without me, in a window of months. I stopped maintaining my versions and used theirs. This is not a forecast about platforms eating orchestration. It already happened, to my own code.

This is the part that took me three rounds to learn. Orchestration is sand. Whoever owns the model owns the harness, and whoever owns the harness eventually ships the orchestration as a feature. Every hour you spend building “how the agents coordinate” is an hour spent on the layer the platform is most likely to give away next quarter. I had reached that conclusion the hard way a few weeks earlier, auditing my own work against the field: the hammers are becoming commodities, the anvil is the work.

What did not get absorbed was the other thing I had built and almost ignored: the durable record. The store of every task, every claim, and every piece of evidence. The thing that knows what is true about the project after the chat window closes, after the model gets swapped, and after the orchestrator changes names again. In the fakoli-plugins monorepo I had called it fakoli-state.

So on June 17th I pulled fakoli-state out of the monorepo, renamed it, and gave it its own life. I called it Anvil.

The name is the whole thesis, and I did not fully appreciate that until later. Anvil is the fixed surface the work is done on. Orchestration is the hammer. The agents are the hand. The hammer and the hand change constantly, with a new model every few weeks and a new orchestrator every few months. The anvil stays.

The idea: a pull queue for heterogeneous agents

Here is what I wanted to build on top of the box.

Not a scheduler that assigns work to agents, but a queue that agents pull from. Codex runs a loop: poll Anvil, ask whether there is a task here that fits what I can do, and if yes, claim it, do it, submit the evidence, and repeat. Claude does the same. OpenClaw on the local box does the same with the local models. Nobody assigns anything. Each runtime sees the backlog, decides what is for it, and takes it.

If you have run CI, you already have the mental model: GitHub Actions self-hosted runners, where the task carries a label and the runner pulls the job it qualifies for. The point is strategic, not tidy. Because agents self-select, the runtimes and the state layer never have to evolve together. I can swap the whole fleet, throw out one runtime, add a local model, let a platform ship whatever is next, and the contract underneath does not move. Which work is the bounded, low-risk middle versus the high-stakes work that needs the worldview is a routing call I worked out separately: route by what the task can afford to get wrong, not by which model is best.

I had the box, the thesis, and the whiteboard. The responsible thing to do next was to not build it.

Red-teaming my own plan

I have shipped enough things I was sure about to distrust the feeling of being sure. So before writing the code, I ran an adversarial review of my own thesis: several independent passes whose only job was to find the holes in feasibility, economics, the trust model, operational risk, and the one I was most afraid of, which was “am I overbuilding this?”

The review was not gentle. Here is what it found, and every one of these landed.

The overflow valve is only worth building if the buckets actually run dry, and I had never measured how often they do. The capacity story rests on one number: how frequently I am throttled across both flat-rate pools at once, with real work waiting. If that happens rarely, the local box mostly idles, and its true marginal cost, meaning power plus the very real time I spend maintaining bleeding-edge serving stacks, is higher than the approximately-zero a maxed subscription already charges me for the next task. The honest correction is that splitting to $100 Claude plus $100 Codex also doubles my odds that at least one bucket has headroom, which makes the local overflow needed less often, not more. I had designed an overflow engine without ever measuring how often the tank overflows. I had a vibe where I needed a meter.

The safety net I was counting on does not catch the thing I am afraid of. My plan was to keep weak local models on a leash by routing only “low-risk” work to them. But when I read the scorer, the risk number mostly tracks complexity (file count, description length, a few refactor and migration keywords), and everything that looks at the path is just name-matching: a blocklist of scary names (schema, migration, config), a small bonus for living under src/, another for being a public-API file. That asymmetry is the hole. A genuinely dangerous one-line change in an innocuously named file trips none of it (low complexity, unalarming name), so it scores low and sails straight to the weakest model on the rack. The leash reads size and a handful of name patterns. Neither one reads behavior.

The “no babysitting” dream was contradicted by my own shipped code. The reaper does catch a dead agent: a claim whose lease has expired gets released on the next operation. What it cannot catch is a busy-but-useless one. The heartbeat renews on a timer with no proof of progress, so an agent stuck re-running a failing test keeps renewing a perfectly valid lease. The lease never goes stale because the agent never goes quiet. On a one-box fleet, that is a local runner pinning a GPU at 100% all night, producing nothing, while I sleep. Worse, the actor identity that ties a claim to its heartbeat resolves three different ways across the codebase, so the owner recorded on a claim and the actor a hook sees can disagree, and that hook is advisory by design anyway: it warns, it never blocks, with real enforcement deferred to the apply gate. And the evidence gate? Off by default. Advisory. The completion record had no field for whether the command actually passed, only that it was reported to have run.

That last one is the knife, and it points at the center of the whole thesis. My pitch is that you can trust work done by an agent you did not watch because the work is verified. But:

You cannot be the trust layer with an unproven trust layer.

I was about to route the most volume, through the weakest executor, under the least-proven gate. That is not a fleet of helpful coworkers. That is a machine for generating fast, plausible, dumb work that looks done, which is the exact 32K nightmare again, now with more VRAM and less supervision.

What survived

Here is the thing about a good adversarial review. It does not kill the idea. It kills the grandiose version of the idea, which is the only version worth killing, and it hands you back something smaller and true.

What survived is sharper than what I started with.

Durable state is not the moat anymore. While I was admiring my own substrate, the platforms shipped theirs. OpenAI’s Symphony makes Linear the durable control plane for Codex agents. GitHub’s Agent HQ bakes the same idea into the repo host. MCP added a Tasks primitive for durable, pollable work. Steve Yegge’s Beads already proved a durable canonical work record, separate from chat and from GitHub Issues, should exist, and it does it with a Dolt-versioned store richer than mine. The “durable record separate from the chat” argument is won, by everyone. So durability is table stakes, not a wedge. I was also behind on the parts I thought I led: Temporal has shipped durable execution, leases, heartbeats, and orphan recovery for years, multi-host, across clouds. On the individual pillars, I am matched or behind in more places than I am ahead.

The wedge that is left is neutrality plus verification, and I want to be honest about the word “wedge,” because it is contested, not won. Neutrality only survives as a schema, not an app. The one record no single vendor will ever build is the one that spans GitHub and Linear and a local GPU box and a runtime they did not bless, with no account and no telemetry, because building it would help their competitors’ agents plug in. That is a thing only a neutral party builds. That is the lane.

Verification is the other half, and here is where I have to resist my own pitch. The non-gameable part already ships: the gate will not pass a command that exited nonzero, and the proof carries the command’s real exit code and an output hash, not the model’s say-so. The open bet is the part that survives leaving my machine, a portable, signed proof a runtime I did not build can trust without trusting the box that produced it. Plenty of systems own one of these columns, durable state, leases, a proof format; the interesting position, and the bet, is being the only one that attempts all of them at once, local-first and neutral. That is the move I went on to test the hard way, and only the ledger survived it.

The version I am actually building

So I am not building the fleet. I am building two shell loops.

One eligibility flag on the “give me the next task” command, which returns work whose risk is within a runner’s ceilings, wired to scoring that already exists. Two loops distinguished only by which models they point at and what ceilings they respect: a local loop that only takes low-risk work and never auto-merges it, and a cloud loop with no ceiling. The three safety fixes that are non-negotiable even for two loops: make the heartbeat prove progress before it renews, give every runner one coherent identity, and make the evidence gate strict by default for the loop that runs while I sleep. Keep the local loop’s output in a “needs review” state until I have actually measured how often it is wrong.

Then I run it on my real backlog and measure the numbers I had been assuming. How often are both flat-rate pools throttled at once with work waiting, the thing that actually justifies the overflow box? How often is the local model’s “done” actually done, and does a tighter packet move that number? Does the per-token API spend I would otherwise pay actually drop?

Every richer thing, including a first-class notion of capacity pools, capability profiles, model tiers, and a shared backend so the fleet can span machines, gets pulled into existence by a measured failure of the simple version, or it does not get built. The whiteboard does not get a vote. The data does.

And I wrote down the condition under which I quit. If a platform ships a portable, vendor-neutral proof-and-state format that my off-cloud, non-blessed local runners can read and write, the standalone tool is dead and I should collapse it into a schema spec and a couple of adapters. Naming your own kill switch is how you keep from becoming the person still defending the orchestration layer in 2027.

What happened when I actually built it

The plan survived contact, mostly because it had already been cut down to the part that could survive it. The two-loop, risk-pinned version got built. The routing rule, volume to the cheap tier and stakes to the frontier, became its own post. And then I ran the test I had not planned: I moved the whole crew onto a second runtime, OpenClaw against local models, to see whether the cheap tier could carry real work. The orchestration did not survive the move, so I rebuilt it. The ledger ported untouched, which is the post that finding deserves.

The part I had called least-proven is the part that earned its keep. The evidence gate I was most afraid of caught a real escape in the wild: it returned FAIL on a change a text-only reviewer had already passed, the file momentarily inconsistent on disk, and only the gate that re-ran the command saw it. One instance, not a statistic, but it is exactly the failure the whole thesis exists to catch, caught while I was not watching.

The actual lesson

I bought a fast GPU to stop hitting rate limits. What it actually bought me was a forcing function to figure out which layer is worth my time, and the answer turned out to be the smallest, most boring one. Not the orchestration the platforms keep eating. Not even the durable state they have now all shipped. The neutral substrate underneath all of it: the fixed surface that spans my flat-rate pools, my local box, and whatever runtime comes next, with no account and no vendor blessing. Two things make that substrate worth a person’s time. A tight packet that lets a fast cheap model do bounded work without a frontier worldview, which is what buys the capacity back. And a verified record that survives leaving the machine that made it. The local half is no longer a hope; it has already caught work that looked done and was not. The bet now is the other half, a proof a runtime I do not control can trust off my box, and I am making it because the first half earned it.

The most useful engineering I did this week was talking myself out of building the impressive version, and out of headlining the one capability I had not actually built yet. The win condition is not market share or a clever fleet. It is personal infrastructure that survives the churn, that one person can maintain, that is still standing after the next three orchestrators come and go.

Build the anvil. Let the hammers change.

The state engine in this post is Anvil; its ancestors fakoli-flow, fakoli-crew, and fakoli-state live in fakoli-plugins.