I Bet on Owning My Own Substrate

The views expressed here are my own and do not represent those of any current or former employer.

The first time I ran several agents at once against my own hardware and watched them all hold real context, I sat back and grinned like a kid who just got the thing to work. No metered counter ticking somewhere I could not see. No quiet downgrade in the model behind an endpoint I do not own. Just my box, my weights, my context, answering. I had bet that this would matter, and here it was, mattering.

I want to be clear about what this was, because the framing decides everything that follows. This was not a hedge. It was not me covering myself against some downside. It was a deliberate, long-term bet on where this technology is going, placed early on purpose, with my own money and my own weekends, because being early is the whole advantage. The future is not written yet. That is exactly why you put your stake down now instead of waiting for the map to be finished by someone else.

The Timing Is a Tailwind, Not a Fear

I did not build this because I am scared of the cloud. I built it because the signals all point the same direction, and they point now.

A recent US government order took the most capable frontier models offline for every customer at once. No warning. No restoration date. One morning the most capable tier of a thing a lot of people had quietly wired into their daily work was simply gone, for everyone, by decree. That is not a knock on any vendor. It is a fact about what happens when your most important tool lives entirely on a substrate you do not control.

Look at the other signals and they rhyme. The newest models ship to a short list of trusted partners first, and the rest of us wait. The tokens an individual gets handed back are getting less performant and less predictable, quietly, in ways you only notice when the work suddenly feels heavier. Meanwhile the open and local models keep getting better, month over month, and the serving optimizations keep fitting more real capability onto hardware a person can actually own.

Put those together and the conclusion writes itself. The capability you rent is getting more conditional. The capability you can own is getting more powerful. When two curves cross like that, the move is not to wait for them to finish crossing. The move is to be standing on the right side of the line before the crowd notices there is a line.

Plenty of people are in this same boat right now, builders making the same read and the same bet. I think that is a good sign, not a crowded trade. The edge is not owning a GPU. Anyone can buy a GPU. The edge is the craft of making local actually useful, and that craft is transferable to whatever comes next.

I Planned Two Machines, Then I Built One

Here is the honest plot of the build, because the turn in it is the part worth keeping.

I reasoned my way, cleanly and confidently, to a two-machine split. The big new card would go in a new box. The older card, still perfectly good, would get relocated into a second machine, and the two would sit on the network and cooperate. It was an elegant plan. It had symmetry. I liked it.

Then I consolidated both cards into a single chassis instead, and the consolidated version is better, so let me tell you why I moved.

I was never going to tensor-parallel a model across those two specific cards. They are not matched, and splitting one model’s layers across both to run as a single larger brain was never on the table. So the supposed benefit of giving each card its own machine evaporated under inspection. What I actually wanted was to run different models on different cards at the same time, and to do that with the lowest possible latency between them. In one box I can split models across the two cards locally and never pay the network-call tax of shipping tensors or requests between two machines. Same room is good. Same PCIe fabric is better.

And the arithmetic of the second machine was lopsided once I looked straight at it. A second card pulling roughly three hundred more watts in a chassis I already have is cheap next to standing up an entire second system: another case, another power supply, another board, another set of everything, all to host a card that runs better sitting next to its sibling anyway. The elegant plan lost to the pragmatic one. That happens a lot when you actually build the thing instead of admiring the diagram. I am glad it did.

Here is what ended up in the one box, run as two independent replicas rather than one pooled brain:

Card	VRAM	Role	What it runs
RTX PRO 6000	96 GB	heavy tier, long context	a strong coder MoE at 4-bit, with room left for a big shared KV cache
RTX 5090	32 GB	fast tier, bounded work	a smaller model for high-volume, low-stakes tasks

Bandwidth Runs the Model, Capacity Only Holds It

The other early decision was memory, and it is the one people second-guess most, so let me defend it plainly.

I chose a single high-bandwidth 96GB card over the cheaper unified-memory boxes that advertise more total capacity. On a spec sheet, more capacity looks like the smarter buy. In practice it is the wrong axis to optimize. Capacity decides whether a model fits. Bandwidth decides whether it runs. A model that loads but generates at a crawl is not a model you will actually use, it is a demo you ran once. My card moves memory many times faster than those slower boxes, and that speed is the difference between a model you reach for fifty times a day and a model you reach for to prove a point.

So I bought the fast 96 and did not flinch. It has to be quick enough to live inside your actual loop, or it does not change how you work.

I Paid a Shop to Build It, on Purpose

I have been building my own machines for two decades. I could have assembled this one in an evening with a beer and good music. I paid a shop to build and validate it anyway, and I would make that call again every time.

This is not laziness, it is judgment about where my risk should sit. The card is expensive and the board is new and finicky, the kind where firmware and memory training and thermals all want a known-good bench. Paying a shop to certify that it posts, trains memory clean, and survives a burn-in is cheap insurance against a single careless seat becoming a very bad afternoon. I did the part that is actually mine, deciding what to build and what to run on it, and handed off the part that is mostly risk. Knowing which work to keep and which to give away is its own skill.

Prefix Caching Is Where the Bet Paid Off

Now the part where the thing actually worked, because a bet is just a story until something clicks.

The painful piece was never the model. It was context. Running a few agents at once felt cramped, especially on the smaller card, and the reason was not raw capability. It was that every agent dragged the same heavy scaffolding into its window. The system prompt. The skills. The shared project state. That common preamble is large, and when each of three agents pays for the whole thing independently, the shared overhead eats the runway before any agent gets room to actually think. Three agents felt like the ceiling, and it felt like a low one.

Prefix caching is what broke that ceiling open. The big shared prefix gets computed once and cached, and then it is served as a cache hit to every agent that shares it. Each agent stops re-paying for the common scaffolding and pays only for its own unique tail, the part that is genuinely different about its task. That single change turned three cramped agents into several agents each carrying real, large context, on a single card.

That is the moment the whole bet stopped being a thesis and became a tool. Not a benchmark I screenshotted. A capability I now use, that I own, that nobody can quietly throttle or revoke on a Tuesday. When local stops being a compromise you tolerate and becomes a thing that is plainly better for the work in front of you, the argument is over.

Prefix caching: the math that broke the ceiling

Several agents share one large prefix (system prompt, skills, project state). Cache it once and each agent only pays for its own unique context. Drag the sliders.

KV headroom on the card 30 GB Shared prefix (system prompt + skills + state) 16K tokens Per-agent unique context 8K tokens

Without prefix caching

20

agents, each re-paying for the prefix

With prefix caching

58

agents, prefix paid once

Same card

2.9×

more agents, for free

Illustrative. The multiplier holds regardless of the exact bytes-per-token, because the KV rate cancels in the ratio. The point is the shape: a big shared prefix that every agent re-pays for is the ceiling, and caching it once is what lifts it.

I Wrote Down What Would Make Me Wrong

Conviction without instrumentation is just stubbornness wearing a nice coat. So I wrote the trip-wires down, the specific conditions that would reverse a specific decision, before I had any emotional reason to argue with them.

If a second matched big card ever comes within reach, that changes the whole topology, because then tensor-parallel across two equals is real and the single-box layout gets rethought from scratch. If the shared-prefix cache stops reliably hitting, that gets fixed before I add a single new agent, because more agents on a broken cache is just multiplying a leak. And if the box sits idle, if I find I am not actually reaching for it, then the bet is not paying and I will say so out loud and act on it, instead of letting a quiet machine become a monument to a decision I was too proud to revisit.

If this happens	What it changes
A second matched 96GB card comes within reach	Rethink the single-box layout, real tensor-parallel is on the table
The shared-prefix cache stops reliably hitting	Fix it before adding a single new agent
The box sits idle, I am not reaching for it	Admit the bet is not paying, and act on it

Writing those down is not doubt. It is the opposite. It is how you hold a strong opinion honestly: you name the evidence that would change your mind, in advance, and then you go all in knowing exactly when you would pull back. That is the discipline that lets you bet hard without fooling yourself.

I will be straight about the uncertainty, because it is the reason to move early, not a reason to wait. I do not know exactly how the next two years break, which local models win, which serving stack wins, or which orchestration pattern outlives the others. Nobody does. That uncertainty is precisely why you want to be inside it early, with your hands on real hardware, building the judgment that only comes from running the thing. The people who understand local serving in 2028 will be the ones who were wrist-deep in it in 2026. You cannot buy that understanding later. You can only earn it now.

Build the Anvil, Let the Hammers Change

The reason all of this holds together is the layer I chose to own.

Models will churn. The best local coder I run today will be embarrassing in a year, replaced by something I have not heard of yet. Orchestration will churn too, the frameworks and the agent patterns and the serving flags, all of it. None of that is the durable thing, and I did not bet on any of it. I bet on the substrate underneath, the machine and the serving tier and the hard-won knowledge of how to make weights run fast on metal I control. That is the anvil. The models are hammers, and hammers are supposed to get swapped out as better ones show up. You do not rebuild the anvil every time you pick up a new hammer. You shape the new hammer against the same solid thing you already own.

Sovereignty, in the end, is just a floor you control, the layer that keeps answering when the rented layers above it get gated or throttled or pulled offline by an order nobody saw coming. I did not build this box to save money, and I did not build it out of fear. I built it because owning your substrate is how you stay a builder instead of a tenant, and because the engineering itself is a genuine pleasure.

So here is the principle I would hand you, the part worth keeping after the specific cards and watts are obsolete. When a capability you depend on starts living entirely on ground you do not own, build yourself a floor. Own the durable layer and let everything above it churn. Be early, on purpose, and write down what would prove you wrong. The map is not finished. That is the opportunity, not the risk.