The Compounding Stack is a reference architecture for AI that runs on local models and gets cheaper the more it runs. Governed data sits at the base, a small local model turns a request into a structured query over it, agents run defined processes assembled from a shared component bank, and every execution feeds back into that bank and into the data layer, so each loop is smaller than the last. It is the AI system I would build if the constraint were reliability and cost rather than access to the biggest model, and it is the shape most production AI is quietly converging on.
The diagram above is the whole thing. It is interactive: tap any layer to open it, and read it from the bottom up, because that is the direction the system is built in.
Ross Jones, Founder, The Hopium Lab. Last modified 4 August 2026.
What problem does The Compounding Stack solve?
Two problems that almost every current AI architecture has, and that get worse the more you rely on it.
The first is cost. If every meaningful action calls a metered frontier API, then the system gets more expensive the more useful it becomes. Success is punished. You end up rationing the thing you built to help people, because scale is the same word as bill.
The second is reliability. In most systems, the thing keeping the output correct is a prompt, an instruction written in English and hoped to hold. That is not a guarantee, it is a wish with good intentions. The moment a model update, an edge case, or a longer context arrives, the wish breaks quietly and you find out from a user.
The Compounding Stack inverts both. Cost falls as reuse rises, because execution returns reusable parts to a bank the next run assembles from. And correctness lives inside the components as enforced behaviour, not in a comment above them. The architecture does the work the prompt was being asked to do on trust.
Why does it run on a local model?
Because when the architecture does the lifting, the model does not have to be large.
A small local model is enough to turn a request into a structured query when it reasons over exact, governed data instead of guessing at it. The weaker the model, the more the layers around it earn their keep, and that is a feature, not a compromise. Running locally buys three things a hosted frontier call cannot: the data never leaves your control, the marginal cost of a call is electricity rather than a metered token, and the latency is a round trip to your own hardware.
This is the practical half of two arguments made elsewhere on the Whiteboard: that a bigger model will not close your gap if the gap is data and architecture, and that you can run a surprising amount of production AI on a local model once the structure around it is right.
What are the seven layers?
Seven layers, read from the base up. Each one delivers a single property to the layer above it.
| Layer | What it is | What it delivers |
|---|---|---|
| 01 Data | Integri.us: unified, governed data products | The foundation everything reasons over, and where results return |
| 02 Query | A lightweight local model | A structured query over exact context, not a guess |
| 03 Components + Code | The shared bank, guarantees enforced inside each piece | Reusable parts that carry their own safety |
| 04 Skills | A discoverable agent skill base | The vocabulary of actions processes are built from |
| 05 Orchestrate + Flow | Agents running processes you define, a loop | Defined execution, assembled from components |
| 06 Execute | FLOW, the system of record | A tracked, replayable run, and the start of the feedback |
| 07 Render | A local model generating an ephemeral UI | The interface a task needs, then thrown away |
The layers are not a menu. Each depends on the one beneath it, which is why the render at the top can be assembled on demand and thrown away safely: it inherits a guarantee from every layer below it.
Why does it get cheaper the more it runs?
This is the point of the whole architecture, and it is the two orange arcs on the diagram.
Every execution feeds two things back. The code that ran returns to the component bank, so the next loop that needs something similar assembles from parts that already exist rather than building them again. And the results return to the data layer, so the system starts its next run warm, reasoning over what it already learned instead of cold.
Put those together and reuse compounds. A process you have run something like before is smaller this time, and smaller again the time after. Token cost and latency are not things you tune down with a setting, they fall out of the architecture as a consequence of reuse. That is the difference between a system that gets more expensive as it scales and one that gets cheaper.
It is also why the agent loop actually works here where it fails elsewhere: the loop is bounded by defined processes and shrinks with reuse, instead of grinding open-endedly and burning tokens on iterations a decent process would have avoided.
What makes it reliable?
Guarantees that are structural, not written in English and trusted.
Each component in the bank carries its safety inside it: retrieval, memory, audit, a deterministic shell where the output has to be reproducible. The correctness is enforced by the code, so it cannot be forgotten in the next feature request the way a prompt instruction can. Above that, FLOW is the system of record: every execution is tracked, replayable and auditable, so when you need to answer "why did it do that," the run is there to replay rather than reconstruct.
A probabilistic model at the front, a deterministic shell behind it. Embrace the messiness of language at the door, and enforce strict, measurable governance behind it. That split is what lets the system use a small, imperfect model and still behave predictably.
Where does The Compounding Stack not apply?
It does not apply to genuinely frontier reasoning, and pretending otherwise would be dishonest.
Some tasks need the largest model available, and the honest move is to route those out to one rather than insist the local model can cover them. The architecture is built to make that routing cheap and rare, not to deny it exists.
It also earns nothing on genuinely novel, one-off work. The compounding only compounds where there is reuse, so a task you will run exactly once, that resembles nothing you have run before, gets none of the benefit and carries all of the setup. And the base assumption is that you control your data. Where the data is somebody else's, ungoverned, or arrives too fast to structure, the foundation is missing and the layers above it wobble.
Use it where the work repeats, the data is yours, and reliability matters more than raw frontier capability. That describes most real production systems, which is the point.
The transferable core
The architecture is a lot of parts, but the idea underneath it is three rules.
THE COMPOUNDING STACK, THE TEST
The Hopium Lab · v1.0 · 4 August 2026 · take it, fork it, argue with it
1. DOES THE SYSTEM GET CHEAPER OR DEARER AS IT SCALES?
[ ] Does a successful run leave behind a reusable part, or nothing?
[ ] Is the next similar loop smaller than the last, or the same size?
2. IS CORRECTNESS A GUARANTEE OR A WISH?
[ ] Is the safety enforced inside the component, or written above it?
[ ] Could a prompt change silently break it, and would you find out?
3. IS THE MODEL DOING THE WORK, OR THE ARCHITECTURE?
[ ] Does the model reason over exact, governed data, or guess at it?
[ ] If you swapped in a weaker model, what breaks?
THE TEST: if your system gets more expensive and less predictable the more
you use it, you have built the normal thing. This is the other thing.
Most AI systems fail the first question. They are built to call out to a frontier model on every meaningful step, which means cost scales with success and correctness scales with luck. Ask instead whether each run makes the next one cheaper, and whether the guarantee is in the code or in a comment, and the whole design changes shape.
Ross Jones, Founder, The Hopium Lab. The Compounding Stack is applied in Integri.us and FLOW.