RAW
THINKING.
Not a blog. System designs, theses, and the patterns you only see from inside the Lab, shared before they're polished.
Why can you keep a 500 day Duolingo streak and still not speak the language?
Because the streak certifies attendance, not understanding. The habit engine is the best thing Duolingo has and it is pointed at the wrong measurement. The fix is not to demand harder answers, it is to build a supported progression where the scaffold fades and the tick is earned for learning rather than completion.
Why your multi-source report hallucinates, and the architecture that fixes it
The most common way people try to build a report generator is to pour every source into one model and ask for the answer. Over enough sectors and steps it drifts, invents figures, and cannot tell you why. The fix is not a bigger model. It is an architecture.
Inside Project H: measure hard, surface nothing
Project H measures continuously and rigorously, and guarantees by construction that no number it computes ever reaches a human surface. Here is the architecture that makes that possible, and why nearly every decision follows from that one inversion.
The AI system that gets cheaper the more it runs
People keep asking me what the AI system of the future looks like. Here is mine, drawn out in full. Seven layers, two feedback loops, and one property almost nobody designs for: it gets cheaper every time it runs.
Will a bigger model close the gap in your AI system?
The gap you hit in production is that the model does not know your data, your process, your edge cases, and none of that was ever in anyone's training set. A bigger model trained on more of the public internet does not fix a problem made of your private context. This is a bet, and I will tell you where it might be wrong.
Why did our AI pilot never reach production?
Six autopsies, a triage tree in which 'kill it' is a legitimate verdict, and a hostile read of the failure statistic the whole industry quotes without opening.
Which parts of an AI system should be deterministic and which should use an LLM?
Berkeley posed the question in February 2024 and left it open. Two years of vendor blogs answered 'a spectrum'. Here is the forced choice, the file that enforces it, and the throughput bill for holding the line.
Which business processes should not use AI?
Plenty of people say don't automate everything. Nobody names a case, because naming one means checking it. Six functions, five named failure modes plus the one nobody names, four questions that decide it, and the corrections to three disasters everybody miscites as AI.
Where has SEO gone in the age of AI answers?
Classic search optimisation still works and still arrives fast. But the target is splitting in two, and the new half is not won on your own page, it is won by being the source a model reaches for, which almost nobody is optimising for on purpose.
How do you cap OpenAI spend per customer?
The budget setting is an email. The only native hard stop kills your whole organisation. The cost API buckets by the day. Four controls that hold, and the counter that enforces them.
What does formal verification actually prove about seL4?
seL4 carries the strongest correctness proof any general-purpose kernel has ever had, and the most valuable page the project publishes is the one listing what the proof assumes rather than proves.
What does an RFC 3161 timestamp actually prove?
It proves an upper bound, not a moment, and half the TSAs you'd reach for omit the field you need to compute it. Six live tokens, pulled and dumped.
How do you validate an AI model under GxP?
The EU already answered the architecture question and almost nobody has read the answer. Static, deterministic, behind a boundary, and a human still signs, with the clause numbers, a worked temperature excursion, and the first FDA warning letter for letting an agent write your SOPs.
How do you test a prompt change before shipping it?
Vendors sell a drift dashboard. The number that decides the deploy is the one the dashboard averages away, a single rubric inverting while the total sits flat.
How do you make it impossible for a system to output the wrong thing?
Prompt guardrails are a request. Removing the surface is a guarantee. What it takes to build a system where the output you fear cannot be represented, let alone emitted.
Why is my new site getting impressions but no clicks?
Because showing up and ranking are two different fights, and the first one is won in days. Two of my own sites went from zero to five figures of weekly impressions in under a fortnight, here is the search-console data, the method, and why the low click count is the wrong thing to panic about.
Who is building the shared components for AI?
Everyone is building the tenth orchestration framework. Almost nobody is building the reusable, trustworthy components those frameworks are supposed to wire together, and that missing layer is the thing standing between us and apps that spin up for a task and vanish.
How do you make an LLM system reproducible when the model is non-deterministic?
Temperature zero is now a 400 error on Anthropic's newest models, and the model that answered in March retires before your retention floor does. The engineering that outlives both sits in storage.
RAG vs fine-tuning vs long context: which do I actually need?
Name the thing that changes and the argument ends. Knowledge changes, retrieve. Behaviour changes, fine-tune. Neither, and the corpus fits, use the window. With the dated arithmetic, the cache modelled, and the multiple everyone repeats corrected.
What questions should you ask an AI vendor about their architecture?
Six questions. The published checklists ask none of them, I counted. Which of the standard RFP questions are theatre, and the two-week protocol that replaces them.
How do you prove an AI decision was not altered?
Every governance page tells you to keep a tamper-evident record. None says who holds the signing key, whether it can leave the machine, or who witnesses the log. The standards landed in June 2026. Here is the whole chain, and the one link nobody has built.
Can you run production AI on a local model?
Small models now score what needed thirty billion parameters two years ago. If your data architecture does the heavy lifting, a model on your own hardware covers a surprising amount of production, and the smaller the model, the more the architecture has to earn its keep.
How does one person run two dozen products?
Not by working harder. By building the infrastructure once and paying almost nothing for each product after it, and by accepting a constraint most teams never hit.
Should an LLM do calculations?
A wrong total looks exactly like a right total. The pattern that keeps model-generated arithmetic off invoices and dosages, and the evidence that 'just give it Python' is only the second-best answer.
What are the most common MCP server design mistakes?
Twenty-four products, twenty-four MCP servers, and the same six mistakes in nearly all of them. Tool count is the one everybody names and the one that matters least.
What are LLM-as-a-judge best practices?
One prompt scoring eight things at once returns a number nobody can check. Many narrow judges that must quote the span they are judging can be checked in seconds, and a calibration harness tells you whether the product got worse or the judge did.
Hash chain vs Merkle tree for audit logs: which one do you need?
A chain costs O(n−k) because the verifier replays every event between the record and the commitment. A tree costs O(log n). Crosby and Wallach put the gap at 800 MB against 3 KB, and neither structure stops you rewriting entry five.
Is it a hallucination or a retrieval failure?
Everyone says check retrieval first. Nobody says what to do when the check comes back ambiguous, so here is the position-controlled replay, the seven branches, and the decision tree that ends the argument.
How does GPS correct for relativity?
Two corrections, two places, one constant printed to ten significant figures in a US Space Force specification, and why your AI system's known bias should never be a dashboard's problem.
Should every product ship an MCP server?
For thirty years software assumed a human would operate it. The thing arriving to use your product increasingly isn't one, and if it can't get in, your product may as well not exist to it.
How do you use event sourcing for LLM applications?
The pages that own this query are vendor content. Here is the pattern as actually run, the command/event split, corrections without data loss, and the three operations everybody keeps confusing.
What is the difference between embeddings and a vector database?
An embedding is a representation. A vector database is storage. Teams buy the second to fix problems that live in the first, and no index can beat exact search over the vectors you already have.
How do you measure something without teaching people to fake it?
Name the quality in the question and you have contaminated the answer. The rubric design that separates a genuine signal from a well-trained echo, and why unprompted evidence is worth more than the same words after a prompt.
Does AI take your job or move it up a level?
Mostly it moves it, up a rung, from doing the task to directing and judging it. Same domain, higher altitude, same same but different. The genuinely hard part is the one nobody in either camp will say: it is the bottom rung that erodes, and you cannot climb a ladder with no bottom.
Do self-correcting agent loops actually work?
The loop is being sold as the fix for everything AI gets wrong. It works under exactly two conditions, almost nobody checks the second one, and when both are missing a loop is a more expensive way to get worse output than a bad offshore team.
How do you use content-addressable storage for AI records?
Two logically identical records, two different digests, and an audit trail that quietly stops meaning anything. Canonicalise the encoding, never the content, and know what a hash does not buy you.
Is it risky to buy software from a one-person company?
Yes, and so is buying from the incumbent, you just cannot see it. The paperwork that makes a solo vendor auditable, and the seven places where one person is genuinely worse.
Should you build a product to solve your own problem?
Being your own first user removes the translation layer that ruins most products. It also hides the one question that decides whether anyone else will pay.
What should an AI decision audit trail contain?
Collibra, PwC and ISACA all tell you to keep one, and not one of them names a field. Here is the record, the properties, the law each field answers to, and a schema you can fork.
Are agent memory benchmarks reproducible?
I ran them to decide whether to keep building my own memory layer. I came back unable to rank anything, and with one of the field's most-quoted numbers attached to the wrong model.
When should you not use an LLM at all?
Almost nobody with an incentive will answer this, because the answer costs the commercial internet money. Five jobs a model is doing that it should not be doing, the arithmetic that settles the argument, and the four gates I run before anything gets a token budget.
What is the Replay Test?
One question that separates an AI system from a demo with error handling. It takes about a minute to apply and most production systems fail it.
How do you build a tamper-proof audit log without a blockchain?
Consensus settles disputed ordering between parties who will not accept an operator. Your log has an operator and nobody arguing. Five primitives, three questions, and the regulator that stopped treating WORM as the only answer.
How do you build RAG over FAR Part 25 and EASA CS-25 certification regulations?
One backbone doing four jobs, ingest and snapshotting, authority tiering, point-in-time resolution, a refusal path. The model was never the hard part. A regulation with a date on it is a different document from the same regulation today.
What does a production-readiness audit actually check?
Seventeen categories and a synthesis pass. Here is the whole taxonomy, what each one is really looking for, and the failure modes that survive every checklist I have ever been handed.
How do you do process mining without event logs?
You don't. But the diagram library nobody has opened since the last reorg is a graph, and diffing it against itself surfaces contradictions a discovery tool has no input to see.
What is a Merkle tree second preimage attack and how does domain separation stop it?
Your inclusion proof verifier returns true for something that was never a leaf. Failing test first, two-line fix second, and the reason the fix works.
How do you make agent tool calls idempotent?
A retry sent the email twice, and the guarantee belongs to the deterministic shell around the model, never to the model's reasoning. Here is where the key comes from, and the half of the problem it does not solve.
How do you implement EU AI Act Article 12 logging?
The regulation tells you the log must exist and not one word about how to stop it being rewritten. Here is the mechanism, the named failure modes, and a checklist you can take into Monday's meeting.
Agent or Process?
Most people reach for an agent when they need a process, and a process when they need judgement. Here's the fork, and when to put a human on it.