A production-readiness audit checks seventeen distinct categories and then runs a synthesis pass over the findings. The categories are architecture, security, compliance, testing, dependencies, API readiness, infrastructure, data handling, performance, observability, resilience, runbooks, environment config, database, accessibility, code quality and documentation. The eighteenth pass, summary, audits nothing — it reconciles the other seventeen and produces the verdict. That distinction matters more than the list.
Ross Jones — Founder, The Hopium Lab. Last modified 21 July 2026.
This is the taxonomy I run against my own portfolio before anything ships. It went to its current eighteen-pass shape on 15 April 2026. I am publishing the whole thing because a checklist you cannot see is a checklist you cannot argue with, and the arguing is where the value is.
Why seventeen categories and not one big checklist?
Because a single list lets a strong area hide a weak one. Run one hundred and forty mixed checks, score 88%, and the seventeen failures cluster invisibly — you ship with an untested payment path and a green dashboard.
Scoring per category makes clustering visible. A repository can hold excellent architecture and no observability at all, and that combination is far more dangerous than a mediocre score everywhere, because nobody will find out until production tells them. The category is the unit that makes the shape of the risk legible.
A single aggregate score is the most expensive number in engineering. It averages away exactly the thing you needed to see.
What is each category actually looking for?
Each category asks one question that the others cannot answer.
| Category | The question it answers |
|---|---|
| Architecture | Do the boundaries hold, or does everything import everything? |
| Security | What is exposed, and what happens when a credential leaks? |
| Compliance | What obligations attach to this data, and can you evidence them? |
| Testing | Is the untested code the code that matters? |
| Dependencies | What did you install, who controls it, and what runs on install? |
| API readiness | Versioning, contracts, error shapes — can a consumer depend on this? |
| Infrastructure | Is it reproducible, or did somebody click it into existence? |
| Data handling | What personal data moves through, and where does it stop? |
| Performance | What falls over first, and at what load? |
| Observability | When it breaks at 03:00, can you tell what broke? |
| Resilience | What happens on partial failure — retry, degrade, or cascade? |
| Runbooks | Can somebody who is not you operate it? |
| Environment config | What differs between staging and production, and who knows? |
| Database | Migrations, indexes, and the query that is fine until it is not. |
| Accessibility | Who is structurally excluded from using it? |
| Code quality | Is the next change cheap or expensive? |
| Documentation | Does the knowledge exist outside one person's head? |
Seventeen questions. Then the synthesis pass reconciles the answers, because a finding in dependencies and a finding in security are frequently the same finding wearing different clothes.
What is the second pass for, and what is it not?
The second pass exists because a first-pass finding is a hypothesis, not a fact. A checker that only generates findings generates confident nonsense at volume, so every finding gets challenged before it counts and either survives with evidence attached or is struck.
Be precise about what that is and is not. It is an adversarial review stage inside my own tooling — the same operator, a different pass, arguing with the first result. It is not independent assurance. Independence in this context is an accredited property with a specific meaning under conformity-assessment standards, and I am not an accredited body. Anyone describing their own second look as independent is either confused or hoping you are.
An unchallenged finding is an opinion with a file path attached. The challenge stage is what converts it into evidence.
Which failures survive every checklist?
Four failure modes survive the checklists I have been handed, and every one of them is structural rather than technical.
The green dashboard. Observability exists, the dashboard is green, and nothing alerts on the failure that actually occurs — because the metric was chosen for availability rather than for the thing customers experience. Checked under observability, and it fails more often than any other single item.
The dependency you did not install. Your direct dependencies are fine. Something four levels down runs a lifecycle script on install. In May 2026 a malicious postinstall reached my own repositories through exactly this path — one line that fetched a binary and executed it. Detection, remediation across every repository and a standing policy took a day; the policy is now that no lifecycle script may download or execute anything, ever. Checked under dependencies, and almost nobody checks it before it happens to them.
The environment that only exists once. Production was configured by hand in 2023 by somebody who has left. It is not in code, the differences from staging are undocumented, and the first person to discover them will do so during an incident. Checked under environment config and infrastructure.
The runbook written by the person who does not need it. Every runbook is legible to its author. The test is whether somebody else can operate the system at 03:00 using only that document, and the honest answer is usually no. Checked under runbooks.
None of these are exotic. All of them ship.
How should you use this taxonomy?
Take it and run it yourself — that is the point of publishing it. Score each category red, amber or green on the system you are most worried about, then look at the shape rather than the total.
PRODUCTION READINESS — 17 CATEGORIES + SYNTHESIS
The Hopium Lab · v1.0 · 21 July 2026 · take it, fork it, argue with it
Score each R / A / G. Do not average them.
STRUCTURE [ ] Architecture [ ] Code quality [ ] Documentation
EXPOSURE [ ] Security [ ] Compliance [ ] Data handling
CORRECTNESS [ ] Testing [ ] Dependencies [ ] API readiness
OPERABILITY [ ] Infrastructure [ ] Environment config
[ ] Runbooks [ ] Observability
BEHAVIOUR UNDER [ ] Performance [ ] Resilience [ ] Database
REACH [ ] Accessibility
SYNTHESIS [ ] Do any two findings share a root cause?
[ ] Which single fix closes the most categories?
[ ] What is the one thing you would not want asked in an incident review?
THE RULE: two reds in OPERABILITY is a launch blocker regardless of every
other score. You cannot run what you cannot see and cannot restart.
The rule at the bottom is the one worth keeping. Structure problems are expensive later. Operability problems are expensive at three in the morning, on the day, in front of everyone.
What does a good verdict look like?
A verdict is one word, on the first page, followed by the reasoning — not a score out of a hundred and not a traffic-light grid the reader has to interpret.
Percentages invite negotiation, and negotiation is how an amber becomes a green in a meeting. Ship or do not ship, and then the findings, each with a file path and a named failure mode, ordered by what happens if you ignore it. If a finding cannot name the consequence, it is a preference, and preferences do not belong in an audit.
Ross Jones, Founder, The Hopium Lab. Last modified 21 July 2026. The taxonomy above is the one I run; the tooling that runs it is my own and is not an accredited assurance service.