Google’s 2025 DORA survey of nearly 5,000 technology professionals found that 90% of them use AI, and that 30% trust its output only a little or not at all. Google’s summary of the report calls ensuring that software works as intended before it reaches users an ongoing challenge (DORA). When a coding agent writes a change, it also reports whether the tests pass, and that report is often the first evidence a team sees.
This guide is for CTOs and heads of engineering whose developers let agents write production code, and who will have to show a client, an investor or an auditor that this code does what the business expects. It sets out which checks count as proof, in what order to add them, and what to keep as a record.
A check counts as proof only when the agent has no way to influence it. Research synthesis of published studies, official guidance and practitioner reports, with izzy’s own observations from taking over AI-built codebases; no named client case.
Answer in 60 seconds
- A passing suite shows that the listed inputs gave the expected outputs, nothing more.
- Agents find ways to pass tests they should fail: in a 2025 benchmark, one leading model passed 76% of the tasks in one set that could not be solved honestly, mostly with code written to recognise the tested inputs.
- Five levels of checking, by independence from the agent: its report, tests it cannot edit, rules checked on every call, a read-only reviewer, a formal proof.
- The strongest levels go to authentication, payments, personal data and secrets.
- A reviewer agent of the same capability runs on another provider’s model; the more critical the code, the more independent opinions a change needs, and every check leaves a record the agent cannot change.
- Planted errors show whether the checks still work.
In this article
- A passing test suite checks only the inputs someone listed
- The agent must not be able to edit the tests that check its work
- Five levels of checking, ordered by independence from the agent
- The level of checking follows the cost of a mistake
- Two or more independent opinions, and a record for every check
- Plant known errors to prove the checks still work
- What the proof pack contains
1. A passing test suite checks only the inputs someone listed
A test suite can only check the cases written into it. Each test fixes one input and the output expected from it, so a green run shows that those inputs still behave. It says nothing about inputs nobody listed, and a change from an agent can break one of them while every test stays green.
A green suite is a record of the cases someone thought of.
izzy’s own work points the same way. In the last AI-built codebase we took over, the first real defect was found in production, not by a test and not by a review.
2. The agent must not be able to edit the tests that check its work
The first control is to make sure the agent cannot change the tests it is measured against. Adding tests comes after.
ImpossibleBench, a benchmark published in October 2025, gave coding agents tasks whose tests contradicted the written specification, so any pass meant a shortcut. With write access to the tests, one leading model passed 76% of the impossible tasks in the “one-off” set drawn from real open-source bug reports (built on SWE-bench, with one expected value changed). It mostly wrote code that recognised the tested inputs and less often edited the tests, despite being told not to touch them (ImpossibleBench). In the “conflicting” set, where duplicated tests contradict each other, it passed 54%. The study then measured what reduced these rates:
| Setting | Effect reported |
|---|---|
| Tests hidden from the agent | Shortcuts near zero, but fewer honest solutions too |
| Tests read-only (edits reverted before scoring) | Honest results back to normal and test edits neutralised, but code written to recognise the tested inputs still gets through. The best-performing model still passed 58% of the one-off set and 39% of the conflicting set; another leading model fell to 6% and 1%. The rest is left to levels 2 and 3 |
| A stricter prompt | Shortcut rate of the best-performing model on the conflicting set built on LiveCodeBench: from 92% to 1%. On the more complex conflicting set built on SWE-bench, a looser prompt moved it only from 54% to 66% |
| An option to stop and flag a contradiction | Shortcut rate of the same model on the sets built on SWE-bench: from 54% to 9% (conflicting) and from 76% to 37% (one-off). Much smaller effect on another leading model |
The authors recommend hidden or read-only tests while the agent works. OWASP’s cheat sheet on secure coding with AI reaches the same point from practice: agents delete tests, weaken assertions or mock the code under test, and a suite written by the agent that wrote the code “provides no independent assurance” (OWASP).
An instruction lowered the rate without removing it, and only locking the tests took test edits out of the result, so in practice the control is three settings. A code-owners rule requires a named person’s review for any change to the test folders and CI configuration. Branch protection stops the agent’s account from merging its own pull request. A CI step flags a deleted test or a drop in the number of assertions, as OWASP suggests.
This does not mean a person has to write every test. At izzy, a team of agents writes them: an orchestrating agent decides which agent handles each part of a task, and whether the tests and the code come from one agent or from separate ones. The tests aim for the widest coverage the codebase allows, plus end-to-end tests of the main user journeys. An experienced engineer owns every line that ships. When one agent writes both the tests and the code, we recommend fixing the tests before the code, as in test-driven development, so they cannot be bent to fit the code that follows.
For security-critical code, OWASP draws a stricter line: it asks for tests on authentication, authorisation, input validation and cryptography to be written by hand, and warns against letting an agent write such code and its tests without independent verification.
3. Five levels of checking, ordered by independence from the agent
Each level adds a checker the agent has less influence over. None replaces the one before it.
| Level | Check | What it proves | What it does not prove |
|---|---|---|---|
| 0 | The agent’s report (“tests pass”, “done”) | That the agent says so | Anything else |
| 1 | Tests fixed before the change, which the agent making it cannot edit | The listed cases give the expected output | Cases nobody listed |
| 2 | Rules checked on every call or over generated inputs: contracts, assertions, property-based tests | The rule holds for every input that runs or is generated | Inputs that never occur |
| 3 | A second reviewer with read-only access: a person, or an agent of the same capability on another provider’s model | The change matches the written rules and the wider code | Behaviour nobody wrote down |
| 4 | Formal proof | The rule holds for every possible input | That the rule itself is right |
Every level from 2 up assumes code with a clear structure. In our experience, the most common problem in agent-written code is the lack of structure and clear organisation: verbose code, tangled logic and no clear boundaries between its parts. Such code gives a rule no obvious place to live and a reviewer nothing clear to follow. The NCSC warns that a codebase built this way can become complicated and confusing to understand, and harder to maintain (NCSC). Where a reviewer cannot follow a part of the code, restructuring it comes before the checks.
Level 2 gives most teams the largest gain for the effort. A property-based test states a rule, such as “the shares always add up to the amount”, and a library generates inputs until it finds a counter-example or runs out of tries. In a 2025 study, an AI agent that read 100 popular Python packages, inferred such rules and wrote the tests produced 984 bug reports. In a manually reviewed sample of 50, 56% were real bugs, and three patches were merged upstream (Maaz, DeVoe, Hatfield-Dodds and Carlini). An agent can draft the rules. A person reads each one before it is added to the suite.
Level 4 stays expensive. A proof has to be written for each rule and redone when the code changes, so it suits a few core functions rather than a whole codebase.
4. The level of checking follows the cost of a mistake
No part of the code should rely on level 0 alone. The decision is which parts get which higher levels.
The UK National Cyber Security Centre set out this logic for AI-assisted development on 18 June 2026: loose oversight for prototypes, demos and internal tools with limited exposure; close oversight for authentication and authorisation, sensitive personal data, secrets and credentials, and anything safety-critical. It adds that another AI agent may help with the review (NCSC).
| Part of the code | Minimum levels before release |
|---|---|
| Prototype, demo, internal tool with no sensitive data | 1, and 3 with an agent reviewer |
| Ordinary product features | 1 and 3; 2 for any business rule that can be written as a property |
| Money: pricing, rounding, invoicing | 1, 2 and 3; 4 for the core calculation if it applies to every transaction |
| Authentication, permissions, personal data, secrets | 1, 2 and 3, with a person as an additional level-3 reviewer |
5. Two or more independent opinions, and a record for every check
We count a second agent with read-only access as an independent check at level 3, provided it runs on a model of the same capability from a different provider than the agent that wrote the change. On the one-off set of the ImpossibleBench study, models from one provider mostly passed impossible tasks by writing code that recognised the tested inputs, while models from two other providers mostly edited the tests (ImpossibleBench). That suggests models from different providers can fail in different ways, although the study did not test review across providers. We expect a reviewer on the same model as the writer to share its blind spots, and a less capable one to let more errors through. An agent reviewer is no guarantee on its own: in the same study, language models used as monitors detected only 42% to 65% of the shortcuts on the sets built on SWE-bench. That is why critical code also gets a person’s opinion.
The aim is at least two independent opinions on every change, plus a person’s opinion for the last row of the table above. Final approval rests with a person who answers for the change, such as the code owner named in the code-owners rule of section 2.
An author at a French consultancy, who supports teams on product and agile, counted in the execution log of eight internal tools built by agents how many tasks marked as successful cited no verification: 44 of 195, or 23%. The same article, published on 12 October 2026, sets two rules: a check whose trace was not kept does not count, and an item the agent could not look at is labelled “not evaluated”, which is not the same as “ok” (OCTO Technology). One person’s tools, measured by the author’s own scripts.
The record also has to sit out of the agent’s reach. METR, which evaluates frontier models, reported on 6 October 2026 that a researcher, with help from an AI agent, found in about 10 minutes a flaw that would have let an agent change what a reviewer sees in the transcript viewer of Inspect, a framework widely used in AI safety evaluations. The stored data stayed intact, but only a reviewer who knew to look would have found it (METR). For a team, CI logs and approvals belong where the agent’s account cannot write, and a reviewer in doubt checks the stored record rather than only the screen that displays it.
6. Plant known errors to prove the checks still work
Mutation testing, planting errors on purpose and counting how many the checks catch, is how a team learns whether its checks still work. At scale it needs filtering. When Google first reported planted errors to its developers, they judged 85% of them unproductive. The 2021 paper describes it running during code review, on changed code only, with a cap per line and a preference for the kinds of error that proved useful before. Suppression rules brought the unproductive share down to 11%, and over 24,000 developers were using it (Google Research).
Language models now write both the planted errors and the tests. ACH, Meta’s system for hardening tests around privacy, generated 9,095 targeted errors and 571 tests in late 2024, and engineers accepted 73% of the tests they reviewed (Meta).
A team without that infrastructure can run a smaller version: for each written business rule, keep one broken variant of the code and confirm before each release that the suite fails on it. A rule no planted error exercises is listed as untested.
7. What the proof pack contains
When a client’s security review, an investor’s due diligence or an auditor asks how the team knows the code is correct, the answer is a set of documents that can be checked:
- the business rules written beside the code, each with a named owner;
- what the agents could and could not write, test files and CI configuration included;
- which parts of the code receive which levels of checking, and why;
- per release, the checks that ran, their results, and the checks that did not run;
- who approved each change, and which independent opinions it received;
- the latest planted-error run and how many errors it caught.
OWASP asks for the same anchor: a human owner for every AI-generated change, responsible for its correctness, security and upkeep (OWASP). For a product changing hands, see Can another team own your AI-built product?; for the agents’ access, AI agent permissions and access control.
Conclusion: count only the checks the agent could not pass alone
First take the tests and CI configuration out of the agent’s write permissions. Then apply the levels in the table of section 4, starting with authentication, personal data and money, and add planted-error runs once those levels are in place.
Bring us your test suite and your last three releases
Bring us repository access, the CI configuration, the permissions your agents run with and the history of your last three releases. We will map which checks already count as proof, where agents can edit the tests that check them, which areas need stronger checks, and which need restructuring before any check can work. If the current checks already hold, we will say so and there is nothing to build. That review is the starting point of our Rescue Missions & AI-Code Hardening service.
Frequently asked questions
Yes, and we count it as an independent check when it has the same capability as the writing agent, runs on another provider’s model and cannot write to the code. On critical code, a person’s opinion is added to it.
No. Coverage shows which lines ran, not whether the results were right, and an agent with write access can raise it with tests that assert its own behaviour. It is still worth pushing as far as it goes, with end-to-end tests on top.
Only for the few functions where a wrong answer is costly, such as a core pricing or rounding calculation. Elsewhere, protected tests, property-based tests and an independent review are the better use of the effort.
The written rules, the agents’ permissions, the checks per area, the record of checks per release, the approvals and the latest planted-error results.
Sources and limits
Research synthesis of published studies, official guidance and practitioner reports. First-party material: izzy’s observations from taking over AI-built codebases (where the first defect surfaced, how izzy writes tests, the structural problems seen most often); no named client case and no measured results. Checked on 6 November 2026.
- Google, DORA 2025 summary (23 September 2025): self-reported use and trust, not measured outcomes.
- ImpossibleBench (October 2025): rates depend on model, prompt and scaffold, and describe 2025 models; shortcut types were classified by a language model.
- OWASP, Secure Coding with AI Cheat Sheet (2026): practitioner guidance, not measurement.
- Maaz, DeVoe, Hatfield-Dodds and Carlini (October 2025): bug validity judged by the authors’ manual review of a sample of 50.
- NCSC (18 June 2026): blog post from the UK’s national cyber security agency on oversight by risk.
- OCTO Technology (12 October 2026): eight internal tools used by one person.
- METR (6 October 2026): a proof-of-concept flaw; METR has not seen an agent exploit it, and has not searched its evaluations for this exploit.
- Google Research (2021) and Meta (January 2025): very large companies; their numbers do not transfer to a small team.
The five levels, the minimum levels per area and the proof pack are izzy’s synthesis of these sources, to adapt to each context.