AI code quality is how well code written with AI coding assistants holds up after it merges. That means it works in production, passes review, is tested where it changed, has no known security flaws and isn't rewritten a few weeks later. You measure it with five signals and keep it high with gates that every change has to pass.
Why does AI-written code need its own quality metrics?#
AI makes code faster than it makes code correct, and the usual speed metrics hide that gap.
The evidence comes from several directions:
- Security. Veracode's 2025 GenAI Code Security Report tested more than 100 large language models. AI-generated code introduced security vulnerabilities in 45% of cases, and the models missed 86% of cross-site scripting threats.
- Trust. In the 2025 Stack Overflow Developer Survey, 84% of developers said they use or plan to use AI tools. Only 29% trust AI output to be accurate, down from 40% in 2024.
- Perception. In a randomized controlled trial by METR, experienced open-source developers believed AI made them 20% faster. In fact they were 19% slower. If people can't judge their own speed, they can't judge their own quality either.
- Team effects. The 2025 DORA report surveyed nearly 5,000 technology professionals. It found that AI doesn't fix a team: it makes strong teams stronger and makes a struggling team's problems worse.
Maintainability is also slipping. In GitClear's 2026 code-change research, moved code (a sign of refactoring) fell from 21% of changed lines in 2022 to 13% in 2023 and 3.8% so far in 2026. Over the same years, copy/pasted code rose from 9.4% to 15.7%. Duplicated code can run fine today and still turn every future fix into a hunt for copies.
Which signals show AI code quality?#
Track five signals on every change. Then check them against two outcome metrics from DORA.
| Signal | What it tells you | How to calculate it | Where the data lives |
|---|---|---|---|
| Defect escape rate | How many problems get past your gates | Defects found in production ÷ all defects found, per period | Issue tracker, incident log |
| Review findings per change | Whether reviewers are catching real problems | Blocking review comments per merged PR, grouped by type (logic, security, design) | Code review tool |
| Changed-line coverage | Whether the new code is actually tested | Lines changed in the PR that tests execute ÷ lines changed | CI coverage report (diff coverage) |
| Security findings per change | Whether AI is adding vulnerabilities | New high or critical findings from static analysis, dependency and secrets scans, per merged PR | Security scanners in CI |
| Short-term churn | Whether code was right the first time | Lines rewritten or reverted within a fixed window (for example 14 or 21 days) ÷ lines added | Git history |
Next to these five, keep the delivery outcomes. DORA's metrics history explains that change fail rate counts the share of deployments that need a hotfix or rollback. In 2024, DORA added a fifth metric, deployment rework rate, to measure unplanned rework more directly. Your five signals are leading indicators. These two outcomes tell you whether those indicators are right.
Three practical rules make the numbers useful:
- Tag AI-assisted changes. A PR label or commit trailer lets you compare AI-assisted and other work on the same signals. Without the tag you only see averages.
- Divide by number of changes, not by time. If AI doubles the number of PRs, total security findings can double while the rate per change stays flat. Rates show whether quality is actually changing.
- Measure per team and per service, never per developer. Leaderboards teach people to game the metric.
How do you read the signals together?#
Read them in pairs. Each signal can be gamed on its own, but it's hard to game two that check each other.
- Coverage rises, escape rate doesn't fall. The tests run the code but check very little. This often happens when an AI writes the tests and nobody reads the assertions.
- Review findings drop, churn rises. Reviewers are approving too quickly, and the rework shows up later as rewrites.
- Security findings per change are flat, but dependency count is climbing. Your scanners may not cover new packages. Check what each scan actually inspects.
- Change fail rate is low, deployment rework rate is rising. Releases aren't failing outright, but teams keep shipping unplanned follow-up fixes.
What gates keep AI code quality high?#
Three gates in the pipeline: tests, security checks and senior engineer review. They apply to every change, whether a person or an AI wrote it, and each release leaves behind evidence of what ran.
This is the standard behind Calidad's Quality Engineering service: "Tests prove it works. Security checks prove it's safe. A senior engineer's review proves it's right." Here's how each gate works in practice:
- Keep changes small. A reviewer can reason about a 200-line diff. A 3,000-line AI-generated diff gets skimmed.
- Test gate. The full suite must pass, and changed-line coverage must meet a threshold your team sets from its baseline. Treat AI-generated tests like any other code: someone reviews what they assert.
- Security gate. Static analysis, dependency checks and secrets scanning run on every PR. Confirm every new package exists and is approved, because AI assistants can suggest packages that don't exist. Block merges on new high-severity findings.
- Review gate. A senior engineer approves every change. Changes that touch authorization, payments or personal data go to a deeper review, because those are exactly the areas where AI code reviewers miss problems.
- Release evidence. Store what ran, what it found and who approved it with each release, so an incident review starts from facts.
For a step-by-step guide to each gate, see how to verify AI-generated code before production.
How do you roll this out without slowing the team?#
Measure first, turn on one gate at a time, and set targets from your own baseline rather than an industry number.
If you switch on every gate at once, a team gets a wall of red builds and starts looking for ways around them. A phased rollout of about two months works better:
Along the way, agree on shared rules for AI tool use: which assistants are approved, what has to be reviewed by hand, and what counts as done. Calidad's dedicated teams agree on these rules up front and check them every sprint, because AI-assisted output varies from engineer to engineer and review can become the bottleneck.
Three things are worth avoiding:
- Percentage of code written by AI. It measures adoption, not quality.
- Lines of code or PR count as a productivity score. AI inflates both.
- Gates that can be skipped "just this once." If you can override a gate without leaving a record, it isn't really a gate.
For the pipeline practices that keep changes small enough to verify, see our guide to continuous delivery practices.
Conclusion#
AI code quality only shows up when you measure it. Track defect escape rate, review findings, changed-line coverage, security findings and short-term churn for each change. Read them in pairs and check them against change fail rate and rework rate. Then enforce the same three gates on every change, whoever or whatever wrote it, and keep evidence for each release.
Calidad starts with a paid pilot: one pod, one outcome, with success criteria agreed up front. Talk to an engineering lead →
Frequently asked questions
What is a good defect escape rate for AI-assisted teams?
No universal benchmark exists, so take your own baseline over the first few weeks and aim to bring it down from there. Compare AI-assisted and other changes within the same team. If AI-assisted changes escape more often, tighten the gate where those defects should have been caught.
Should AI-generated code have a higher test coverage requirement than human-written code?
Use the same threshold for both, applied to changed lines rather than the whole codebase. The difference is review: AI-generated tests can run the code without checking much, so a reviewer should read their assertions. The pairing to watch is coverage going up while escape rate stays flat.
Can an AI code reviewer replace senior engineer review?
No. AI reviewers can flag common patterns like injection and hard-coded secrets, but they tend to miss authorization logic and business rules. Use them as an extra check before human review, and send changes that touch authorization, payments or personal data to a senior engineer with domain knowledge.
How do we know which changes were written with AI?
Ask engineers to add a PR label or commit trailer when an assistant wrote a meaningful part of the change. It doesn't need to be exact: a consistent tag is enough to compare churn, review findings and escape rate between groups. Use the tag to learn, never to rank individual engineers.
What is short-term code churn and why does it matter for AI code?
Short-term churn is the share of newly added lines that get rewritten or reverted within a fixed window, such as 14 or 21 days. High churn means code merged before it was right, which happens when AI output is accepted without enough thought. Rising churn alongside falling review findings is an early warning that reviews are too light.
- AI code quality
- code review
- software quality metrics
- DevSecOps
- DORA metrics
Newsletter
Get new articles by email
A short monthly round-up of what we've written on engineering pods, quality and AI. No spam.
