Calidad Infotech
    Insights

    Is QA Dead in the Age of AI Coding?

    AI writes much of the code now, but QA isn't dead. The job shifts from manual bug hunting to designing the tests, gates and release evidence every change must pass.

    · 7 min read · Calidad Infotech

    No, QA isn't dead. What's ending is manual checking at the end of a sprint. When AI writes much of the code, someone still has to prove each change works. The job moves from finding bugs by hand to designing the tests, review gates, security checks and release evidence that every change must pass before it ships.

    Is QA dead now that AI writes so much of the code?#

    No. AI has increased the amount of code that needs checking, and the research shows that unchecked AI output makes delivery less stable.

    Look at what's happening on engineering teams:

    • Adoption is high, but trust is low. In the Stack Overflow 2025 Developer Survey, 84% of developers said they use or plan to use AI tools. 46% said they don't trust the accuracy of the output, up from 31% the year before.
    • Stability drops. Google's 2024 DORA research estimated that a 25% increase in AI adoption goes with a 7.2% decrease in delivery stability and a 1.5% decrease in throughput (RedMonk summary).
    • Security flaws get through. Veracode's 2025 GenAI Code Security Report found that AI-generated code introduced security flaws in 45% of its tests.
    • Maintainability gets worse. GitClear's analysis of 211 million changed lines found more duplicated code blocks and more short-term churn as AI assistants spread.
    Share of developers who distrust AI output accuracy, Stack Overflow surveys 2024 vs 2025
    Share of developers who distrust AI output accuracy, Stack Overflow surveys 2024 vs 2025

    So the need for QA isn't going away. The old operating model is: a separate testing phase at the end, people clicking through regression suites, and bugs reported after the code is written. That model can't keep up when code arrives faster than anyone can check it by hand.

    What does AI-generated testing do well, and what does it miss?#

    AI is good at writing a lot of tests quickly. It's weak at knowing what the software is supposed to do.

    Where it does well:

    • Drafting unit tests at scale. In a study of 2,232 test-related commits, AI agents wrote 16.4% of all commits that added tests. Their tests had higher assertion density and coverage gains similar to tests written by people.
    • Individual test quality. A separate study checked Claude-written Python tests with mutation testing and historical-revert testing. They were broadly comparable to tests written by people.
    • Filling coverage gaps in boilerplate, edge-case inputs and well-specified functions.

    Where it misses:

    • Intent. A test generated from the code checks what the code does, not what it should do. If one flawed prompt produced both the code and the test, the test confirms the bug.
    • Scope in complex changes. The same commit study found that AI tests stayed more localized in complex contributions, so they cover the changed lines and miss the behaviour around them.
    • Coverage that proves nothing. Coverage only shows that code ran, not that anything checked the result. One mutation-testing study found that 14% of 2,041 methods were "pseudo-tested": tests ran them, but no assertion would fail if their logic broke. Generated suites aimed at coverage targets can make this problem worse.
    • Business rules, security abuse cases and cross-system flows, which depend on knowledge that isn't in the repo.

    How does the QA role shift from finding bugs to designing quality systems?#

    The quality engineer stops being the last checkpoint. They now own the automated checks that every change has to pass, whoever or whatever wrote it.

    In practice the system has four parts:

    1. Executable acceptance criteria. Requirements become tests before or alongside the code, written with the subject-matter experts who know what "correct" means.
    2. Pipeline gates. Every pull request runs tests, secrets scanning, static analysis and dependency checks, and failures block the merge. Our guide to CI/CD security gates for AI-written code explains how to set these up and enforce them.
    3. Review rules. Required code-owner review, with extra attention on AI-generated tests.
    4. Release evidence. A record of what ran, what passed and who approved, so you can show why a release was safe to ship.
    Who hands what to whom as one AI-assisted change moves from pull request to release
    Who hands what to whom as one AI-assisted change moves from pull request to release

    At Calidad this is the Quality Engineering service: AI-generated tests that people review, code review gates, security and regression checks, and release evidence. Every release passes three gates. Tests show it works, security checks show it's safe, and a senior engineer's review shows it's right. Nothing ships without all three, whether a person or an AI wrote the change.

    What skills do quality engineers need when AI is writing the code?#

    They need more engineering skill and more judgement about risk, and they spend less time writing scripts by hand.

    • Reading code and diffs. Reviewing a pull request quickly enough to spot untested behaviour.
    • Oracle design. Deciding what the right answer is and stating it as a specific assertion, not "it didn't crash."
    • Mutation testing. Measuring whether a suite actually catches faults, rather than trusting coverage percentages.
    • Pipeline engineering. Writing CI configuration, choosing what blocks a merge and keeping suites fast enough to run on every change.
    • Security basics. Common vulnerability classes, dependency risk and secrets handling. The Veracode numbers above are why this is now part of the job.
    • Directing and reviewing AI tools. Prompting for tests, then rejecting weak ones.
    • Delivery metrics. Reading change failure rate and recovery time together with deployment frequency. Our post on DORA metrics when AI writes your code shows how.
    • Domain knowledge. Working with business experts on acceptance criteria. AI can't supply this.

    If your product has LLM features of its own, add evaluation sets: scored test questions that run on every prompt or model change.

    Who reviews the tests that AI writes?#

    A person who didn't produce the code under test reviews them, usually the quality engineer or the module's code owner. A mutation tool then checks whether the tests catch real faults.

    Rules that work:

    • Separate author and verifier. Never merge code and its only tests from the same AI session without independent review.
    • Check every assertion. Does it compare a specific expected value? Does it come from the requirement or just from the current output?
    • Look for missing negative paths. Invalid input, permission failures, timeouts.
    • Treat surviving mutants as a review finding. If you can break the logic and the test still passes, the test goes back for revision.
    • Quarantine flaky tests. A test that sometimes fails teaches people to ignore red builds. Fix it or delete it.
    Lifecycle of one AI-written test from draft to merge, quarantine or retirement
    Lifecycle of one AI-written test from draft to merge, quarantine or retirement

    How should you restructure a QA team for AI-assisted delivery?#

    Put quality engineers inside the delivery teams, make them owners of the gates, and measure them on escaped defects and change failure rate instead of test cases executed.

    A practical sequence:

    1. Audit where QA time goes. Find how much goes to manual regression that could be automated.
    2. Automate the regression suite. Let AI draft the tests and have people review them, with the rules above.
    3. Embed quality in each team. A separate QA department that receives finished work recreates the end-of-cycle phase. In Calidad's Product Engineering Pods, QA and automation sit in the same team as the lead, engineers, DevOps and product.
    4. Name gate owners. Each pipeline check needs someone responsible for its rules, its false positives and its speed.
    5. Change the scorecard. Test cases executed and bugs logged reward activity. Escaped defects, change failure rate and recovery time reward outcomes.
    6. Pilot on one team before rolling out, and compare its stability metrics against teams that haven't changed.
    Responsibility Old model AI-assisted model
    When testing happens Phase at the end of the sprint On every pull request
    Who writes tests QA, by hand AI drafts, people review
    What QA owns Test cases and bug reports Gates, rules and release evidence
    Security Separate, often late Scans block the merge
    Success measure Tests run, bugs found Escaped defects, change failure rate
    Team placement Separate QA group Inside each delivery team

    Conclusion#

    AI made code cheap and fast to produce, but it didn't make it correct. The QA function survives by changing its job: less checking by hand, more designing the system that proves every change works. That means human-reviewed tests, enforced gates, security checks and release evidence. Teams that make this change get AI's speed without the drop in stability.

    If you want to see how this works on your own codebase, Calidad starts with a paid pilot: one pod, one outcome and success criteria agreed up front. Talk to an engineering lead →

    Frequently asked questions

    Can AI replace exploratory testing?

    Not fully. AI can suggest scenarios and generate inputs. Exploratory testing depends on someone noticing behaviour that seems wrong for the business, and that needs domain knowledge AI doesn't have. Keep it as a focused human activity on high-risk flows, and turn what you find into automated tests.

    What is mutation testing, and why does it matter for AI-written tests?

    Mutation testing makes small deliberate changes to your code, such as flipping a condition, and checks whether your tests fail. If they still pass, the tests run the code without actually verifying it. That makes it a better measure than line coverage for judging AI-generated suites, which can reach high coverage with weak assertions.

    Should AI-written code get different review rules from human-written code?

    The gates should be the same: tests, security checks and senior review on every change. What changes is where reviewers look. With AI-written changes, check that the tests come from the requirement and not the generated code, check dependencies the AI added, and look for duplicated logic.

    How do we know if our quality system is working?

    Track escaped defects alongside the DORA stability metrics, change failure rate and time to recover, not just deployment frequency. If you ship more often while failure rate and recovery time stay flat or improve, the gates are working. If speed rises and stability falls, AI output is getting through unchecked.

    Do we still need dedicated quality engineers if developers write the tests?

    Usually yes, but in a different role. Someone has to own the gate rules, review AI-generated tests independently, keep suites fast and reliable, and read the stability metrics. Without a named owner, those jobs tend to get done by no one.

    • QA
    • AI coding
    • Software testing
    • Quality engineering

    Newsletter

    Get new articles by email

    A short monthly round-up of what we've written on engineering pods, quality and AI. No spam.

    Keep reading

    Start with proof

    Skip the pitch.
    See it work.

    A paid pilot: one pod, one outcome, judged on working software — results in days. No long-term contracts.