A well-run continuous delivery pipeline keeps every change releasable on any day. That means small changes merged often, automated tests fast enough to run on every commit, security and dependency checks inside the pipeline, a required human review, and a record of evidence for each deploy. Feature flags, rollbacks and monitoring then keep releases safe once they are live.
These basics matter more now that AI writes a large share of the code. In DORA's 2024 research, a 25% rise in AI adoption was linked to an estimated 1.5% drop in delivery throughput and a 7.2% drop in delivery stability. One reading of that report is that faster coding doesn't improve delivery when teams skip basics like small batch sizes and solid testing. This guide describes the standard our pods work to. Every release passes three gates: tests, security checks and a senior engineer's review. That applies whether a person or an AI wrote the change.
What is a continuous delivery pipeline, and how is it different from continuous deployment?#
A continuous delivery pipeline is the automated path that brings every commit to a releasable state. Continuous deployment goes one step further and releases each passing change to production with no human step.
Martin Fowler puts the difference in one line: "Continuous Delivery means you ensure every change can be deployed to production. Continuous Deployment means you deploy every change." He also explains that continuous delivery is a prerequisite for continuous deployment, because it proves automatic release is possible.
| Practice | What happens to each change | Human step before production | Main requirement |
|---|---|---|---|
| Continuous integration | Merged to mainline at least daily and verified by an automated build | Not applicable | Fast, reliable automated build and tests |
| Continuous delivery | Kept releasable at all times | Yes: someone chooses when to release | Full pipeline automation and release evidence |
| Continuous deployment | Released automatically when every check passes | No | Strong tests, fast rollback, good monitoring |
Teams with change-approval rules or fixed release windows often stop at continuous delivery. Teams that trust their tests and can roll back quickly can go on to continuous deployment. Most gates are the same either way:
Why should every change be small enough to release on any day?#
Small changes are easier to test, review and roll back, so each release carries less risk.
A large release mixes dozens of changes together. When it fails, nobody knows which change caused it, and the fix is usually "roll back everything." Small changes avoid this:
- Merge often. Fowler's definition of continuous integration has each developer merging into the mainline at least daily. Short-lived branches stop a merge from growing into a project.
- Hide unfinished work. Put incomplete features behind a flag so code can merge before the feature is ready for users.
- Keep diffs readable. A good rule of thumb is that a reviewer should be able to understand the whole change in one sitting. If they can't, split it.
- Watch AI-generated batch size. An assistant can produce a 2,000-line change in minutes. Batch size used to be limited by how long code took to write. That limit is gone, so the pipeline and review policy now have to enforce it.
Our DevOps engineering practice works toward exactly this: CI/CD and releases small enough to ship all day.
Which tests belong in the pipeline, and how do you keep them fast and stable?#
The pipeline should run tests that catch real regressions within minutes: many unit tests, fewer integration and contract tests, and a small set of end-to-end user journeys. Any test that fails at random gets quarantined and fixed.
Speed decides whether people trust the pipeline. As Fowler writes, "Nothing sucks the blood of Continuous Integration more than a build that takes a long time." Arrange tests by how fast they run:
- Commit stage, on every pull request: unit tests, linting, type checks and fast contract tests. Aim for feedback in about ten minutes.
- Acceptance stage, after merge: integration tests against real dependencies or close stand-ins, plus database migration tests.
- Release stage: a few end-to-end tests that cover the journeys that earn revenue, such as sign-in, checkout and core workflows.
To keep tests stable:
- Track flaky tests and quarantine any test that fails without a code change. Give it an owner and a deadline.
- Give every test its own data, so tests can run in parallel.
- Review AI-generated tests as carefully as AI-generated code. A test that only repeats what the implementation does proves nothing.
Where do security and dependency checks fit without slowing releases?#
Put fast checks on every pull request and slower scans on the main branch or on a schedule. Only block a release for findings above an agreed severity level.
NIST's Secure Software Development Framework (SP 800-218) is a good reference. Two of its practice groups match pipeline work directly: protecting all parts of the software from tampering, and producing software with few vulnerabilities in its releases. NIST has also published SP 800-218A, a profile for generative AI and foundation models.
A layout that works in practice:
- Every pull request: secret scanning, static analysis on changed files, and dependency checks for known vulnerabilities and licence problems.
- Every merge to main: a full static scan, a container image scan and infrastructure-as-code checks.
- Every build: provenance that records how the artifact was made. The SLSA framework's build track goes from "provenance exists" to provenance signed by a hosted build platform, then to hardened, isolated builds.
- Exceptions: an accepted risk gets an owner, a reason and an expiry date. It should never be a silent override.
Our Security & DevSecOps consulting follows this order: find the gaps that AI-written code and sprawling dependencies opened, rank them by risk, then build the checks into the pipeline.
How do you enforce code review when AI writes much of the code?#
Make review a required, branch-protected step that nobody can bypass, human or agent. Treat AI-written changes exactly like human ones, with a named engineer accountable for each.
Two-person review is an old control. SLSA's original specification required it for all changes at its top level and called it an industry best practice for catching mistakes and deterring bad behaviour. Here is how to keep it working when AI produces most of the diffs:
- Protect the main branch: no direct pushes, and approval required before merge.
- Require code-owner approval for sensitive areas such as authentication, payments and data access.
- Never let an AI agent's account approve a change, including one it didn't write.
- Ask authors to note which parts were AI-generated and how they checked them.
- Turn back oversized pull requests instead of skimming them.
We cover this in more depth in how to verify AI-generated code before production.
What release evidence should the pipeline produce for each deploy?#
Each deploy should leave a record that links the artifact to its commit, tests, scans, approvals and the person who released it.
The pipeline should generate this record itself, not a release manager filling in a spreadsheet. It answers an auditor's or customer's questions in minutes. The SSDF notes that its shared vocabulary helps software buyers talk to suppliers about security, and evidence gives that conversation substance.
| Evidence | What it proves | Generated by |
|---|---|---|
| Commit SHA and pull request link | Exactly what changed | Version control |
| Test results | The change works as specified | CI test stages |
| Security and dependency scan results, with accepted exceptions | Known risks were checked and decided on | Scanners in the pipeline |
| Review approvals | A named engineer took responsibility | Branch protection |
| Build provenance and component list | The artifact came from this source, built this way | Build platform |
| Deploy record: who, when, which environment | Who released it and where | Deployment tool |
| Post-deploy health check | It behaved correctly once live | Monitoring |
Our Quality Engineering service treats this record as part of the release, together with tests, review gates and security checks.
How do feature flags, rollbacks and observability make releases safe after deploy?#
Feature flags separate deploying from releasing. One-step rollbacks limit the damage from a bad change. Monitoring built on SLOs spots the problem before most users do.
- Release gradually. Turn a feature on for staff first, then a small share of users, then everyone. Have each step check error rates and latency before moving on.
- Make rollback dull. Rolling back should be one command or one button, and you should practise it. Plan database changes so old and new code can both run against the same schema.
- Clean up flags. Remove each flag once its feature is fully on. Old flags pile up into technical debt.
- Measure outcomes. DORA tracks deployment frequency, change lead time, change failure percentage and failed deployment recovery time. The 2024 report also added rework rate as a stability measure.
Our Observability & SRE practice sets SLOs users would recognise, instruments with OpenTelemetry and cuts alerts down to the ones that matter. Those are the signals that should trigger an automatic rollback.
How do you move a manual release process to continuous delivery step by step?#
Automate the release you already have, one stage and one service at a time. Measure lead time and failure rate before and after each step.
- Map today's release. Write down every manual step, handoff and approval, and record the four DORA metrics as a baseline.
- Pick one low-risk service. Prove the approach where mistakes are cheap.
- Put build and unit tests on every pull request, and turn on branch protection.
- Build once and promote the same artifact through each environment.
- Add security and dependency checks, first in warning mode, then blocking above a severity level.
- Automate deployment, keeping a manual "release" button for production at first.
- Add flags, rollback and monitoring.
- Generate release evidence automatically and retire the manual checklist.
- Expand to the next service once the metrics show improvement.
We often start this kind of work as a paid pilot: one pod, one outcome, with success criteria agreed up front. For a pipeline, the outcome might be one service going from manual releases to on-demand releases with evidence for every deploy.
Conclusion#
A continuous delivery pipeline that holds up isn't a special toolchain. It's a set of rules the pipeline enforces: small changes, fast and trustworthy tests, security checks on every change, required review and evidence for every release. AI has made code cheap to write, so these gates now do more work. The goal is a pipeline your team trusts enough to release from on a Friday afternoon.
Want to see what this looks like on your own release process? Talk to an engineering lead →
Frequently asked questions
Do we need continuous deployment, or is continuous delivery enough?
Continuous delivery is enough for most enterprises. The pipeline does all the same checks, and a person decides when to release. Move to continuous deployment once your tests, rollback and monitoring catch problems reliably without anyone stepping in. Either way, the gates stay the same.
How long should a CI pipeline take?
Aim for about ten minutes of feedback on each pull request, from unit tests, linting and fast contract checks. Slower integration and end-to-end suites can run after merge. If the commit stage takes much longer, developers start batching changes and skipping runs.
Should security scans block a release?
Yes, but only for findings above an agreed severity level, and with a clear exception process. Every accepted risk gets an owner, a reason and an expiry date. Blocking on every low-severity finding teaches teams to ignore the scanner.
Does AI-generated code need different review rules?
The rules should be the same, applied strictly. Every change needs a named human approver, AI agents should never approve changes, and oversized AI-generated pull requests should be split up. Reviewers should also check that AI-written tests actually test behaviour.
Which metrics show whether our pipeline is improving?
Use DORA's measures: deployment frequency, change lead time, change failure percentage and failed deployment recovery time, plus rework rate. Record a baseline before you change anything and compare after each step. Better throughput with worse stability means the gates need tightening.
- Continuous Delivery
- CI/CD
- DevOps
- DevSecOps
- AI-Generated Code
- Release Engineering
Newsletter
Get new articles by email
A short monthly round-up of what we've written on engineering pods, quality and AI. No spam.
