Before an aircraft leaves the ramp, someone walks it with a printed checklist and treats every line as a question that has exactly two honest answers. Nothing on that list is skipped because the last ten flights went fine, and nothing depends on anyone’s mood that morning. Software pipelines have earned the right to the same treatment. DevSecOps is the discipline that puts security checks inside the delivery flow and makes the delivery team own them, so nothing ships without examination or without a trail — and no separate security phase waits at the end. It works best when you can point at specific controls and ask whether each one exists, where it sits, and what it does when it fires. What follows is a preflight-style audit built on exactly that logic: six checks, each answerable in writing, that together tell you whether a pipeline would notice an attack, prove its own history, and fail safely.
Why pipelines need a DevSecOps preflight
The aviation parallel holds because both worlds deal with failure that is rare, expensive, and easy to normalise. A checklist differs from a survey in one respect: it demands a location for every control. Practitioners model this as the placed-controls view — every check has a position in the flow, a confidence level, a cost, and an action such as block, warn, or record. Those families line up with the practices in NIST’s SSDF, which comes as near as anything in this field to a shared vocabulary. When you audit against positions rather than intentions, the conversation stops being “are we secure?” and becomes “which of these six places is unguarded?” — a question a team can actually answer before lunch.
Check one: the gate policy, written down
Find the document that states what blocks a merge. If no such document exists, that is your first finding, and the audit has already paid for itself. A defensible policy blocks merges only on high-confidence, fixable, severe findings that are new in the change under review. Everything else turns into a tracked item, assigned to somebody, carrying a deadline that reference templates cap at ninety days. Baseline debt, the pile of issues that predate the programme, never blocks; it gets scheduled instead.
- Is the policy written, dated, and owned by a specific person?
- Does it separate new findings from pre-existing debt?
- Does every exception carry an owner and an expiry, and does the register get reviewed?
- Do expired exceptions fail loudly, or quietly renew themselves?
- Can anyone state, from memory, the three conditions that justify a block?
That last question is the tell. On pipelines where the gate works, the rule is boring and known; where it fails, the rule is a rumour.
Check two: what the scanners have earned
Static analysis is the control most often switched to blocking far too early, and it is where the audit should spend its longest look. Evidence drawn from Google’s Tricorder-derived adoption work is blunt about the conditions under which a scanner earns blocking rights: an effective false-positive rate near zero for the rule in question, review comments kept under ten percent of findings, results judged on changed lines only, and a thirty-finding sample per rule before anyone trusts it. The monthly override rate then acts as the referee — when it climbs past roughly five percent, developers have stopped believing the gate. A pipeline that blocks on a rule with none of this measured is not protected; it is training people to bypass. Until the numbers exist, report-only is the correct setting, and the findings belong in a queryable store, ideally in SARIF 2.1.0 format so the history survives tool changes.
Check three: credentials and their escape routes
Ask where secrets live and what happens in the first hour after one leaks. The working pattern pairs a pre-commit scanner with server-side push protection, so the key is caught twice before it lands anywhere durable. One detail separates careful setups from careless ones: on GitHub, the push-protection bypass is open to any writer by default, so check who can actually click past the warning. And when a leak does occur, the order of operations is not negotiable — rotate the credential first, rewrite history second. Teams that reverse those steps spend a week scrubbing a key that is still live.
Check four: the machinery that runs the builds
The pipeline itself is an attack surface, a lesson three public incidents have taught at considerable cost — a point the practical guide to site reliability engineering on this site arrives at from the operations side. The OWASP CI/CD Top Ten (CICD-SEC-1 through 10) catalogues the failure modes; the audit reduces them to five mechanical checks. Third-party actions and workflows should be pinned to full commit SHAs, not moving tags. Every job should run under least-privilege permissions rather than the default shotgun. Long-lived cloud keys in CI variables should be replaced with OIDC or workload identity, where the platform supports it. Fork pull requests must never receive secrets, and nothing untrusted should ever execute under pull_request_target. Gate rules themselves should be read from the base commit, so a hostile change cannot rewrite its own referee. The tj-actions/changed-files compromise of March 2025 — where a widely used GitHub Action leaked secrets and CISA advised rotation — is the standard answer to anyone who calls this section paranoia.
Check five: receipts for what you ship
Does every build produce evidence that survives into production? The set to look for is an SBOM, a signature, and provenance per artifact. Supply-chain maturity has a ladder — SLSA Build levels zero through three — and GitHub artifact attestations currently land at Build L2, reaching L3 when reusable workflows are pinned. The check that catches most teams out is verification: signing proves who built a thing, but verifying at deploy time proves somebody looked, and the second act is the one that contains value. Overtrusting stored attestations is a documented anti-pattern for good reason; a compromised Syft installation can falsify an SBOM as easily as it produces an honest one. In the audit, ask to see a deployment log where a verification step either passed or failed visibly.
Check six: DevSecOps people, hours, and exception health
Pipelines are run by humans on budgets, so the audit asks about them too. Security champions embedded in delivery teams and given ten to twenty percent protected time are the staffing model that recurs across mature adopters. The culture research is sharper than the tooling marketing: DORA’s 2022 findings tied weak security practice to odds of high burnout 1.4 times higher, and found that culture predicted adoption better than any purchase did. Read those two facts together and the audit question writes itself — is security work in this team a scheduled, bounded activity, or an after-hours tax? Anyone who has worked through an aviation entrance exam guide knows the pattern: unstructured pressure produces poor results, while timetabled study produces pilots. The exception register counts as people evidence too: a healthy one shows owners, expiry dates, and closures; an unhealthy one shows the same five waivers renewed since spring.
Scoring the walk and keeping the log
Resist the urge to leave with a percentage. Leave with the seven numbers worth tracking instead: gate override rate, effective false-positive rate, fix time for new findings at the median and the ninetieth percentile, how often defects escape, what share of artifacts carry provenance, pipeline overhead, and whether exceptions are healthy. The identity model, the SARIF store, and the written gate policy are the three artifacts to create before anything else changes — they cost an afternoon and they make every later audit comparable. A familiar rhythm helps here: preparation and repetition beat talent and improvisation, which is the same reason our guide to preparing for pilot aptitude tests insists on practice over cramming. Structured rehearsal is how any checklist stops being paper.
When a DevSecOps check fails
A failed check is not a scandal; it is a finding. Record it with severity, owner, and expiry, then let the gate policy decide whether it blocks anything — which, under the rules above, it usually will not. What it will do is become visible, scheduled work instead of an unknown. Rehearse the compromise runbook while nothing is wrong, retire the noisy rules on a monthly cadence, and treat the whole audit as something you redo each quarter, not something you survive once. For the fuller discipline behind each check — definitions, the placed-controls model, standards from SSDF to SLSA, and the metrics that separate working programmes from theatre — consult The Ops Atlas chapter on DevSecOps, which this checklist condenses. Walk the machine, mark the card honestly, and the pipeline will fly the way it is documented to.
