The DevSecOps CI Pipeline Audit

Why this matters

The most common pattern we see in CI security is what we call "shelfware-by-acquisition." A security tool gets bought, a pipeline step gets added, the build still passes, and the findings go to a dashboard nobody opens. Six months later there are five tools running, none of them block a deploy, and a developer is googling how to disable one because it's slowing the build down.

A CI pipeline audit is not about adding more scanners. It's about answering one question on every existing check: does this finding stop a deploy, file a ticket, or get ignored on purpose — and is that the right choice? Once you can answer that for every check, you have a security pipeline. Until then, you have a collection of tools.

1. Map the pipeline before you change anything

Get a one-page diagram of every stage from git push to production. For each stage, write down what's running, what it's checking, and what happens if it fails.

• Pre-commit / pre-push: linters, formatters, secret scanners, fast SAST. Runs on the developer's machine
• PR / merge request: full SAST, dependency scanning (SCA), license check, unit tests, coverage gates
• Build: reproducible build, SBOM generation, image build, image scan, image signing
• Pre-deploy: infrastructure-as-code scanning, policy-as-code checks, configuration drift detection, integration tests
• Deploy: environment-specific config, secret injection, deployment to staging
• Post-deploy: DAST against staging, smoke tests, security regression suite, runtime policy verification
• Production: canary, monitoring, runtime detection

Most pipelines we audit are missing 2-3 of these stages entirely and double-running another 2-3. The diagram alone is usually enough to start a productive conversation with the platform team.

2. Secret scanning — where it actually has to be

Secret scanning is the cheapest, highest-value check in any CI pipeline, and it has to run in more than one place. A single scan in CI is too late — by the time CI runs, the secret is already in git history. Defense in depth here is mandatory.

• Pre-commit: a hook that catches obvious patterns before the developer ever pushes. gitleaks or trufflehog with fast rules, opt-in for the dev's own benefit
• Server-side push hook: a hard block at the git server (GitHub push protection, GitLab equivalent). This is the one that actually prevents the breach
• CI scan on every PR: historical scan of the diff and a periodic full-history scan. Catches anything the previous two layers missed
• Repository scan as a scheduled job: daily, full history. Runs against everything you own and reports drift
• And, separately: a runbook for what happens when a secret is found. Rotate first, then investigate, then audit usage. The order matters

3. SAST, SCA, DAST — placement and gates

The three families of code-level scanning are different tools with different placement, and they're often confused. The audit question for each is the same: where does it run, what does it block, and is the noise level survivable.

SAST (static analysis)

• Run on every PR, against the diff, with a hard block on new high-severity findings introduced by the change. Existing findings get triaged separately, not blocked
• Run a full-codebase scan as a scheduled job, with results going to a backlog the security team owns. Don't use this to gate deploys
• Tune ruthlessly. An untuned SAST tool will produce 10x noise to signal. The first audit task is usually deciding which 30% of rules are actually relevant to your stack
• Make the tool's findings actionable in the IDE, not just in CI. Developers should never have to context-switch to a separate dashboard to see why their PR failed

SCA (dependency scanning)

• Run on every PR. Fail the build if the change introduces a new dependency with a known critical CVE. Existing CVEs get a separate, time-bounded SLA
• Distinguish between direct and transitive dependencies in the report. Developers can act on direct ones; transitive ones often need a different fix path
• Have a documented exception process. There will be CVEs you can't fix immediately. The audit tracks the exception, not the violation
• Watch for the trap of upgrading for the sake of upgrading. A "fix" that introduces a breaking change in a dependency you depend on heavily is not a fix

DAST (dynamic analysis)

• Run against staging, post-deploy. Don't try to run a full DAST in PR — it's too slow and the false positive rate is too high
• Use as a regression check, not a discovery tool. Discovery is what your pentest is for; DAST is what catches the regression next month
• Focus on authenticated DAST. Most of the interesting attack surface is behind login, and most default DAST configurations are unauthenticated

4. Container image supply chain

If you're shipping containers, the image supply chain is its own audit subject. Everything from base image selection to admission control falls in scope. The pattern that survives audit and production both:

• Pinned base images by digest: FROM nginx@sha256:..., not FROM nginx:latest. The pin lives in source control and is bumped by automated PR
• Reproducible builds: the same source produces the same image bytes. This isn't a nice-to-have; it's the prerequisite for everything else
• SBOM at build time: Syft attached to the image as an attestation. The SBOM is what you grep when the next OpenSSL CVE drops
• Vulnerability scan on the built image: Trivy or Grype, with a build-time gate on critical CVEs older than your SLA window
• Image signing: Sigstore/cosign, signed at build, verified at admission time (Kyverno or Gatekeeper enforce). Unsigned images don't run
• Provenance attestation: SLSA build provenance attached as an attestation. Audit can answer "where did this image come from" without trusting the registry alone
• Private registry with pull auth: no anonymous pulls, no public registries in production manifests, no fallback to Docker Hub

5. IaC and policy-as-code

Infrastructure-as-code is the highest-leverage place to catch security issues, because the same scanner runs against every change for every environment, before anything is provisioned. The audit question is whether the IaC scanner is enforcing or advising.

• Scan on every PR that touches infra: Checkov, tfsec, terrascan, or a policy engine (OPA/Conftest, Sentinel)
• Block, don't warn, on a defined critical set: publicly readable buckets, security groups open to the world, encryption disabled, public RDS, IAM wildcards. The exact list is small and the policy is unambiguous
• Plan-time policy enforcement: if you're running Terraform Cloud, Atlantis, or similar, enforce policies on the plan output, not just the source
• Drift detection: what's in the cluster or cloud account that doesn't match the IaC? Drift is a security signal. Resources created out-of-band are usually the riskiest ones
• Module enforcement: approved modules for sensitive resources (databases, IAM, networking). Free-form modules for anything else, but with policy gates

6. The "block vs warn" decision matrix

This is the hardest part of the audit and the place where teams are most resistant to honesty. Every check has to be either blocking or non-blocking. There is no middle ground that works in practice.

• Block when: the finding is unambiguous, the false positive rate is under 5%, the fix is well-understood, and the cost of shipping the issue is higher than the cost of a delayed deploy. Examples: known critical CVEs in direct dependencies, secrets in source, IaC public-bucket exposure
• Warn (file ticket) when: the finding is real but the fix has uncertain blast radius, or the false positive rate is in the 5-30% range. Examples: medium-severity SAST findings, transitive CVEs without a direct upgrade path
• Track only (no ticket) when: the finding is informational, the false positive rate is over 30%, or the team has decided the class of finding is not worth acting on. Examples: low-severity SAST, license findings on permissive licenses, untuned SCA noise
• Ignore when: the rule is wrong for your stack. Suppress at the rule level, not the finding level. Document why

Common failure mode: everything is in the "warn" bucket because nobody wants to be the person who blocked a deploy. The audit fix is to move 5-10 specific finding classes into "block," and accept that the rest are tickets

7. Ownership and the broken-window problem

A pipeline with security checks but no owner decays. The audit assigns ownership at three levels:

• The platform team owns the pipeline definition. Adding, removing, or changing a check is a platform team decision
• The security team owns the rule sets and exception process. They're not the ones writing pipeline YAML, but they decide what passes for "high severity" and they sign off on suppressions
• The developing team owns the findings on their own code. The pipeline shouldn't put findings in front of anyone but the team that wrote the change

Without all three, you get the broken-window state: warnings accumulate, the dashboard turns red, the team learns to ignore red, and the next real finding is invisible.

8. Metrics that actually mean something

Most CI security metrics are vanity. "Number of vulnerabilities found this quarter" goes up whenever you add a tool. "Time to fix" is meaningless without severity and exploitability context. The four metrics we look for in an audit:

• Mean time to remediate critical findings: measured from finding-creation to merge, broken out by direct-vs-transitive
• Build break rate by check: if a check breaks more than 5% of builds and most of those breaks are false positives, the check is broken
• Suppression rate: what percentage of findings get suppressed. A high number is either a tuning problem or a process problem. Investigate either way
• Coverage: what fraction of repos in scope are actually running the standard pipeline. The honest number is usually lower than the security team thinks

The short version

A CI pipeline audit produces three kinds of output: rule-tuning recommendations, missing-check additions, and an ownership map. Sequence them in that order:

• Tune existing checks first. Cut the noise from what's already running. This is usually the highest-leverage week of work in the entire engagement
• Add the small set of missing critical gates. Secret push protection if it's missing, a real signing step if there isn't one, an IaC public-bucket block if it's not enforced
• Fix ownership. A documented owner per stage and per rule set. This is the part that makes the audit stick six months later
• Don't add new tools in the first pass. The temptation is to bring in a new SAST or a new SCA. Resist. Make the existing tools work first, then re-evaluate