Agents in CI

Running an agent-driven scan without a human in the loop, and where to stop.

5 sections2 min read

An agent in CI can scan, triage, and explain. It should not publish or merge, and Cefense is built so that it cannot do either by accident.

When this makes sense

  • Summarising what a scan found in terms of what this service actually does, rather than a list of rule names.
  • Deciding whether a new finding is worth blocking a release for, given what changed.
  • Answering when a vulnerability was introduced, from the scanned commit history.
  • Preparing a patch and a written explanation for a person to review in the morning.

Setup

.github/workflows/cefense-agent.ymlyaml
name: cefense
on:
  push:
    branches: [main]

jobs:
  scan:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/setup-node@v4
        with:
          node-version: 22
      - run: npm install -g @cefense-npm/cefense-cli
      - name: Scan and report
        env:
          CEFENSE_TOKEN: ${{ secrets.CEFENSE_TOKEN }}
          CEFENSE_ORG: acme
        run: |
          cf scan --repo ${{ github.repository }} --wait --agent
          cf reproduced --repo ${{ github.repository }} --severity critical,high --agent

CEFENSE_TOKEN replaces signing in: cf auth login needs a browser and cannot work here. --repo is required because agent mode resolves no directory. See Running in CI.

Failing the build

No parsing neededbash
cf reproduced --repo acme/api --severity critical,high --exit-code --agent

Exit 1 when any Critical or High finding is present. Use this rather than having the agent decide whether to fail the job by reading text.

Keeping it bounded

  • Let CI scan, list, read, and explain. Let a person publish and merge.
  • Cap what the agent reads. --severity critical,high --limit 20 keeps the payload and the run bounded.
  • Give it one job per run. An agent asked to fix everything on a schedule will eventually do something surprising at three in the morning.
  • Log the JSON. When an agent reports something odd, the envelope is the record of what it was actually told.

A safer division of labour

StepCI agentPerson
Name the organization, once, in the job's environmentYesSets it up
Scan on pushYesNo
List and filter findingsYesNo
Read the evidence and explain itYesReviews it
Generate a patchYesNo
Open a pull requestNoYes
MergeNoYes

Have a correction or need help?

Email the docs team