Cloudflare’s AI Security Auditor Does Not Trust the Agent That Found the Bug

Cloudflare has open-sourced the coding-agent skill that became the starting point for its internal vulnerability-discovery system. The most useful rule in the repository is simple: the agent that finds a security bug is not allowed to confirm it.

That sounds obvious. It is also missing from a great deal of AI-assisted review, where one model proposes a problem, writes a convincing explanation, and then grades its own work. Confidence travels through the whole process even when the original premise is wrong.

Cloudflare built the skill around the opposite assumption. A finding remains a candidate until a separate verifier tries to disprove it, deterministic checks confirm the cited files and evidence, and the result fits a structured record. The design is less impressed by how many bugs an agent can announce than by how few bad claims survive.

My first formal technology job was software testing. A test that passes because the test changed the thing being tested is not evidence. Apparently an agent needs that rule written down too.

The agent is expected to produce plausible junk

Cloudflare is unusually direct about the failure modes it encountered. An agent may edit source code so its own exploit works. It may demonstrate that a dangerous function behaves dangerously without showing that an attacker can reach it. It may describe a missing secondary control as a vulnerability even though the primary control already stops the attack.

The open-source skill responds by requiring a concrete attacker, affected resource, crossed trust boundary, and security outcome. Confirmed findings need a source trace and a bounded observed result. Unresolved claims become needs_validation records without a severity. Disproved candidates are retained as rejected rather than quietly disappearing.

That last state is useful. Security review produces repeated ideas, and forgetting why an attractive theory failed guarantees that another person—or another agent—will spend time rediscovering it.

For Cloudflare’s larger harness, a confirmed finding also carries a proof-of-concept test that runs against the original, untouched codebase and a proposed patch. Plain code verifies that the referenced paths exist and that the test and patch parse. A validator that cannot create findings of its own then tries to break the Hunter’s argument.

A different model performs the next validation stage. That reduces the chance that one model’s habits flow through the entire pipeline. I would not call the second model unbiased, as Cloudflare does in its post; it has different biases. Independence is still better than asking the author to approve its own work.

The safety model is doing real work

Installing the skill does not make arbitrary code safe to run. Its requirements call for an operating-system-enforced sandbox with external networking disabled, a sanitized environment, strict resource limits, a read-only target, and writes confined to assigned scratch space.

If those controls are unavailable, the workflow is supposed to leave execution-dependent claims at needs_validation. That is an important failure mode. It is much safer than quietly running a repository’s build scripts with the credentials and filesystem access of whoever launched the audit.

The separation continues into output handling. Agents write only to scratch locations. Trusted parent-side code promotes an allowlisted set of regular files after checking paths, file types, links, sizes, and ownership boundaries. This is more elaborate than most teams will reproduce on a first attempt, but the threat is real: source code under review is untrusted input, and so are artifacts created while examining it.

The architecture also keeps agents away from production changes. Cloudflare’s fleet pipeline can prepare fixes, but a human must approve a dry run before anything writes to production.

One run is not a security audit completed

The repository says a single run found roughly half of the vulnerabilities eventually discovered across repeated runs in Cloudflare’s tests. The company does not claim a false-negative rate because there is no complete list of undiscovered bugs against which to measure one.

That is the honest limitation. An agent can improve coverage without proving coverage. Cloudflare records what each hunting pass inspected, uses later runs to target gaps, and revalidates old evidence against current source. The coverage ledger matters at least as much as the prose report.

Cost matters too. Cloudflare says its fleet system uses worker pools of 50 to 200 agents. A full scan of a complex repository has taken more than 14 hours, so the large runs are periodic backlog sweeps rather than per-pull-request checks. The downloadable skill is a single-repository starting point, not the entire fleet harness described in the blog post.

Those qualifications do not weaken the project. They make it possible to decide where it belongs. A small team can run focused audits on exposed parsers, authentication paths, or risky changes without pretending it has recreated Cloudflare’s security operation.

What I would copy first

1. Record the attacker, resource, trust boundary, and demonstrated outcome before assigning severity.
2. Give validation to a fresh agent that cannot create its own findings.
3. Require reproduction against untouched source and retain rejected theories.
4. Keep a coverage ledger so “the agent reviewed it” can be replaced with a list of what was actually inspected.
5. Run target-controlled code without ambient credentials, network access, or write access to the repository.

The multi-agent part will get most of the attention. The distrust is the better contribution. Most AI security demos optimize for how many findings the model can produce. Cloudflare has published a process for making each finding difficult to keep.

Sources

• Cloudflare, security-audit skill repository.
• Cloudflare, Build your own vulnerability harness.