Codex Security Preview: AppSec Agent for Real Repos

OpenAI shipped Codex Security in research preview, and the framing matters. This is not a glorified linter and it is not a wrapper around Bandit, Semgrep, or CodeQL. It is an agent that reads your repository, understands the call graph, hypothesizes attack paths, and writes up findings the way a junior application security engineer would. I pointed it at three open source repositories with known CVEs, plus a private one we use internally for CI tooling, and tracked exactly what it caught and what it missed.

This post is the truth about its catch rate, where it fits in a real SDLC, and the operating posture I would recommend if you get into the preview.

What Codex Security Actually Is

Most static analysis is pattern matching. The tool has a database of known dangerous patterns, scans your code, and reports any matches. This catches obvious bugs but misses anything that requires reasoning about how data flows across files, how authentication is structured, or how a feature is actually used.

For the security frame around this, see OpenAI Codex: Cloud AI Coding With GPT-5.3 and OpenAI vs Anthropic in 2026 - Models, Tools, and Developer Experience; both focus on the places where agent autonomy needs explicit boundaries.

Codex Security is agentic. It treats your repository the same way a Codex coding agent would: it can read files, follow imports, execute searches, and build a working model of the application. It then proposes hypotheses about where vulnerabilities could exist, investigates each one, and reports findings with reproduction steps and suggested fixes.

The shift from "look for known patterns" to "reason about this specific application" is the same shift that happened from rule-based to ML-based tooling a decade ago, but the unit of reasoning is now the application itself rather than the line. That is what makes it interesting and also what makes it dangerous.

Joining the Research Preview

Access is gated. You request an org-level enrollment and get a quota of agent runs per day. The preview is rate-limited per repo, which means you cannot point it at every repo in your org on day one. I would recommend prioritizing your highest blast-radius services: anything that handles auth, billing, or PII. Save the rest for when GA pricing lands.

The integration is straightforward. You point Codex Security at a repo, optionally scope it to a path or a PR diff, and it streams findings back. I run it from the API directly because I want findings in our existing security pipeline, not in a separate dashboard.

from openai import OpenAI

client = OpenAI()

scan = client.responses.create(
    model="gpt-5.3-codex",
    input=[
        {
            "role": "developer",
            "content": (
                "You are a senior application security engineer. "
                "Audit the attached repository for vulnerabilities in: "
                "authentication, authorization, input validation, "
                "SSRF, SQL injection, and secrets handling. "
                "For each finding, return JSON with: severity, file, line, "
                "description, exploit_scenario, and suggested_fix."
            ),
        },
        {
            "role": "user",
            "content": "Repository: github.com/acme/payments-api at commit 8a3f9c2",
        },
    ],
    tools=[
        {"type": "code_interpreter", "container": {"type": "auto"}},
        {"type": "file_search"},
    ],
    reasoning={"effort": "high"},
    response_format={"type": "json_object"},
)

findings = scan.output_text

That is essentially the agent loop. Behind the scenes the model uses tools to clone the repo into a sandboxed container, walks the codebase, and emits structured findings. You can wire that JSON into your existing tracker, dedupe against last week's run, and only surface new issues to humans.

My Benchmark Setup

Three open source repositories, each seeded with a small number of intentionally broken commits I authored, plus the existing CVE history of the project. Each commit introduced exactly one bug from a known vulnerability class, with the file and line documented in a private ground-truth list.

Repo A was a Node.js Express API with seeded SQL injection, an SSRF in a webhook handler, and an authorization-bypass where a user-controlled role field was trusted from the request body.

Repo B was a Python FastAPI service with seeded path traversal in a file download endpoint, a hardcoded HMAC secret, and a race condition in a balance update.

Repo C was a Go HTTP service with seeded server-side template injection, an open redirect, and a misconfigured CORS that allowed arbitrary origins with credentials.

Plus the natural CVE history each project carried before I touched it. Total ground truth: 23 issues across the three repos.

I also wired the agent runs through DD Traces so I could replay each scan, look at the tool calls, and audit which files the agent read. Without traces you cannot tell whether a missed vulnerability was a reasoning failure or simply a file the agent never opened. With traces, the post-mortem is straightforward.

What It Found

Codex Security caught 16 of the 23 seeded or historical issues across the three repos. That is a 70% catch rate, which sounds modest until you read the findings.

The SQL injection in Repo A was caught with a clean exploit scenario, including the exact crafted payload, and a fix that used the parameterized query API the codebase already had elsewhere. The agent had clearly read three other endpoints in the same router and noticed the pattern was inconsistent.

The SSRF in Repo A was caught and, more impressively, the agent flagged a second related issue I had not seeded. The webhook handler trusted a user-supplied URL without validating the scheme. I had seeded an SSRF where the URL was used directly. The agent noticed the same URL was logged later via a different code path that also performed an HTTP fetch in a debug branch. That second path was a real issue I had not noticed.

The hardcoded HMAC secret in Repo B was caught immediately and the suggested fix was correct: move to environment variables, rotate the key, document the rotation procedure. The agent also noticed the secret had been committed for two years and recommended a git history rewrite, which is the right call.

The CORS misconfiguration in Repo C was caught with a clear writeup of why Access-Control-Allow-Origin: * combined with Allow-Credentials: true is not just wrong but actively dangerous, and a fix that locked the origin list down to the canonical domains.

There were two surprising catches that were not on my ground-truth list. In Repo B the agent noticed that a JWT verification path skipped the audience claim, which meant tokens issued for a sibling service would validate. In Repo C it flagged a time.Now().UnixNano() used as a session token seed, which is a real entropy issue. Both were legitimate, both were not seeded, and both were findings I would have wanted from a human reviewer.

What It Missed

The misses tell you more about the boundaries than the hits do.

It missed the authorization bypass in Repo A. The user-controlled role field in the request body was trusted without verification, but the bug only surfaced if you traced the request through three middleware layers and noticed that the role check happened against the request body rather than the session. Codex Security read the endpoint and the immediate handler but did not climb up into the middleware to verify the auth assumption. This is the classic auth-flow blind spot for any tool that reasons primarily about a single file or a single endpoint.

It missed the race condition in Repo B. The balance update path read, computed, and wrote without a transaction. Static-analysis tools usually miss these because the bug only exists in concurrent execution, not in the source. Codex Security is no different. The fix would require either explicit concurrency reasoning prompts or a runtime fuzz harness, neither of which is in the preview today.

It missed the path traversal in Repo B. This one was a tooling issue more than a reasoning issue. The agent never opened the file containing the download handler because it was nested in a deprecated module that was still wired in via a router include. The trace made that clear. If your repo has dead-looking modules that are actually live, scope the scan explicitly.

It missed the open redirect in Repo C. This was the closest to a true reasoning failure. The agent identified the redirect endpoint, noted that it accepted a query parameter, and concluded that an allowlist check existed elsewhere. There was no allowlist, but the agent had seen a similarly named function in another file and assumed it applied. Honest mistake, and one that a careful human reviewer would also have to verify.

It missed business-logic vulnerabilities entirely. None of the three repos had seeded business-logic bugs because I was not testing for them, but I want to call this out: agentic AppSec today is good at vulnerability classes that have a recognizable code shape and weak at vulnerabilities that depend on understanding intent. A coupon system that allows negative discounts will not be flagged because there is nothing in the code that looks dangerous. That requires product context the model does not have.

Where It Slots Into A Real SDLC

Pre-commit is the wrong slot. Run times are too long, the agent needs to understand the full repo to be useful, and most pre-commit checks should be deterministic. Skip it.

PR-bot is the right slot for findings discovery, but with discipline. Run Codex Security on the diff plus the immediate import graph, treat all findings as draft, and have a human triage before anything blocks the merge. Auto-blocking on agent findings is not ready for the preview. False positive rates are still high enough that you will erode trust quickly.

CI is the right slot for full-repo scans, scheduled nightly or weekly, with results piped into your existing ticketing system. Dedupe against the previous run, only file new tickets for new findings, and tag findings with the agent's confidence level if you can extract it from the response.

The patterns I use here are borrowed wholesale from SkillForge CI, which we built for general agent CI workflows. The same primitives, scoped runs, structured outputs, dedup-against-baseline, apply to AppSec agents directly. If you are wiring this into your own pipeline, that repo is where I would start.

For the visual walkthrough of running Codex Security against a real repo and triaging findings live, the DevDigest YouTube hands-on video covers the full flow, including the trace replay of one of the catches I described above.

Final Take

Codex Security is the first AppSec agent I have used that I would describe as a genuine review partner rather than a noisy scanner. It catches things a junior engineer would catch, occasionally things a mid-level engineer would miss, and it explains itself well enough that triaging is fast.

It is not yet a replacement for a security engineer. It misses auth flows, business logic, and concurrency. It will hallucinate the existence of safety checks that are not there. The preview rate limits will stop you from running it across an entire org on day one.

But the trajectory is real. If you have access to the preview, the highest-value thing you can do this week is wire it into your CI on one critical service, run it nightly for a month, and build your own ground-truth list of what it catches. By the time GA lands, you will have a calibrated sense of what to trust and what to verify, and that is worth more than any benchmark anyone else publishes.