Most security tooling in the CI pipeline is signature-based: it knows about a vulnerability because someone already found it, named it, and published a CVE. That's useful — it catches known-bad dependency versions and known-bad patterns — but it's fundamentally reactive. It cannot catch the vulnerability class that hasn't been discovered yet, because by definition there's no signature for it.
The category of finding we care most about is the one signature scanners structurally cannot produce: a novel exploitable pattern in your code, introduced in this PR, that doesn't match anything in a known-vulnerability database yet.
Why pattern reasoning beats pattern matching
A signature scanner asks: "does this code match a known-bad string or AST shape?" A reasoning-based security agent asks a different question: "given what this code does, what could go wrong?"
The second question generalizes to code nobody has seen before, which is exactly the property you want when the whole point is catching things before they're catalogued.
This is why we run a dedicated security agent as part of the per-PR review, focused specifically on patterns like:
- Authentication & Authorization Logic: Code that looks correct on the happy path but contains a bypassable edge case or missing tenant isolation check.
- Credential Leaks: Secrets or environment credentials that leak into logs, CI output, or verbose error responses.
- Supply-Chain Integrity: Dependencies or build artifacts that don't match their expected cryptographic provenance.
Consolidating the toolchain, not expanding it
A counterintuitive lesson from building this: more scanners isn't more security. We started with seven overlapping tools and retired four of them down to three that cover the same territory with meaningfully less redundant noise per PR.
A security agent whose output is 60% duplicate or low-confidence findings trains developers to skim past all of it, including the finding that actually mattered. The three we kept share a critical property: each targets a distinct, high-signal category — authentication pattern integrity, CI log leak detection, and software bill-of-materials integrity.
Sandboxing is a security decision, not just an infra decision
Every PR we scan runs inside an isolated Firecracker microVM, created fresh per scan and destroyed after. This matters specifically for zero-day detection, because a security agent that's actually reasoning about exploitability sometimes needs to execute or trace code paths, not just read them statically. That only becomes safe to do if the execution environment cannot touch anything outside itself.
We treat this as seriously as the findings themselves: the sandbox creation path has its own guard clauses specifically to prevent a scanned PR's code from achieving remote code execution against the host.
What happens after a finding
A zero-day-class finding in a PR is exactly the kind of thing that shouldn't live only as a comment that vanishes when the PR merges or gets closed. High-severity findings get escalated into a tracked GitHub issue, fingerprinted so the same underlying issue doesn't spawn duplicate reports across repeated scans of the same PR as it gets updated.
The honest limitation
No PR-time security agent, however good, replaces defense in depth. Static and semi-dynamic analysis at PR time catches a meaningful class of issues before they ship, but it's not a substitute for runtime monitoring, dependency auditing, or a real incident response process. We're explicit about this with customers: Codeward is the gate that stops the avoidable stuff from reaching production, not a claim that nothing bad can ever reach production.
Want a second set of eyes on your next PR before it merges? Codeward's security agent is tuned to catch the pattern, not just the CVE.