Imported from dinosn/raptor-loop-hunt (
SKILL.md). Install upstream withnpx skills add dinosn/raptor-loop-hunt. Copyright stays with the author.
RAPTOR Auto-Research Vuln Hunt
A natural-language agent loop for security research. A generic "find bugs" prompt lets the model fall back to its defaults — single pass, mid-level altitude, converge fast, summarize, stop. Those defaults are wrong for vuln hunting. This skill replaces them with an explicit search procedure: traverse every altitude, generate then adversarially verify from raw, run isolated parallel reasoners, and keep a persistent ledger so each loop is net-new coverage instead of rediscovery.
The prompt is the program. In an agentic system the model's "algorithm" is whatever you tell it to be. This skill specifies that algorithm. Follow the structure; the quality comes from the structure, not from any single clever instruction.
When to use
Use for any "go deep / find everything / audit this thoroughly" security request against a
codebase. Don't use for a quick triage, a single known-CVE reproduction, or a one-file
sanity check — those want /scan or /understand --hunt directly. This is the heavy,
looping, high-coverage mode.
The core loop
Run this as a loop, not a one-shot. Each round:
- Pick an altitude and a slice you have not exhausted (see traversal below).
- Generate candidate findings on that slice with an independent reasoner working from raw source (no prior summaries in its context — summaries cause anchoring).
- Judge each candidate with a separate reasoner, also from raw, prompted to refute.
- Live-verify survivors against the actual code before they count (see guardrails).
- Record everything tried and everything found in the ledger.
- Vary the approach next round — new altitude, new bug-class lens, new slice. Novelty is mandatory; repeating a round wastes budget.
- Check the stop condition. Loop until dry, not literally forever.
Multi-altitude traversal (the coverage guarantee)
Different bug classes live at different zoom levels. A single-pass scan implicitly picks one altitude and is structurally blind to the others. Cover all four, in order, and revisit:
- Whole project — architecture, trust boundaries, auth model, data flows across modules. Catches the broken-object-level-authz / IDOR / missing-authz class that dominates real web audits, plus deserialization sinks, SSRF, and design-level bypasses.
- File by file — each file's responsibilities, its inputs, its exported surface.
- Functionality by functionality — each feature end to end (upload, import, export, templating, auth, admin actions). Trace source → sink for every bug class, not just the headline one. A mapped-but-untraced entry point is an uncovered entry point.
- Function by function — parsing, memory, encoding, length math, crypto comparisons,
format strings, integer handling. Line-level bugs only surface here. This is the altitude
with the weakest mechanical support — the Semgrep anchors above are seeded from whole-project
trust boundaries, so nothing systematically walks the interior functions no entry-point cell
anchors. RAPTOR's
/auditcan be used as an experimental candidate source over that interior: it works from its own checklist-derived gap set and, for functions that reach LLM review, forms a hypothesis and may invoke an applicable analyser. Neither a hypothesis nor a sweep is guaranteed per function — triage and prefilter can short-circuit tocleanfirst. Wire it in the generator seat only, and read the whole contract in "Mapping to RAPTOR's machinery" before using any of its output.
Track which (altitude × slice) cells you've covered. The point of "start with the whole, then file by file, then functionality, then function" is that you sweep the whole grid, not one band.
Component inventory first — the completeness gate (MANDATORY)
Before round 1, enumerate the entire project as a flat list of components — every top-level
module, package, service, transport, and deployable unit — not just the subsystems that look
interesting. For a multi-module build, list every module directory (ls modules/, every Maven
pom.xml, every top-level source package). This inventory is the denominator for coverage.
Then maintain a coverage matrix: every component maps to a hunt cell — now, or a named, logged later round. The hunt is not "done", and the report must not read as done, until every component is either covered or explicitly listed as omitted with a reason (out of scope, build-time-only tool, generated code, third-party vendored copy). Silent omission is the exact failure this gate prevents: slicing by hot-spots and quietly skipping whole modules (transports, databinding, the newer/less-audited modules) is how a shallow audit masquerades as a complete one — especially on a mature project that has had many issues over the years, where the unexamined module is often where the next bug lives.
Rules:
- The per-round slices must be drawn from the full inventory; you may prioritise, but every
component not yet assigned to a cell is recorded as
UNCOVEREDinTRIED.md— never absent from it. The ledger's component list must equal the project's component list. - A component is the unit of accountability. "I audited the interesting parts" is not a complete assessment. When the codebase is large, scale the number of rounds to cover all of it rather than narrowing the inventory to what is convenient.
- Re-run the enumeration when the target changes (new clone, new release) — modules get added (e.g. a new protocol bridge, an OpenAPI/REST surface) and a stale inventory silently drops them.
The entrypoint manifest is the coverage SPINE (mechanical, not a hand list). A component list is
too coarse — a route can fall between hand-picked cell anchors and be silently skipped (the miss:
GET /users/:id/calendar-heatmap was never read because one cell listed six other controllers and
another opened the file at the wrong route). Before scheduling cells, deterministically extract
every externally-reachable entrypoint — controller routes (method, path, class+method decorators,
inherited auth defaults, handler span), framework filesystem routes (e.g. SvelteKit +page.ts /
+page.server.ts loads & actions), and statically-enumerable RPC/event handlers — each with a stable
entry_id. A cell may claim coverage of an entrypoint only by recording its exact entry_id +
handler span + effective audience/auth + primary callee + the boundary-scout check-ids run; opening
one line in a controller does not cover the controller, and a wildcard "all routes covered" receipt is
forbidden. Round closure fails when manifest_entry_ids − covered − approved_exceptions is non-empty
— a deterministic diff, not a model "completeness critic" re-reading its own work. Keep hand-picked
non-route anchors (repositories, background workers, parser/process sinks, state transitions) —
routes are the spine, not the whole skeleton.
Deterministic front-load — cheap ground truth before the LLM loop (Round 0)
Deterministic tools are fast, hallucination-free, and refusal-free. Run them before the first LLM round and don't spend inference rediscovering what they already know:
- Cross-run Knowledge Base (
kb/) — a MONOTONIC-SCRUTINY signal: it can only ever make you hunt MORE. If this target was hunted before, a durable KB sits beside the ledger ($KB/kb.json). It stores no coverage and no "this is safe" signal; it can only raise scrutiny. Load it into the planner context only, AFTER you have freshly enumerated the whole inventory this run (the completeness gate below) intoinv.txt: scripts/raptor-loop-kb load --kb "$KB" --target "" --inventory inv.txt- Priority, never coverage.
priority_orderputs confirmed-dirty (a prior confirmed/corrected finding) first, then prior-rejection components (recheck), then everything else. Everycurrent_stateisuncoveredand nothing is ever deprioritized — historical work NEVER counts as current coverage and never pushes a component out of scope. Draw this run's slices from the fresh inventory; the KB only changes the order. - Rejections are recheck ANNOTATIONS, never an exclusion. Each
annotations[]entry says "previously rejected for X — recheck X and ALL delivery vectors (path/query/body/cookie/header/enc)"; astaleone (tree changed since) reads "FULLY OPEN, recheck from scratch." The candidate still runs the full generate → judge → live-verify chain from raw — the annotation only tells you where to look harder. - Round 0 is never skipped. The KB seeds
/sca, inventory enumeration, prior-art recon, and mapping — it does not replace them. New CVEs, new lockfiles, new advisories, and new modules are seen every run. - Isolation (MANDATORY). The payload stays in the planner/orchestrator context. Never feed a prior
rejection or summary into the independent generator, judge, or live-verifier — they reason from raw source
- current scope. Injecting history re-creates the anchoring "Generate → judge, both from raw" prevents.
- Priority, never coverage.
- Known-CVE deps (
/sca). SBOM + dependency-CVE audit is deterministic and cheap. Run it first, log the hits, and exclude those packages from the LLM hunt scope — the model's budget is for the bespoke bugs a scanner can't find, not for re-deriving a public CVE in a pinned dep. - Native-target reachability ground truth (binary-oracle). For C/C++/Rust/Go targets with a
locally-built debug binary, the binary-oracle is deterministic reachability: it joins the source
inventory to the binary via DWARF + nm and marks each function
symbol_present/inlined/folded(survived compilation) orabsent(compiler/linker removed it). It is auto-detected and on by default in/agentic,/codeqland/audit— pass--binary <path>for an explicit build,--binary-autofor a louder auto-detect,--target-kind auto|library|hybrid|application(defaultauto). Anabsentverdict hard-suppresses the finding before the LLM ever sees it (/agenticand/codeqllog it tosuppressions.jsonl;/auditsuppresses without logging, so that path has no audit trail). Asymbol_present/inlinedverdict refutes a later "that's dead code" kill — but note it proves survival in that binary, not reachability; "no caller" is a separate claim needing--binary-edgesor a source call graph.--no-binary-oracleis/codeqland/auditonly./agenticnever registered it (its own argparse block declares--binary,--binary-auto,--binary-edgesand nothing else), so the flag is silently ignored there (worse:/agentic's own code reads and recommends the flag it never registered) —/agentic's escape hatch is--allow-unreachable, which bypasses the reachability suppression chokepoint wholesale, not just this oracle. It does not stop binary auto-detection or inventory enrichment: the verdicts are still computed and still annotate the inventory.- This is build-specific compilation-survival evidence, not source ground truth.
--binaryis validated as "is a file" and nothing binds it to the current commit, dirty tree or build config, so a stale or partial binary can hard-suppress live source findings. Treat a verdict as evidence about that build; when the binary's provenance is not pinned to the audited tree, do not let it suppress. /auditsuppresses on MIXED-TIER evidence — prefer/audit --no-binary-oracle. The canonical reachability path refuses to suppress when any contributing binary is below full-DWARF tier, but/audit's extraction keeps anabsentverdict when merely one binary has full DWARF, then drops path, line and tier and keys by bare function name. That defeats both thesymbol_only-never-demotes rule and the two-signal gate.- The oracle is not the only pre-LLM kill.
module_abortsandlexical_deadare also suppression-eligible structural witnesses, and/agenticdeterministically marks test fixturescleanand skips the LLM entirely, with no CLI switch to disable it./audit's small-function batch path has its own gate: a function hitting_dead_code_reasonis committeddormantwithevidence_tool="reachability:dead_code"without any LLM review at all. "Binaryabsentis the one permitted hard-suppressor" is this methodology's policy, not the framework's behaviour — import every fixture/structural suppression as anopencandidate unless a loop-owned receipt backs it.
- CodeQL on C/C++ is BUILDLESS by default — its negatives are scope-limited. RAPTOR now creates
C/C++ databases with
codeql database create --build-mode=none, so the untrusted repo's build scripts never execute; detected build commands are ignored unless traced extraction was explicitly selected, and a CodeQL CLI older than 2.16 fails the create rather than silently falling back to a traced build. Traced extraction is opt-in —--traced-build, or an explicit per-language--build-command— and is deliberately independent of--trust-repo: trusting the repo buys you no traced build, and a traced run still refuses on unsafe CodeQL pack config. The cost is coverage: build-generated headers are invisible. RAPTOR surfaces this, but weakly — the logged number is a regex count of extractor-diagnostic lines mentioning unresolved includes, it collapses to zero on a parse failure, zero prints a generic warning with no number, the summary runs only after a fresh successful create (a cached database returns before it), and the count is never persisted into database metadata. Consequence for this loop: a "CodeQL found nothing here" receipt on a C/C++ target must record whether extraction was buildless or traced and, when buildless, retain the original creation log (or an independently captured extractor-diagnostic inventory). A zero or a missing count is not proof that headers resolved. Without that provenance the sweep is partial coverage presented as complete — the same fraud class as a capped rule sweep. Other languages are unaffected. /scancan silently lose registry rule packs. The Semgrep preflight opens a 3-second TCP probe (to the configured HTTPS proxy's first hop when one is set, elsesemgrep.dev:443); on failure it logs a warning, drops every uncached registry pack, and scans on with what remains — exiting 0. A/scanreceipt must carry the resolved pack list, the applicable-rule count, and any "dropping uncached registry pack(s)" warning. A clean exit after pack-dropping is not a full sweep.- Project trust markers change later runs — announced, but easy to miss.
/project trust config|build|dynamic(libexec/raptor-project-manager trust <marker>) persists an operator assertion that RAPTOR applies to every subsequent run:config→--trust-repoandbuild→--traced-buildfor/agenticand/codeql;dynamic→ dynamic validation for/audit. Direct/scandoes not consume them (only flags forwarded from/agenticor/codeqlcarry that state). Precedence is explicit-negative > explicit-positive > marker > off, and RAPTOR prints a one-line banner whenever a marker actually affected the run — so this is visible, not silent, provided you read it. Record the effective flags and the active marker set per round inTRIED.md: flipping a marker mid-engagement re-characterises the provenance of every receipt that follows it. - Target's OWN known vulns + upstream fixes (prior-art recon) — MANDATORY, not optional. Pull
the TARGET application's history, not just its dependencies, BEFORE the LLM loop:
- Its CVE/GHSA record — OSV (
POST https://api.osv.dev/v1/query{"package":{"name":..,"ecosystem":..}}), NVD (keywordSearch=<name>),gh api repos/<o>/<r>/security-advisories. Every past CVE names a vulnerable sink class + file; treat each as a post-fix variant re-audit lead (is the fix complete? does a sibling path / different delivery vector bypass it? is the sink still reachable?). It also calibrates severity — if the vendor/CVE process treated an authenticated/admin-area bug of class X as CVE-worthy before, don't dismiss your class-X finding as "admin-only, informational." - Upstream open + recently-merged security PRs and recent security commits —
gh pr list --state open,gh search,git log <last-release>..HEAD -- <hot files>, patch-diff<vuln-tag>..<fixed-tag>. An open or queued fix is a vendor-acknowledged live bug in the current release: seed it as a known-real finding, and NEVER reject a candidate that an upstream PR is actively fixing (this is exactly how a real finding gets wrongly killed — the vendor was patching it while the audit rejected it). - Public PoC/exploit search —
gh search repos/code <name>, WebSearch<name> exploit/PoC. This is the standing "search existing PoCs / CVE-breadth" discipline applied to a FROM-SCRATCH AUDIT, not only to known-CVE reproduction. Skipping it means re-deriving — or wrongly rejecting — bugs the vendor already flagged. Log what you checked inTRIED.mdas methodology evidence.
- Its CVE/GHSA record — OSV (
- STRIDE template (
/threat-model build). One pass over the/understand --maprecon that yields trust boundaries, entry points, and a per-boundary STRIDE classification. Its output is a reusable template that seeds every later round's bug-class lenses — it tells the generators where privilege changes and which STRIDE class to hunt at each boundary. Recon and the threat model are infrastructure for the loop, not outputs — never report them as findings. - Semgrep anchors seeded from the trust boundaries. Turn each threat-model trust boundary into
a Semgrep pattern and run those rules alongside the generators every round. Semgrep gives
fast deterministic anchors at exactly the boundaries the threat model flagged; the LLM reasons
about the semantics, data flow, and exploitability pattern-matching can't reach. The union of
Semgrep hits and LLM candidates is the candidate pool (high recall, high noise — by design). A
Semgrep hit is a lead, not a finding: it enters the same finding contract and the same
from-raw judge as any LLM candidate. Keep Semgrep in the generator (recall) seat only — a
pattern match never stands in for the judge.
- Standing anchor — check-view ≠ use-view. Beyond the per-boundary rules, always seed one anchor
for the policy–execution interpretation differential class (
references/vuln-class-discovery.md): a security-relevant boolean derived fromequalsIgnoreCase/regionMatches(true,…)/startsWith/indexOf(x)==0/ a decode-then-match on a name/URI/id that guards a privileged branch. It is a high-recall/high-noise candidate generator, never a detector: promote a hit only when a judge can exhibit a witnessxthe control interpretation misses but the action interpretation accepts, and suppress it when the identifier is not attacker-influenced or both consumers share one canonicalization. (Both our own scheduler-alias false-negative and the CVE-2026-45505 RCE reduce to this class — and note grep alone under-matchesregionMatches(trueand over-matches benignequalsIgnoreCase, so treat it as an AST/taint-aware anchor, not a text scan.)
- Standing anchor — check-view ≠ use-view. Beyond the per-boundary rules, always seed one anchor
for the policy–execution interpretation differential class (
Per-class discovery method (load when a bug-class lens is active)
The altitude traversal says where to look; references/vuln-class-discovery.md says how to
find and confirm each vulnerability class, generically. For every bug-class lens a round runs,
load that file's matching section and fill its five slots against the target — source class,
sink class (by mechanism), violated invariant (the finding's root cause), enumeration strategy
(derive the complete sink set from the inventory), confirmation oracle. Stack-agnostic by design;
concrete signatures stay in the deterministic layer (Semgrep, /sca).
Generate → judge, both from raw
Separating generation from verification is the single biggest quality lever:
- The judge kills false positives — independent skeptic, prompted to refute. It defaults to "not a bug" only when the defect itself is uncertain (the code path isn't real, the entry isn't attacker-controlled, the sink isn't dangerous). It does not default to "not a bug" because some unobserved layer might mitigate it — see below.
- The act of re-reading from raw to verify surfaces new findings the generator missed.
- "From raw" is load-bearing. A judge that reads the generator's summary rubber-stamps it. A judge that reads the source re-examines it. Always feed the judge the code, not the claim.
What counts as a valid refutation (and what doesn't). The judge may only kill or downgrade a finding for a reason it can see in the artifacts:
- VALID: the cited code doesn't do what the claim says; the entry point isn't attacker-reachable; a mitigating check is present in code you can read; it's designed behavior under the established trust model.
- INVALID: "a well-built server probably enforces this," "the framework likely handles it," "presumably there's authz upstream," "the server surely re-hashes/re-validates." Assuming an unseen layer is secure is not refutation — it is the single most common false-negative in this methodology. The whole reason for the test is that the unseen layer might be broken. Don't assume the unseen layer is secure.
- INVALID (reachability / gating): "that component isn't built/loaded," "it needs a non-default
config," "it's behind a flag," "the trigger needs an impractical number of iterations" — asserted
without checking. A reachability or gating claim kills a finding, so it carries the finding's own
proof burden: verify it against the observable build + default config (the build flag, whether
a default deployment exposes the entry point, the real trigger path) before it counts as
refutation. A capability that ships and is enabled by default is in scope even when it's packaged
as an optional-looking "module" or "plugin." Don't assume the gate is closed. For native
(C/C++/Rust/Go) targets, the binary-oracle mechanizes exactly this check: an
absentverdict is an observed dead-code reject (the function isn't in the shipped binary), whilesymbol_present/inlined/foldedrefutes a "compiled away / dead code" kill — so a compiled-away claim is only a valid refutation when the oracle (or an equivalent DWARF/nm check on the actual build) confirms it, never on assertion.
When the only barrier between a finding and exploitation is a layer you cannot observe (server
code you don't have, a runtime you can't run, a config out of scope), the correct verdict is
not "rejected/low" — it is needs-live-validation (see the disposition below). Preserve
the worst-case severity and emit the exact, safe test that would confirm it.
For higher-stakes hunts, use a small panel (e.g. 3 skeptics with distinct lenses —
correctness, reachability/exploitability, does-it-actually-reproduce) and require a majority
to keep a finding. A panel may move a finding to needs-live-validation, but a finding is only
rejected when a majority can point to an observed reason it cannot be a bug.
Concurrency and isolation
Run N independent reasoners (default N=10) with full isolation — each works from raw, blind to the others' conclusions. This is an ensemble: isolation preserves diversity, which is the coverage. Drop isolation and they converge on the first finding (groupthink). Use a pipeline so generation, judging, and verification stream rather than barrier-block.
The per-anchor boundary scout + lossless leads (the recall-miss and false-clear fixes)
A cell scoped to ONE headline bug-class lens is structurally blind to a different class living at
the same code. This is the dominant real-world recall failure — measured, on a re-hunt that used
stronger models than the run it was compared against: an SQLi-lens cell read searchStatistics and
missed the cross-tenant authz omission there; an XSS-lens cell read the license route and missed
the CSRF state-changing-GET; an authz-gate cell proved a shared-link "correctly gated" and never
looked at the response mapper leaking emails; an IDOR cell read tag.addAssets and mistook
Permission.AssetShare (owner OR partner) for an owner-only gate. Every one of those was read
and still didn't surface. Two mechanisms fix it — neither is "run five full hunts per anchor":
1. A short, isolated boundary scout after the primary lens. For each externally-reachable or
security-sensitive anchor, run ONE extra reasoner that is blind to the primary generator's verdict
and coverage prose (feed it the anchors and the shared cards below, never "authz is soundly
implemented" — otherwise it rationalizes the same omission). It evaluates only the boundary checks
that apply, emitting a structured observation or a source-backed N/A for each:
- ENTRY — the real audience/auth decorator (incl. inherited class defaults); any state-changing
GET / page-load /
load()that a link click triggers (CSRF / forced-action). (F-17, F-25) - SUBJECT — bind the actor to every path/body id, session id, and target resource; expand a
permission's semantics, never trust its name (
AssetShare= owner-or-partner-or-shared-link, resolved to its predicate spans). (F-20, F-24) - SECURITY STATE — visibility (
Locked/Hidden), elevation/PIN, revocation, lease/session renewal, stale memberships. (F-34, F-02/F-08) - OUTPUT — follow the final serializer/mapper graph and enumerate the sensitive fields it emits against every route audience (unauth / shared-link / user / partner / admin). (F-18, F-19)
- DANGEROUS SINK — for any interpreter/parser/process, resolve the data channel, invocation
mode/options, interpreter grammar/metacommands, and execution identity. (F-22
psql \!) Reuse capability/mapper cards across cells so this is cheap: one card perPermission.*(its real predicate + accepted principals), per response mapper (fields × audiences), per visibility/elevation check, per process sink (argv/stdin/identity). Cost is ~+20–35% generator work, not 5×.
2. Leads are lossless — a concrete defect cannot die in prose. Cell output is typed: a
hypothesis (a question — "does statistics have the same visibility bug?") may stay a lead until
traced, but a defect_observation — a reachable operation + a specific missing/mismatched
binding, filter, cleanup, elevation, or policy condition, with source spans for both the operation and
the control/asymmetry — automatically opens or joins a candidate and runs the full
generate→judge→cross-vendor chain. A defect_observation may never be marked safe / latent /
intended / defense-in-depth in a coverage_note; coverage notes are non-dispositive (they carry
coverage-receipt references, not security verdicts). This is the exact gap behind the three
false-clears: the HLS sessionId-not-bound-to-caller (F-24), the "duplicate groups are always
single-owner" invariant (F-07), and the "mapUser email is intended baseline" dismissal (F-18) were
all present in the run's prose and never became candidates, so no gate ever fired. Two of those are
now typed rejection receipts the ledger enforces: an invariant kill must inventory every writer
to the invariant-bearing field (incl. bulk/import/restore/deser — the F-07 miss was the unlisted PUT /assets {duplicateId} writer) plus a counterexample attempt; an intended-behavior kill must cite a
real policy artifact — "another endpoint already exposes it" is a second bug, not intent, and a
contrary privacy control (publicUsers) is evidence against. Dedup observations by root
span/resource/control so one defect is one candidate, not two chains.
3. Bounded security-differential cards feed the scout. Several misses were near-sibling
contradictions a compact mechanical diff puts straight in front of the reasoner: search() vs
getStatistics() visibility predicates (F-42), searchStatistics omitting visibility its siblings
pass (F-23), bulk updateAll cleanup vs the singular update (F-02/F-08), a download path lacking the
elevation its sibling timeline/search paths enforce (F-34). Generate security_contrast cards —
singular-vs-bulk on a resource, sibling query methods differing in owner/visibility/deleted/elevation/
session/permission predicates, omitted security args at sibling callsites, all writers to an
invariant field — restricted to the same resource/operation family (AST/call-fingerprint + LSP
references; batch, never top-K truncate), and attach them to the scout. Don't launch a separate full
generator.
Discovery is severity-neutral. "No LOW-padding" is a reporting rule, not a generation filter:
a concrete reachable disclosure/control defect enters the candidate pipeline regardless of tentative
severity — severity is assigned after evidence. Pre-judging externalDomain as "by-design public"
suppressed a credential-bearing-URL leak (F-33) before it could be evaluated. The concrete-defect
threshold (not a severity floor) is what controls candidate volume.
Invalid-cell lint. A cell whose only output is a placeholder/stub candidate (fields literal
"test", evidence file:"a" line 1, no source spans) is an invalid execution, not a dry cell — it
fails closure and reruns (the ledger's add refuses it). A "dry" verdict requires source-backed
coverage receipts.
The attempt ledger (what makes "loop forever" productive)
This is the part most people omit, and it's what turns an infinite loop from waste into a converging search. Maintain two files in the run's output directory:
TRIED.md— every (altitude, slice, bug-class, approach) attempted this engagement, with the outcome (nothing / lead / confirmed / refuted). Before each round, read it; never repeat a cell within the current engagement.TRIED.mdis engagement-scoped — it is NOT the cross-engagement coverage authority. On a new engagement (fresh clone / new release / changed tree — a changed target identity) start a newTRIED.md: every component beginsUNCOVEREDregardless of what any prior engagement covered. Cross-engagement history enters only as the KB's monotonic priority/recheck signal, never as coverage.FINDINGS.md(orEXISTING-FINDINGS.md) — confirmed findings, used as an exclusion set so the loop doesn't re-surface what's already logged.
Dedup new candidates against FINDINGS.md and against the judge-rejected set for the current engagement
only — this is what stops a rejected finding reappearing every round within a run so the loop converges.
This dedup is strictly engagement-scoped; it is NOT a cross-engagement exclusion. A candidate is suppressed
only when it is the same candidate instance already adjudicated in this engagement, against the same tree
identity. Across engagements — or the moment the tree identity changes — confirmed/rejected history is not
an exclusion set: it becomes a KB recheck annotation and the resurfaced candidate re-enters the full
generate → judge → live-verify chain from raw. A changed-code variant that merely shares an old signature must
never be dropped on that signature. The ledger gives "try something new every time" a reference point — but
that reference point resets at the engagement boundary.
Cross-run Knowledge Base — a monotonic-scrutiny layer above the ledger
TRIED.md / FINDINGS.md are the engagement-scoped ledger (above). The Knowledge Base (kb/) is the
durable cross-engagement layer, and it obeys one rule: it can only ever RAISE scrutiny. It stores no
coverage and no "safe" signal, does no deprioritization, and never excludes a candidate. It emits exactly two
things: confirmed-dirty components (hunt first) and prior-rejection recheck annotations. Because every signal
only adds effort, the KB cannot semantically suppress, cover, or clear a candidate: given the freshly
enumerated inventory is accepted in full (the helper fails closed on any inventory cap/invalid entry) and the
completeness gate is honoured, a maximally poisoned or stale KB can only change hunt order and add recheck
work — the worst case is wasted budget re-hunting a real component, never a skipped one.
Only structured, locally-recomputed facts steer it. Component identity from a fresh enumeration (a name not
in this run's inventory is ignored); source identity + drift from local digests; finding disposition from
typed, schema-validated records the orchestrator writes; evidence spans resolved under the canonical target
root. Trajectory final_summary / "gave-up" / tool-error prose is attacker-influenced in origin, so it is
display-only telemetry and can never become a rejection, a priority, or any durable state.
Where it lives (RAPTOR-owned, OUTSIDE the scanned tree): <project_output_dir>/kb/ (or
out/kb/<target_path_id>/). The helper refuses a symlinked KB path, a KB inside the target, or an
unowned/corrupt store; writes are atomic + locked + fsynced; the inbox is race-free (all appends take the
lock; synthesize rotates it aside before folding). Schema + learnings.jsonl grammar: references/kb-schema.md.
Enable trajectory capture. Pass the run's output directory to every /understand and /cve-diff
call — the FLAG is what sets the location, and the flag name differs per command:
/understand --out "$OUTPUT_DIR", /cve-diff --output-dir "$OUTPUT_DIR" (--out is not a
/cve-diff flag and is rejected). Exporting RAPTOR_TRAJECTORY_DIR yourself is a no-op: both
libexecs assign os.environ["RAPTOR_TRAJECTORY_DIR"] from their own resolved output dir, overwriting
whatever you exported. But the flag only sets the location — it does not guarantee a trajectory.
A trajectory is written only by an agent tool-use loop, and /understand reaches one on a minority
of its routes. /cve-diff opts its agent loop in under --output-dir. /understand is mode-routed
(dispatch: skill): source-tree --map, --hunt, --trace, --teach and --study run in-session
(you are the LLM) and write lifecycle and result artifacts — variants.json, context-map.json — but
no trajectory; the libexec rejects source-tree --map and refuses --hunt / --trace without
--model. Even on the libexec path only the LLM hunt/trace loop persists: with --hunt-tool auto (the
default) a C/C++ target with spatch on PATH selects the Coccinelle backend, which makes one
rule-writing model call and no tool-use loop; binary --map sets the env var but never runs one either.
So for a hunt trajectory pass --model <name> --hunt-tool llm, and keep the Coccinelle rule + match
output as a mechanical sweep artifact when auto picks that backend instead. Verify by existence:
no trajectories/<run_id>/trajectory.json means no reflect input for that round — never assume the
record landed. (/agentic does not persist trajectories either; for that stage reflect has no input
and the steering records come from adjudication.)
Stop condition — loop until dry, not literally forever
"Continue no matter what" is the right attitude but the wrong literal rule — it burns budget on diminishing returns. Concretely: stop after K consecutive rounds (default K=2) that surface nothing new across all remaining uncovered cells. If a round is dry, increment the counter and switch to an under-explored altitude or bug class before giving up. Report what was left uncovered rather than implying the grid was exhausted.
What counts as a finding — the bar, and how to rate it
The loop generates candidates aggressively. This is the bar a candidate must clear before it counts. The loop has no schema validator, so enforce this contract yourself.
The finding contract. A reportable finding states, every time:
- Root cause, templated: "
<function>in<file>does not<missing check>, allowing<consequence>." Name the function and file where the defect actually lives. - A trace that starts at an attacker-controlled entry point and ends at a dangerous sink. If you can't name both endpoints from the map, the trace is incomplete — that's a lead, not a finding. Check this invariant by hand on every finding (the first hop is an entry point, the last is a sink).
- A concrete attack: who the attacker is, the exact input / request / action sequence, and the observable result. "An attacker could theoretically…" is not a finding.
- Severity as likelihood × impact (below) and confidence with the reason you scored it.
Severity = likelihood × impact. Rate on both axes; don't inflate:
- Critical — unauth RCE, full data dump, admin takeover without credentials.
- High — authed RCE, SQLi with exfiltration, stored XSS firing for all users, auth bypass, or an explicit role/permission boundary completely defeated for a consequential action.
- Medium — XSS needing specific conditions, CSRF with real state change, secret/credential disclosure, logic bypass confined to the attacker's own data or needing uncommon conditions.
- Low — non-secret info disclosure, DoS needing sustained effort, hardening gaps.
- If you can't describe the concrete damage, the severity is lower than you think.
Two-axis rating when evidence is partial. When you can see only part of the system (client code without the server, one service of many, no running instance), rate two numbers — never one collapsed number:
- Confirmed severity — what you can prove from the artifacts in hand.
- Potential severity — the worst case if the unobserved assumption turns out insecure.
Carry the higher one into triage as needs-live-validation. Never collapse potential severity
to Low just because the confirming layer is out of view. A client-side trace showing an
unauthenticated password-reset request with a client-controlled target id and a client-controlled
"skip the email check" flag is a potential critical the moment the path is real — even though the
server isn't in scope. That is a validate-now item, not a low hardening note. (This is not
hypothetical: that exact finding was downgraded to "low/server-dependent" by an over-eager judge
and later confirmed as live, unauthenticated mass account takeover.)
Rate the sink, not the symptom — trace consequence before any final Low. When a finding's claimed
impact depends on attacker-influenced data or control reaching a security-sensitive operation
(command execution, unsafe query construction, unsafe deserialization, template eval, arbitrary file
access, or restore/import content being interpreted as code), record the entry→operation trace and
rate the operation's consequence, preconditions, barriers, and execution privilege — never the entry
symptom alone. A finding whose disposition is written from the symptom ("missing @Authenticated",
"unauth endpoint", "missing check") without following the invoked operation is a lead, not a rated
finding: do not finalize it Low / hardening / not-security while the downstream consequence or an
alternate ingress is unresolved — record it as a lead with a potential-impact hypothesis in a
needs-trace/needs-live state. Operation-first, even for an "auth" finding: when you flag an
endpoint as unauth/missing-auth, inspect what it invokes — if that is a restore/import/eval/subprocess
or a query builder, trace it before rating. (Two honest boundaries: a protected operation whose impact
is already evident supports an authorization finding without tracing to a separate sink; and ordinary
parameterized SQL, safe deserialization, or routine file access is not automatically a
dangerous-sink finding — the trigger is attacker-influence reaching the operation with plausible
High/Critical impact, not the mere presence of the API.)
Disposition — every survivor gets exactly one:
- confirmed — proven exploitable from the artifacts or a run. Rate confirmed severity.
- needs-live-validation — the defect pattern is real and reachable in what you can see, but the final proof depends on an unobserved layer. Required fields: the exact minimal, safe validation step (the request/command/test, plus the expected vulnerable-vs-safe response) and the potential severity. A first-class outcome, not a soft reject.
- corrected — real but mis-scoped/mis-rated; give the accurate version. Use this to fix observed errors, not to downgrade for unseen mitigations.
- rejected — a majority can cite an observed reason it is not a bug (wrong code, unreachable, mitigation present in the code, designed behavior). "Probably handled elsewhere" is not such a reason.
Disposition receipts — the reasoner proposes, the orchestrator certifies
A disposition written by the same generator/judge that reasoned the finding is a claim, not
proof. The weak-tier seats this loop fans out to (Sonnet judges, uncensored local generators) will
state a discipline in prose and then skip it — so a disposition is not a sentence a reasoner emits,
it is a state transition the orchestrator refuses until an evidence receipt exists. Division of
labour: the reasoner proposes the disposition and the search recipe; the tool harness executes and
emits the observation; the orchestrator (this trusted session) owns the transition and populates the
fields a model could otherwise fabricate. A cross_vendor verdict or an oracle result is filled
from an actual dispatched job, never from the reasoner's own text — a self-attributed
cross_vendor=…:UPHOLD line only makes the fraud easier to format. Enforce this at the moment of
the disposition, not at end-of-run: the KB already validates the rejection bundle at synthesize
(_classify_outcome), which is too late — a real bug rejected in round 3 is gone long before round
20's fold runs.
Each terminal disposition carries a typed receipt, stored in a run-dir sidecar (an event log),
not dumped into the human report — the report renders a one-line reference (C-123 confirmed [R-91]); verbose receipt blocks in the report are their own artifact-fatigue failure. The three:
- confirmation receipt —
oracle_class, the replay command / fixture id, the input hash, a machine predicate for the expected signal, and the observed-artifact hashes, emitted by the verifier harness. "It crashed / looks exploitable" cannot fill the predicate. A bare crash may confirm a reproducible DoS (its own oracle class), but it does not fill the predicate for memory-corruption exploitability — that wants the sanitizer class + a stack fingerprint. Oracle classes are open, not the three-item list a first draft reaches for: sanitizer, differential, authorization, state-change/integrity, disclosure, execution-marker, dos, timing, crypto-failure, parser-differential, policy-bypass, race-invariant. - rejection receipt — a reason-specific counter-hypothesis (the concrete proposition that
kills the finding); a per-(vector × transform) result — an execution receipt or a rationale'd
N/A, because a bare list of vectors proves no test ran, and
encis a transform dimension across vectors, not a vector; build/config digests wherever the kill is a reachability claim; and an orchestrator-dispatched cross-vendor verdict that is neitheroverturnnorinconclusive. Miss any and the transition fails — the candidate staysopen/needs-live-validation, never silently rejected. (Rejection is not a severity — do not "downgrade" an incomplete one.) - material-downgrade receipt — a severity downgrade is a partial rejection, so it carries the
same burden. A material downgrade is DERIVED, never a label you pick: a finding whose adopted
potential high-water-mark is High/Critical landing at an effective-final Low/hardening/
not-security (via
corrected, or extinguished asduplicate/out-of-scopewithout a preserving canonical link) — regardless of the transition name, and the high-water-mark is retained so aHigh→Medium→Lowstair-step still trips it. It requires: the severe hypothesis being negated; a consequence/impact-cap trace (what the sink actually does / why the ceiling is Low); a bounded inventory of the sink's statically-discoverable distinct trigger paths, each with path-scoped evidence (a failed test on one ingress is evidence for that path only; a path leftneeds-live/needs-reviewis not cleared); gating digests; and an orchestrator-dispatched cross-vendor verdict ofconcur_downgrade(anoverturn/inconclusive/upholddoes not permit it). This is the gate the F-22 restore-RCE downgrade escaped: aHigh→Lowre-rate recorded as acorrectedslid past a reject-only gate.Critical→HighandHigh→Mediumare ordinary corrections (not material); a High-potential finding kept open asneeds-livewith a Low observed result on one path is fine — it is alive, not downgraded. Enforced byraptor-loop-ledger transition; the closure gate re-flags any severe finding sitting at effective-Low with no such receipt (anti-laundering). - dirty-sweep receipt — the enumeration recipe (the commands / semantic queries / call-graph
root), the discovered-site set with each site's own disposition, the subsystem (de)serialization
loader identified and checked, and a re-runnable site-set hash.
deser_loader=yeswith no denominator is worthless; the judge re-runs the recipe and diffs the resulting site set.
PENDING actions — the follow-up that must not vanish. A confirmation event auto-enqueues its
required follow-up as an orchestrator-owned action: a confirmed memory-safety bug → a DIRTY_SWEEP
on its file; a server-dependent finding → a live_validation; an incomplete rejection → a
complete_rejection_bundle; an incomplete material downgrade → a complete_material_downgrade_bundle;
and a severe-sink hypothesis (any candidate whose potential ≥ High names a sink) → a sink-keyed
enumerate_trigger_paths (shared by every candidate reaching that sink, so it enumerates the sink's
callers/routes/dispatchers/loaders once — this is what stops a re-sweep from silently omitting the
files that actually reach the sink). A file cannot be marked closed, and the report cannot render, while
its action is open. This is deliberately event-driven, not memory-driven: fable-method's evals show
a forced artifact transfers when it annotates an action in hand and fails when it asks a model to
notice an absence — so the sweep is enqueued by the confirmation, never left for a later "did I
forget to sweep?" pass (which is exactly the pass that doesn't transfer).
Bias note: this methodology is tuned against false positives. A false negative — dismissing a real, high-impact, server-dependent finding as "theoretical/low" — is just as much a failure, and is the easier mistake to make once you are in refute mode. Hold both error types in view.
Comparable-baseline triage. For each candidate ask: what mainstream software is comparable, does it carry this same pattern, and has it ever been exploited there? Same pattern + exploited elsewhere → stronger finding. Same pattern + never exploited in years → understand why before reporting. Use the baseline to focus effort, never to auto-dismiss.
Anti-patterns — these turn a hunt into noise:
| Anti-pattern | Why it's wrong |
|---|---|
| Rating a defense-in-depth gap as a real finding — High, or Medium "to be safe" | If another layer you can observe already blocks the attack, the missing layer is a hardening note / Low at most. E.g. a cookie missing secure/sameSite when CSRF tokens already stop CSRF and HSTS + TLS-redirect already stop downgrade is Low/hardening. But the blocking layer must be visible in the artifacts — "a server probably validates this" is not an observed mitigation (see the false-negative row below). |
| Treating designed behavior as a bug | If the trust model says admins are fully trusted, admin-does-admin-thing isn't a finding. Establish the trust model first. |
| Using "potential" / "theoretical" as a synonym for "dismissed" | Two different cases — don't conflate. (a) Theoretical because you haven't finished reasoning over code you do have → finish it; if inert, drop it. (b) Unconfirmable because the deciding layer is out of scope (server you don't have, runtime you can't run) → that is not "theoretical/low," it is needs-live-validation with worst-case severity and an exact safe test. |
| Padding with LOWs | Three real MEDIUMs beat ten LOWs. Volume is not thoroughness. |
| Exploits built on assumed parser/runtime behavior | The most convincing false positives. If the exploit depends on how a parser/runtime treats input, cite the spec or test it — don't reason from intuition. |
| Listing every OWASP/checklist deviation | A checklist is not a bug list; every real application makes tradeoffs. |
| Dismissing a finding because an unseen layer "probably" handles it | The defining false-negative of refutation. "A real server would enforce authz / verify the OTP / re-hash the password" is an assumption, not evidence. If you cannot see the enforcing layer, the verdict is needs-live-validation, not rejected — the test exists because that layer may be broken. (Canonical miss: an unauthenticated UpdateUserPassword that trusts a client-supplied user id + a client-supplied "skip email check" boolean, dismissed as "the server surely binds the reset" — it did not.) |
| Killing a finding (or a whole subsystem) on an unverified gating / reachability claim | The mirror of the row above. "That component isn't built/loaded," "needs a non-default config," "the trigger needs an impractical number of ops" are finding-killers, so they carry the finding's own proof burden — verify against the observable build + default config before dismissing. A capability enabled by default is not "gated" just because it's packaged as an optional-looking module/plugin. |
Fraud classes the closure gate hunts (security-specialized)
The anti-patterns above are reasoning errors. These are dishonesty / costume-rigor signals — a report claiming work that the receipts and logs do not back. The closure gate (see Reporting) and the cross-vendor judge actively search for them; they are not just formatting checks. Each has a mechanical tell:
| Fraud class | The tell (what the gate re-derives) |
|---|---|
| Fabricated PoC output | The claimed observation has no matching artifact hash / harness run in the event log. |
| Stale PoC | The confirmation receipt's build_digest / commit ≠ the report's target identity — it reproduced against a different build. |
| Crash inflated to RCE | oracle_class=dos (bare crash) but the finding is rated as memory-corruption exploitability with no sanitizer-class receipt. |
| Unexecuted vector listed as tested | A delivery_vectors_tested entry with no per-vector execution receipt behind it. |
Rationale-free N/A |
A vector marked N/A with no route/sink reason for why it cannot reach the sink. |
| Self-attributed cross-vendor | cross_vendor_result present but no orchestrator-dispatched review job produced it. |
| Read-derived coverage | A component marked covered from file reads alone, with no logged hunt action (generate→judge→verify) against it. |
| Grep-only sweep | A DIRTY_SWEEP whose enumeration recipe is a text grep where the class demands a semantic / call-graph enumeration. |
| Server-dependent finding collapsed | An auth/IDOR/reset/deser finding marked rejected/Low when the deciding layer was out of view (must be needs-live-validation). |
| Severe finding downgraded to Low without the bundle | A High/Critical-potential finding re-rated to Low (via corrected/duplicate), or its severe sink not swept, on a single-ingress test — no material-downgrade receipt (trigger-path inventory + concur_downgrade). The F-22 restore-RCE failure: gated one entry, missed the sink + the second entry. |
| Assumed-default config | "Default deployment exposes this" with no observed config/build evidence. |
| Dedup across distinct entries | One sanitizer finding used to cover several distinct attacker entry points reaching the same sink. |
| Dropped pending/inconclusive | The report omits candidates still open / needs-live-validation, reading as done while work is outstanding. |
| Skip counted as review | A cell marked covered off an /audit record whose status=clean carries evidence_tool=triage or prefilter — no rule ran, no loop-owned hunt receipt. Triage can fire before the source context is built; prefilter is a bounded heuristic, not a hypothesis-directed hunt. The tell is that pairing in .audit-log.jsonl with no receipt joined to it. |
| Capped or completeness-unknown sweep sold as complete | A rule sweep used as the discovered-site denominator when completeness is not positively established — capped=true, or complete≠true, or total_matches absent, or no re-runnable site-set hash. A hit count landing exactly on a known cap is a mandatory re-run trigger, not proof on its own; and absence of a cap flag proves nothing, because the /audit adapter drops it and replayed rules never set it. Fail closed. |
| Tacit workaround sold as reproduction | The author needed an undocumented retry, dependency, privilege/config change, remembered path, disabled control, or manual repair, but no intervention + resolution receipt exists. |
| Author-coached handoff sold as owner-ready | The PoC works only while its author supplies missing steps, or the latest handoff predates a newly discovered intervention. delivery_readiness is recomputed and cannot be ready. |
Guardrails (where this methodology bites back)
These are hard-won failure modes. Bake them in:
- Live-verify every survivor with a fresh, independent reader — by default, not on request.
Novelty pressure rewards confident-but-wrong hypotheses; the judge reduces this but doesn't
eliminate it. A claimed-exploitable finding is a hypothesis until its PoC or trace is checked
against the real code. Make the verifier independent of both the generator and the judge on
every run: a fresh context that did NOT write the finding re-reads each cited
file:linefrom raw and re-derives the attack, returning verified / corrected / rejected. With a second vendor key, use it as that reader (see below); with none, use the orchestrating Claude Code session itself — it is a second vendor even with no extra key. Independence is the default; cross-vendor is the upgrade. Cross-model "I found a bug" claims are hypotheses, full stop. - A REJECTION carries the finding's full proof burden — test every delivery vector AND cross-vendor-judge
the kill. When you kill a candidate (especially one the verifier graded
needs-live— i.e. you are OVERRIDING it downward — or one you kill by asserting a mitigation / "not reachable" barrier), two hard requirements before it counts: (1) enumerate the sink's ACTUAL input sources and prove the barrier holds on ALL of them — a request value arrives via URL path, query string, POST body (form/JSON/XML), cookie, header, or a decrypted blob (APIenc_request); a mitigation that blocks ONE vector rarely blocks the rest (ApacheAllowEncodedSlashes=off404s%2fin the path but?x=../../fin the query is literal../that reachesrequire_onceuntouched). A single-vector non-repro is NOT a refutation. (2) Route the kill through the cross-vendor judge — applying the outside model only to findings you want to confirm leaves your rejections unchecked, which is exactly where the blind spot lives. (Real miss: an authenticated API controller-param LFI→RCE rejected on one URL-path test + own reasoning; the query-param vector executed live, and OpenAI called the rejection "not sound" — a High shipped as "rejected.") Also beware sink-order reasoning:require_once/includeruns the file's top-level code BEFORE any class instantiation, so "the class won't match" does not neutralize an arbitrary-include. This burden is now a disposition-time transition gate, not an end-of-run KB check (see "Disposition receipts"): arejectedtransition is refused unless its receipt carries the per-vector results, the gating digests, and an orchestrator-dispatched cross-vendor verdict — the judge cannot mark its own kill cross-vendor-clean. False rejections are the costliest error here (they silently erase real bugs while a false confirmation stays visible and retestable), so the gate bites hardest at exactly this transition. - Don't assume the unseen layer is secure — escalate, don't dismiss. When exploitability
hinges on code/config/runtime you can't observe (a server behind client code, one service of
many, no live instance), the verdict is
needs-live-validationcarrying the worst-case severity and an exact, safe test — never "rejected/low." This applies especially to server-enforced classes: authentication, authorization / IDOR, account recovery & password reset, OTP/2FA, mass-assignment, deserialization, SSRF, credential handling. In an authorized owner / pentest context (the common case for this skill), a server-dependent auth or access-control finding is a validate-now item by default. The judge refutes the evidence — not by trusting an invisible mitigation. A "live validation" can be acurl/script you propose for the user to run, or one you run yourself when you have authorized access; the point is to resolve the unknown, not bury it. - Don't let per-unit judging kill cross-function bugs. A judge looking at one function in isolation will reject a real bug whose source and sink span two files. Keep a whole-flow / pairing pass at the functionality altitude so multi-step findings survive single-unit refutation.
- Hunt the classes the headline lens skips. For every entry point, trace XSS / CRLF / SSRF / path-traversal / auth-bypass, not only the RCE you came for.
- Don't scope-lock after the first hit. Proving one bug biases the whole hunt toward that bug's shape. A broad re-audit must be a separate pass with fresh scope.
- A flow that reaches a dangerous sink is proven-dirty — sweep the sink's other trigger paths
before rating, downgrading, or closing it. This is not memory-safety-only: it fires the moment
attacker-influenced data or control reaches a security-sensitive sink with plausible High/Critical
impact — memory-corruption, OS/command exec (a
\!/system/exec/spawn, apsql/shell that runs uploaded content), SQL, deserialization, template eval, arbitrary file R/W, or a restore/import/eval primitive. One bug's low trigger probability (needs many iterations, a rare state)
Truncated - read the full file at https://github.com/dinosn/raptor-loop-hunt/blob/f4c419fd0315fe9203962ff28ebbfd88f74380a7/SKILL.md.