Imported from chli09/scagent (AGENTS.md). Install upstream with npx skills add chli09/scagent. Copyright stays with the author.

AGENTS.md

This repository builds an automated pipeline to discover and validate GPU side channel attack interfaces using LLM agents plus deterministic measurement and scoring.

How Codex reads this file

Codex loads AGENTS.md before doing work, and it can merge multiple AGENTS.md files from the repository root down to the current directory, where later files override earlier ones. You can also use AGENTS.override.md in a directory to override the local AGENTS.md. Keep project specific rules here at the repo root, and add narrower rules in subdirectories when needed.

Project goal

Produce an evidence based, reproducible ranking of risky GPU facing APIs.

The pipeline has four conceptual agents to implement:

1. Curator Agent
   Build prior knowledge from papers and manuals.

2. Discovery Agent
   Find candidate APIs on a target stack and emit testable hypotheses.

3. Evaluation Agent
   Convert hypotheses into experiments, run them, score leakage, and minimize to a reproducible core.

4. Orchestrator Agent
   Coordinate Curator → Discovery → Evaluation and capture execution status.

A deterministic layer runs experiments and computes metrics.

Working agreements

1. Prefer structure over prose
   Write outputs to the artifact files defined below. Avoid keeping essential information only in chat.

2. Deterministic metrics are the source of truth
   Do not change scoring formulas, thresholds, or pass gates without an explicit request.

3. Always include controls
   Every new measurement signal or template must include a negative control.

4. Reproducibility first
   Record environment, versions, seeds, and exact configs for every experiment.

5. Human editable knowledge is preserved
   Never overwrite human notes. Only append or merge with care.

6. Directory updates must stay in sync
   If directory structure changes, update both AGENTS.md and README.md to keep them aligned, and promptly remove unused directories.

7. Agent code must be documented
   When adding Python agent source files, also add documentation explaining the design logic. If source code changes, update the corresponding documentation in sync.

8. API connection error handling
   If an API connection error occurs, automatically run the test script in test/api_smoke_test.py to verify endpoint reachability (token exhaustion is a common cause).

Repository artifacts and folders

Use this layout. Create it if missing.

• artifacts/knowledge_base/
• artifacts/candidates/
• artifacts/experiments/
• artifacts/results/
• artifacts/logs/ • test/ (ad-hoc and integration test scripts)

Use JSONL for large collections, one record per line.

Agent contracts

Agent 0 Curator Agent

Mission
Build and update the prior knowledge base that captures what kinds of APIs become attack interfaces, and what evaluation recipes work.

Inputs
• papers/ PDFs or structured paper notes
• docs/ optional vendor manuals, specs, blog posts with technical details
• artifacts/knowledge_base/ existing cards for incremental updates
• human provided notes

Outputs

1. artifacts/knowledge_base/papers.jsonl
   One paper per line, called a Paper Extraction Card.

Required fields
• paper_id, title, venue, year
• threat_model summary
• attack_interfaces list with interface_type mapping
• observables and prerequisites
• feature engineering and evaluation summary
• mitigations summary
• takeaways for Discovery and Evaluation
• human_notes field, may be empty but must exist

2. artifacts/knowledge_base/interface_types.jsonl
   One interface type per line, called an Interface Type Card.

Required fields
• type_id, type_name, description
• why_selected list
• typical_observables list
• typical_prereqs list
• common_keywords list
• recommended_templates list
• example_paper_ids list
• human_notes field

Optional
3) artifacts/knowledge_base/rules.json
Lightweight keyword and heuristic rules that assist Discovery.

Quality gates
• Every interface_type claim should reference at least one example_paper_id
• Conflicts are allowed but must be recorded in human_notes
• Mark uncertainty explicitly in notes fields

Agent 1 Discovery Agent Step 1

Mission
Enumerate the target platform API surface and output a ranked list of candidate APIs with evidence, hypotheses, and template assignments.

Inputs
• target sources such as headers, docs, extension specs, profiling tool docs
• artifacts/knowledge_base/interface_types.jsonl
• artifacts/knowledge_base/rules.json if present

Outputs

1. artifacts/candidates/candidate_apis.jsonl
   One candidate API per line.

Required fields
• api_id, api_name, signature, stack
• source_location and evidence snippets
• access_scope and capability_tags
• mapped_interface_types
• hypotheses list, 1 to 3 entries
• priority_score and confidence
• human_overrides block

Hypothesis required fields
• hypothesis_id
• description
• secret_variable and secret_values
• expected_observable_shift
• recommended_template

2. artifacts/candidates/discovery_report.md optional
   Short narrative summary of what was scanned and top findings.

Quality gates
• Every candidate must include evidence strings copied from sources
• Every hypothesis must name a secret variable and expected shift
• If no viable template exists, tag needs_template and rank lower

Agent 2 Evaluation Agent Step 2

Mission
Turn hypotheses into experiment specs, drive iterative refinement, and produce reproducible evidence for leakage risk.

Inputs
• artifacts/candidates/candidate_apis.jsonl
• environment capability probes produced by Executor
• artifacts/results/result_summaries.jsonl produced by Oracle

Outputs

1. artifacts/experiments/experiment_specs.jsonl
   One experiment instance per line.

Required fields
• experiment_id, api_id, hypothesis_id, template_id
• environment summary
• setup including repetitions and randomization
• secret definition
• victim, attacker, observer blocks
• controls including negative_control
• outputs paths to raw traces and metadata

2. artifacts/results/triage_report.md optional
   Ranking of risky APIs, with stability and minimization status.

Quality gates
• Every new template or signal must include negative control
• High leakage is not accepted until stability checks pass
• Minimization must be attempted for any high risk candidate

Agent 3 Orchestrator Agent

Mission
Coordinate Curator → Discovery → Evaluation using a stable, reproducible command interface.

Inputs
• agent directories under agents/
• environment variables and CLI flags that select which agents to run

Outputs
• process exit codes and run status (logs may be written to artifacts/logs/ if needed)

Deterministic layer contracts

Executor

Mission
Run experiment_specs exactly as written and store raw artifacts.

Inputs
• artifacts/experiments/experiment_specs.jsonl

Outputs
• raw traces and metadata at paths referenced in experiment_specs
• environment capability probe results
• logs in artifacts/logs/

Oracle

Mission
Compute leakage metrics and stability metrics using fixed algorithms.

Inputs
• raw traces and metadata from Executor

Outputs
• artifacts/results/result_summaries.jsonl
One result per experiment_id, includes leakage_metrics and control results.

Template ids

Use stable template ids that map to deterministic implementations.

Suggested initial set
• T_memory_query_trace
• T_counter_time_series
• T_sync_latency_diff
• T_sensor_time_series
• T_io_congestion_proxy

Each template must define
• required signals
• required controls
• default feature extraction
• oracle metrics

Human in the loop edits

Humans may edit these fields freely
• knowledge_base cards human_notes
• candidate_apis human_overrides
• optional overrides files under artifacts/knowledge_base/

Curator updates must preserve human edits.