offload-harness

Imported from dmmdea/offload-harness (AGENTS.md). Install upstream with npx skills add dmmdea/offload-harness. Copyright stays with the author.

AGENTS.md

Vendor-neutral entry point for any AI coding agent working in this repository. (Claude Code also auto-loads CLAUDE.md, which carries the same orientation in more detail.)

What this repo is

offload-harness — a local-first Go harness that delegates short-context, low-judgment work (summarize / classify / extract / triage, plus vision / OCR / transcription / media generation) to a free Gemma-4 cascade served by llama.cpp. It ships as a CLI, an MCP server (local-offload), and an optional local coding agent (local-agent). It never calls a cloud model; on low confidence it returns a structured defer so the calling agent does that task itself.

Documentation map

Detailed documentation lives in docs/. This file only routes you there.

• docs/README.md — the index. Start here.
• docs/systems/ — how each part of the harness works (offload pipeline, coding agent, MCP server, media generation, fleet node, installer, accelerators).
• docs/tiers/ — one page per hardware tier (GENERATED from setup/templates/profiles.json by cmd/gentiers; run go generate ./... after changing the profile table) plus checked-in capability reports from real machines.
• docs/flows/ — behavior that crosses systems (cascade escalation and defer, run-graph manifest satisfaction, fleet job lifecycle, zero-warm generation).
• docs/systems/printed-clis.md — the tools/ printed CLIs: what they are, why they are separate Go modules, and the regenerate/re-vendor cycle.
• docs/systems/cache-server.md — the optional second-device KV tier (kv_cache_server): off by default, capacity at parity cost, the pipeline-layout constraint.
• docs/systems/composite-tier.md — a box that is several hardware tiers at once (ADR 0052): its device layers, the one placement table, the display-card guards that fail closed, and the placed block every result carries.
• docs/systems/fleet-overview.md — fleet-ui / top / fleet-smoke: the read-only operator page on the delegator, its poller, and the jobs/errors feeds.
• docs/systems/opencode-integration.md — the harness inside opencode: MCP registration, rules parity, and the opencode-local-offload plugin.
• docs/architecture/decisions/ — Architecture Decision Records. Only Accepted status is current guidance.
• docs/glossary.md — terms with a specific meaning here (Defer, Tier, Zero-Warm, Node Manifest, …).

Workflow:

1. Read the system and flow docs for the area you are about to change.
2. Inspect the source for implementation detail.
3. Make the change.
4. Update the affected docs in the same pull request when behavior, responsibilities, flows, invariants, interfaces, or glossary concepts change.
5. Make sure docs and code agree before you finish. go test -run TestDocsLint . checks structure; you check meaning. If they disagree and you cannot resolve it, say so explicitly in the PR.

Do not put detailed system behavior in this file — it belongs in docs/.

Installing the stack

Point yourself at setup/SETUP-AGENT.md and follow it exactly. It is an agent-executable runbook: run setup/detect.ps1 → setup/install.ps1 → setup/selftest.ps1 (Windows, PowerShell), read each script's final JSON line, and branch on the decision tables. Do not substitute pinned assets, install ROCm/CUDA, or start the unauthenticated agent server beyond loopback without asking the human.

Operating the stack

• Orientation map (ports, model tiers, golden commands, do-not-break invariants, where things live): CLAUDE.md in the repo root.
• Task walkthroughs (start/stop the stack, chat, run an offload task, drive the coding agent, add a model, diagnose failures, update): docs/OPERATOR-GUIDE.md.

Repo tooling (tools/)

tools/ holds printed CLIs — standalone Go modules generated by CLI Printing Press and vendored here. They are separate modules, so a root go build ./... / go test ./... does not cover them; cd into the module.

• Driving a local ComfyUI server — read tools/comfyui/SKILL.md first. It is the agent entry point for comfyui-pp-cli: submitting graphs, node schemas and COMBO options, model-visibility triage, honest run timing, output provenance, and replay. Prefer it over hand-rolled HTTP against the ComfyUI API. Note this is the operator/agent tool for ComfyUI — the harness's own render path is docs/systems/media-generation.md.
• Inspecting or operating llama-swap — read tools/llamaswap/SKILL.md first. It is the agent entry point for llamaswap-pp-cli: what is loaded, what a swap cost, seat and model bindings, config and its backup history, and measurement. Prefer it over curling the llama-swap API by hand. This is the operator/agent tool — the harness's own serving path is docs/systems/offload-pipeline.md.
• Never hand-edit a generated file under tools/. Fix it upstream in the Printing Press and re-vendor, or the next reprint silently drops your change. Full rules: docs/systems/printed-clis.md.

Working in the code

• Build: go build ./... · Test: go test ./... · Vet: go vet ./... — all must stay green.
• Go 1.26+. TDD for Go changes (failing test first). Keep changes scoped and minimal.
• Do not break the serving invariants or the agent safety defaults — see CLAUDE.md → "Invariants".