engram-digest

Imported from techtheist/engram (claude-plugin/skills/engram-digest/SKILL.md). Install upstream with npx skills add techtheist/engram --skill engram-digest. Copyright stays with the author.

Engram digest — ingest an existing project

Digestion turns a codebase that predates its memory graph into a first graph. It is explicit and opt-in: it runs only when the user asks, never automatically. One digestion is a bounded pass, audibly reported at the end — the opposite of the engram skill's silent trickle capture.

The target is the current working tree — the code and docs that exist now on this branch. Git history is supporting material only (the why behind what you see), never a deep crawl of old commits. And Engram is reasoning memory, not a code map: digest the decisions, constraints, and gotchas the code embodies — not its structure, which the code itself already records. Quality over coverage; a good first digest is dozens of well-linked nodes, not hundreds of thin ones.

Procedure

0. Preflight. The engram MCP tools must be available (if not, the repo isn't wired — run engram-alpha setup, or the plugin's /engram:setup, and stop). Agree the scope with the user if they gave one ("just the backend"); default is the whole tree. A near-empty tree (fresh scaffold, no docs, no real code) has nothing to digest — say so and stop; never pad a graph to have something to show. Choose the ingest session id now: digest-<YYYY-MM-DD> — every write in this pass carries it (see Session marking).

1. Recall first. Call brief (or read the injected one). If the graph already has nodes, digestion tops up: search each area before writing about it, and lean on the dupe guard — never re-state what the graph already holds.

1b. Choose the shape before the first write. A digest pours dozens of nodes into whatever ontology the graph has, so review the settings with the user first: GET /projects/<name>/config on the core (the manual is GET /guide — read it once), then ask whether the shipped 9-type set fits this project, or whether a preset (research, minimal, general) or a custom ontology fits better, and whether custom fields should carry what the user would otherwise repeat in every body (an owner, a component, a ticket, an effective date). If anything is customized: PUT /config per the guide, set brief.ontology.show = true so every later session learns the vocabulary, reinstall the skill (POST /skills/install), and translate the examples below into that vocabulary. The user decides; you propose.

2. Offline marker scan (tier 1). The daemon scans the tree for FIXME/TODO markers, gitignore-aware:

PORT=$(sed -n 's/.*"port"[: ]*\([0-9]*\).*/\1/p' .engram/daemon.json)
curl -s -X POST "http://127.0.0.1:${PORT}/digest/scan"

(No .engram/daemon.json or the health check fails → run engram-alpha serve from the repo root: it ensures the machine core is up, registers this repo, and exits on its own — no backgrounding. A 404 from /digest/scan means an older binary — engram-alpha update, then engram-alpha stop and engram-alpha serve to restart the core, retry once.)

The response is nominations, not nodes: candidates with marker, suggested_type (FIXME → Problem, TODO → Intent), redacted text, file, line. You are the judge. For each candidate: skip vendored/generated/dead markers and trivial notes-to-self; when the text is too thin to write a real title, read the surrounding code first; then write the survivors — Problems get status: "open", Intents stay volatile, code_refs = the candidate's file (path only, never the line number). truncated: true means the cap cut the walk short — digest what you have, then run the scan again.

3. Read the reasoning surfaces (tier 2 — the real material). List the tree first, then read individual FILES — a directory is never a readable target. README, design/plan docs and ADRs if present, top-level configs and manifests (lockfiles and CI reveal locked tool choices), the heads of the main modules. Use git log --oneline -30 and targeted git log --follow <file> only to recover the why behind something you found in the tree. From this, extract: choices made for a reason (Decisions), stated or evident conventions (Principles), constraints and gotchas (Cautions), non-obvious realizations (Insights), unfinished threads (Intents), known open issues (Problems).

4. Author nodes by the worked examples below. Batch related notes into add_notes calls (≤100 items, per-item verdicts) — items carry no links; link is always a second pass over the ids the response returns. An unlinked pile is not a graph, so make that second pass immediately, batch by batch. (The worked examples show single add_note calls for clarity — the item shape inside add_notes is identical.)

Date historical knowledge. Digestion writes history, and history deserves its real place in time: when a source tells you when something happened — a commit date from git log, a dated ADR, a changelog entry — pass it as created_at: "YYYY-MM-DD" on the write, so the node lands on the timeline where the decision was made, not where the ingest ran. Omit created_at whenever the date is unknown or the knowledge is being stated fresh — never guess a date. (Old dates interact with durability honestly: a year-old episodic note starts stale, which is correct; historical decisions and conventions are stable and keep full trust regardless of age.)

5. Act on every verdict. Digestion gets no bulk bypass: each write's response carries the same duties as normal capture. {matched, created: false} → merge into the match with update_node, never re-add. warnings → read the flagged node before proceeding. suspects → judge each with resolve_suspect now (contradiction → conflict and tell the user; your note is the fresher truth → replaces; fine together → dismiss). missing_code_refs → fix or drop those paths. Leaving verdicts unhandled is how an ingest rots the graph on day one.

6. Report. Digestion is audible: summarize what was ingested (counts by type, the anchors created, anything contradictory found), remind the user everything is provisional and reviewable in the pane (/engram:pane or http://127.0.0.1:<port>), and that the audit log filters by this ingest's session id.

Session marking

Pass session_id: "digest-<YYYY-MM-DD>" explicitly on every add_note / add_notes item. The whole ingest then reads as one batch in the audit journal — reviewable, attributable, and revocable as a unit if it turns out bad. One digestion, one session id; a re-run on another day gets its own.

Ingested nodes start provisional at 50% trust like any Claude-authored node and earn trust only through later reconfirmation or user approval. Never approve_node your own ingest, and never present digested content as settled canon.

The eight authored node types — worked examples

One example per type; each also teaches one feature of the graph. The shapes are load-bearing: copy the example, replace the content with your project's fact, and keep everything else — type names, edge verbs, durabilities, and statuses are exactly as written here (the shipped ontology has 9 types and 7 verbs — the ninth, Tombstone, marks knowledge the user deliberately deleted and is never authored by a digest; inventing others fails the write). (0.7+: if this graph runs a customized ontology — the write errors with "unknown node type", or the brief teaches different names — call describe_ontology and translate these examples into that vocabulary; the roles map 1:1.)

Principle — a stable conviction the project keeps honoring. Teaches: durability, and that principles are what because edges point at.

add_note {"type": "Principle", "title": "Prefer boring, inspectable storage over clever caching",
  "body": "Every past cache layer here got stale-data bugs; plain reads from the source of truth were always fast enough. Optimize only with a measured need.",
  "durability": "stable", "session_id": "digest-2026-07-13"}

Decision — a choice made for a reason. Teaches: the core triple — Decision because Principle — link every decision to its reason when the reason is in the graph.

add_note {"type": "Decision", "title": "Sessions live in httpOnly cookies, not localStorage",
  "body": "Chosen over localStorage tokens: XSS cannot read httpOnly cookies, and the CSRF cost is covered by same-site=strict.",
  "session_id": "digest-2026-07-13"}
link {"from": "<decision-id>", "to": "<principle-id>", "type": "because"}

Caution — a constraint or gotcha that bites. Teaches: extract the why into the body (a rule without its reason gets relitigated), and tags for the slice it belongs to.

add_note {"type": "Caution", "title": "The payment webhook redelivers without idempotency keys",
  "body": "The provider retries on any non-200 for 24h. Every handler must be replay-safe or invoices double-post — bit us in production twice.",
  "tags": ["payments", "webhooks"], "session_id": "digest-2026-07-13"}

Problem — something known to be wrong, not yet solved. Teaches: open status — open Problems are the live worklist and never decay while open. This is what a judged FIXME candidate becomes.

add_note {"type": "Problem", "title": "Invoice import drops rows on duplicate external ids",
  "body": "From FIXME in the importer: the upsert silently ignores conflicts instead of merging. Affects re-imports after provider outages.",
  "status": "open", "code_refs": ["src/billing/import.py"], "session_id": "digest-2026-07-13"}

Resolution — how a problem actually got solved. Teaches: answers + closing the loop, and supersession: when the tree shows the current shape and history shows an older approach it replaced, capture the current one and link it replaces the old node with the why in the edge note.

add_note {"type": "Resolution", "title": "Imports are keyed by (provider, external_id) with explicit merge on conflict",
  "body": "Replaced the silent ignore with a merge that keeps the newest line items; re-imports are now idempotent.",
  "session_id": "digest-2026-07-13"}
link {"from": "<resolution-id>", "to": "<problem-id>", "type": "answers"}
update_node {"id": "<problem-id>", "status": "resolved"}

Insight — a non-obvious realization worth carrying forward. Teaches: builds-on chains insight to the insight it extends.

add_note {"type": "Insight", "title": "The importer is single-threaded by design — ordering is the contract",
  "body": "Line items must apply in provider order; a parallel importer passed every unit test and still corrupted running balances.",
  "session_id": "digest-2026-07-13"}
link {"from": "<insight-id>", "to": "<earlier-insight-id>", "type": "builds-on"}

Intent — deferred work worth surviving the session. Teaches: volatile durability (the Intent default — let it default; unresolved month-old maybes should fade). This is what a judged TODO candidate becomes.

add_note {"type": "Intent", "title": "Add backoff with jitter to the export job's retries",
  "body": "From TODO in the exporter: fixed 1s retries hammer the API during provider incidents.",
  "code_refs": ["src/export/job.ts"], "session_id": "digest-2026-07-13"}

Anchor — a free-text subject that clusters related nodes. Teaches: about (the only edge that points at Anchors, and Anchors are its only valid target) and code_refs — repo-relative paths that get drift-checked when code moves, so keep them real and never include line numbers. Create an Anchor the moment three digested nodes share a subject.

add_note {"type": "Anchor", "title": "billing pipeline",
  "code_refs": ["src/billing/"], "session_id": "digest-2026-07-13"}
link {"from": "<problem-id>", "to": "<anchor-id>", "type": "about"}
link {"from": "<decision-id>", "to": "<anchor-id>", "type": "about"}

Tombstone — a record that knowledge was deliberately removed (an existing one means: don't resurrect this). Digesting a living codebase rarely produces one — mint it only when the material itself documents a removal (a changelog entry, a reverted ADR, a "we dropped X" comment).

If no edge verb completes an honest English sentence between two nodes, leave them unlinked — a forced edge is worse than none.

Guardrails

• Bias hard toward reasoning types. Decisions, Principles, Cautions, Insights are the payload; marker-derived Problems/Intents are the floor, not the ceiling. Never digest code structure, file inventories, or volatile implementation trivia.
• No secrets, ever. Old docs and history are where keys leak. The backend scrubs every field and the scan redacts marker text, but you are the first line — if a fact can't be stated without its credential, it isn't memory.
• Idempotency. Re-running a digest must not duplicate: the dupe guard catches near-identical restatements, and step 1's search-before-write covers the rest. On a re-run, prefer update_node (which also reconfirms) over new nodes.
• Don't inflate. If the tree offers little reasoning to digest (a fresh scaffold, a mirror of vendored code), say so and stop — an honest small graph beats a padded one.