Imported from proudzhu/llm-wiki (.agents/skills/paper-reader/SKILL.md). Install upstream with npx skills add proudzhu/llm-wiki --skill paper-reader. Copyright stays with the author.

Paper Reader

End-to-end workflow for ingesting academic papers from Zotero into the LLM Wiki knowledge base.

When to Invoke

• "ingest paper X from Zotero"
• "re-ingest paper X from Zotero"
• "read paper X from Zotero"
• Add a paper from the user's Zotero library to the wiki

When NOT to Invoke

• Paper already in wiki and up-to-date — Grep wiki/log.md and wiki/sources/*.md for author/title first
• User wants a quick summary — read the existing source page or abstract; full ingest is only for deep analysis
• Zotero not running — verify: curl -s http://localhost:23119/connector/ping
• Paper not in Zotero — skip Zotero steps, use extraction scripts directly with an arXiv ID or local PDF
• Source is a web page/blog/informal HTML — use the standard raw article workflow. Exception: substantive non-academic PDFs (transcripts, slides, presentations) DO qualify — adapt the source page template with themed sections (Vision & Strategy / Q&A / Key Quotes); slug follows thought-piece convention without a year (e.g., liang-wenfeng-investor-exchange-meeting)
• Re-ingesting an identical version — only if PDF was updated (camera-ready replaces preprint) or the wiki page is significantly incomplete

Checking Existing Pages (Windows Glob Caveat)

Several steps require checking whether a page already exists. On Windows, Glob brace expansion ({a,b,c}.md) does not work and returns "No file found" even when files exist. Use one of these patterns instead:

• LS the directory (ground truth for existence — filenames cannot lie): LS wiki/entities returns the full file list; scan for the target slugs. One call checks any number of candidates.
• Multiple parallel Glob calls: one Glob per slug (no braces), e.g. wiki/entities/ernst-seidel.md.
• Grep with alternation — content search only, not an existence check. Grep matches file contents, not filenames: a page rarely contains its own slug (e.g. mvdr-beamformer.md has H1 "MVDR Beamformer", so grepping mvdr-beamformer misses it — false "missing"), while index.md contains every slug (false "exists"). Use Grep only to find pages that mention a name, e.g. pattern seidel|fingscheidt|mowlaee finds those authors' entity pages because the names appear in the body text. See pitfalls.md #43.

Always use forward slashes in Glob patterns and prefer relative paths from project root over absolute Windows paths.

Do not use RunCommand with PowerShell Where-Object { $_.Name -match '...' } for batch existence checks — the shell wrapper strips $_ and other $-prefixed automatic variables, producing "item not recognized" errors (see pitfalls.md #31). Use the LS/Glob/Grep tools directly instead; they avoid PowerShell quoting entirely.

Same-File Edit Discipline (applies to EVERY step)

Never issue parallel Edit/Write calls to the same file. A batch of N parallel edits to one file races: each call reads the file as it was when the batch started, and last-write-wins discards the others — silently (every call reports success). Which edit survives is nondeterministic; sometimes a later batch member persists on top of an earlier write, producing duplicated content on re-apply (see pitfalls.md #46).

This has now bitten three ingests — Apostolidis 2026 (Step 8 concept pages), Xiao 2023 and Sun 2024 (Step 9 synthesis page and Step 10 wiki/index.md, where four parallel row-edits were lost and only the Statistics edit survived) — despite the warning previously living only in Step 8. Hence this top-level rule.

Rules:

1. Parallelize across files, never within a file. Reads, Globs, Greps, and Edits to different files are safe and encouraged in one message.
2. Multiple changes to one file = sequential Edit calls in separate messages. Synthesis updates (Step 9) and hand-edits to any index (Step 10) typically need 4–6 edits to the same file — frontmatter sources:, tags:, a table row, an insight paragraph, a takeaway. Do them one per message, or combine into fewer, larger edits with unique anchors.
3. Verify after a suspected race: Grep the file for the expected new content (e.g. the new slug). If missing, re-read the target region before re-applying — a prefix-matching old_string can match text that already contains your addition and duplicate it.
4. Prefer scripts over hand-edits wherever one exists (update_indexes.py, append_log.py, commit_ingest.py) — they are immune to the race by construction.

Terminal Output Black Hole (RunCommand recovery)

Sometimes RunCommand executes a command successfully but its output never comes back: the tool result reports command_id not found, a redirected output file is never created, or a PowerShell pipe capture (git log | Set-Content file) truncates to a single line. The command did run — side effects (files written, commits made) are real; only the output channel is broken. This has now consumed whole sessions twice in a row (Zotero search in the pre-summary session, then every script call in the Zhu 2025 ingest — ~10 wasted turns of retries before the workaround was adopted).

Rules:

1. Switch after the first failure — do not retry. If one RunCommand call loses its output, assume the channel is broken for the rest of the session. Retrying the same call, adding Start-Sleep waits, or re-running with different syntax all waste turns. (The flip side also holds: don't stop running commands — verification via Test-Path/marker files confirms they execute fine.)

2. Recovery wrapper: route the command through capture.py, which captures stdout/stderr (UTF-8, exit code) to .tmp_capture_out.txt; then Read that file.

   uv run python .agents/skills/paper-reader/scripts/capture.py -- git status --porcelain
   uv run python .agents/skills/paper-reader/scripts/capture.py -- uv run python .agents/skills/paper-reader/scripts/append_log.py --op ingest --title "..." --body "..."

3. Keep using the skill scripts through the wrapper — do not fall back to hand-editing wiki/index.md/wiki/log.md or raw git commit just because the terminal is broken. Hand-edited index tables are the known cause of lost rows (pitfalls.md #35); raw git bypasses commit_ingest.py's guards (pitfalls.md #48). The wrapper restores full script output, so the normal Steps 10–13 scripts remain usable.

4. If wrapper-invoked uv run calls hang for minutes: an earlier uv run probably died holding the uv lock. List hung processes via the wrapper (capture.py -- tasklist /FI "IMAGENAME eq uv.exe"), Stop-Process -Id <PID> the stale one, then retry through uv run as usual. Always invoke Python via uv run python (per pitfalls.md #18) — it guarantees the environment matches pyproject.toml/uv.lock; direct .venv\Scripts\python.exe invocation is a last resort only when uv remains hung after the stale process was killed, and must be noted in the session summary.

Prerequisites

• Zotero running with "Allow other applications" enabled
• mineru-open-api CLI (npm install -g mineru-open-api) — verify token: mineru-open-api auth --show
• defuddle CLI (npm install -g defuddle) for arXiv HTML extraction
• All scripts run from the project root via uv run python .agents/skills/paper-reader/scripts/<script>.py

Script Command Quick Reference

Exact invocations for every script (all prefixed with uv run python .agents/skills/paper-reader/scripts/). In sessions resumed from a summary, consult this table before each script's first invocation — handoff summaries preserve step order but lose exact CLI signatures: update_indexes.py --slug SLUG (missing the mandatory subcommand) and append_log.py ingest "Title" (positional args) both exit 2 (pitfalls.md #50, #51).

References (load on demand)

Review/Survey Paper Routing

The default workflow (Steps 4–9) is shaped around research papers — Problem Formulation / Methodology / Experimental Setup / Results. Review/survey/tutorial papers need different analysis targets, source-page sections, concept-page thresholds, and figure selection. Detect them early and route accordingly.

How to detect a review/survey paper

Any of the following is a strong signal (load references/review-papers.md on the first match):

• Title contains: survey, overview, review, comprehensive, taxonomy, tutorial, primer, introduction to
• Abstract phrases: "we survey", "we review", "we present an overview", "this paper reviews", "a comprehensive review"
• Zotero metadata: paperType is journalArticle and the venue is a magazine (IEEE SPM, IEEE Comms. Surveys & Tutorials, ACM Comput. Surv.) rather than conference proceedings
• Structure: the paper's primary contribution is a taxonomy/comparison table, not a single proposed method; "Section II: Background" appears before any "Proposed Method"
• Reference count: 60+ references (reviews typically cite 50–200+; research papers usually 20–40)

Step-by-step routing differences

Once you've identified a review/survey, these steps change:

Tutorials are a special case

Tutorial papers survey methods AND teach them. Treat tutorials as reviews for structure (Taxonomy / Applications Survey / Limitations), but treat them as research papers for figure inclusion — include every figure that aids comprehension of a surveyed method or concept (a tutorial with 7 application areas may legitimately warrant 15–25 figures).

Workflow

Step 1-2: Search Zotero & Fetch Metadata

uv run python .agents/skills/paper-reader/scripts/zotero_fetch.py search "SEARCH_TERMS"
uv run python .agents/skills/paper-reader/scripts/zotero_fetch.py metadata ZOTERO_KEY

Note the Zotero key (e.g., 8ZWV2E4T) and PDF attachment key (e.g., 5H7GWRF3).

Keep search terms short (3–5 distinctive words) — searching with the full paper title times out (Sun 2024 ingest: the complete title timed out; "directional voice activity detection" found it instantly, pitfalls.md #45).

If the paper has an arXiv ID but is not in Zotero, note the arXiv ID and proceed to Step 3b directly (skip prepare_paper.py).

Multiple attachments: Zotero items often have both an HTML attachment and a PDF attachment (e.g., IEEE Xplore saves both). Always pick the PDF attachment (application/pdf) for prepare_paper.py — HTML attachments from publisher sites are typically cluttered with navigation/ads and not suitable for extraction. The zotero_fetch.py metadata output lists all attachments with their MIME types; choose the one whose type is application/pdf.

Step 3: Extract Paper Content

Extraction priority: arXiv HTML > MinerU > pdftotext. All scripts delete the PDF after extraction (so run pdfimages first if you need standalone images).

3a. Prepare directory & copy PDF (skip for arXiv-only papers with HTML)

uv run python .agents/skills/paper-reader/scripts/prepare_paper.py --slug SLUG --pdf-key PDF_KEY

Slug format: author-year-short-title (lowercase, hyphenated).

3b. arXiv HTML (preferred for arXiv papers — better text quality than PDF)

uv run python .agents/skills/paper-reader/scripts/extract_arxiv_html.py --arxiv-id ARXIV_ID --slug SLUG

Auto-creates raw/papers/{slug}/. Falls back to MinerU (exit code 2) if HTML 404 or defuddle missing — then run 3a + 3c.

3c. MinerU (non-arXiv papers or arXiv fallback)

uv run python .agents/skills/paper-reader/scripts/extract_mineru.py --slug SLUG [--language en --model vlm --timeout 600]

• --model vlm (default, layout analysis) or --model pipeline (zero-hallucination). Token required: mineru-open-api auth.
• Language codes (MinerU convention, NOT ISO 639 — zh is INVALID, use ch): ch (Chinese), en (English), chinese_cht, japan, korean, latin, arabic, cyrillic, east_slavic, devanagari, ta/te/ka. Script validates locally; invalid codes exit 2 with the valid list.
• Post-processing: images/ → figures/, refs updated in full-text.md.
• Verify quality: Read first 200 + last 100 lines. Mermaid code blocks for diagrams are normal.

3d. pdftotext/pypdf fallback (plain text, no images)

uv run python .agents/skills/paper-reader/scripts/extract_pdftotext.py --slug SLUG

Uses pdftotext (poppler) if on PATH; automatically falls back to pypdf when poppler is not installed — common on Windows (both the Xiao 2023 and Sun 2024 ingests hit MinerU failures and had no poppler, forcing ad-hoc Python extraction; pypdf is now a project dependency so this path always works, pitfalls.md #47). If you resort to a custom extraction script anyway, it must still (i) write full-text.txt/full-text.md into raw/papers/{slug}/ and (ii) delete paper.pdf afterward — every standard script enforces this, and commit_ingest.py refuses to stage paper.pdf; a manual git add raw/papers/{slug}/ bypasses that guard (Sun 2024 committed its PDF this way, pitfalls.md #48).

3e. Map figures to captions (after 3b or 3c)

uv run python .agents/skills/paper-reader/scripts/map_figures.py --slug SLUG

Pairs each hash-named crop with its "Fig. N." caption (line proximity), prints dimensions, flags axis/colorbar strips, exits 2 on referenced-but-missing hashes. One-call replacement for manual figure forensics (pitfalls.md #25). If no caption-style lines exist, it falls back automatically to the first in-text "Fig. N" reference of each figure (heuristic — verify multi-panel figures against the sub-labels it prints); only when it reports neither captions nor in-text references do you match manually (pitfalls.md #32).

Step 4: Read and Analyze the Full Paper Content

Read the extracted text in chunks (head 200 + tail 100 + targeted range reads). Extract: core problem/motivation, key contributions (numbered), methodology (architecture/algorithms/losses/equations), experimental setup (datasets/metrics/hyperparameters), results (quantitative tables), key concepts warranting wiki pages, authors warranting entity pages.

Neural-network papers (any paper whose method includes a DNN/RNN/CNN/transformer/vocoder etc.): additionally extract the model architecture (layer-by-layer structure with sizes/densities), the input features (exact feature representation, frame rate, window length) and output (what the network produces, at what rate), and the training losses (equations with coefficient values). These feed the mandatory model-documentation section in Step 5 — see the Neural-Network Model Documentation rules in references/page-templates.md.

Review/survey paper: use the Analysis Targets in references/review-papers.md — taxonomy, comparison tables, application domains, open challenges, coverage gaps — instead of the research-paper-shaped list above. See the Review/Survey Paper Routing section earlier in this file for the full routing table.

Numeric sign check (MinerU): MinerU can silently drop minus signs on negative numbers — the Ke 2021 SNR range −5…10 dB extracted as "5 dB to 10 dB", and the test SNRs {−5, 0, 5, 10} as "5, 0, 5, 10". Suspicious patterns: a 1 dB-step SNR sweep that appears to start positive, or a value list with a repeated entry after a sign flip. Cross-check numeric signs against the publisher page / abstract / PDF before writing them into the source page.

If you encounter graphical-only results, citation discrepancies, loose review classifications, or cross-references to already-ingested papers: load references/edge-cases.md.

Step 5: Create/Update Source Page

Create wiki/sources/{slug}.md. Load references/page-templates.md for frontmatter, required sections (Summary / Problem Formulation / Methodology / Experimental Setup / Results / Key Contributions / Related Concepts / Related Synthesis), figure-usage criteria, and figure-filename verification rules. H1 is Author1, Author2 & Author3 Year: Short Title.

Neural-network papers: the source page MUST include a Model Structure, Inputs, and Outputs section with a mermaid architecture block diagram and per-network spec tables (structure / inputs / outputs / training data / role), plus a Training Losses section with the loss equations — see the Neural-Network Model Documentation rules in references/page-templates.md for the template and mermaid syntax constraints.

Review/survey paper: use the review-paper source-page template in references/review-papers.md — sections are Summary / Taxonomy / Methodology (surveyed methods) / Applications Survey / Key Contributions / Limitations and Caveats / Related Concepts / Related Sources. The research-paper sections (Problem Formulation / Experimental Setup / Results) do not apply unless the review itself reports original experiments.

Figure embeds: use map_figures.py output (Step 3e) to write ![[raw/papers/{slug}/figures/HASH.jpg|caption]] — never markdown ![alt](path) (pitfalls.md #27). Multi-panel figures: one ![[...]] per (a)/(b) crop above the shared *Figure N: ...* caption (pitfalls.md #26).

Unreferenced figure files: map_figures.py lists files in figures/ that full-text.md never references — these are typically axis/colorbar strips split off by MinerU (flagged as <-- likely axis/colorbar strip, skip). Do not embed unreferenced files. Only embed figures that are (i) referenced in full-text.md AND (ii) paired with a caption or an in-text "Fig. N" reference by map_figures.py (the in-text fallback runs automatically when no caption lines exist), or manually matched. If map_figures.py reports neither captions nor in-text references, fall back to reading the text for "Fig. N" / "Figure N" mentions and matching by position (see pitfalls.md #32).

For re-ingestion: overwrite the existing source page with updated comprehensive content.

Step 6: Create or Update Entity Pages

For each author not already in wiki/entities/, create a new page. For existing authors, make append-only edits (update updated:, append a bullet to ## Key Contributions, do not touch created: or rewrite existing bullets). Load references/page-templates.md for the full template and the append-only update rules. Check first: LS wiki/entities and scan the filenames for the author slug (entity pages don't contain their own slug, so a content Grep gives false "missing").

For existing authors, use the script — it performs the mechanical append-only pattern (frontmatter updated:, Key/Notable Contributions bullet, Related Sources bullet) on all listed pages in one call, idempotently:

uv run python .agents/skills/paper-reader/scripts/update_entities.py \
    --slug SLUG --entities author1 author2 [--note "contribution description"] [--role "First author of"]

It derives title/venue/year/authors from the source page's H1 and **Venue**: line. After running, review the appended bullets and polish the Key Contributions wording with one sequential Edit per file (e.g., a paper-specific description — the --note flag covers this at insert time). New-author pages are still created by hand from the template.

Step 7: Create Missing Concept Pages

For each key concept referenced via wikilink in the source page but lacking a dedicated page, create wiki/concepts/{concept-name}.md. Load references/page-templates.md for the template and concept-page threshold (novelty / distinctive formulation / central-to-contribution). Do not create pages for generic ML/DL primitives (Adam, ReLU, dropout, gradient clipping) — link them as plain text.

Review/survey paper: apply the stricter concept-page threshold described in references/review-papers.md. A review surveys many terms, but only warrants creating a concept page when the review itself contributes a distinctive taxonomy or synthesis of that concept — not merely because the concept is mentioned. Tutorials are an exception (a tutorial that introduces/formulates a concept distinctly warrants a page).

Batch existence check (one LS call, not N Globs): before creating any concept page, list all candidate concept slugs from the source page and check them against the directory listing:

LS wiki/concepts

Scan the returned filenames for each candidate. Candidates absent from the list are confirmed missing and should be created (if they pass the concept-page threshold). Do not use Grep for this — it matches file contents: a page like mvdr-beamformer.md doesn't contain its own slug (false "missing"), and any page mentioning a slug in a wikilink matches (false "exists"). See the "Checking Existing Pages" section above and pitfalls.md #43.

Step 8: Update Existing Concept Pages

For each existing concept page touched by this paper: add the paper to sources: in frontmatter, update updated: date, add new sections with findings, extend ## Related Concepts and ## Related Sources with new wikilinks.

Identify existing concept pages to update from the same Step 7 LS listing — the candidates that did appear in wiki/concepts are the existing pages to update here. No separate existence check needed.

Efficient batched-update pattern (when updating >2 existing concept pages in one ingest):

1. Read all target pages in parallel — one Read per file in a single message (different files, safe). Typical ingest touches 4–8 existing concept pages.
2. Round 1 — frontmatter edits in parallel — one Edit per file (different files, safe).
3. Round 2 — content additions in parallel — one Edit per file.
4. Round 3 — ## Related Concepts / ## Related Sources extensions in parallel — one Edit per file.

Parallel Edits to the same file race and silently drop each other (pitfalls.md #13; see the top-level "Same-File Edit Discipline" section). Parallelize across files, never within a file. If a single page needs frontmatter + content + Related Sources changes, apply them as sequential Edit calls in separate messages. commit_ingest.py emits WARN: uncommitted changes remain after commit when an edit was silently dropped.

Step 9: Update Synthesis Pages

Triage first (cheap — avoids reading 200–400-line synthesis pages). Precondition: the source page wiki/sources/{slug}.md must already exist (Step 5) — the script reads its frontmatter tags and exits with an error if the page is missing:

uv run python .agents/skills/paper-reader/scripts/triage_synthesis.py --slug SLUG

• No matches → skip Step 9 entirely (common for single-method papers).
• Candidates with Shared tags (1): <broad-topic> (e.g. speech-enhancement, beamforming, audio-processing, signal-processing, machine-learning) → skip without reading — topical coincidence. Read only if N ≥ 2, or N = 1 with a contribution-specific tag (e.g. lpcnet, packet-loss-concealment).
• Surviving candidates → read, then update if any trigger fires:
  1. New data point on an existing frontier (params/MACs/quality tuple)
  2. Fills a gap in an existing comparison
  3. Refutes or refines an existing synthesis claim
  4. Introduces a new axis of comparison

Concrete trigger examples (from real ingests):

When in doubt: prefer not updating. A thin synthesis addition adds clutter; a substantive one (1–2 sentences + a table row) is valuable. If you cannot write at least one substantive sentence about what the paper contributes to the cross-source analysis, skip.

Applying a synthesis update typically means 4–6 edits to the same file: frontmatter sources: + updated:, frontmatter tags:, a row in the Sources Synthesized table, a paragraph in the matching insight, possibly a takeaway and an open question. Apply these sequentially, one Edit per message — parallel edits to one file race and silently drop each other (see "Same-File Edit Discipline" above; the Sun 2024 ingest lost 3 of 5 synthesis edits this way).

Verify synthesis edits landed (mandatory): after the last edit to each synthesis page, Grep the file for the new slug and confirm it appears in both places you intended — the frontmatter sources: list and the Sources Synthesized table row (plus any insight paragraph). The frontmatter edit is the one most often lost: it is the first of the sequential edits and produces no visible change in the rendered page. In the ADL-MVDR ingest, an entire synthesis update (frontmatter + row) was believed complete but never landed in deep-speech-enhancement.md — caught only because this Grep was re-run at the start of the next session (pitfalls.md #49).

Resumed sessions: re-verify before Step 10. When a conversation continues from a summary (context was compacted), treat every "completed" edit claim as unverified: Grep each synthesis page (and any concept page claimed updated) for the new slug before moving on. A handoff summary can describe an edit that was never applied, or an edit that raced and was silently dropped. This has now bitten twice — ADL-MVDR and Souden 2011, where the summary claimed the synthesis frontmatter was already updated while the re-verify Grep found no trace of the slug in the file (pitfalls.md #49). The subtlety: a summary can be accurate about what remains while wrong about what is done — only the Grep distinguishes them.

Step 10: Update Indexes

Do not hand-edit index tables for new entries — use the scripts below. Hand-editing is the known cause of lost rows: in the Sun 2024 ingest, four parallel manual edits to wiki/index.md all reported success but only the Statistics edit survived, leaving the three new slugs unindexed (caught by the mandatory Grep check below and check_index_drift.py). The scripts write rows atomically and recompute statistics; hand-edits are acceptable only for modifying an existing row's summary/date.

For the source-page row, use the source subcommand — it derives all fields from the source page (display from the H1, summary from the first sentence of ## Summary, date from created:) and runs stats automatically, so there is nothing to type and no add-then-forget-stats hazard (pitfalls.md #7):

uv run python .agents/skills/paper-reader/scripts/update_indexes.py source --slug SLUG

Use --summary "..." to override the derived first sentence if it is too long or vague. Review the printed Derived: line before accepting.

For entity/concept rows (and re-ingests), use add then stats, or for ingests creating multiple pages (typical: 2–4 entities + 5–15 concepts), prefer batch with a YAML manifest:

# .tmp_ingest_manifest.yaml
entries:
  - category: sources
    slug: author-year-short-title
    display: "Author Year: Short Title"
    summary: "One-line summary"
    date: YYYY-MM-DD
  # ... repeat for each entity and concept

uv run python .agents/skills/paper-reader/scripts/update_indexes.py batch \
    --manifest .tmp_ingest_manifest.yaml --stats

--stats recounts statistics automatically. Delete the temp manifest afterward.

For one-off additions or re-ingests, use add --category <cat> --slug <slug> --display "..." --summary "..." --date YYYY-MM-DD, then run stats.

Verify after updating indexes (mandatory, one Grep call): confirm every new slug actually landed in both the main index and its subdirectory index. Long ingests can lose track of which entries were added — statistics alone do not prove the table rows exist (the Guldenschuh 2014 ingest had correct statistics but missing table rows, caught only by this check):

Grep pattern: "new-slug-1|new-slug-2|new-slug-3"
      path: wiki
      output_mode: content, -n: true

Each new slug must appear in two files: wiki/index.md and wiki/{category}/index.md. If a slug is missing from either, add the row before proceeding.

Verify statistics (one cheap call — the stats recount writes numbers, this proves they match reality):

uv run python .agents/skills/wiki-lint/scripts/check_statistics.py

Exit 0 = stated counts match actual file counts for every category.

Step 11: Update Log

Write the entry body to a temp file, then call the script:

uv run python .agents/skills/paper-reader/scripts/append_log.py --op ingest \
    --title "Paper Title (Author Year)" --file .tmp_log_entry.md

For short entries, --body "..." avoids the temp file:

uv run python .agents/skills/paper-reader/scripts/append_log.py --op ingest \
    --title "Paper Title (Author Year)" --body "Created ...; updated ... ."

The script takes no positional arguments — --op, --title, and one of --body/--file/--stdin are required flags. Calling it as append_log.py ingest "Title" --details "..." exits 2 with the usage message (pitfalls.md #50); check --help first if unsure of a script's signature.

Entry body format (temp file):

- **Source**: `raw/papers/{slug}/full-text.md` (Zotero: KEY)
- **Authors**: Author1, Author2, ...
- **Published**: Venue Year, pp. XXX–XXX
- **DOI**: 10.xxxx/xxxxx
- **Summary**: One-line summary
- **Pages created**: list each new file path
- **Pages updated**: list each modified file with a short note (e.g., "added cross-refs")

For re-ingestion, use ingest (re) in --title.

Link format in log bodies (pitfalls.md #40): reference wiki pages with vault-absolute wikilinks ([[sources/slug|Title]]) or backticked plain paths (wiki/sources/slug.md) — never ../-relative markdown links like [Title](../sources/slug.md), which do not resolve from wiki/log.md and abort mkdocs build --strict. append_log.py validates the body and rejects such links (exit 2) before appending.

Step 12: Build Verification

Step 12a — Wikilink verification (fast pre-check, catches broken links before the build):

uv run python .agents/skills/paper-reader/scripts/verify_wikilinks.py --slug SLUG

Checks both [[category/slug]] wikilinks and ![[raw/...]] figure embeds against the filesystem, and flags LaTeX math ($...$) in wikilink/embed display text — math aliases mangle the pipe-escaping and abort the build (pitfalls.md #44). A single-character hash typo in a MinerU figure filename (...151105... vs ...158105...) aborts the mkdocs build — this check catches it here, before the 60+ second build cycle. Scans the source page + all new/modified wiki/*.md files; exits 0 if all links resolve, 1 if broken links found. Fix any broken links (create the missing page, correct the slug, or use plain text; for embeds, Glob the figures/ dir with a hash prefix and copy the exact filename) before proceeding to 12b.

Do not substitute the wiki-lint check_broken_links.py here: it scans the whole vault and reports hundreds of pre-existing convention violations (bare slugs, wiki/ prefixes, ../ prefixes from legacy pages), drowning the signal for the files this ingest touched. verify_wikilinks.py --slug is scoped to exactly the new/modified pages; if you do run the vault-wide checker, grep its output for the new slugs rather than reading the full report.

Step 12b — Backlink audit (catches one-way link omissions from Steps 8–9):

uv run python .agents/skills/paper-reader/scripts/check_backlinks.py --slug SLUG

Audits the ingested source's curated links (## Related Concepts / ## Related Synthesis on the source page) against the target pages: every concept/synthesis page the source links to must reference the source back — via frontmatter sources:, a body wikilink, or a ## Related Sources entry. A target is flagged only if it already references ≥50% of its in-linkers (partial reciprocation — hub pages like beamforming that reference few by design are excluded) and shares ≥1 distinctive frontmatter tag with the source. mentioned-unlinked = author+year appear in the body without a wikilink; not-mentioned = omitted entirely.

Exit 0 = clean; exit 1 = omissions found. For each flagged target, apply the Step 8 update pattern (frontmatter sources: + body reference + ## Related Sources wikilink) and re-run until clean. This catches the "Wang 2022" failure mode: a source links a concept page that curates comparable sources but omits this one — usually a Step 8/9 edit silently dropped by a parallel-edit race (pitfalls.md #46) or a target page missed during candidate identification. Without --slug, the script audits the entire vault (slower; also usable during periodic wiki-lint passes).

Step 12c — MkDocs strict build:

uv run python .agents/skills/paper-reader/scripts/build_check.py

If the build fails, resolve broken links/missing pages before proceeding. Pre-existing INFO messages about log.md links and the Material "MkDocs 2.0" banner do not fail the build (pitfalls.md #8, #28). The check intentionally runs without --quiet: mkdocs's --quiet sets the log level to ERROR, which filters WARNING records before strict-mode's counter sees them — a --quiet build can exit 0 despite strict violations (observed in the Ke 2021 ingest). INFO-level nav lines in the output are normal.

Step 13: Commit Changes

uv run python .agents/skills/paper-reader/scripts/commit_ingest.py \
    --slug SLUG --message "ingest: Short Title (Author Year)" \
    --entities author1 author2 --concepts concept1 concept2 --synthesis synth1

Stages raw/papers/{slug}/, wiki/sources/{slug}.md, all index files, wiki/log.md, and the specified entity/concept/synthesis pages. Verifies no paper.pdf is staged. Auto-stages any other modified/untracked file under wiki/ (catches Step 8 edits to existing concept/entity pages that would otherwise be dropped) — pass --strict to disable. Use --no-verify only if the pre-commit hook has environment issues unrelated to your changes.

Duplicate-build skip (automatic): if Step 12c's build_check.py passed within the last hour and no build input (wiki/, raw/papers/, mkdocs.yml, plugins/, hooks.py) has been modified since, the script commits with --no-verify itself and prints a NOTE: — the pre-commit hook's ~60 s rebuild is guaranteed redundant. If you edit anything after 12c, the script detects it and runs the hook normally.

Expected post-commit noise: WARN: uncommitted changes remain after commit listing .obsidian/* files is normal — Obsidian config is deliberately not part of ingests. Only investigate if a wiki/ or raw/ path appears in the remaining list.

Important Notes

• raw/ immutability exception: replacing remote image URLs with local paths in full-text.md is allowed.
• Avoid \bm{} in LaTeX math — MathJax does not load the bm package. Use \mathbf{x} or \boldsymbol{x} instead.
• Never put LaTeX math in a wikilink alias — [[concepts/foo|$\mathcal{L}$]] breaks the fix_obsidian_escapes pipe-escaping and aborts mkdocs build --strict (pitfalls.md #44). Use a plain-text alias ([[concepts/foo|Spectrally Adaptive Loss]]) and keep the math outside the wikilink.
• Always commit via commit_ingest.py — manual git add/git commit bypasses its guards: it refuses to stage paper.pdf, auto-stages all wiki/ modifications, and avoids PowerShell quoting entirely (heredoc <<'EOF' is a parse error on PowerShell 5.1, pitfalls.md #37). If a manual commit is unavoidable: exclude paper.pdf and .obsidian/, and pass multi-paragraph messages as multiple -m flags — never a heredoc.
• Todo list structure: one todo per workflow step (1–13), in numerical order. Treat Steps 3a–3e as a single "extract content" todo. Treat Step 12a–12c as a single "build verification" todo. If Step 9 triage finds no candidates, mark that todo completed with "none relevant — grep triage" rather than leaving it pending.