Imported from krivitsky/professional-agentic-product-engineering (
plugins/pape/skills/harness-audit/SKILL.md). Install upstream withnpx skills add krivitsky/professional-agentic-product-engineering --skill harness-audit. Copyright stays with the author.
Harness audit — a static review of the agent configuration in a repo
The guide's Big Idea: a harness is a workflow bounded by three constraints — guidelines (how the agent should behave), autotests (ground truth from the environment), guardrails (limits it can't cross). This skill reviews the one a repo actually has.
The full guide ships at ${CLAUDE_PLUGIN_ROOT}/guide.md. The check catalogue is ${CLAUDE_PLUGIN_ROOT}/skills/harness-audit/checks.md — read all of it before auditing.
Read-only, except the report. Never edit, move, or delete a file. Never run tests, builds, installs, or migrations. Never git add/commit/checkout/stash/clean. The only writes are the two report files in harness-audits/.
The distinction that makes this an audit and not an inventory
Presence is the cheap half. "Is there a hook?" is a glob. The findings worth the reader's time come from opening files and cross-referencing them:
- two
CLAUDE.mdfiles giving contradictory orders, so behaviour depends on working directory - nine rules written as "never", one of them enforced
- a hook that
|| trues its own failure — so the harness looks gated and is not - a subagent instructed to dispatch subagents, which the runtime cannot do
- a
.gitignorepattern that near-misses the file it was written to exclude
Every one passes a presence check. checks.md has two classes and Class C is the point — a run that reports only Class P has done the easy work and stopped.
Three passes: find wide, then falsify
An independent review of a shipped report re-checked every claim against the working tree and returned: diagnosis sound · three of seven findings misstated · one false. Its architecture held; its citations did not. The single pass that wrote it had used 14 of its 35 allowed reads — budget was never the constraint. A grep hit was being treated as a read.
That is not fixable by trying harder in one context, and the skill already knew why: the session that wrote a recommendation should not be the only one that checks it. This is Tip 4.7 — review with fresh eyes, not the context that wrote it — applied to the audit itself.
| Pass | Job | Count |
|---|---|---|
| 1 · Finders | Produce candidate findings. Run in parallel, each with a different lens, each read-only | 2 by default |
| 1b · Re-checker | Re-test the previous run's open issues. Runs beside the finders and returns first | 1, when a prior report exists |
| 2 · Pooling | Merge, dedupe, flag conflicts. Mechanical — done in the orchestrating context, no agent | — |
| 3 · Verifier | Falsify every pooled finding against the tree. Returns verdicts, not opinions | 1, always |
Pass 1 — finders, for recall
Fan-out buys coverage, not correctness. Class C checks are open-ended — "find contradictions between instruction files" has no fixed answer set, and one pass finds what its lens is shaped for. The shipped report cited ci.yml and playwright.config.ts and never noticed that a comment in one contradicts the ternary in the other; a finder told to cross-read pairs would plausibly have caught it.
Spawn each as a subagent with the read-only constraint, checks.md, and one lens:
| Lens | Told to look for |
|---|---|
| Enforcement | Class P absences, C-2, C-4, C-5, C-10, C-12 — what is stated versus what holds |
| Coherence | C-1, C-3, C-6, C-8, C-9, C-11 — contradictions between files. Every claim must rest on two files read together |
Under --quick the single finder carries both lenses, not one of them. An earlier version ran Enforcement alone, which quietly made the cheap mode the shallow mode: Coherence owns the cross-file reading — C-1, C-3, C-6, C-8, C-9, C-11 — and dropping it drops the half this skill opens by calling the point. Class P absences are the easy half; a --quick run that returns only those has done the inventory and called it an audit.
Specialisation is a two-finder technique. With two, divergence is the product: different lenses read differently, and where they clash pooling has a signal. With one, there is nothing to diverge from, so a narrow lens is loss with no compensating gain. Give the single finder the union and a smaller budget — it sees less of everything rather than none of something.
Under --deep, add Hygiene (C-7, secrets, tracked state, machine paths) and Economy (context budget, always-loaded cost, duplication between instruction files and skills).
Each finder returns candidate findings in the §Findings shape, each with the file:line it rests on. Finders do not write the report.
Pass 1b — the re-checker, when a prior report exists
Spawn it alongside the finders, not after them. It re-tests the previous run's open issues against the tree, and because every one of them arrives with a file:line and a stated condition, its work is targeted reads rather than an open-ended sweep. It finishes long before the finders do.
Give it the issues inline — never the report path. The orchestrating context has already opened the previous report; it extracts each open issue's subject, file:line and the condition claimed, and passes that list in the prompt. Handing over a path instead makes the re-checker read a hundred-kilobyte document to find its own worklist — an observed run spent more than the pass it was supporting this way, more than the finder it was supposed to undercut, and most of it before any re-testing began.
The whole premise of this pass is that it is cheap because its targets are known. Make them known to it.
Worst first, capped at ten. All Highs, then fill by severity. More than that and it stops being the fast pass.
Three verdicts per issue, and the third is not a failure:
| Verdict | Means |
|---|---|
| fixed | The condition is gone. Name what changed — "the gate now exits non-zero" |
| still open | The condition holds. It carries into this run's findings with its original open since date |
| can't tell | The evidence moved or the check no longer applies. Say so; do not guess either way |
This is what makes Trend evidence instead of inference. Trend has been deriving fixed from a key's absence in the new run — but a key can vanish because the problem was solved, or because nothing looked there this time. Under --quick, with one finder on a smaller budget, the second is likely and indistinguishable. A re-checked fixed is a fact; a diffed fixed is a guess wearing the same word. Where the two disagree, the re-checker wins.
A correction to the previous report gets its own line, not a subordinate clause. A re-check re-derived one issue's measurement to 66% where the earlier report said 69%, and reported it at the tail of a sentence about what was still open. The audit correcting its own prior number is the strongest trust signal it has — stronger than any finding, because it is the only evidence a reader gets that the process catches itself. Give it the line:
Correction to the 27 Jul 11:28 report:
ISSUE-3's measurement re-derives to 66%, not 69% —CLAUDE.mdhas grown ~800 characters since.
Report it the moment it lands, before the finders return:
Re-checked the last run's 7 Highs: 2 fixed, 8 still open.
Closed: the pre-push hook now blocks on a failing build, and
verifyexits non-zero when the link checker can't reach the network. Still open, worst first:CLAUDE.mdandSTYLE.mdstill define disjoint palettes · the outbound-link check is still skipped on every automated path ·"lint": "eslint"is still invoked by nothing.Harness audit · krivitskydotcom — Standard · 2 finders + verifier · light
✓ setup → ✓ re-check → ▸ research → · cross-check → · fact-check → · report· 12m · 287k + 1 runningNow finding what's new. Next update when research returns.
Name them. A count is not a finding. "8 still open" tells the reader a number about a document they have not opened; three named conditions tell them about their repo. These are the most reportable facts the whole run produces — verified against files minutes ago, about issues the reader already agreed were real, available before anything else exists. Reporting them as an integer is the single largest waste in the display.
Three to five still-open, worst first, one clause each. Never all eight — the tail of a re-check list is where interest dies. Take five only when the fifth is a distinct condition rather than another face of the first. The full set is in the report.
Never build a live tracker of the candidates. Two designs were tried and both failed on their own terms:
| Tried | What happened |
|---|---|
| A four-column markdown table | The tip column carried a full https://…/tier-3#tip-3-5, ate half a hundred-character terminal, and squeezed the claim to three words a line |
Tip: / Claim: / Where: / Status: blocks |
Four lines per finding — forty-four for eleven — with trailing empty fields where a row had no data |
But the layout was the symptom. The content was the problem: every row read ⏳, because nothing resolves until the verifier runs. A status column that says pending on every line for ten minutes carries no information, and a reader shown one asked "what's the value?" There isn't one.
A third of candidates have historically been misstated or cut, so anything shown before the verifier runs must say so plainly and once — "none verified yet" — and must not carry a severity, which is a judgement about a claim that is not yet settled.
But not naming the candidates was over-correction. The claims are known at cross-check — pooled, deduped, ranked. Withholding them is not caution, it is a fifteen-minute wait with nothing in it. What failed was tracking them through states in a live table, not saying what they are.
So name three at cross-check, in prose, once:
Strongest of the eleven, none verified yet:
CLAUDE.mdcalls its palette the single source of truth and forbids going outside it, whilespecs/slides/STYLE.mddefines a second, disjoint one — 31 off-palette hexes already ship. The outbound-link checker is skipped on every automated path, and the one invocation that runs it is the one your own docs say not to trust."lint": "eslint"is defined, configured and installed — and invoked by nothing.Now re-opening the file behind each of these. If any is wrong, you'll know before I do — say so.
| Kept | Dropped |
|---|---|
| Three claims, prose, at one boundary | Rows, a status column, …N more, a live tracker |
| "none verified yet", said once for all three | ⏳ repeated on every line, which was the only thing the column ever said |
| The claim in the reader's terms | A https://…/tier-3#tip-3-5 column eating a third of the width |
The invitation is the point. A reader who works in that repo settles "eslint is invoked by nothing" in two seconds — faster and more reliably than a subagent re-deriving it. Withholding the claim to protect them from a possible error also denies the run its best available check.
Whatever is named must be accounted for in the report. If the verifier cuts one, §6 says so by name. A claim shown live and then never mentioned again reads as quietly true, which is worse than never showing it.
Three to five, and only at cross-check. Same rule as the re-check list: five only if the fifth says something the other four did not. Not at research (nothing is pooled yet, so the ranking is one lens's opinion) and not repeated at fact-check (the reader has them; what they want next is which survived).
The budget is facts about their repo, not lines. An earlier version of this rule capped each boundary at one line, which is how "8 still open" got written where three named conditions belonged. Prose that says what is wrong with their repo is never the thing to cut — cut the sentences about how the audit works, the reassurances about options they already chose, and the counts standing in for content.
Do not name the display. A run said "three rows on the board already" — coining a noun for its own progress readout, which the reader had never heard and could not look up. The reader has a screen, not a board; there is nothing here that needs a name.
Name the phases in the reader's words, not this file's. Finders, pooling and verifier are the vocabulary of the design; nobody watching a progress strip knows what pooling is. The strip shows five phases, and each says what is happening to the repo rather than which agent is running:
| Shown | Is | Means |
|---|---|---|
| setup | flags + confirmation | what shape was agreed, before anything spawns |
| re-check | pass 1b · re-checker | re-testing last run's open issues — skipped when there is no prior report |
| research | pass 1 · finders | reading the repo and collecting candidate claims |
| cross-check | pass 2 · pooling | comparing what the lenses found, resolving disagreements |
| fact-check | pass 3 · verifier | re-opening the file behind every claim, trying to break it |
| report | writing | both files land in harness-audits/ |
These six words are the only words for these six things — in the strip, in the prose beside it, and in every agent description. A run's strip read ▸ research and the sentence directly beneath it said "waiting on the re-check, then the reading pass": a synonym invented one line below the word it was a synonym for, leaving the reader to work out whether the reading pass meant research, cross-check, or both.
❌ "Presence checks done here. Waiting on the re-check, then the reading pass." ✅ "The quick file-and-config checks are done in this session. Waiting on re-check, then research."
This is the Enforcement-under-▸ research defect one layer in. The strip was rewritten into the reader's vocabulary and the sentence beside it was left in the design's — so the fix moved the leak rather than closing it.
It binds every term in the block, not only the phase nouns. "Presence checks" shipped in that same line: it is this file's name for a glob that answers a question without opening a file, and it names nothing the reader has been told about.
Test before sending: every noun in the message either appears in the six-row legend the introduction printed, or is ordinary English. A third label is never the answer — use the phase name, or describe the activity plainly.
fact-check carries a counter — fact-check 12/19 — because it is the only phase whose progress is a number the run actually knows. Every other phase is one agent working until it is done; this one has a claim list with a length. It is also the phase where the reader has just been shown five claims and is waiting to learn which survive, so a moving number answers the question they actually have.
It is not the longest phase — research is. An earlier version of this line justified the counter by duration, which was an assumption rather than anything measured. Do not reason about which phase is slow, and do not state it either — the strip reports elapsed time as it accrues, and that is the only duration this skill ever shows.
A phase that will not run this time is not shown at all. Under --quick there is one reading pass, so cross-check has nothing to compare; with no prior report, re-check has nothing to re-test. Both come out of the strip and out of the legend, and the strip is five phases instead of six.
Do not show it greyed and then explain it. An earlier version kept the phase visible so the reader could see which step they traded away — and the run, having a visible-but-empty phase to account for, wrote a sentence accounting for it:
❌ "One lens ran, so cross-check has nothing to compare — that's the coverage you traded away, and it's the only thing you traded: every claim above still goes to the verifier."
That is the run arguing with a decision the reader made thirteen minutes earlier, and it is the third time the same fact has been stated: the option label said one lens instead of two, the confirmation block said it again, and this says it a third time with a defence attached. Nobody needs reassurance about the option they chose; they need the run to get on with it.
Where the traded coverage belongs is the record, not the wait. The strip header carries the resolved mode (Quick look · 1 finder + verifier) and so does the report cover — so a reader comparing two reports can see the difference in the place where comparing happens. A ghost phase mid-run reaches only the one person who already knows.
The strip closes every message; substance only at boundaries
These are two things with two different costs, and bundling them was a mistake. The strip is two lines. The substance a boundary carries is longer, and there is only something to say when a boundary has actually changed.
| Emitted | |
|---|---|
| The strip — identity, phases, meter, and the line naming what comes back next | at the end of every message this skill sends during a run, without exception |
| The substance above it — what closed, what conflicted, what got cut | only when a boundary actually changed something: setup, re-check, research, cross-check, fact-check |
The closing line says what comes back, never what was dispatched. A run closed cross-check with:
❌ "Thirteen go to the verifier — the eleven new ones in full, plus the two carried claims whose framing is most contestable." ✅ "Next update is the last one: which of these survived."
Routing arithmetic is the audit describing its own plumbing. Which claims were forwarded, and on what basis one was judged more contestable than another, is a decision the reader neither made nor can act on — and it arrives at the exact moment their attention is on the five claims they were just shown. The only thing they want to know is what the next message will contain.
A run sent a bare one-line status with no strip beneath it, because that moment was not one of the boundaries — so a reader who had just been given a phase display was shown a loose sentence and left to wonder whether the display still applied. Any message without the strip under it is a message that abandons the reader mid-run.
Two lines is not a token sink. Repeating a boundary's full content on every message would be, which is why only the strip repeats.
Boundaries stay at five blocks — one per phase, never one per finding. A block for each claim would bury the report under its own progress before the report existed.
Say what the audit looks for, once, while it looks
A user thirty minutes into a real run asked "what is this audit? common issues it finds?" — which is the run's own fault. It had shown them agent names, tool counts and a token meter, and never once said what it was for.
So the block that appears when research finishes carries one short paragraph, once, naming the shapes it hunts. Absences are the cheap half and everyone expects them; name the other half:
What this is looking for. Not "you're missing a file" — that half is easy. The valuable findings are where the setup looks complete: two instruction files giving opposite orders · a rule written as NEVER with nothing that could catch a violation · a gate that exits clean when it couldn't run · the command you're told to run not being the command CI runs · reference material in the file that loads every session, crowding out the real rules.
Then, if the claims already show it, say so: "Your repo has four of those five." That single sentence is the moment the reader stops watching a tool and starts reading about themselves — and by then the run has the evidence to say it honestly.
Once, and never again. It is orientation, not a lesson: a paragraph on the second block and nothing on the third or fourth. A run that re-teaches at every boundary is a run that has confused the reader's attention with its own.
This is the same content the tip links serve, at a different altitude. The paragraph says what kind of thing goes wrong; the per-row links say what to do about the specific one. Neither replaces the other, and both cost almost nothing next to a thirty-minute wait.
Three lines that show the reader their repo, not the machinery
The strip shows agents, phases and tokens — all of it about the audit. These three are about them, and each costs one line at a boundary that is already emitting one.
Open with the delta, when there is one. A repo with a prior report gets this in the first block, before any finding exists:
Last audit, 25 Jul: 16 issues, 5 High. Let's see what moved.
This is the retention line and it is currently buried in §4 of a document nobody has opened yet. The audit's product is not a report — it is the difference between two of them, and a reader who never sees that difference has no reason to run a third. Close the loop at the end: "5 fixed since 25 Jul, 3 still open, 16 new."
But it may not claim a fix count — only §Pass 1b may do that. A run opened with "you've since shipped 15207ce closing three of them", read off the commit; the re-checker then tested them and found two. Commit messages describe intent, and intent overshoots. Say what is observable without opening a file — that commits have landed since — and let the re-check supply the number a minute later with the files behind it.
✅ "Last audit, 27 Jul 11:28: 24 issues, 7 High. You've shipped since — let's see what actually moved." ❌ "…and you've since shipped
15207ceclosing three of them."
The second reads better and is wrong, which is the whole reason this pass exists.
Say one thing the repo does well, at cross-check. The scorecard's credit line is the only place the report says anything good, and it lands at the very end:
Worth saying: 561 commits, median two files — T5 checkpointing already holds.
One line, drawn from what the finders actually counted, never invented. It changes the shape of the wait from how bad is this to here is where you stand — and it is what makes the criticism land instead of bounce. A run that lists twenty-four problems and never once says what works has written an accusation.
Show where the gaps cluster, at fact-check, once the tiers are known:
Clustering: T3 (9) · T4 (7) · T5 (5) · T8 (1).
That histogram is the most transferable thing the audit produces. A specific issue is about this repo; the shape of the distribution is about how the reader works, and it will look similar in their next one. It also previews the lever honestly — the reader can see T3 leading before §1 says so, which makes the verdict feel derived rather than pronounced.
All three are facts the run already holds. None requires an extra pass, an estimate, or a judgement it has not already made.
Carry elapsed time and tokens in the strip. "12m · 287k" is a measurement of what has already happened, which is the only kind of cost figure this skill is allowed to show. There is nothing to compare it against, by design — see §Say nothing about how long it takes.
The token figure must be on the same basis as the estimate, or it is noise. Count every pass — each finder, the verifier, and this context — not the orchestrator's own slice. A run showed 1m · 34k while its finder was still working: the finder's ~120k had not landed, so the number was a third of the truth and would have stayed flat for ten minutes before jumping. A meter that cannot be compared to the quoted figure fails the only job it has.
Subagent cost is unknown until the agent returns. That is a fact about the runtime, so state it rather than hiding it:
✓ setup → ▸ research → · cross-check → · fact-check → · report · 1m · 34k + 1 finder running
Then drop the qualifier the moment it resolves — 12m · 287k once every spawned agent has reported. Never print a bare total while an agent is outstanding; a number that silently excludes the largest contributor is worse than no number, because it reads as reassurance.
Elapsed time needs no qualifier — it is wall clock and always true.
Keep the subagents' tool calls legible
Nothing cds — not the subagents, and not the orchestrating context. Everything already starts in the repo root. Runs have been observed emitting cd /Users/alexey/src/aidy/repos/<repo> before every command, which spends the visible width of the line on a constant, and then the actual command is what gets truncated. The reader is left with a path they already know and an ellipsis where the evidence was.
This binds the orchestrating context too. It now runs the presence checks itself while research is out, and a run was observed prefixing each of those with cd /Users/alexey/src/aidy/repos/<repo> — the same waste, on the line the reader is actually watching, since these are the only tool calls visible during the wait.
Ask for short tool descriptions too. A finder's Bash calls are the only window into what it is doing while it runs; rg for hook definitions tells the watcher something, and a truncated multi-line echo banner does not.
Name each subagent after the phase it belongs to
The runtime prints every subagent's description directly beneath the strip, so the two are read together. A run showed ▸ research in the strip and Agent(Enforcement lens finder) on the next line — two vocabularies for one activity, stacked, with nothing connecting them. A reader who has just been taught five phase names is then shown a word from none of them and cannot tell whether it is the same step or a different one.
Every subagent description begins with its phase name:
| Phase | Agent description |
|---|---|
| re-check | re-check · last run's open issues |
| research | research · what's declared vs what actually holds · research · contradictions between files |
| fact-check | fact-check · try to break each claim |
The phase word does the joining; the clause after it says which pass this is. Under --quick there is one research agent, so it carries both jobs: research · what holds, and what contradicts.
Lens names never appear here either. Enforcement and Coherence are this file's labels for bundles of check IDs — useful when spawning, meaningless to whoever is watching. The same rule that keeps pooling out of the strip keeps it out of the agent list.
Only three of the five phases spawn anything — re-check, research, fact-check. cross-check and report run in the orchestrating context, so the screen shows no agent activity at all while they work, and those are the two phases most likely to look hung. They are also, not coincidentally, where a reader has most recently been promised something: the candidates have just appeared, or every claim has just resolved and the file is being written.
So the strip is the only signal during those two, which is why the block must be emitted at both boundaries — and why the report phase says where the file will land rather than going quiet until it exists.
Flags
Determine the mode before spawning anything. Two ways a user arrives, and both must resolve to exactly one row of the table below:
| How they invoke | Resolve by |
|---|---|
Slash command — /pape:harness-audit --deep |
Read the literal flag. It wins over everything, including contradicting prose |
| Natural language — "audit my harness", "is this repo agent-ready" | Map intent: quick · fast · just a look · rough → --quick · thorough · deep · comprehensive · don't miss anything · be exhaustive → --deep · anything else, including a bare request, is the default |
Never upgrade silently. A plain "audit this repo" may pre-select two finders, and "don't miss anything" may pre-select four — but neither spawns anything until the user has seen the shape and its cost and said go. The word doing the work is silently: mapping intent is fine, spending on it unasked is not.
Say what you are before you touch anything
Introduce yourself before doing any work. A run was observed reading two files, listing a directory and running three shell commands before it said what it was — so the user's first information about the skill was a tool-use summary.
One exception, and only one: locating a previous report. A single glob of harness-audits/ costs nothing and buys a better opening — "There's a run from earlier today" tells the reader more than "checking for a previous run", and they were never going to notice the intervening second. Reading that report's contents waits until after the introduction; so does everything else.
The test is what the user sees first. A directory listing that resolves instantly, then the introduction, is fine. A tool-use summary standing alone at the top of the transcript is not.
harness-audit v0.44 · from the Professional Agentic Product Engineering (PAPE) guide
I read the agent setup checked into this repo — instruction files, skills,
hooks, permissions, tests, CI — and rate it against the guide's eight tiers.
I change nothing: read-only, apart from the report I write to harness-audits/.
Six phases. This strip closes every message, so you always know where it is:
setup agree the depth and the output format — nothing spawns until you do
re-check re-test the issues the last run here left open
research read the repo, collect candidate findings
cross-check line up what research found, resolve where it disagrees with itself
fact-check re-open the file behind every claim and try to break it
report write it to harness-audits/
· setup → · re-check → · research → · cross-check → · fact-check → · report
There's a run from earlier today. Reading its cover for the baseline, then three
questions before anything spawns.
Spell the phases out here, once, before anything runs. A reader watching ▸ research → · cross-check → · fact-check has been given six words and no key to them, and cross-check and fact-check are near-synonyms in ordinary English — nothing in the strip says which one re-opens the files. The strip is a position indicator, not an explanation, and it was being asked to be both.
It costs six lines at the one moment a reader is deciding whether to commit to the run at all, and it buys everything after: every later message can then say research on its own, because the word has a definition the reader has already seen. This is the same trade as §Say what the audit looks for — orientation once, early, never repeated.
Say the strip will recur. "This strip closes every message" is what turns it from a thing that appeared once into a thing the reader tracks — and a run that then drops it has visibly broken a promise rather than quietly omitted a display.
Print only the phases this run will actually reach — no re-check row without a prior report, no cross-check row under --quick. A phase that never fills teaches the reader to distrust the strip, and a phase that has to be explained costs more than it shows. One clause in the closing line covers the absence: "No previous run here, so nothing to re-check — starting from scratch."
Name the version, and read it from ${CLAUDE_PLUGIN_ROOT}/.claude-plugin/plugin.json — the same source the report cover uses. A user comparing two reports, or reporting a bug, needs to know which version produced what, and the cover is too late: it arrives at the end, and only if the run finishes.
Say what happens next, in one clause. Not a plan, not a numbered procedure — "checking for a previous run, then three questions" is enough to tell someone the tool calls they are about to see are not the audit starting without them.
Confirm before spawning — every run
This run is not cheap and the user has not agreed to it yet — so the introduction is followed by a two-line frame and one AskUserQuestion. Short: they came for an audit, not a form.
State no duration and no token figure here. Not a range, not a hedge, not a prior run's measurement — see §Say nothing about how long it takes for why every figure this file ever carried was wrong. Say what it does and what it writes; the live meter reports the rest as it happens.
Harness audit — rates this repo's agent config against the eight tiers.
Read-only; writes to harness-audits/.
Then one AskUserQuestion call carrying two questions, so it is a single interaction:
| Question | Options | Default |
|---|---|---|
| How deep? | Quick look · Standard · Thorough | pre-select whatever the flag or the phrasing implied — Standard when nothing did |
| What output? | Markdown only · Markdown + light HTML · Markdown + dark HTML | Markdown + light HTML, or whichever theme --theme: named |
| Rate T1 too? | No — repo only · Yes, read this project's prompt history | No. The only read outside the repo, and the one most worth refusing |
Three questions, still one interaction. The T1 question is the difference between eight rated rungs and seven, and it cannot be inferred from anything in the tree — so it is asked once, here, where the reader is already deciding what the run may do.
Format and theme are one question, not two. Theme is the least consequential axis in the run and asking it separately spends a decision on it. Folded in, it is only asked where it means something — and the Markdown only path never has to answer it at all.
Say what Markdown only buys structurally: it skips the layout pass entirely. The HTML is generated serially by the same context after the markdown, so choosing markdown-only removes a whole phase of writing. Do not quantify it — no fraction, no minute count, no token figure. What runs is a fact; what it costs is not one this file gets to state.
Recommend Standard, and say why it is the recommendation — the reason is not "more coverage", and stating it wrong oversells the other two:
| Option | What to tell them |
|---|---|
| Quick look | One lens. Every finding is still verified, so nothing unchecked ships — you just see less of the repo, and no finding gets flagged as contested, because there is no second reading to disagree with it. |
| Standard — recommended | Two lenses read independently. Where they disagree, the clash itself is the signal: pooling flags it and the verifier takes it first. A real run had one finder report site/eslint.config.mjs missing when it exists — the second lens is what turned that from a shipped finding into a caught error. |
| Thorough | Four lenses, adding hygiene and context-economy — twice the finders, so materially more time and spend. Worth it when you want the sweep exhaustive, not when you want it right. |
Do not claim the second finder is what makes findings true. The verifier does that, and it runs in every mode. What the second lens buys is disagreement — a contested claim surfaced before the verifier sees it, and prioritised when it gets there. Say that, not more.
Pre-select, don't interrogate. A user who typed --deep has answered the first question; show it answered and let them confirm with one tap. The interview exists to make the spend visible, not to make them specify it twice.
Ask for intent, never for a number. "Quick look / Standard / Thorough", not "how many finders?" — nobody's first question is how many agents they want, because nothing has told them what a finder is. The mapping to --quick / default / --deep is this skill's job.
Say the cost structurally, in the option labels — "one lens instead of two", "twice the finders". That is a fact about what runs, not a prediction about the clock, and it is the honest way to make the trade visible where the choice is made. A number nobody can stand behind belongs nowhere; a number two paragraphs above the buttons belongs nowhere either.
Say what the T1 option reads, in the option label itself. "reads this project's prompt history" — not "enables T1". A permission question whose label describes the feature rather than the access is not a permission question.
Say what it writes before it writes it. Two files into harness-audits/, and nothing else touched. That is the only write this skill makes, and it lands in the user's repo — it is the real consent moment, more than the finder count is.
Say nothing about how long it takes
No duration, no token figure, no comparison to a previous run — at setup, in the report, or anywhere in between. Not a range, not a hedge, not "a few minutes", and not "the last run here took 38m" either. A measured past run is still presented as guidance for this one, and repo size and depth move it enough that the number misleads exactly when it matters.
Every figure this file has carried was wrong. It said 5–10 minutes, then 15–20, then 18, against real runs of 26 and 38. Each correction was written by someone who believed the new number was the accurate one. The pattern is the evidence: the problem is not which number, it is that a number is being stated at all.
What replaces it is the live meter in the strip — elapsed time and tokens as they actually accrue. That reports what is happening rather than predicting what will, and it needs no baseline to be honest.
Then emit the block — it is the confirmation
The moment the answers come back, emit the progress block and nothing else. Its header line already carries the resolved config, so a separate "Running Standard — 2 finders + verifier…" sentence says the same thing twice:
Harness audit · krivitskydotcom — Quick look · 1 finder + verifier · markdown only
✓ setup → · re-check → · research → · cross-check → · fact-check → · reportSpawning both passes now.
A run given both forms emitted the sentence and skipped the block, then produced the strip two minutes later once the agents were already out — so the reader spent the opening with no phase display at all, which is the one moment it is most needed. One mechanism: the block is the confirmation.
The header echoes the resolved config, so a misread is catchable while it is still cheap.
Describe the lenses, never name them. "1 finder (Enforcement)" tells the reader nothing — Enforcement is this file's word for a bundle of check IDs, and it leaks the same way pooling and finders did before the strip was rewritten. Say what it is doing, anchored to the phase word: "research reads what's declared against what actually holds". Under Standard, "research runs twice, independently — one for what holds, one for what contradicts".
Anchor it to research, because a loose description becomes a name. An earlier version of this line said "two passes reading independently", and a run compressed that into "then the reading pass" — a seventh phase word, invented from a sentence meant to explain the third one. Descriptions get shortened by whoever repeats them; if the phase name is not the thing being shortened, the shortening replaces it.
Never mention ctrl+b. The runtime already prints (ctrl+b to run in background) beneath every agent it spawns, so saying it again is a second spinner — the same duplication §Say something at each pass boundary forbids for progress bars, applied to a keybinding.
And it is not reliably true. That affordance backgrounds the agent the runtime is showing, not the skill invocation; the orchestrating context still holds the session. A run that promises ctrl+b "backgrounds it safely" is describing behaviour it does not control.
The wait no longer needs escaping anyway. Findings stream, the re-check lands in the first minutes, and the strip closes every message. Telling the reader to look away is an argument against everything else in this section.
The first characters you emit are harness-audit v. Not a sentence about what you are about to do — that string, literally, as the opening of the first message.
This is stated as a test rather than a prohibition because the prohibition failed three times, each with a new phrasing:
❌ "I'll run the harness audit on this repo." ❌ "I'll start with the frame, then confirm the shape before spawning anything." ❌ "I'll start with the audit intro, then two questions before anything spawns."
Banning examples teaches the shape, not the rule, and the shape has infinite variants — every one an announcement that the next line is coming. If your first message does not begin harness-audit v, delete whatever is above it.
The introduction is the first thing said; the frame is the second. Nothing precedes them.
❌ "I'll run the harness audit on this repo." — restates the command back at the person who typed it ❌ "I'll start with the frame, then confirm the shape before spawning anything." — narrates the procedure it is about to perform
The second is the one that keeps returning, because it reads as helpful orientation. It is not: the reader learns the shape from the frame one line later, and announcing an intention to do a thing is never worth a line when the thing itself is the next line.
The auditor field on the report cover carries the resolved shape, so the mode is also verifiable after the fact.
| Flag | Finders | Gauntlet | Budget | Verifier |
|---|---|---|---|---|
| (none) | 2 — Enforcement, Coherence | rendered | ≤20 reads each | always |
--quick |
1 — both lenses in one pass | dropped | ≤12 reads | always |
--deep |
4 — adds Hygiene, Economy | rendered | ≤25 reads each | always |
--theme:light · --theme:dark is orthogonal to the three above and combines with any of them. It changes the report's appearance and nothing about the audit.
| Value | Build |
|---|---|
--theme:light (default) |
inline report.css |
--theme:dark |
inline report.css, then report-dark.css after it |
Both themes are named. Not a bare --dark — a boolean flag leaves the other theme with no way to ask for it, so the report can only tell a reader to remove something rather than to request what they want.
The report never advertises its own theme. A line reading "Dark theme. For the light version, re-run the audit with --theme:light" spends the most-read position in the document telling the reader how to regenerate what they are already holding. The auditor field on the cover records the resolved flags, which is where a fact about how the run was configured belongs.
Cheap mode reduces recall, never precision. --quick buys speed by looking at less, never by checking less — there is no flag that skips verification, because unverified findings are the failure this design exists to prevent. A user who wants it faster gets fewer lenses and the same standard of evidence.
The verifier is the last thing to cut, not the first. It is the only phase whose removal changes what the report is, rather than how much of the repo it covers. Roughly a third of candidates get cut or corrected there. A run without it does not produce a quicker audit; it produces a document where one finding in three is wrong, addressed to someone who will act on it in their own repo. Speed that costs correctness is not a cheaper version of this skill, it is a different and worse one.
Nor may the finder verify its own claims to save the pass. The context that formed a claim is the worst available judge of it — it already has the reasoning that produced the error. Verification works because the verifier arrives with no memory of how the claim was reached and only the file in front of it. Folding the two together removes the mechanism while keeping the word.
Write it plainly
The report's default register drifts abstract, and abstract prose reads as authoritative while saying nothing. A shipped scorecard note read: "Two instruction layers, neither authoritative and both resident, with no step that reconciles them when one changes." The reader's verdict was "I don't understand this." They were right — three nominalisations, no picture, nothing a person could point at.
❌ "Two instruction layers, neither authoritative and both resident, with no step that reconciles them when one changes." ✅ "
CLAUDE.mdand five skills both hold rules. Neither one wins, both load every session, and when one changes nothing updates the other."
Same claim, same length, and the second one can be pictured.
- Name the thing.
CLAUDE.md, not "the instruction layer". Five skills, not "the extension surface". - Verbs, not nominalisations. "nothing updates the other" beats "no reconciliation step exists".
- One idea per sentence. A clause stack joined by commas is where meaning goes to hide.
- Words to distrust in your own draft: surface · layer · resident · authoritative · reconcile · instance of · structural · leverage · posture · in practice. Each is sometimes right and usually a sign you stopped at the first phrasing.
- No compressed triplets. "right context, proof it's done, a way back" — the reader asked what "a way back" meant. It meant rollback. Three noun phrases in a row read as cadence, and cadence hides that one of them stopped being a description. Write the sentence: "giving the agent the right context, checking that work is actually finished, and being able to undo a bad change."
- No aphorisms. "You can't harden a gate you don't have." "Climb only as far as the work needs." They sound settled and carry nothing a reader can act on. Cut them; the sentence before them was already finished.
- Never use a severity word as a bare noun. "every High below" — High what, and below where? The scorecard opens the report, so a reader meets the severity scale a section later. Write "all five high-severity issues." The same goes for
Confirmed,ProbableandNot assessed: they are labels in a table, not nouns in a sentence. - "Below" must point at something on the same screen. Once sections moved, half the cross-references pointed a section away. Name the section or name the thing.
This applies everywhere — scorecard notes, summary, issue prose, backlog items. Evidence gets to be technical; the sentence around it does not. LINK_CHECK_SKIP=1 is precise and belongs; "disables a deterministic verification pathway" is fog around it.
Findings
Every finding gets a display ID — ISSUE-1, ISSUE-2, … — unpadded, carried over from the previous report where the key matches and allocated fresh otherwise; see §Display IDs are allocated once and never reused. They will not run in table order, and should not. And this shape; the middle three are what separate a report from a linter.
Sort by severity, then by tier ascending — in the issues table and the backlog alike, and check the backlog against §1 before shipping. A run shipped a §1 reading "the next lever is T3" above a backlog opening T5 · T5 · T4, with the first T3 item fourth. Both halves were defensible on their own — the backlog had ordered itself by cost, cheapest-and-loudest first, and said so in its own lead — and together they told the reader two different things to do first. The lever is the report's single most actionable sentence; a backlog that opens somewhere else has overruled it silently.
The check is mechanical: the first backlog item must sit on the rung §1 named. Where it does not, one of the two is wrong — fix that, don't reword around it. Severity-then-tier produces the right order by itself, because the lever is by definition the lowest rung that doesn't hold, so its Highs sort above every other High.
And §3's lead must describe the order it actually used. The same run's lead promised "items early on are cheap and stop something from lying to you; items later are larger reorganisations" — a third ordering, matching neither the sort above nor the lever, and not consistently followed either. A lead that describes a different sort than the list below it is worse than no lead.
Severity alone interleaves rungs inside a band — a High on T3 sitting under a High on T5 — which contradicts the scorecard's lever and the backlog's order in the same document. The lower rung comes first because the higher one rests on it. The backlog uses the same key, and each backlog item carries its rung (— T3 · ISSUE-1) so the ordering is visible rather than asserted.
Not H-. An earlier version used it (for harness), never defined it anywhere, and put it one column away from a severity column containing High — so H-013 · Low read as a contradiction to anyone scanning the table. A prefix that collides with a value in the neighbouring column is a bad prefix however defensible its etymology, and an abbreviation the report never expands is a decoding tax on every reader to save four characters. Spell it.
| Criteria | The standard, stated positively — not "you're missing X". Take it from the guide tip the finding cites, so the bar is the guide's and not this pass's invention. A criteria line the audit made up is the first place a wrong finding hides |
| Condition | What is true, with file:line and an excerpt quoted from the file read in full context — never lifted from a grep hit. Where two files bear on one claim, read both and say what the pair shows; a contradiction between them is usually the better finding |
| Cause | Why it drifted. Almost always process, not ignorance: "the allowlist grew by accretion — each entry added to unblock one session, and nothing treats a permission change differently from any other config change." Guess honestly and say you're guessing |
| Consequence | What it costs in practice. Name the failure mode, not the rule |
| Corrective action — now | The smallest thing that closes it, with the literal lines to paste |
| Corrective action — structural | What stops it recurring |
The finding asserts. The fix proposes. Keep the voices apart.
A report once recommended the wrong hook event with total confidence. The reader's verdict — "trust the report's problem, not its wiring" — is the whole design brief: when diagnosis and prescription speak in one voice, a wrong prescription discredits a correct diagnosis. Separate them and a bad fix costs only the fix.
Every corrective action carries three things the finding does not:
| The requirement, stated before the mechanism | "The rule fires at push time; the gate must fire there too." A reader can then accept the requirement and reject your mechanism — which is the point. Fuse them and they can only accept or reject both. |
| The assumption it rests on, written as a refutable sentence | "This assumes npm run verify is deterministic outside CI — checked at playwright.config.ts:38." This line is the actual control: you cannot write it without opening the file, and once written it can be proved wrong by someone who knows more than you. Unstated assumptions are the ones that ship. |
| Its own confidence | A Confirmed finding routinely carries a Probable fix. Say so: "Fix: Probable — I have not run this against your dev-server setup." Collapsing them into one number is how a solid diagnosis inherits a shaky prescription's risk. |
Where the reader knows more than the audit, ask instead of answer. A static review cannot see which server is warm on :3007, how long the suite takes, or which rules the team has already given up on. Turning those into questions — "before wiring this into a gate, check how your Playwright config resolves the server outside CI" — gets the right answer from the person holding it, and costs nothing when they already know.
Ship runnable content, at the altitude you can defend. This is not a licence to hand back homework — a corrective action with nothing executable in it is worthless. But the mechanism is the part this pass is most often wrong about, and the acting reader can check it in seconds where the audit could not. So the requirement binds, the lines illustrate, and the check that would falsify them is named. "Baseline — confirm X before applying" is worth more than a confident block of JSON, because the confident block is what shipped wrong twice.
Write for a reader who can do better than you
Design target: a report an agent with more context can trust and act on. Whoever acts on this — a person who has worked in the repo for a year, or an agent that can run the suite, read every file, and try three approaches — is very often better positioned than this pass was. That sets the altitude for everything below. This audit is a budgeted, read-only, single-pass review; where it descends into specifics it cannot check, it manufactures exactly the claims a better-positioned reader will refute, and each refuted claim spends trust the sound findings needed.
So state the standard, state what was seen, and mark the uncertainty — then stop descending. Requirement-level statements anchored in a guide tip survive review; mechanism-level assertions are what get overturned.
The failure to avoid is a ceiling: hand an agent working JSON and it pastes the JSON. It will not look for the better mechanism, because nothing asked it to — and the one thing this audit was most confident about is the thing it got wrong.
So each corrective action makes the requirement the contract and the code the baseline:
Requirement: the gate must fire at push time, run the same command CI runs, and not tax commits. Baseline that meets it: (the literal lines) Better welcome: if you can satisfy the requirement more cheaply or more robustly with what you can see from inside the repo, do that instead — the requirement is the contract, not this implementation.
And give them what they need to go past you. The .md twin's job is not "the same report, machine-readable" — it is a briefing, so it carries what the HTML doesn't need:
open-questions:
- Does `npm run verify` complete in under 60s on a warm machine?
Not measurable statically; decides whether a push gate is tolerable.
- Are any of the 56 rules already known-dead? Killing them beats enforcing them.
- Is `edge-tts` expected on PATH for other contributors, or is audio a solo task?
budget-hit: no — finders 18 + 14 of 20 each · verifier 21 of 25
cost: 18m · 340k tokens · 2 finders + verifier
cost: records what this run took — wall time from start to report written, total tokens across every pass, and the shape that produced them. It is a measurement of something that already happened, which is the only kind of cost figure this skill states. Nothing reads it back as an estimate, and no run quotes it at the reader; see §Say nothing about how long it takes.
Open questions are not hedging. Each one names something the audit genuinely could not determine and says what it would change. A question that would not change a recommendation is noise; cut it.
Then say so in §6: "If you find a better solution than the one proposed, the finding is what mattered — take it."
The correction is altitude, not breadth. Do not offer three options and let the reader choose — that is decision-dumping, and it is the same mistake as gating by tier. Climb one rung above the mechanism, state the requirement there, then commit to a single mechanism that meets it:
❌ "Add a
Stophook runningnpm run verify." ✅ "The rule fires at push time, so the gate must too — that meansPreToolUsematched ongit push, the only event that fires there."
Same one recommendation, one sentence longer, and now refutable. The failure it prevents is pattern-matching: enforcement means hook reaches for the hook in the tip's example instead of the hook the rule needs. Writing the requirement first forces the mechanism; skipping it lets the example pick.
Say it once in §6: corrective actions are proposals; a finding stands even if its fix is wrong.
Say when findings compound. "Combined with ISSUE-1, the configuration reads as gated to anyone reviewing it, while in practice nothing blocks." Two findings that multiply are worth more than their sum, and a reader who fixes one and not the other has fixed nothing. Look for these deliberately.
Severity — the cost if it goes wrong
| Critical | A single bad turn is unrecoverable — destroyed work, rewritten shared history, leaked live credential |
| High | Silently wrong behaviour, or a guardrail that doesn't hold |
| Medium | Real cost, recoverable — waste, drift, inconsistency between sessions |
| Low | Friction or a latent trap |
| Info | Worth knowing, no action |
Confidence — how sure you are, a separate axis
- Confirmed — read directly from a file. Quote it.
- Probable — inferred, and say from what: "whether this server is loaded in practice cannot be determined from configuration alone."
Never collapse these. Medium / Probable and Medium / Confirmed are different asks. A reader deciding what to do this afternoon needs both numbers.
A finding may only assert what a file shows. Absence of a positive is not a negative — if the tests might be in an unusual place, that's Probable with the search printed, never a confident miss.
Redact secrets. A credential is reported by shape and location, never reproduced: "sk_live_••••••••••••". The report gets committed; do not put the key in it twice.
Observations (O-1…) are not findings. Things with no measurable effect on behaviour, recorded so they're not mistaken for oversights — including praise that's actionable: "the API file uses worked examples rather than prose rules; that's the strongest writing here and worth copying to root." Say explicitly they need no response.
The scorecard rates the guide's eight tiers
The axis is the tier ladder, not the check catalogue. checks.md categories stay where they are good — filing a finding — but the scorecard is indexed T1–T8, so a reader who sees a weak rung knows which part of the guide to open. Take the names from guide.md's ## Tier N — headings, never from a mirror:
T1 Professional Prompting · T2 Shaping & Slicing · T3 Context Management · T4 Loop Until Done · T5 Checkpointing & Hardening · T6 Orchestration · T7 Fleet Ops · T8 Agent Execution Layer
Each finding already cites a tip; the tip number gives its tier. A finding that blocks two tiers counts against both — a missing .claude/settings.json genuinely stops you doing T5's hardening and T8's permission work, and a rating reads as "can this repo do this tier's work", not as a fault tally.
Rate each Strong · Satisfactory · Moderate · Weak · Missing · Not assessed, with a one-line note.
This is not tier-gating. ## Method forbids inferring *how amb
Truncated - read the full file at https://github.com/krivitsky/professional-agentic-product-engineering/blob/fafbc694a3be8692b3f092d4be0c800ed21944f0/plugins/pape/skills/harness-audit/SKILL.md.