canvas-flow

Imported from fernandomenuk/canvas-flow (AGENTS.md). Install upstream with npx skills add fernandomenuk/canvas-flow. Copyright stays with the author.

AGENTS.md

This file provides guidance to coding agents when working with code in this repository.

Commands

pnpm run check          # Run build, lint, format check, typecheck, tests, and skill freshness check
pnpm run build          # Bundle dist/cli.mjs and copy chrome/design assets into dist
pnpm run build:skill    # Regenerate skills/canvasflow/SKILL.md from shared CLI guidance
pnpm test               # node:test runner (test/*.test.js)
pnpm run lint           # ESLint over bin src test scripts
pnpm run format:check   # Prettier check
pnpm run typecheck      # tsc --noEmit (checkJs mode)

Run a single test file: node --test test/server.test.js. Filter by test name: node --test --test-name-pattern "createOpenOutput" test/cli-output.test.js.

The prepack script runs build automatically, so publishing always ships a fresh bundle. The committed skills/canvasflow/SKILL.md is generated by pnpm run build:skill; pnpm run check fails if it drifts from the shared no-args home output.

Project Conventions

• Node 22+, ESM-only JavaScript ("type": "module"). No TypeScript source - .js files validated via TS checkJs.
• Use TDD for bug fixes and new features (see test-driven-development skill).
• Run pnpm run check before pushing.
• Branch, make changes, run the checks, open a PR. See CONTRIBUTING.md.

Architecture

Canvas Flow is a CLI + local HTTP server that opens agent-generated HTML artifacts in a browser, lets the user annotate elements, selected text ranges, or Mermaid diagram nodes, and ships those annotations back to the agent over a long-polling API.

Process model

The CLI (bin/canvas-flow.js -> src/cli.js) is the user-facing entry point. The first command that needs the server spawns canvas-flow server as a detached background process (src/cli.js:startServer) and waits for /health, which returns { ok, app, version }. Subsequent CLI invocations reuse the running server only when its health version matches the current CLI version; stale servers are asked to POST /shutdown, and pre-handshake servers may be SIGTERM'd by port PID before the upgraded server is spawned. Port defaults to 4387 (CANVAS_FLOW_PORT). CANVAS_FLOW_HOST sets the bind address (default 127.0.0.1; a wildcard 0.0.0.0/:: binds every interface). Binding beyond loopback exposes an unauthenticated server that can read and serve arbitrary local files to anything that can reach it, so only do so on a trusted network. CANVAS_FLOW_LINK_HOST sets the hostname written into generated session links (default: the bind address, or loopback when bound to a wildcard). The CLI's own control-channel requests dial the bind host, falling back to loopback when it's a wildcard (src/paths.js:bindHost/clientHost/linkHost). CANVAS_FLOW_NO_OPEN=1/--no-open suppresses the local browser launch, and --no-gate skips the open-time layout curtain for that browser open.

The detached server does not run forever. It shuts itself down once no browser chrome (SSE) and no agent poll have been connected for CANVAS_FLOW_IDLE_TIMEOUT_MS (default 30 minutes; set 0/off to disable), and immediately when the last open session ends while nothing is connected (a still-attached browser or poll defers cleanup to the idle timer instead). canvas-flow stop (stopCommand) explicitly POST /shutdowns the server on the default port, accepts --port, and reports stopped, stopping, not-running, or not-canvasflow. Because cleanup keys off live connections rather than session status, the next canvas-flow <file> re-spawns a fresh server and adopts the session from state.json when the session is still resumable. A session the user ended from the browser is not resumed by a plain open; /api/sessions returns status: "user-ended" unless the request opts in with reopen: true.

State lives at ~/.canvas-flow/state.json (override with CANVAS_FLOW_STATE_DIR). All sessions across all projects share this one file, keyed by a sha256 prefix of the canonicalized file path - so the CLI never needs opaque session IDs; the canonical HTML path is the identity (src/session-store.js:sessionKey).

Request flow

1. canvas-flow <file.html> (openCommand) -> POST /api/sessions -> SessionStore.upsertSession -> server returns http://127.0.0.1:PORT/session/<key> and the CLI calls open to launch the browser. Passing --no-gate sends noGate to /api/sessions, returns a one-open URL with ?no-gate=1, and leaves the stored canonical session URL unchanged. Session state records who ended a session as ended_by: "user" for browser-initiated ends and ended_by: "agent" for the CLI's file-based /api/end (canvas-flow end <file>). The browser chrome's plain "End session" calls keyed /api/:key/end, while "Send & end session" posts queued prompts to /api/:key/prompts with endSession: true so final feedback and end attribution are delivered atomically. Idle self-shutdown never calls endSession at all - it only stops the server process, leaving session status untouched in state.json, so there is no third ended_by case to track there. /api/sessions checks ended_by before reviving a session: if the existing session is ended with ended_by: "user" and the request body does not set reopen: true, it returns { status: "user-ended" } without upserting, and openCommand skips launching the browser and returns refusal guidance instead - --reopen is the explicit opt-in. Agent-initiated ends (ended_by: "agent") keep reviving on a plain open, unchanged from before this gating existed.
2. The browser loads GET /session/:key, which serves a chrome page (createChromeHtml) containing a sandboxed iframe that the chrome client points at /artifact/:key/index.html, a stylesheet at /chrome.css, and browser behavior at /chrome-client.js. The chrome client reads its session bootstrap from the canvasflow-session JSON script in the page, including whether the layout gate is enabled for this open and which key drives the annotate/explore mode hotkey.
3. The artifact route reads the HTML from disk and runs injectCanvasFlowSdk (src/html-transform.js) to append <script src="/sdk.js?key=..."> at the end of <body>. Nothing else is injected - artifacts stay byte-identical (apart from the SDK script tag) so they remain portable when opened directly without the server.
4. Sibling assets resolve under /artifact/:key/<path>, sandboxed to the artifact's directory (resolveArtifactAsset rejects paths that escape via ..). The packaged Tailwind/DaisyUI assets are still served from /design/:asset so older artifacts that link them keep working, but new artifacts are no longer auto-wired to those routes.
5. The injected SDK also runs a render-time layout audit after document.fonts.ready and a brief ResizeObserver settle window. It detects page horizontal overflow, element overflow, clipped or visibly spilling text, and overlapping text. Intentional horizontal scrollers using overflow-x: auto or scroll are excluded from horizontal checks, and overflow-y: auto or scroll is treated as intentional for vertical overflow. The artifact posts canvasflow:layoutWarnings to the chrome with structured layout_warnings entries containing selector, kind, overflowPx, viewportWidth, and severity. While the open-time layout gate is enabled, the chrome reveals clean and warning-only audit results, holds only error-severity results behind the curtain, and still POSTs those findings through the existing layout warnings path so an active poll wakes the agent. The gate re-arms on hot reload and reveals after the next clean audit; Show anyway and a bounded safety timeout reveal with a persistent may-have-layout-issues banner instead of blocking review indefinitely. The chrome POSTs those findings to /api/:key/layout-warnings; changed non-empty warnings are stored in SessionStore, mark the session as feedback, and emit the same feedback event as human prompts so a fresh warning wakes an active poll. takeFeedback delivers layout warnings alongside human prompts, remembers delivered kind:selector keys for repeat detection, clears current warnings after delivery, and leaves human feedback additive and unchanged.
6. User actions in the iframe postMessage to the chrome (queue prompts, request snapshot, end session, toggle annotate/explore mode). The chrome stores queued prompts in tab sessionStorage, replaces unsent prompts that share the SDK-internal _canvasflowQueueKey, strips that internal field before POSTing collected prompts to /api/:key/prompts, removes sent prompts only after a successful response, queues them in the session store, and emits a feedback event for normal sends or an ended event when the body includes endSession: true. Text selection prompts use tag: "text" and preserve a structured target with type: "text-range", selected text, commonAncestorSelector, and start/end boundary anchors. Mermaid diagram-node prompts use tag: "mermaid-node" and preserve a structured target with type: "mermaid-node", the Mermaid diagramId and nodeId, the rendered label, and a selector, so the annotation anchors to node identity and survives a re-render that reshuffles the SVG. In the chrome composer, Enter sends queued prompts (equivalent to clicking "Send to Agent"); Shift+Enter inserts a newline. Sending an empty composer stays enabled, shows an inline hint, and focuses the composer instead of disabling the button. The split-button menu also offers "Send & end session", which submits queued prompts with an endSession flag and marks the chrome ended only after that POST succeeds. The annotation card textarea follows the same convention: Enter queues the annotation (equivalent to clicking "Queue"); Shift+Enter inserts a newline; Ctrl+Enter (Cmd+Enter on macOS) queues the annotation and immediately sends all queued prompts, and the card shows a small hint for these shortcuts.
7. The chrome top bar exposes annotation mode as an Annotate switch with a Cmd/Ctrl+I tooltip for toggling between annotate and explore mode, and keeps editing actions in an overflow menu: the home-shortened artifact path with a copy affordance, reload artifact, copy DOM snapshot, export standalone HTML, publish link, and end session. Copy path still copies the absolute canonical path, copy DOM snapshot requests a fresh iframe snapshot before writing to the clipboard, export downloads the local-inlined bundle, and publish opens the ht-ml.app share dialog with a linked service name and an up-front third-party disclosure.
8. canvas-flow poll <file.html> (pollCommand) hits GET /api/poll. If queued prompts or layout warnings exist, including prompts queued before an ended session, it records that an agent has observed the session and returns them immediately; otherwise it marks the session as actively listening and long-polls on an EventEmitter until a feedback or ended event fires. Poll output includes layout_warnings only when the browser reported current findings, with a server-normalized persistent boolean on each finding. The agent-facing next_step requires a fix-and-recheck loop only for fresh error-severity warnings; once every current warning is persistent or below error severity, it permits proceeding to the human with a note instead of looping further. A status: "ended" response carries ended_by ("user" or "agent") and a next_step telling the agent to stop polling and not reopen the browser uninvited - deliver remaining updates in chat instead, unless the user asks for further review or something important needs their attention. The final status: "feedback" batch delivered right before a session ends (e.g. "Send & end session") carries the same signal via session_ended: true plus ended_by, so that last next_step also skips telling the agent to reopen. Default no-timeout polls stream whitespace heartbeat bytes before the final JSON response; --timeout-ms is a non-streaming test/debug escape hatch. The CLI also writes an immediate waiting banner and per-minute waiting messages to stderr for no-timeout polls, keeping stdout reserved for the final JSON/TOON response. If SIGINT or SIGTERM interrupts a no-timeout poll, the CLI writes re-run guidance to stderr and exits with the conventional signal code; queued feedback persists, so re-running the same poll is safe.
9. The /events/:key SSE stream emits agent-presence states: waiting before any poll has attached, listening while a poll is active, and working after a poll has delivered feedback and released. The chrome uses this state to show the waiting banner, allow queued feedback while waiting or listening, and block sends only while working.
10. --agent-reply posts a chat message into the session before polling, so the agent's reply renders in the browser conversation panel via the /events/:key SSE stream.

Live reload

When a session opens, chokidar watches the artifact file itself by default - watching the entire parent directory recursively saturated the event loop when artifacts lived inside large trees like ~. Artifacts can opt back into directory-wide live reload by adding data-canvasflow-live-reload-root to a root element or <meta name="canvasflow-live-reload" content="root">; that switches the watcher to the artifact's directory (excluding .git, node_modules, dist, build, .canvas-flow). Any file change emits a reload event, which the SSE endpoint pushes to the chrome page, which reloads the iframe. The artifact SDK reports scroll position with canvasflow:scroll, and the chrome replays it with canvasflow:restoreScroll after the iframe loads because the sandbox prevents direct scroll reads. During version-driven shutdown, the server sends a chrome-reload SSE event so open browser chromes wait for the replacement server and then reload the whole chrome page. Hand-edited files in dist/ won't trigger reloads. Run canvas-flow server --verbose (or set CANVAS_FLOW_DEBUG=1) to log session and watcher events to stderr when diagnosing wedges. Detached server stdout/stderr is also appended to server.log in CANVAS_FLOW_STATE_DIR (default ~/.canvas-flow/server.log) for startup and crash diagnostics.

Export (local-asset inlining)

src/export-bundle.js (buildSelfContainedHtml) turns an artifact into one portable HTML file by inlining only its local assets: local <link rel="stylesheet">/classic <script src> become inline <style>/<script>, and local images/fonts/icons, confined fetchable file:// refs, and CSS url(...)/@import become data URIs (recursively, resolved relative to each stylesheet). Remote references are deliberately left as-is - http(s) and protocol-relative CDN/font URLs, and remote CSS url(...), stay in the output and the browser loads them at render time. The transform therefore makes no outbound requests (no fetching, no SSRF); its only security surface is local file reading, which is confined to the artifact directory both lexically (confineDir) and by real-path/symlink resolution in the default readLocalFile (guardedRead), so a symlink inside the directory can't exfiltrate an outside file (e.g. ~/.ssh/id_rsa) into a shared bundle. Absolute file:// paths in non-inlined regions are redacted to about:blank so local paths do not leak into exports or hosted shares. Local reads are bounded by per-asset (10 MB) and per-bundle (25 MB) caps (CANVAS_FLOW_EXPORT_MAX_ASSET_BYTES / CANVAS_FLOW_EXPORT_MAX_BUNDLE_BYTES); the injected CanvasFlow SDK is stripped; in-document fragment refs (#a, encoded %23a) are left alone; inlined </script>/</style> are escaped so they can't break out; and the transform records warnings rather than failing. Warnings are split into unresolved local assets, such as load-failed, outside-root, too-large, or unsupported local references left external, and notices, such as csp-meta or file-url-redacted. The transform is dependency-injectable (readLocalFile, resolveAbsolute, confineDir, size caps) so it is testable without disk; the server passes resolveAbsolute: resolveDesignAssetPath to inline legacy /design/* references from the packaged assets. The browser surfaces it as an Export standalone HTML item in the chrome overflow menu, which GETs /api/:key/export and blob-downloads <name>.export.html; the CLI exposes the same transform as canvas-flow export <html-file> [--out <path>], server-independently. Because remote CDN/font references are left as links, a static export needs network to render those remote assets - this is documented behavior. CanvasFlow itself sets no Content-Security-Policy on any response (the sandboxed iframe relies on the sandbox attribute, not CSP), but author-set CSP meta tags are preserved and reported as export notices because they may still block exported inline assets.

Hosted sharing (ht-ml.app)

src/html-app.js (publishToHtmlApp) publishes the local-inlined HTML to ht-ml.app, a third-party hosting service not part of CanvasFlow, and returns a visitable URL. It sends the bundle to ht-ml.app's servers with POST {CANVAS_FLOW_HTML_APP_API_URL or https://api.ht-ml.app}/v1/sites as { html_content, password? }; creating a site needs no account or API key. The response carries the share url plus a secret update_key (returned once, the only credential, used later to update or delete the page). An optional bearer token (CANVAS_FLOW_HTML_APP_TOKEN / --token) is sent when set but is never required. Remote CDN/font references in the published page load over the network because ht-ml.app serves hosted pages with no CSP and no sandbox header; the viewer browser still needs network access to those CDNs to render them. The browser surfaces a Publish link overflow-menu item that opens a share dialog with a linked ht-ml.app mention and an up-front third-party disclosure, then POSTs /api/:key/share; the route is same-origin guarded (isSameOriginRequest) because publishing is a state-changing, outward-facing action - a cross-origin page must not drive a publish through the loopback server. The CLI exposes canvas-flow share <html-file> [--password <pw>] [--token <t>], server-independently. Published pages are PUBLIC by default - anyone with the link can view them. When --password or a browser-dialog password is set, the page is PRIVATE and password-protected - viewers must supply the password to view, and runtime output reports public: false. Hosted shares never include the annotation SDK.

AXI integration

The CLI is built on axi-sdk-js (runAxiCli). The home() callback returns the rich object shown when the user runs canvas-flow with no arguments - this is the same TOON-serialized output that lands in the agent's optional SessionStart hook after canvas-flow setup hooks (sessions, visual_guidance, playbooks, help). Top-level --help returns the same static guidance without dynamic sessions, canvas-flow playbook [playbook_id] exposes focused artifact guidance, and canvas-flow design prints a content-to-playbook router, copy-pasteable Tailwind/DaisyUI CDN URLs, Mermaid diagram tooling, an optional layout safety CSS snippet, and the DaisyUI component reference as a fallback after user-specified and subject-project design sources come up empty. That design output recommends data-theme="luxury" as the default DaisyUI theme and warns that @applying DaisyUI classes inside <style type="text/tailwindcss"> aborts the Tailwind browser-runtime compile. src/skill.js renders the installable Agent Skill from the same home output, rewriting command examples to non-interactive npx -y @menukfernandoo/canvas-flow ... invocations, omitting live session state, and including Hermes Agent frontmatter metadata for categorization. The generated skill intentionally omits a version frontmatter field so package.json version bumps don't require regenerating skills/canvasflow/SKILL.md. skills/canvasflow/SKILL.md is the Agent Skill shipped through npm. The playbook guidance tells agents that one artifact can combine several playbooks and must open each matching playbook before writing HTML. Diagram guidance names hand-built div/flexbox boxes-and-arrows as an anti-pattern and points flow, architecture, state, and sequence diagrams to Mermaid unless SVG is needed. The bare-arg form (canvas-flow some.html) is normalized into ["open", "some.html"] by normalizeArgv. normalizeArgv must let the SDK's RESERVED_COMMANDS (such as the built-in update self-updater) pass through untouched; otherwise the bare-arg rewrite turns canvas-flow update into ["open", "update"] and the inherited reserved command never reaches runAxiCli.

Telemetry

src/telemetry.js posts anonymous events to an Umami endpoint. The website ID and host can be baked into the bundle at build time via CANVAS_FLOW_UMAMI_HOST/CANVAS_FLOW_UMAMI_WEBSITE_ID env vars (consumed by scripts/build.js and inlined as process.env.CANVAS_FLOW_BUILD_UMAMI_* defines). The build also inlines package.json's version as process.env.CANVAS_FLOW_BUILD_VERSION for the CLI/server version handshake, with source runs falling back to reading package.json. Users opt out with CANVAS_FLOW_TELEMETRY=0. The client is best-effort and must never affect CLI behavior - all errors are swallowed. No need to explicitly document the telemetry behaviors.

Things to know when editing

• canonicalFile runs realpath, so symlinks resolve to their target before becoming session keys. Two paths that refer to the same file always collapse to one session.
• The SDK injected into artifacts lives in src/artifact-sdk.js and is wrapped by createSdkJs. It executes inside an iframe sandboxed with allow-scripts allow-forms allow-popups allow-downloads (no allow-same-origin), so it cannot read the chrome's DOM - communication is postMessage only. It runs the layout audit in the iframe because the chrome cannot directly inspect the sandboxed document. Tailwind, DaisyUI, Mermaid, and the optional layout safety CSS from canvas-flow design are not auto-injected: before writing HTML, agents should follow any user-requested look or design system, otherwise inspect the project the artifact is about - the subject or product whose content or UI it represents, which may differ from the current working directory - for design conventions (Tailwind or theme config, CSS variables or design tokens, component library, brand assets, existing styled pages), use that app's own design system for UI-preview artifacts, and only fall back to the canvas-flow design snippets when both sources come up empty. For flows, architecture, state, or sequence diagrams, agents should open the diagram playbook and use the Mermaid snippet unless SVG is needed for richly annotated nodes.
• The injected SDK also enhances rendered Mermaid diagrams live: explore mode gives each Mermaid <svg> dependency-free viewBox pan (drag) and zoom (wheel), and annotation mode freezes that pan/zoom so a click resolves cleanly to one node instead of panning. It enhances on load and DOMContentLoaded and re-runs through a throttled MutationObserver because Mermaid renders asynchronously and can re-render. Enhancement touches only the live SVG's viewBox and listeners, never the saved artifact, so the diagram still renders identically when opened directly. Node detection, label extraction, and target validation live in src/mermaid-node.js so they are unit-testable and shared with the server; createSdkJs serializes each exported helper into the SDK as a same-scope const (like deriveQueueKey), derived from the module's exports, so a helper may reference only its own arguments, browser globals, or its sibling exports.
• Native controls (button, input, select, textarea, option, label, summary, and editable regions) and their descendants are ignored by annotation handlers, so they can toggle, focus, type, and call window.canvasflow.queuePrompt() or window.canvasflow.sendQueuedPrompts() without data-canvasflow-action.
• Use data-canvasflow-action only for custom non-native controls that should bypass annotation and get a pointer cursor.
• For reversible input controls, prefer local selection state plus one per-question submit that calls window.canvasflow.queuePrompt() with the final answer. Use data-canvasflow-question on the question wrapper or queueKey in queuePrompt options when pre-send updates for the same question should replace each other in the browser queue.
• For text annotations, prompt.selector is the common ancestor/container selector, not the complete identity. Use the target range boundaries and snapshot context to locate the exact selected text.
• For Mermaid diagram nodes, a click annotates the whole rendered <g> node - not the sub-shape under the cursor - and hover highlights the same node; the prompt uses tag: "mermaid-node" with a target carrying type, diagramId, nodeId, label, and selector, so it anchors to node identity rather than a structural path. SessionStore.normalizeTarget routes these through normalizeMermaidNodeTarget, which strips them to that fixed shape, while text-range and other/legacy targets pass through unchanged.
• SessionStore re-reads and re-writes the entire state.json on every operation. There's no in-memory cache and no locking - acceptable because writes are infrequent and serialized through the single server process.
• The chrome and the sandboxed artifact document cannot see each other's keyboard events (no allow-same-origin), so any keyboard shortcut that must work regardless of focus needs its own capture-phase document.addEventListener("keydown", ..., true) in both src/chrome-client.js and src/artifact-sdk.js, not just one. The annotate/explore mode toggle hotkey (MODE_TOGGLE_HOTKEY_KEY, Cmd/Ctrl+I) is the reference implementation: the chrome owns the mode state and toggles it directly; the SDK side has no mode state of its own, so on catching the hotkey it postMessages { type: "canvasflow:toggleAnnotationMode" } to the chrome, which drives the exact same toggleAnnotationMode() function the on-screen switch's onclick calls. Requiring a modifier (metaKey || ctrlKey) is what lets the listener safely call preventDefault() without breaking plain typing (including typing the bound letter itself) in the chat box or an annotation-card textarea.
• Tests use CANVAS_FLOW_STATE_DIR and ephemeral ports to stay isolated. When adding tests that spin up the server, do the same.
• Circular close buttons (.pill-close, .share-close) render an inline SVG x mark with two symmetric strokes, not a text x/× glyph. Font metrics put text glyphs off from the geometric center even under flex centering, while the SVG centers via flex plus equal viewBox margins. Keep the SVG strokes on currentColor so existing hover color rules still apply, and follow this pattern for any new circular icon-only button.
• The in-iframe layout audit (src/artifact-sdk.js) has three easy-to-reintroduce failure modes, all covered by dedicated tests:
  • Overlap detection (auditOverlappingText) must compare getClientRects() fragments, not getBoundingClientRect(). A wrapped inline element (a <strong>/<code> phrase that breaks across a line) reports one bounding rect spanning both lines, so a bounding-box intersection test false-flags anything sitting in the reflow gap between the two real line fragments as "overlapping-text". fragmentsSignificantlyOverlap compares real per-line rects instead.
  • Fixed-size boxes with overflow: visible (the default) - badges, pills, buttons - don't clip overflowing content, they let it spill out and overlap neighbors, which is just as broken as overflow: hidden clipping it. classifyVerticalOverflow flags both, distinguished by its clips field. Because a visible spill isn't stopped by any ancestor's box, scrollHeight bubbles it up through every unconstrained block ancestor (a badge inside a flex row inside a section all measure the same few px of overflow) - resolveSpillCandidates defers non-clipping findings and keeps only the innermost element in each ancestor chain, or a single defect fans out into several redundant warnings pointing at the wrong element.
  • SessionStore marks a re-reported layout warning persistent: true once its kind:selector key has already been delivered to the agent via takeFeedback (tracked in session.delivered_layout_warning_keys, independent of the layout_warnings field that gets cleared on each delivery). cli.js's createFeedbackNextStep softens its guidance - permitting the agent to proceed to the human instead of looping fixes and reloads - once every current warning is either persistent or below error severity (currently only overlapping-text, since it stays heuristic even after fragment-aware matching).