vidra

Imported from yegamble/vidra (AGENTS.md). Install upstream with npx skills add yegamble/vidra. Copyright stays with the author.

AGENTS.md — vidra (meta repo)

Deployment/orchestration repo for the vidra platform: docker-compose files, Caddy config, deploy/backup/restore/rollback scripts, env templates, and audit docs. The application code lives in the component repos (vidra-core, vidra-user, vidra-search), which are checked out NESTED inside this repo on operator machines — those nested checkouts are gitignored here and pinned DETACHED at release tags.

Verification gates (run before opening any PR)

bash -n <every touched script>
shellcheck <every touched script>     # if available
cp env/production.env.example /tmp/check.env   # fill the ${VAR:?} keys with dummies
docker compose -f docker-compose.yml -f docker-compose.prod.yml --env-file /tmp/check.env config -q

config -q above is a complete check: Compose validates the whole model, and the ${VAR:?} asserts fire, whether or not profiles are selected (verified 2026-09-03 by breaking a service key and by removing JWT_SECRET — both forms exit 1). But every service sits behind a profile, so if you drop -q to READ the render, add --profile core --profile frontend or the output is a bare services: {} and you will think the file is empty. Use the full form when you need to assert what the prod overlay actually produces — e.g. that postgres, redis and search publish no ports and api/frontend publish on 127.0.0.1 only:

docker compose -f docker-compose.yml -f docker-compose.prod.yml --env-file /tmp/check.env \
  --profile core --profile frontend config

CI: what "required for merge" means

One check stands for the whole required set: ci-required — the only name that belongs in branch protection for this repo. It reads .github/required-checks.txt, the checked-in definition of required, and fails if any listed lane failed, was cancelled, timed out, or never ran. Required today: validate, bundle, boot. Removing an entry from that file fails validate (scripts/ci/check-required-manifest-removals.sh, compared against the base branch) unless the same file carries # retired: <name> — <reason> for it: retiring a lane is a deliberate, diff-visible act, never a quiet deletion.

The local gates above are the same assertions validate runs, plus two that had been committed and wired to nothing until A39:

• tests/*.sh is now shell-linted, not just tests/install_test.sh. The acceptance harnesses blank-server-smoke.sh and runtime-smoke.sh were checked by nothing; a syntax error in either would have surfaced on the machine being rehearsed, not in CI. tests/*.mjs gets node --check for the same reason (the smoke DRIVERS need a live stack, so parsing is what CI can honestly assert).
• python3 -m unittest discover -s tests -p '*_test.py' runs the deploy-script unit suites — release_preflight_test.py, rollback_floor_test.py, runtime_smoke_test.py, backup_test.py, caddy_reload_test.py, blank_server_smoke_test.py. They need no stack and ran in no workflow, so every acceptance record citing them was a local claim with nothing keeping it true. A skipped test fails the job: an unrunnable assertion must not read as a passing one.

validate and boot upload their evidence as 14-day artifacts (meta-validate-python-unit, meta-boot-compose-log) so a green check's console record outlives the run page.

Finding F04, half closed: the required boot lane starts the stack in production mode with transcoding disabled and no search integration, so a green required set still certifies nothing about the media path. The non-required stack-e2e lane (.github/workflows/stack-e2e.yml + tests/stack-e2e.mjs) now walks the rest of that chain against a live stack with TRANSCODING_ENABLED=true and SEARCH_SERVICE_URL wired to the compose search service: owner claim → login → channel → multi-chunk resumable upload of an ffmpeg-generated 4 s 320×240 fixture (deliberately larger than the 8 MiB chunk size, and the driver fails if it is not, so the ≥2-chunk path is real) → transcode polled to published with packaging_format=cmaf and a rendition → #EXTM3U master playlist → a variant playlist → the CMAF init segment (first ISOBMFF box ftyp) and the first media segment (first box styp, the CMAF brand box the movflags=+cmaf packager writes — a bare moof is deliberately NOT accepted) fetched anonymously, 200 with a real ISOBMFF header → the video returned by a signed /internal/v1/search against vidra-search, and by the public /api/v1/videos/search with search_total and total_is_lower_bound, the two fields core's local SQL trigram fallback cannot produce — so the read path is proven to have been served BY vidra-search, not merely to have returned the right id. It then waits for vidra_queue_depth{queue="search_outbox",state="pending"} to reach zero — requiring state="dead" to be zero as well, since a dead-lettered event also empties the queue without ever having been applied — and re-asserts the video is still indexed, because "indexed" must survive every event still queued behind it.

What it still does NOT prove — do not over-read a green run:

• No browser and no decoder. It asserts playlist text plus the first ISOBMFF box type and a size floor on each segment. That rules out an empty body or an error page; it does not make the bytes decodable. Real playback stays tests/release-acceptance.mjs's job, on a prepared lab host.
• Rate limiting is off (RATE_LIMIT_ENABLED=false), so it proves nothing about the shipped limits, and scanning is off (MALWARE_SCAN_MODE=disabled — without it every ingestion route answers 503 scanner_not_configured), so it proves nothing about scanning.
• Source builds of the three default branches, not released images or pinned digests: stack coverage, never release qualification.
• Local storage, not S3/Spaces/MinIO, and no CDN, presign or federation.
• No degraded-dependency coverage. The lane boots vidra-search to healthy before the api, because an api whose first outbox drain fails reschedules its boot events behind later ones and reconcile.end then suppresses the freshly indexed video (suppressed_reason=reconcile_orphan) — a real core↔search defect, measured on this branch, that the lane deliberately does not exercise. A green says nothing about recovery from a search outage.
• Not required for merge. It lives in its own workflow, outside .github/required-checks.txt, precisely so one transcode flake on a shared runner cannot block an unrelated PR. It runs nightly, on workflow_dispatch, and on a PR that touches its own inputs — a lane that runs on neither push nor PR cannot go red in the PR that breaks it. If it earns promotion after several weeks of green nightlies, the manifest already supports the graduation: a ?name entry means "required only IF it ran", which fits a path-filtered lane exactly. That is a separate, diff-visible decision and is deliberately not taken here.

Hard rules

1. One small PR per session. Deploy tooling failures cost real downtime — keep diffs surgical and explain the failure mode each change closes.
2. Ordering in deploy/deploy.sh is sacred: pre-deploy dump (abort on failure) → pull → migrate as discrete exit-code-gated steps → up -d --no-build → health probes. Never fold migrations into up -d.
3. Nested checkouts are a real trap (incident 2026-08-10). Mechanism corrected 2026-09-03 — the original bind-mount is gone, the rule is not. The trap WAS that the migrate service bind-mounted ./vidra-core/migrations from the nested checkout, which git pull on this repo does not advance, so a deploy could run new images against old migrations and exit 0. That is now architecturally impossible: migrations are compiled into the release binaries, and the rendered prod migrate service has no volumes at all — it runs migrate up on the same VIDRA_CORE_TAG image as the api, so image and migrations agree by construction. What survives is narrower and still load-bearing: the nested checkout is what deploy.sh reads to compute the EXPECTED migration version for its independent ledger assertion (a deliberate second opinion — reading it out of the migrator would only prove the migrator agrees with itself). So a drifted checkout no longer runs the wrong migrations, but it does invalidate the check that would catch the wrong ones. Any change touching migrations flow must keep the checkout-pinning + ledger-assertion guards intact, and must not reintroduce a migrations bind mount.
4. Compose >= 2.24 assumptions: docker-compose.prod.yml uses !reset/!override merge tags; older Compose silently ignores them and publishes Postgres/Redis on 0.0.0.0. Never remove the version check.
5. Keep the house script style: set -euo pipefail, log()/die()/step() helpers, comments that explain WHY (failure modes), POSIX-ish bash.
6. Never commit secrets: env/*.env stays untracked; only *.env.example files are committed. Never commit .env.bak files either.
7. Do not touch .github/workflows or bump pinned image digests/versions unless that is the task.

Git hygiene — finished means merged (all agents / AI tools)

These rules bind every AI tool working in this repo (Claude, Jules, Codex, …):

1. Commit early, push often. Work on a short-lived branch off main. Prefer several small, scoped commits over one session-end mega-commit, and push the branch at every green checkpoint — unpushed work does not exist.
2. A task is finished only when its work is merged to main and pushed. Once the verification gates and the PR's CI are green, merge the PR before declaring the task done. If you cannot merge (no permission, review requested, red CI), report the task as open — awaiting merge, never as finished/complete/done.
3. Delete merged branches. Immediately after a merge: delete the work branch on the remote (git push origin --delete <branch>), delete it locally (git branch -d <branch>), then git fetch --prune. Also sweep for leftovers each session: delete any local (git branch --merged origin/main) or remote (git branch -r --merged origin/main) branch already merged into origin/main. Never delete main, the branch you are on, or an unmerged branch — an unmerged stray is reported for triage, not deleted.

Enforcement: .claude/hooks/finished-means-merged.sh runs as a Claude Code Stop hook and refuses to let a session end while any repo here has uncommitted tracked changes or unpushed commits. It ignores untracked files (so env/*.env never trips it), skips detached checkouts (the pinned nested repos), and gives up after 3 attempts. It is Claude-only — Jules, Codex and every other tool are bound by the rules above regardless, with nothing mechanical to catch them.

Layout

• deploy/ — deploy.sh, rollback.sh, backup.sh, restore.sh, release.sh (release.sh cuts GitHub releases in the component repos and verifies GHCR images), Caddyfile, systemd units.
• env/ — production env template; VIDRA_*_TAG pins the deployed images.
• docs/ — platform audits, parity reports, runbooks; treat them as the source of truth for known gaps and keep them honest.