dance-step

Imported from EnriqueZepedaH/dance-step (AGENTS.md). Install upstream with npx skills add EnriqueZepedaH/dance-step. Copyright stays with the author.

AGENTS.md

Repository Layout

This repository is a monorepo with three workspaces:

• apps/web/ — the Next.js frontend (the consumer-facing app + admin).
• apps/ingest-worker/ — standalone Node service deployed to Railway. Runs on a cron schedule, pulls events from public sources, gates them through quality checks, and publishes to Supabase. Repo-root Dockerfile + railway.json own the deploy.
• packages/db/ — shared @dancestep/db workspace exposing the generated Supabase Database type. Both apps consume it.

When file paths appear bare (app/..., lib/..., components/..., proxy.ts, next.config.mjs, .env.local, supabase/...), read them as relative to apps/web/, not the repo root. Root-level files are: AGENTS.md, PROJECT_PROPOSAL.md, .gitignore, Dockerfile, railway.json, root package.json + package-lock.json, scripts/gen-types.sh.

Run npm commands from the repo root with workspace flags (npm run -w web build, npm run -w ingest-worker test, etc.). Per-app .env.local files live at apps/web/.env.local and apps/ingest-worker/.env.local.

Repo Structure — Deferred Until Triggered

The following monorepo conventions are intentionally not adopted in v1. Each one is real engineering value once a second consumer exists, but for a single-app repo they pay configuration tax (build/types/lint/paths) for benefits that don't activate yet. Pull each one forward only when its trigger fires; do not adopt preemptively.

What we did adopt in v1 (Tier 1 — done or pending): root npm workspaces with delegated scripts, zod-validated env at module load (apps/web/lib/env.ts), db:types script wrapping the typegen call, apps/web/.env.example, GitHub Actions running typecheck + lint + build on PR.

Rule of thumb: if you find yourself proposing one of the deferred items, point at the trigger row first. If the trigger hasn't fired, don't adopt — write a one-paragraph note in the relevant phase doc explaining what would have triggered it, and move on.

Project Context

DanceStep is a Next.js app for Latin dancers. It recently pivoted from a single-purpose Cuban Casino move analyzer into a broader v1 platform:

• Library: YouTube-powered video search, bookmarks, and playlists.
• Scene: Mapbox-backed Chicago event explorer.
• Lab: Deferred research preview for the original analyzer. In v1, /upload becomes a waitlist page. Do not build pose tracking, video uploads, FastAPI, MediaPipe, or Claude Vision in v1.

The current app is mostly a polished static landing page plus a placeholder upload page. The v1 goal is an account-bearing deployable product with Library + Scene functional, and Lab honestly framed as "coming in v2."

Important proposal alignment: the original PROJECT_PROPOSAL.md Week 5 goal still mentions an end-to-end analyzer POC. Phase 6 must update the proposal so Week 5 reflects "Library + Scene live, Lab waitlist published" and explains the analyzer deferral.

Current Baseline

• Framework: Next.js App Router.
• Current installed versions include Next 16 and React 19. Use proxy.ts, not middleware.ts.
• Styling is custom CSS in app/globals.css. Do not migrate to Tailwind unless explicitly requested.
• Existing files (under apps/web/ unless noted):
  • app/page.tsx
  • app/upload/page.tsx
  • app/layout.tsx
  • app/globals.css
  • package.json
  • next.config.mjs
  • tsconfig.json
  • PROJECT_PROPOSAL.md (repo root)
  • AGENTS.md (repo root)
• .gitignore should ignore node_modules/, .next/, out/, env files, Vercel metadata, logs, caches, and local assistant metadata.

Locked Decisions

• Auth: Clerk.
• Database/storage: Supabase Postgres + RLS.
• Supabase auth integration: Clerk native Supabase third-party auth. No custom JWT template.
• Map: Mapbox GL JS via react-map-gl.
• Events: manual Chicago seed data plus admin-only /admin/events.
• Language: migrate current JSX files to TypeScript first; new code should be .ts/.tsx.
• Hosting: Vercel frontend only for v1.
• Edge auth file: proxy.ts, because this project is on Next 16.

Implementation Rules

• Make focused commits by phase. Do not mix TypeScript migration, auth setup, schema, and feature work in one change.
• Preserve the landing page visual output when extracting shared components.
• Keep v1 scope disciplined. The ingest-worker MAY scrape public event sources (e.g. WordPress event pages) when no structured surface exists, per the Scene ingestion plan. Supabase Storage was adopted 2026-05-19 — the admin flyer-extraction flow is the first consumer (private event-flyers bucket, signed URLs only; see the Flyer Extraction Runbook below). Otherwise do not add scrapers, comments, profiles, shared playlists, PWA/mobile, video upload, or analyzer backend work.
• Do not put DB queries in proxy.ts. Use it only for Clerk route gating.
• Use explicit server-side admin checks in app/admin/layout.tsx and admin API handlers.
• Keep service-role Supabase usage isolated in lib/supabase/admin.ts and narrow helpers only. Never import service-role code into client components.
• The normal browser/server Supabase clients must be RLS-bound.
• Use UTC timestamps in the database; render event times for America/Chicago.
• Restrict the public Mapbox token in Mapbox account settings to local/dev and Vercel domains.
• YouTube search must be cached and quota-aware. Friendly quota limit UI, never a crash.

Scene Ingestion Pipeline

The Scene's events come from a Railway-deployed Node worker (apps/ingest-worker) that runs on cron. Daily flow:

sources (gcal iCal + lsd HTML scrape)
   │
   ▼
ingest_runs (audit log)        raw_events (immutable per-run extract)
   │                              │
   │  clean (normalize, geocode cache) ─► venue_geocode_cache
   │  quality (validation gates)  ─► quality_rejections
   │  dedup (cross-source match)
   ▼
events (status='published')    ← public /scene reads via RLS

Schedule: 0 12,0 * * * UTC (~06:00 + 18:00 Chicago). Cadence + window are tuned so weekly recurring events show a full horizon. Soft-delete: an event missing from 3 consecutive successful runs of its source flips to status='archived'. Failed runs do not archive — archiveMissing is source-scoped and only runs on success.

Operator runbook

• Trigger a one-off run. Railway dashboard → ingest-worker service → "Run now". No web endpoint trigger exists (deferred).
• Inspect a run. /admin/scene/runs (paginated history) → click a row for the per-raw_event outcome breakdown.
• Promote a rejected event. /admin/scene/rejections → "Promote anyway". Requires the venue to already exist in the venues table (geocoding can't run from the Next side because the Mapbox token is URL-restricted); if it doesn't, create the venue via /admin/events/new first, then promote.
• Resolve a cross-source duplicate. /admin/scene/duplicates → Merge (keep existing) or Replace (archive existing, promote candidate).
• Re-enable a paused source. /admin/scene/sources → toggle.
• Restore an archived event. Direct SQL: update events set status='published', missing_run_count=0 where id=…; (no admin UI knob yet).
• Scraper broke. Run lands partial or failed. Inspect /admin/scene/runs/[id] for the error. The HTML scraper's selectors are concentrated in one SELECTORS constant in apps/ingest-worker/src/ingest/sources/lsd.ts — fix and redeploy.
• Mapbox token note. The worker uses an unrestricted token (MAPBOX_TOKEN env), separate from the browser-side URL-restricted token (NEXT_PUBLIC_MAPBOX_TOKEN). Server-side calls don't send an Origin header, so URL-restricted tokens 403.

Canonical health query

select * from ingest_runs order by started_at desc limit 10;

If you see failed rows or zero promoted_count from a success row, something's off.

Flyer Extraction Runbook

Admin-only flyer-to-event accelerator on /admin/events/new. An admin uploads a Latin dance flyer image; Claude Vision extracts event fields; the admin reviews/edits the prefilled form and submits through the existing /api/admin/events POST. Every extraction is audited in flyer_extractions with full observability (latency, tokens, raw response, edited fields, finalized payload).

Architecture sketch

/admin/events/new
  FlyerDropzone ─► EventForm (prefilled, editable)
       │                       │
       │ POST /api/admin/      │ POST /api/admin/events
       │   flyers/extract      │ { ...event, flyerExtractionId,
       │                       │   fieldsEdited[], city, country, timezone }
       ▼                       ▼
  upload → vision call    lookup flyer_extractions (admin client),
  audit row updated        insert event w/ flyer_extraction_id,
  status: processing →     finalize fields_edited + finalized_payload
   completed | failed

• Storage bucket: event-flyers (private, no select policy, service-role only).
• Audit row inserts BEFORE bytes hit Storage so crashes never leave orphan flyers without a trail.
• flyer_storage_path on events is loaded from the DB on the server side — clients cannot tamper with which image gets stamped.
• Service-role isolation: getSupabaseAdminClient() is the only path to flyer_extractions and Storage; event/venue writes stay on the RLS-bound createSupabaseServerClient().

Operator runbook

• Re-extract a flyer. Upload via the dropzone, then click "Re-extract" to re-POST the same file. Both extractions live in flyer_extractions; finalization links the successful one to the event.
• Audit failed extractions.

  select id, error, latency_ms, created_at
  from flyer_extractions
  where status='failed'
  order by created_at desc;

• Audit slow extractions.

  select id, model, latency_ms
  from flyer_extractions
  where status='completed'
  order by latency_ms desc limit 20;

• Token cost survey.

  select date(created_at), sum(total_tokens)
  from flyer_extractions
  group by 1 order by 1 desc;

• Edit-rate by field (training signal for future prompt tuning).

  select field, count(*) as edit_count
  from flyer_extractions, unnest(fields_edited) as field
  group by field order by edit_count desc;

• Adjust daily cap. Set FLYER_EXTRACT_DAILY_CAP in Vercel (default 100). Cap is enforced UTC-day, globally — not per admin.
• Inspect a flyer manually. Supabase Studio → Storage → event-flyers → drill by storage_path from the row.
• Configuration check. Route returns 503 when ANTHROPIC_API_KEY is absent; dev builds without the key still compile and serve everything else.

Multi-city note. The Scene page filters by events.city against an allowlist in apps/web/lib/scene/cities.ts. Adding a city = append to the list; no schema change. Calendar/list date keys are still Chicago-anchored — non-Chicago events near local midnight may bucket one day off until that helper is refactored.

End-to-End Verification

Before calling v1 done:

1. Anon can browse / and /scene; /library redirects to sign-in.
2. Signup works and creates a Supabase users row.
3. Library search/save/playlist flow works.
4. Scene map pins and event details work.
5. Admin event creation works for admin and is forbidden for non-admin.
6. Lab waitlist insert works.
7. YouTube quota failure shows a friendly message.
8. RLS audit passes for bookmarks, playlists, playlist items, events, venues, and lab waitlist.