Imported from bedkillerspacex-boop/codex-skill-library (javascript-backfill/SKILL.md). Install upstream with npx skills add bedkillerspacex-boop/codex-skill-library --skill javascript-backfill. Copyright stays with the author.

JavaScript Safe Backfill Procedures

Research grounding

Operational patterns for large data backfills from Node/TS jobs: correctness, low blast radius, and resumability. Skill id: javascript-backfill.

Scope And Authorization

• In scope: owned databases and streams, authorized maintenance.
• Out of scope: mass data changes on systems without change control; exfil via “backfill.”
• Prefer staging rehearsal with production-sized samples.
• Redact PII in backfill logs.
• Pair with database-migration-safety, javascript-backpressure, code-quality-standards.

When To Use

• New column needs computed values for historical rows.
• Encrypt-in-place / rehash passwords migration.
• Search index rebuild / denormalized field repair.
• Event replay to projection stores.
• Multi-tenant migrations with uneven sizes.

Do Not Use As Primary

Domain Focus — safety properties

// Keyset chunk sketch
async function backfillChunk({ afterId, limit }) {
  const rows = await db.query(
    `SELECT id, payload FROM items WHERE id > $1 ORDER BY id ASC LIMIT $2`,
    [afterId, limit],
  );
  for (const row of rows) {
    await processIdempotent(row); // upsert by id
  }
  return rows.at(-1)?.id ?? null;
}

Workflow

1. Confirm scope and success criteria

1. Record table/stream size, SLA, maintenance window, owner, rollback plan.
2. Success: 100% eligible rows correct; error rate < threshold; primary p99 within budget; resume after kill works.
3. Data classification and backups current (javascript-backup-verify).

2. Inventory

rg -n "backfill|migrate:data|// TODO fill" --glob "*.{js,ts}"
# Row counts, indexes, hot paths using table

1. Dependencies: triggers, replicas, CDC consumers.
2. Lock risks (avoid table locks).

3. Baseline

1. Snapshot metrics: CPU, IOPS, replication lag.
2. Dry-run on copy: time per 10k rows.
3. Define correctness query (checksum counts, nulls remaining).

4. Implement job

1. Expand/contract: ship code reading both old/new; backfill; switch; cleanup.
2. Idempotent writes (UPSERT).
3. Chunk size adaptive to lag.
4. Dual-run validation sampling.
5. Kill switch env/flag checked each chunk.
6. Don’t run unconstrained UPDATE without WHERE chunking on huge tables.
7. Separate DB user with limited rights when possible.

5. Verify

npm ci && npm test
node jobs/backfill.js --dry-run --limit 1000
node jobs/backfill.js --resume-from-checkpoint
# Correctness SQL: count where new_col is null and eligible

1. Staging full rehearsal.
2. Prod: start low rate; watch lag; ramp.
3. Pause on anomaly.

6. Hand off

1. Record final counts, duration, incidents.
2. Remove temporary dual-read after soak.
3. Document for next backfill template.

Good / Bad

Output Checklist

• Change ticket and rollback
• Staging rehearsal done
• Idempotent chunked job
• Kill switch tested
• Progress metrics live
• Correctness queries pass
• Primary/replica health OK during run
• Checkpoint/resume works
• Dual-write cleanup planned
• code-quality-standards applied

Rules

• Owned data only; backups before irreversible transforms.
• Prefer expand/contract over big-bang rewrites.
• Keep focused on safe backfills for JS services.
• Never paste live secrets into skill notes, commits, or chat.