.agents

Imported from AqsaJabbar/hiring_tool (config/.agents/AGENTS.md). Install upstream with npx skills add AqsaJabbar/hiring_tool --skill .agents. Copyright stays with the author.

AI-Powered Interview Kit Generator — Agent Instructions

Project Overview

A Rails 7 API + React app that turns a plain-language role description into a polished job description, 2 interview questions (1 behavioral + 1 technical), a scorecard (4–6 criteria), and a skills assessment rubric (1–2 skills) — all generated in one OpenAI-compatible call and optionally persisted to Postgres.

Stack

Data Model

JobRequisition
  title, department, level (enum: junior/mid/senior/staff/principal)
  raw_description, polished_description
  ├── has_many :interview_questions  (content, question_type, evaluation_criteria, position)
  ├── has_one  :scorecard            → has_many :scorecard_criteria (name, description, max_score, position)
  └── has_one  :skills_assessment_rubric → has_many :rubric_skills (name, description, proficiency_level, weight)

All artifacts are owned by JobRequisition. Regeneration destroys and recreates child records inside a single ActiveRecord::Base.transaction.

Key Files

app/services/ai_interview_kit_generator.rb           # Core service — prompt, OpenAI call, validation, persistence
app/controllers/api/v1/job_requisitions_controller.rb # POST create (preview or persist), GET show
app/models/job_requisition.rb                         # Central model with associations + level enum
app/models/interview_question.rb                      # question_type enum: technical / behavioral
app/models/scorecard.rb                               # has_many :scorecard_criteria
app/models/scorecard_criterion.rb                     # max_score validation
app/models/skills_assessment_rubric.rb                # has_many :rubric_skills
app/models/rubric_skill.rb                            # proficiency_level enum, weight validation
config/routes.rb                                      # namespace :api / :v1 → job_requisitions [:create, :show]
config/initializers/openai.rb                         # AI provider selection + client config
config/initializers/cors.rb                           # CORS — FRONTEND_ORIGIN allowlist
config/initializers/inflections.rb                    # criterion → criteria irregular plural
frontend/src/pages/HiringManagerDashboard.tsx          # Main UI — form, tabs, loading, print view
frontend/src/lib/api.ts                               # Axios instance (VITE_API_BASE_URL or Vite proxy)
frontend/src/index.css                                # Tailwind directives + @media print rules
qa/scripts/run_framework.rb                           # Generate QA artifacts from scenarios
qa/scripts/evaluate_framework.rb                      # Score runs, produce CSV + release gate report
qa/scenarios/core_scenarios.json                      # Baseline test scenarios (JDIG-001, JDIG-002)

AI Generation Contract

Two Entry Points

Both return a Result struct:

• success? — boolean
• job_requisition — reloaded record (persist mode only)
• payload — preview hash (preview mode only)
• error — string message on failure

Provider & Model Details

• Prompting uses separate system and user messages; temperature 0.7
• Response format requests { type: "json_object" }
• Model fallback: tries AI_MODEL first, then each model in AI_FALLBACK_MODELS (comma-separated)
• Rate-limit retries with exponential backoff (up to 4 retries)
• Server-error retries with backoff (up to 3 retries)
• Content validation retries (up to 2 retries) — re-calls AI if output fails quality checks
• Response text extracted from choices[0].message.content (handles string, array-of-parts, and hash formats)
• Raw response cleaned by extract_json_payload (strips markdown fences, finds first {…} block)

Validation Pipeline

1. Structure validation (validate_response_structure!) — required keys exist, correct types and sizes
2. Content quality (validate_content_quality!):
   • validate_question_type_split! — exactly 1 behavioral + 1 technical
   • validate_role_level_signals! — polished description contains ≥2 level-appropriate tokens (e.g. junior → "support", "guidance", "learn", "collaboration")
3. Failures raise JSON::ParserError, triggering a retry of the full AI call

Vague Description Detection

If raw_description is blank, under 12 words, or contains vague markers ("tbd", "n/a", "various tasks", etc.), the user prompt instructs the AI to infer responsibilities from title/department/level.

Required JSON Shape from AI

{
  "polished_description": "...",
  "interview_questions": [
    { "content": "...", "question_type": "behavioral|technical", "evaluation_criteria": "..." }
  ],
  "scorecard": {
    "title": "...",
    "criteria": [{ "name": "...", "description": "...", "max_score": 5 }]
  },
  "skills_assessment_rubric": {
    "title": "...",
    "skills": [{ "name": "...", "description": "...", "proficiency_level": "novice|intermediate|advanced|expert", "weight": 1 }]
  }
}

Constraints: exactly 2 questions (1 behavioral + 1 technical), 4–6 scorecard criteria, 1–2 rubric skills.

Env Vars

Controller Behavior

Api::V1::JobRequisitionsController handles two modes:

POST /api/v1/job_requisitions

• Preview mode (skip_persistence: true): calls AiInterviewKitGenerator.preview, returns { job_requisition: <payload_hash> }. This is the default frontend flow.
• Persist mode (skip_persistence: false/absent): creates a JobRequisition record, calls .call(id), returns serialized record with nested associations. On AI failure, the requisition is destroyed.

The controller infers title from the first line of role_description if not provided, defaults department to "General", and validates level against the enum.

GET /api/v1/job_requisitions/:id

Returns the full serialized requisition with nested interview_questions, scorecard (with criteria), skills_assessment_rubric (with skills), plus pre-formatted interview_questions_text and scorecard_text strings.

Agent Personas

@architect

Owns: schema, migrations, service objects, API controllers.

• Follow "Thin Controller / Service Object" — controllers delegate to services, models own validations and associations.
• Always use migrations; never touch schema.rb directly.
• New endpoints go under namespace :api do namespace :v1.
• Return { job_requisition: ... } as the JSON root key.
• All DB writes in a single transaction; partial saves are never acceptable.
• Error responses use { error: "message" } with appropriate HTTP status.

@frontend-lead

Owns: React components, Tailwind styling, print layout.

• All UI lives in HiringManagerDashboard.tsx — no separate component/hooks directories.
• Use functional components with useState / useMemo hooks.
• The dashboard posts { role_description, seniority_level, skip_persistence: true } to /api/v1/job_requisitions.
• Vite dev server (port 5173) proxies /api to Rails (port 3000).
• Print layout uses .printable-only (shown on print) and .no-print (hidden on print). Both must have matching @media print rules in index.css.
• Tabs: job_description, interview_questions, scorecard, interview_kit.
• Minimum 20 characters required in the role text before Generate is enabled.

@compliance-agent

Owns: system prompt content, inclusive language validation.

• All AI output must use gender-neutral language. Banned terms: "aggressive", "rockstar", "ninja", "dominant", "fearless", "competitive", "digital native", "young".
• Junior roles → fundamentals-focused questions (core concepts, debugging basics, testing basics).
• Senior/Staff/Principal → system design, architecture trade-offs, scalability, mentorship.
• Level-description conflicts: prioritize the role description over the seniority label.
• Prompts are sent as separate system + user messages.
• New validation steps run after parse_response and before persist!.

@qa-engineer

Owns: QA scenario framework, evaluation rules, release gating.

• Scenarios live in qa/scenarios/core_scenarios.json.
• Run generation: bin/rails runner qa/scripts/run_framework.rb (supports --retest).
• Run evaluation: bin/rails runner qa/scripts/evaluate_framework.rb (supports --retest).
• Evaluation checks: banned terms, role-level token alignment, question type split (1 behavioral + 1 technical), scorecard criteria count (≥4), rubric skills count (≥2).
• Release gate: PASS = 0 high-severity defects + ≥90% pass rate; CONDITIONAL PASS = 0 high-severity but <90%; FAIL = any high-severity defects.
• Outputs go to qa/runs/<timestamp>/: JSON artifacts, results_scored.csv, defect_log.csv, release_gate_report.txt.

Global Rules

1. Inclusive language — never use gender-coded words in prompts or generated content.
2. Schema safety — migrations only; never hand-edit db/schema.rb.
3. Print-first — scorecard and interview kit views must render cleanly via window.print().
4. Role-level precision — tailor question depth and rubric proficiency to the seniority enum.
5. Atomic persistence — all DB writes happen inside a single transaction; partial saves are not acceptable.
6. Error handling — surface Result#error to the frontend; never swallow exceptions silently.
7. Preview is default — the frontend always sends skip_persistence: true; persisted mode is for API-only callers.
8. Question count — exactly 2 interview questions (1 behavioral + 1 technical). Tests and QA evaluator enforce this.
9. Model fallback — if the primary model fails, the service tries each model in AI_FALLBACK_MODELS before giving up.