intelFraud

Imported from nghiahuynh091/intelFraud (AGENTS.md). Install upstream with npx skills add nghiahuynh091/intelFraud. Copyright stays with the author.

AGENTS.md

This file provides guidance to Codex (Codex.ai/code) when working with code in this repository.

Commands

# Frontend
npm i              # Install dependencies
npm run dev        # Start Vite dev server (http://localhost:5173)
npm run build      # Production build — MUST pass before task is considered done

# Backend (standalone)
cd backend
pip install -r requirements.txt
uvicorn app.main:app --reload   # http://localhost:8000
python scripts/train.py         # Pre-train model to models/ directory

# Docker (full stack)
docker-compose up --build       # Frontend :5173 (http://localhost:5173)  ·  Backend :8000

Architecture

Credit Sentinel is a fraud detection dashboard with a React frontend and a Python FastAPI backend.

Data Flow

Browser
  └── Live feed (primary): GET /api/transactions/recent?seconds=120 → poll every 2s (all pages)
                            Empty state shown if backend down — no synthetic fallback
  └── Area chart (Dashboard): GET /api/transactions/hourly-distribution → 24-bucket fraud/legit
  └── Historical chart:       GET /api/transactions/historical → real 30-day daily aggregates from SQLite
  └── KPI cards:              GET /api/transactions/stats → runtime totals from SQLite day aggregates
  └── XAI Drawer:  click transaction → POST /api/predict (real scikit-learn RF + SHAP)
                   rawFeatures forwarded if present (Kaggle mode) → 30-feature model path
  └── Model Health: mount → GET /api/model/health, /history, /model/features/importance
                    retrain → POST /api/model/retrain (random seed each call, returns real metrics)

Backend
  └── Lifespan: init_db() → load_or_train() → _preload_kaggle_cache()
               → clear_transactions() → clear_transaction_aggregates()
               → clear_model_versions() → _seed_initial_history()
               → asyncio.create_task(_transaction_generator())
  └── Background task (every 2s): sample 1–3 Kaggle rows → INSERT raw transaction
                                  → UPSERT hourly/day aggregate buckets
  └── DB prune: 1% chance per tick to DELETE raw transactions older than 2 hours

SQLite Persistence Layer

backend/app/db/database.py — stdlib sqlite3, no new pip deps.

• DB path: data/fraud.db (relative to backend/ working dir; mounted via Docker volume)
• transactions table: hot raw feed kept for 2 hours — id, amount, location, merchant_category, timestamp, card_last4, frequency, probability, confidence, is_fraud, raw_features (JSON), created_at (epoch float)
• transaction_hourly_aggregates table: bucket_start, total_count, fraud_count, prevention_value
• transaction_daily_aggregates table: bucket_date, total_count, fraud_count, prevention_value
• model_versions table: id, version, f1, precision, recall, auc, confusion_matrix (JSON), n_samples, feature_importances (JSON), trained_at
• Key functions: init_db(), save_transactions(), get_recent_transactions(seconds), get_transaction_stats(), get_historical_days(days), save_model_version(), get_model_versions(), count_model_versions(), clear_transactions(), clear_transaction_aggregates(), clear_model_versions(), cleanup_old_transactions()

On every startup, clear_transactions() wipes hot raw rows, clear_transaction_aggregates() wipes runtime hour/day rollups, and clear_model_versions() wipes the model_versions table. _seed_initial_history() then inserts 4 fake earlier model versions (v1.1-v1.4) plus the current loaded model as v1.5. The first user retrain after startup becomes v2.0.

Dual-Mode ML Pipeline

The backend auto-detects training mode at startup and retrain:

rf_engine._n_features (8 or 30) drives which feature extraction path is used in predict(). TransactionInput.rawFeatures carries the real V1–V28 values from stream → XAI drawer.

Key Frontend Modules

• src/app/services/predictionService.ts — Client-side RF simulator (deterministic, seeded LCG). Used only as fallback by XAIDrawer when backend is unreachable.

• src/app/services/apiService.ts — HTTP client for the Python backend. Exports: predictTransactionAPI, retrainModel, getModelHealth, getModelHistory, getFeatureImportance, getRecentTransactions, getTransactionStats, getHourlyDistribution, getHistoricalData, getModelHistoryFull, getServiceHealth. Base URL from VITE_API_URL env var (default http://localhost:8000).

• src/app/components/fraud/mockData.ts — Static fallback data only. generateTransaction() and initialTransactions have been removed. Exports: Transaction interface (with createdAt?: number), historicalData (fallback), areaChartData (fallback), featureImportanceData (fallback).

• src/app/components/fraud/XAIDrawer.tsx — Slide-in XAI panel. Calls predictTransactionAPI on open (forwarding rawFeatures if present); shows loading skeleton while awaiting SHAP results. Falls back to client-side prediction if backend unreachable.

• src/app/routes.tsx — React Router v7 route definitions.

• src/app/components/fraud/Layout.tsx — Sidebar nav + header shell using React Router <Outlet>.

Key Backend Modules

• backend/app/db/database.py — SQLite persistence. init_db() creates tables on startup. save_transactions() writes raw feed rows and updates hour/day aggregates. get_recent_transactions() reads the 2-hour hot table. get_transaction_stats() and get_historical_days() read aggregates. save_model_version() / get_model_versions() manage model history.

• backend/app/core/rf_engine.py — Module-level rf_engine singleton. Owns trained model, scaler, metadata, _n_features (8 or 30), threading.RLock. load_or_train() on startup; predict() for dual-mode inference; retrain() CPU-bound (random seed via int(time.time())). After retrain, saves new version to DB via db.save_model_version().

• backend/app/services/feature_engineering.py — FEATURE_NAMES (8 custom), KAGGLE_FEATURE_NAMES (30 Kaggle). extract_ml_features() for 8-feature path; extract_kaggle_features() for 30-feature path. generate_pca_features() for V1-V28 display values (djb2+LCG, display only).

• backend/app/services/xai.py — SHAP TreeExplainer integration. FEATURE_META (8-feature) and KAGGLE_FEATURE_META (30-feature) dicts. compute_contributions(n_features=...) selects the right mapping.

• backend/app/services/training.py — Auto-detects Kaggle CSV via settings.resolved_kaggle_csv. Kaggle mode: real 284k dataset, class_weight="balanced". Custom mode: 100k synthetic samples. save_artifacts() / load_artifacts() via joblib.

• backend/app/api/health.py — Module-level _kaggle_df cache; get_kaggle_df() accessor used by background task in main.py. Serves /recent, /stats (from day aggregates), /stream (legacy), /hourly-distribution, /historical (from day aggregates), and /health (bootId included for client session sync).

Pages

Backend API Endpoints

Settings

Settings persist via localStorage (intelfraud_settings key) within the current backend runtime. loadSettings() reads on init; handleSave() writes JSON.stringify(s) on click. Clearing browser storage or using a fresh browser resets to DEFAULTS in SettingsPage.tsx. If /health.bootId changes after a backend restart, the frontend clears settings, notifications, and runtime model history cache before reload.

Live Feed — Shared Dataset Pattern

All three live pages (Dashboard, LiveMonitoring, HistoricalAudit) poll /api/transactions/recent from the same SQLite hot table:

• Dashboard / LiveMonitoring: seconds=120, poll every 2s
• HistoricalAudit: seconds=7200, poll every 5s

New transaction IDs (tracked via seenIdsRef = useRef<Set<string>>) get a 700ms scanning animation, then a 1200ms highlight fade.

Age-based row opacity: Math.max(0.15, 1 - (Date.now()/1000 - createdAt) / 120) — rows fade to 15% opacity as they approach 2 minutes old, then disappear from the freshest live query window even though the hot table retains up to 2 hours.

Display cap: txns.slice(0, 100) — both Dashboard and LiveMonitoring render at most 100 rows (backend returns up to 200).

XAIDrawer

Opens for all transactions (High Risk and Verified). Every row in every transaction table must have cursor: pointer and onClick={() => setSelected(txn)}. Do not filter by isFraud.

Recharts cursor pattern

const BarCursor = ({ x = 0, y = 0, width = 0, height = 0 }: { x?: number; y?: number; width?: number; height?: number }) => (
  <rect x={x - 1} y={y} width={width + 1} height={height} fill="rgba(255,255,255,0.06)" rx={4} />
);
// Used in Tooltip: cursor={<BarCursor />}

Combined with itemStyle={{ color: '#fff' }} and labelStyle={{ color: 'rgba(255,255,255,0.5)' }} on all bar chart <Tooltip> elements.

Styling

• Tailwind CSS v4 via @tailwindcss/vite plugin (no tailwind.config.* needed)
• Custom CSS variables in src/styles/theme.css (dark theme, glassmorphism)
• Path alias @ → ./src (configured in vite.config.ts)
• Radix UI + shadcn/ui in src/app/components/ui/

Guardrails (read before touching anything)

• Không dùng Math.random() cho classification — live stream lấy từ /api/transactions/recent (real Kaggle rows). XAI drawer dùng predictTransactionAPI(). Không bao giờ dùng Math.random() để quyết định fraud.
• Không dùng generateTransaction() hay initialTransactions — đã xóa. Nếu backend down, hiển thị empty state, không fallback sinh dữ liệu giả.
• Không hardcode kết quả fraud — forceFraud=true chỉ bias đầu vào, không override output.
• npm run build phải pass trước khi coi task là xong — không có test script cho frontend.
• Backend: pytest tests/ -v phải pass — 43 tests, chạy trong ~15s.
• Tests luôn dùng custom 8-feature mode — conftest.py patches settings.resolved_kaggle_csv trả về None cho toàn bộ session (cả retrain). Không thêm creditcard.csv vào test fixtures.
• KAGGLE_FEATURE_NAMES order trong training.py và feature_engineering.py phải match chính xác — thứ tự V1–V28 + Amount + Time là load order từ CSV. Sai thứ tự → model corrupt.
• rf_engine._n_features drives mode selection — đừng set manually. Được set từ metrics["feature_names"] sau training.
• CSV path là relative to backend/ working dir khi chạy uvicorn — data/creditcard.csv relative to backend/. Fallback D:\creditcard.csv chỉ cho dev local.
• data/*.csv và data/*.db không commit — đã có trong backend/.gitignore.
• DB path là relative to backend/ working dir — data/fraud.db. Docker volume ./backend/data:/backend/data persists cả CSV lẫn DB.
• Inline styles cho business logic, Tailwind chỉ cho layout — xem pattern const glass trong bất kỳ page component nào.
• Không tạo file mới nếu có thể edit file hiện có.
• Single uvicorn worker — rf_engine singleton + threading.RLock là process-local. Không scale multi-worker mà không có external model store.
• cors_origins trong config.py là str (comma-separated), không phải list[str] — pydantic-settings v2 JSON-decodes list fields từ env, gây crash. Dùng settings.cors_origins_list property.
• Settings dùng localStorage (intelfraud_settings) — loadSettings() đọc khi init, handleSave() ghi khi click Save. Không reset khi docker-compose up trừ khi clear browser storage.
• Background task (_transaction_generator) chạy trong asyncio event loop — không block với CPU-bound ops. Retrain (CPU-bound) chạy trong run_in_executor.

Về file này

• Đây là tài liệu sống — cập nhật khi có thay đổi kiến trúc, pattern mới, hoặc guardrail phát sinh từ bug thực tế.
• Commit AGENTS.md cùng với code thay đổi — không để file này drift khỏi codebase.
• Không dump style guide hay API docs vào đây — chỉ giữ những gì Codex cần để không phạm sai lầm lần đầu.