ZRB-groundwater-model-public

Imported from hydrosolutions/ZRB-groundwater-model-public (AGENTS.md). Install upstream with npx skills add hydrosolutions/ZRB-groundwater-model-public. Copyright stays with the author.

AGENTS.md — Project Conventions (single repository-instruction authority)

AGENTS.md is the single repository-instruction authority for every agent runtime. CLAUDE.md must remain a relative symlink to this file.

Working Discipline

Do not assert — validate and check. Deliberate carefully, do not assume anything. Before claiming that a file contains certain data, that a column holds a particular meaning, that a function does something specific, or that a path exists — open it and verify. Read the schema, run a quick count, grep for the actual usage. When stating numbers, derive them in front of the user from the live data; do not recall figures from earlier turns without re-checking against the current code/data. If something is unverified, say so explicitly. The cost of one extra tool call is small; the cost of acting on a wrong assertion is large and erodes trust.

Project Overview

gwswi — Regional groundwater flow model of the Zarafshan River valley (Uzbekistan) using MODFLOW 6 and FloPy. Simulates seasonal GW–SW exchange dynamics over 2010–2024 (180 monthly stress periods, 1 km grid).

Quick Commands

uv sync                    # Install all dependencies (creates .venv)
sh hooks/install.sh        # One-time: wire tracked git hooks into this clone
uv run pytest tests/       # Run all tests
uv run pytest tests/ -v    # Run tests with verbose output
uv run jupyter lab         # Launch JupyterLab (open notebooks/)
uv run ruff check src/     # Lint source code
uv run python scripts/verify_workflow_spec.py   # Check the pinned lean-workflow snapshot
uv run python scripts/ss_run_gate.py <dir>      # Steady-state deterministic gate (spec §6)
uv run marimo run apps/modflow_viewer.py  # Read-only regional MODFLOW viewer (src/gwswi/viewer/)
uv run python run_notebooks.py         # Execute notebooks → .cache/notebooks-executed/
uv run python export_presentations.py  # Cache → bilingual PPTX in presentations/
bash scripts/sync_public_mirror.sh     # Rebuild public workshop mirror from current HEAD

Workflow — Lean Environmental Modeling Orchestration

Work under the Lean Environmental Modeling Orchestration workflow. It is pinned by commit and content hash, never followed from a mutable path:

A Codex runtime may use the adapter skill at that same commit, adapters/codex/orchestrate-environmental-modeling/SKILL.md, SHA-256 587c1610659c68720f615dd7f38032e74dcaee786c617fcaafeaaeebe341d2ca. A Claude runtime follows adapters/claude/README.md at that commit. Moving to a later lean version is an owner decision recorded in GAPS.md §5.

The exact upstream files are retained under vendor/lean-environmental-modeling-orchestration/; paths above are relative to that directory. config/workflow-spec-lock.json records every retained file; uv run python scripts/verify_workflow_spec.py checks the snapshot and this pin offline. Cite the pinned specification; do not restate it here.

Repository-specific bindings:

• Governed record. Owner rulings and iteration entries live once, in GAPS.md §5 (steady-state) or the transient section that succeeds it. The documents under docs/steady_state_*.md are working artifacts and ledgers; they carry pointers to GAPS.md, never restatements of a ruling.
• Assurance mode. E2 scientific for model iterations: one independent critique (Opus, per the Claude adapter) after deterministic gates pass, on a landed iteration. E0 for viewer, notebook styling and workshop material.
• Run contract. Each GAPS.md §5 iteration entry opens with the compact run contract from spec §2 (objective, mode, authoritative inputs with identity, invariants, per-stage predicates, validity precondition, owner-only stops), written before the first run.
• Deterministic gate. uv run python scripts/ss_run_gate.py <scenario_dir> is the steady-state gate: convergence, budget closure, routed-SFR identity, Khazara outflow band, western boundary flux band, cleaned-target error, and flooded-cell fraction, with bands in config/ss_gate_bands.json. It exits non-zero on any failure; --self-test injects known violations and must fail. Bands marked provisional are owner-ratifiable thresholds, not accepted science.
• Calibration readiness. No parameter estimation (pilot points, pestpp-ies) starts on a configuration whose gate fails or whose readiness review is open; see spec §1 "System completeness before calibration".
• Status file. Overwrite the untracked .run-status at each stage boundary during multi-stage work.

Branches and worktrees

• Keep the primary checkout on main. Put every additional worktree under .worktrees/ and remove it when its branch lands.
• One active implementation branch per iteration, named for the GAPS.md entry it implements (iter/<n>-<topic>). Open its pull request when the branch is created, not when the work is finished.
• Land the iteration even when the hypothesis failed; the recorded negative result is the deliverable. Delete the branch locally and on origin on merge. When work outgrows the entry, land what is accepted and open a new entry; do not extend the branch.
• A fresh worktree has no gitignored inputs (sensitive data folders, generated data/model artefacts). Run uv run python scripts/link_worktree_inputs.py once after creating it; the links are read-only inputs, never write targets. Run the full test suite with an unmasked exit status before the first commit.
• Scenario outputs (scratch/ss_rapid_*/) are per-worktree and gitignored. Regenerate the baseline scenarios in a new worktree rather than copying them.
• Nothing in NB08a or NB12a changes during an iteration; promotion into the durable notebooks is its own entry and re-triggers NB15a/NB16a.

Public Mirror

This is the private development repo. Workshop participants clone the public mirror at hydrosolutions/ZRB-groundwater-model-public, which carries a single squashed commit per sync, no LFS, and a workshop-edition banner injected into README. Sensitive folders (data/momigeo, data/observations, data/mow, data/uzhydromet) and rendered presentations/*.pptx are excluded by .gitignore and never reach the public working tree.

After every change to main that should reach participants, run:

bash scripts/sync_public_mirror.sh

The script refuses to run on a dirty tree or non-main branch and force-pushes the new squashed commit to the public repo. Workshop participants who already cloned should git pull --rebase (or just re-clone) to receive updates.

Notebook/Presentation Workflow

Committed notebooks/*.ipynb are stripped of outputs by the pre-commit hook (see hooks/README.md). Executed copies live in the gitignored .cache/notebooks-executed/ and are consumed by export_presentations.py. PPTX files in presentations/ are gitignored derived artefacts — regenerate locally, distribute out-of-band.

The combined May 2026 workshop deck is built from materials/2026_may_workshop/workshop_slide_manifest.yaml via scripts/build_workshop_deck.py (authored title/agenda slides + selected per-notebook figures copied from the auto-exported decks).

Architecture

src/gwswi/
├── domains.py          # Four spatial domains: SWB, RS, ZNP, regional grid
├── georef.py           # GridGeometry dataclass, coordinate transforms, DEM utilities
├── mapviz.py           # Basemap overlays, cell colouring, profile plots
├── climatology.py      # CHELSA precipitation/temperature climatologies
├── surface_water.py    # MoW discharge analysis, OSM waterways, station maps
├── crossvalidation.py  # MoW vs RS ET cross-validation
├── river.py            # River delineation, thalweg processing, SFR data
├── irrigation.py       # Irrigation fraction computation from RS maps
├── well_data.py        # Well registry loading, transmissivity, spatial mapping
├── groundwater_use.py  # Population-based pumping estimation (domestic + industrial)
├── recharge_et.py      # RCHA, EVT, WEL array construction + reservoir mask/EVT
├── aquifer_properties.py # Geology-zoned K, Sy, Ss estimation (Approach 2)
├── observations.py     # GW observation QA/QC, calibration targets, hydrogeology
├── model_setup.py      # Model grid assembly, CHD/GHB boundaries, validation, export
├── model_v2.py         # Build SS & transient MODFLOW 6 simulations, result readers, residuals
├── steady_state_scenarios.py # Rapid SS scenario runner + water-balance/residual diagnostics (NB12d)
├── viewer/             # Read-only marimo viewer backend: catalog, readers, plots, water balance
├── sensitivity.py      # LHS sensitivity analysis, parallel model runner, plots
├── calibration.py      # pestpp-ies calibration helpers (NB14a)
├── visualization_v2.py # Publication-quality plots for v2 model results
├── visualization_budget.py # Water budget visualisation helpers
├── visualization_cal.py # Calibration result visualisation
├── export_pptx.py      # Notebook → PowerPoint presentation export
├── workshop_data.py    # Fetch + unpack workshop data archive from .env URL (NB00 §0)
├── workshop_presentations.py # Fetch + unpack rendered presentations archive from .env URL (NB00 §0)
├── figures.py          # Figure export utilities
├── model.py            # `run_simulation`, `ensure_mf6_binary` — shared MF6 runner (used by NB12a/12b/12c/15a/16a)
├── mf6_schematic.py    # MF6 package-dependency diagrams for NB §0.5 context cells
├── parameters.py       # [legacy] ModelParameters dataclass — ZNP toy model, tests only
├── boundary.py         # [legacy] RIV/CHD builders for ZNP toy model — tests only
├── visualization.py    # [legacy] matplotlib + Plotly plotting for ZNP toy model — tests only
└── _style.py           # Shared matplotlib rcParams

Spatial Domains

Four spatial domains are defined in NB01 and persisted to data/domains/study_domains.gpkg:

• SWB (Surface Water Balance): Full upstream catchment at Khazara, clipped to Uzbekistan
• RS (Remote Sensing): 16 administrative districts (Samarkand, Navoi, Jizzakh)
• ZNP (Zarafshan National Park): Rotated 65 × 40.5 km grid (130 × 81 cells, 500 m, EPSG:32642) — archived local model domain
• Regional grid: Rotated ~230 km × 70 km domain (72 × 223 cells, 1 km resolution, EPSG:32642); covers swb_no_north from Ravathoji to Khazara — the active model domain

Key Patterns

• GridGeometry dataclass: Immutable grid definition with from_json() constructor. Pass to every georeferencing function.
• FloPy MODFLOW 6 API: Uses flopy.mf6.* classes (not MODFLOW-2005). Newton-Raphson solver for unconfined aquifer robustness.
• Budget text discovery: River budget reading uses cbb.get_unique_record_names() to robustly find the RIV text identifier.
• Visualisation split: Static analysis plots use matplotlib; interactive/animated plots use Plotly.
• Basemap convention: All maps use EPSG:3857 for basemap overlay. Use cell_corners_3857(grid) and utm_to_3857() for coordinate transforms.
• Reservoir GHB (Option B): 75 HydroLAKES cells are represented as GHB boundaries at DEM elevation with calibratable lakebed conductance (ghb_cond_res). Built via build_reservoir_ghb_spd() in model_v2.py. Toggle USE_RESERVOIR_GHB in NB12a. EVT arrays in recharge_et.py retained for bookkeeping.
• Transient model: build_transient_simulation() in model_v2.py extends the SS builder with STO (zone-based Sy/Ss), multi-period TDIS, and per-SP forcing dicts (RCHA, EVT, WEL, SFR). Uses calibrated SS heads as IC. Companion functions: build_transient_obs_data() (date→SP mapping), compute_transient_residuals() (time-indexed comparison), read_water_budget_timeseries() (per-SP budget).

Notebook Conventions

• Thin notebooks: Notebooks must not contain reusable logic. Every non-trivial function belongs in src/gwswi/. Notebooks contain only: section markdown, short wiring calls to src/ functions, and output display.
• Active pipeline (XXa-* in notebooks/): Regional scale (Ravathoji → Khazara, 1 km grid).
• Archived prototypes (XXb-* in notebooks/archive/): ZNP scale, superseded, not maintained.
• Section numbering: §0 = imports/setup, §1+ = analysis, final § = export.

Code Style

• Linter: ruff (line-length 100)
• Layout: src-layout (src/gwswi/)
• Docstrings: NumPy-style with Parameters/Returns/Notes sections
• Types: Use Python 3.10+ union syntax (float | None)

Testing

• Tests in tests/ cover: parameters, boundary conditions, model integration, georeferencing, domains, river, and visualisation
• Integration tests use a small 5×5 grid (500×500 m) for speed
• MODFLOW 6 binary is auto-downloaded by ensure_mf6_binary() if not found

Dependencies

Core: flopy, numpy, pandas, geopandas, matplotlib, plotly, scipy, rasterio, ipywidgets, jupyterlab Dev: pytest, pytest-cov, ruff

Calibration Workflow

Steady-state calibration (NB13a → NB14a)

Iterative calibration is tracked in GAPS.md §5 — Steady-State Calibration Log.

1. Pick the top item from the Backlog in GAPS.md §5
2. Run sensitivity analysis via src/gwswi/sensitivity.py and NB13a (LHS)
3. Implement calibration changes in NB14a (pestpp-ies)
4. Add an Iteration N entry to GAPS.md §5 (setup, result, diagnosis)
5. Reorder the Backlog based on what was learned
6. Commit and push

Transient calibration (NB15a → NB16a → next)

NB15a builds and runs the 180-SP transient model (2010–2024) using SS heads as initial conditions. NB16a executes the LHS Sy sensitivity analysis (500 runs across five zone-based Sy values, seasonal head amplitude-MAE as the objective; best run amplitude-MAE 0.23 m, −40 % vs baseline; see GAPS.md §1 for the 2026-04-18 NB15a + NB16a re-run cascade). Next steps:

1. Joint SS + transient calibration (pestpp-ies with both target sets)
2. Independent validation — compare simulated SFR outflow vs observed Khazara discharge

Each iteration entry captures what was tried, what happened, and why — so no knowledge is lost between sessions.

Key References

• GAPS.md: Open issues, data gaps, modelling decisions, and calibration log
• notebooks/README.md: Pipeline overview with per-notebook descriptions
• data/README.md: Data directory structure with per-folder READMEs