africas2s

Imported from ACMAD-Niamey/africas2s (skills/africas2s/SKILL.md). Install upstream with npx skills add ACMAD-Niamey/africas2s --skill africas2s. Copyright stays with the author (MIT).

AfricaS2S — seasonal forecast downscaling, calibration, and verification

AfricaS2S turns coarse GCM seasonal hindcasts/forecasts plus fine-resolution observations into calibrated high-resolution forecasts (continuous fields or below/normal/above tercile probabilities) and scores them with cross-validated skill metrics. It operates purely on xarray and is source-agnostic — it pairs naturally with rosetta for data acquisition but does not require it.

• Install: pip install africas2s — import name is africas2s (conventionally import africas2s as ds).
• Methods, metrics, ensemble strategies, and CV schemes are registries looked up by name strings — you select behavior with plain strings like method="cca", metrics="rpss", strategy="bma", cv="loyo".
• CCA is validated against CPT Fortran 17.8.3 / PyCPT to r≈0.9996 on predictions.
• MCP server: pip install 'africas2s[mcp]' then africas2s-mcp (stdio). Tools mirror the verbs below (downscale, optimize, calibrate, ensemble, skill, to_tercile, plot_terciles, plot_field, plus list_registry and describe_dataset); gridded data is exchanged as NetCDF paths under AFRICAS2S_MCP_WORKDIR (default ~/.africas2s/mcp). The same shapes, method names, and discipline rules apply whether you call the library or the server.

Data conventions (get these right first)

lat/latitude/Y and lon/longitude/X aliases are accepted, but prefer lat/lon. CV years must be consecutive integers (gap of 1) or CV schemes raise. NaN cells (ocean/dry) propagate cleanly. Rosetta's fetch(..., year_index=True) / assemble() produce exactly these shapes.

Quick start — pick a method, forecast, verify

import africas2s as ds
from africas2s.tercile import to_tercile, to_tercile_cv
from africas2s.cv import loyo

# gcm: (year, member, lat, lon) hindcast; obs: (year, lat, lon)
best = ds.optimize(gcm, obs, methods=["bcsd", "cca"], cv="loyo", primary_metric="rpss")
print(best.method, best.score)          # winning method + CV RPSS
tercile_fc = to_tercile(best.forecast, obs)   # (tercile, lat, lon)

# Honest cross-validated verification (see leakage warning below)
cv_preds = []
for train_years, test_year in loyo(gcm.year.values):
    model = ds.train(best.method, gcm.sel(year=train_years), obs.sel(year=train_years))
    cv_preds.append(model.predict(gcm.sel(year=test_year)).expand_dims(year=[test_year]))
import xarray as xr
cv_fcst = xr.concat(cv_preds, dim="year")
cv_terc = to_tercile_cv(cv_fcst, obs, method="bootstrap")

report = ds.skill(cv_terc, obs, metrics=["rpss", "roc", "reliability", "hss"], spatial=True)
report.to_pdf("skill.pdf")               # WMO-SVSLRF verification report
ds.write_terciles(tercile_fc, "forecast.nc", title="MAM precip")
ds.plot_terciles(tercile_fc, title="MAM 2024")

Or run the whole PyCPT-style pipeline in one call:

result = ds.seasonal_mme({"prcp": {"ECMWF": (gcm, None)}}, obs, method="cca", cv="loyo")
result.tercile_forecast; result.skill_report.scores; result.metadata

The core verbs

Downscale methods: bcsd, cca, qm, dqm, delta, climatology, rank-analog, chelsa (precipitation; terrain + wind auxiliaries), corrdiff (GPU). Ensemble strategies: uniform, skill_weighted, bma, drop_worst. CV schemes: loyo, lko, blocked, expanding. Metrics: rpss, roc, groc, reliability, hss, pearson_r, spearman, 2afc, rmse, msss, crpss, spread_error_ratio, spread_error_correlation (+ presets "svslrf", "all"). Also exported: ds.seasonal_coefficients — the fitted Kharin-2017 seasonally-smoothed regression coefficient field behind the smoothed_regression calibrator (see references/methods.md). Full parameter tables: references/methods.md, references/metrics-and-terciles.md.

Analog completion & climate positioning (SMPG)

A second workflow, alongside the downscale/calibrate pipeline: pick the past years that resemble the one being forecast, splice a partly-observed season forward with each, and say where the result falls in the record. All pure xarray, all agnostic to whether the dims are (lat, lon), (region,), or a single series — full detail in references/analog-completion.md.

import africas2s as ds
clim = ds.seasonal_stack(archive, "JJAS")               # (time,…) -> (year, step, …) season-aligned
nino = ds.Index.named("nino34").reduce(sst_hcst)        # a teleconnection index series over year
analogs = ds.analogs_from_index(nino, target=1.4, n=9)  # the 9 years most like a forecast Niño3.4 of 1.4
result = ds.complete(observed_to_date, analogs, climatology=clim,
                     season="JJAS", forecast=next_30_days)  # -> CompletionResult
result.consensus; result.percentile; result.accumulation()  # median outcome + where it sits + curves
ds.plot_accumulation_scenarios(result, climatology=clim)

• Analog selection (ds.analogs_from_years / _from_index / _from_field / _from_evolution / where) → an AnalogSet (scores every candidate, composes with & | .top(n)); _from_evolution ranks years on the shape of an index curve over the months observed so far (Niño3.4 analogues), plotted with ds.plot_index_evolution.
• Climate positioning (ds.percentile_of, ds.percent_of_normal, ds.rank_of_record, ds.frequency_below, ds.accumulate, ds.seasonal_reduce, ds.seasonal_stack) → where a value sits in a reference record.
• Scenario completion (ds.complete → CompletionResult) → one plausible end-of-season per analog; omit forecast= and run twice to isolate what a dynamic forecast adds.
• Scalar-series calibration (ds.quantile_map, ds.error_bounds) → bias-correct / bracket a forecast index, not a field.
• Teleconnection indices (ds.Index) now cover wvg, wvg2, nino12/3/34/4, oni, roni, dmi (iod), wtio, setio, wio, wpac, with configurable transform= ("standardize"/"anomaly"/"raw"), weights= ("cos_lat"), and baseline= — see references/api.md.
• Combine & mask (ds.combine_terciles, ds.mask_by_skill, ds.dry_mask) and pool (ds.pool_ensembles) — see references/methods.md.
• Predictor significance (ds.loo_corr, ds.permutation_test, ds.fdr) — see references/metrics-and-terciles.md.
• Daily aggregations (africas2s.aggregations.onset / cessation / season_length / dry_spell) → rainy-season timing from daily rainfall. Returns (year, lat, lon) fields usable as a predictand on the tercile and skill paths, but handle the failed-season years first: to_tercile_cv reads NaN as missing data, and a NaN step can also mean the season never started, which is a fourth outcome rather than a low tail. occurred (1.0 found / 0.0 failed / NaN no data) is the field that tells them apart: see references/aggregations.md.

Calendar helpers (africas2s.time.season_step, season_bounds, dekad/pentad arithmetic) and daily aggregations (africas2s.aggregations.onset, cessation, season_length, dry_spell) are module-qualified: reference them as africas2s.time.<fn> / africas2s.aggregations.<fn>, not ds.<fn>.

Critical discipline rules

1. Tercile leakage: never score CV hindcasts with to_tercile(pred, obs) — full obs leaks the held-out year. Use to_tercile_cv(cv_predictions, obs, method="bootstrap"|"cpt"|...). to_tercile is for the production forecast only.
2. Grid rule: calibrate() and skill_compare() do not regrid — put the GCM on the obs grid first (gcm.interp(lat=obs.lat, lon=obs.lon)). downscale() methods map coarse→fine themselves.
3. Metric/forecast pairing: probabilistic metrics (rpss, roc, groc, reliability, hss) need a tercile dim of size 3; continuous metrics (pearson_r, spearman, 2afc, rmse, spread_error_*) need the deterministic ensemble and reject tercile input. Score each family in its own skill() call.
4. Don't nest optimize() inside a CV loop — double CV plus non-consecutive inner years breaks the CV schemes. Use train() + .predict() in manual loops (see quick start).
5. primary_metric must be a leaf metric — roc_an, not roc (which expands to a dict).
6. DL methods (requires_training=True) refuse inline fitting: ds.train(name, ..., save_to=path) then ds.downscale(..., weights_path=path).
7. Mask discipline when comparing forecasts: RPSS masks its climatology reference only where obs is NaN, so forecasts with different NaN footprints are silently scored over different cell sets (a uniform-1/3 forecast has scored +0.26 this way). Apply one common valid mask to every forecast and the obs before scoring, and self-check that a uniform [1/3,1/3,1/3] forecast scores ≈ 0 — see references/metrics-and-terciles.md.
8. CHELSA atmospheric leakage: chelsa fits wind/PBL climatologies using training years only. A production forecast_atmosphere= override must already be a 2-D (lat,lon) Dataset; year/time-bearing fields are rejected so a CV call cannot silently use held-out ERA5.

Getting data in (rosetta)

Any xarray source works; rosetta produces the exact shapes africas2s expects:

import rosetta
gcm = rosetta.fetch("c3s/ecmwf-monthly", "precip", init="2024-02", target="MAM",
                    region=[-5, 15, 33, 48], hindcast=(1993, 2016),
                    year_index=True)["precip"]                    # (year, member, lat, lon)
obs = rosetta.fetch("obs/era5", "precip", region=[-5, 15, 33, 48],
                    hindcast=(1993, 2016), target="MAM",
                    seasonal="mean")["precip"]                    # (year, lat, lon)

rosetta.assemble(roster, ...) returns {label: (hindcast, forecast)} already shaped (year, member, lat, lon) with a guaranteed member dim. Rosetta is optional at runtime — only shapefile/geometry Index regions import it.

Common pitfalls (errors + environment: references/troubleshooting.md)

• downscale(gcm=...) is deprecated → predictor_hindcast=.
• ensemble(optimize_ensemble=True) requires obs and can silently fall back to uniform weights (gate; RuntimeWarning).
• Degenerate/dry cells get NaN tercile boundaries and are excluded from skill — expected, not a bug.
• Plotting/reporting/GeoTIFF need the plotting extra (pip install africas2s[plotting]); errors say so.
• CorrDiff GPU deps (torch, earth2studio, nvidia-physicsnemo) are not on PyPI — install manually; its save/load raise NotImplementedError.

Runnable examples (also see the repo's examples/ directory)

• examples/end_to_end_forecast.py — optimize → forecast → honest CV verification → PDF report
• examples/seasonal_mme_pipeline.py — one-call multi-model MME with seasonal_mme
• examples/calibration.py — eReg multi-model calibration, logistic/WVG index calibration, and season-aware smoothed_regression (deterministic + tercile)
• examples/ensemble_and_reporting.py — strategies, safeguards, skill comparison, plots
• examples/smoothed_calibration.py — Kharin-2017 function layer in a manual honest-CV loop: gamma→normal, fit/smooth a·b, CRPSS
• examples/analog_completion.py — SMPG workflow: seasonal_stack → analogs_from_index/where → complete → CompletionResult (consensus, percentile, rank-of-record)
• examples/demo_chelsa.py — real Rwanda MAM precipitation: C3S/CHIRPS + GLO-90 DEM + ERA5 wind/PBL + official CHELSA exposure; compares reduced/full-paper paths, checks conservation, and writes PNG/NetCDF

Reference files

• references/api.md — full signatures for the core forecasting verbs/dataclasses and the generalized Index
• references/analog-completion.md — SMPG subsystem: analog selection, scenario completion, climate positioning, scalar-series calibration, calendar/season-step utilities
• references/aggregations.md: rainy-season onset/cessation/season length and dry-spell statistics from daily rainfall
• references/methods.md — downscale methods, calibrators, ensemble strategies (+ pool_ensembles), tercile combination/masking (combine), CV schemes, registries
• references/metrics-and-terciles.md — every metric's semantics + tercile conversion discipline + predictor-significance tools
• references/plotting-reporting.md — which plot for which artifact, forecast/skill maps, panel grids & components+objective composites (plot_matrix), packaged colour languages (TercileStyle.named: icpac default, icpac-temperature, icpac-onset, noaa-cpc, ghacof, acmad) and workflow JSON style files (TercileStyle.from_json), dominance rules (prob_bins[0], secondary_max), packaged classified field scales (FieldScale.named: noaa-cpc-anomaly, ucsb-chirps-anomaly, ucsb-chirps-total, icpac-onset-date, icpac-onset-spread; scale= on plot_field/plot_matrix), index-evolution plume (plot_index_evolution), field maps & choropleths, scenario/index plots, SVSLRF PDFs, GeoTIFF/NetCDF export, headless figure handling
• references/troubleshooting.md — error → cause table, environment/install setup, test markers, operational scripts, convention caveats