# OpenSmartRoute documentation

> OpenSmartRoute routes every request to the best LLM, agent, skill, persona, tool or human under hard constraints, explains the decision and learns from outcomes. Open source, zero core dependencies.

Every published page of https://opensmartroute.ai/docs as Markdown, in reading order. The short index is https://opensmartroute.ai/llms.txt.

<!-- Section: Get started -->

---

<!-- Page: Overview (https://opensmartroute.ai/docs/readme) -->

<!-- OpenSmartRoute: Overview. Source https://github.com/isathish/OpenSmartRoute/blob/main/README.md; HTML https://opensmartroute.ai/docs/readme -->
<p align="center">
  <picture>
    <source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/isathish/OpenSmartRoute/main/assets/brand/dist/logo-horizontal-reverse-1240.png">
    <img src="https://raw.githubusercontent.com/isathish/OpenSmartRoute/main/assets/brand/dist/logo-horizontal-1240.png" alt="OpenSmartRoute" width="620">
  </picture>
</p>

# OpenSmartRoute

> **An open, intelligent route to the right decision, solution, or destination.**

<p>
  <a href="https://github.com/isathish/OpenSmartRoute/actions/workflows/ci.yml"><img alt="CI" src="https://img.shields.io/github/actions/workflow/status/isathish/OpenSmartRoute/ci.yml?branch=main&logo=githubactions&logoColor=white&label=CI"></a>
  <a href="https://github.com/isathish/OpenSmartRoute/actions/workflows/codeql.yml"><img alt="CodeQL" src="https://img.shields.io/github/actions/workflow/status/isathish/OpenSmartRoute/codeql.yml?branch=main&logo=github&logoColor=white&label=CodeQL"></a>
  <a href="https://github.com/isathish/OpenSmartRoute/releases"><img alt="Release" src="https://img.shields.io/github/v/release/isathish/OpenSmartRoute?logo=semanticrelease&logoColor=white&label=release"></a>
  <a href="https://pypi.org/project/opensmartroute/"><img alt="PyPI" src="https://img.shields.io/pypi/v/opensmartroute?logo=pypi&logoColor=white"></a>
  <a href="pyproject.toml"><img alt="Python 3.10+" src="https://img.shields.io/badge/python-3.10%20%7C%203.11%20%7C%203.12%20%7C%203.13-3776AB?logo=python&logoColor=white"></a>
  <a href="https://github.com/isathish/OpenSmartRoute/pkgs/container/opensmartroute"><img alt="Container image" src="https://img.shields.io/badge/ghcr.io-opensmartroute-2496ED?logo=docker&logoColor=white"></a>
  <a href="LICENSE"><img alt="License: Apache-2.0" src="https://img.shields.io/badge/license-Apache--2.0-D22128?logo=apache&logoColor=white"></a>
  <a href="https://github.com/astral-sh/ruff"><img alt="Ruff" src="https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json"></a>
  <a href="src/opensmartroute/py.typed"><img alt="Typed" src="https://img.shields.io/badge/typing-py.typed-informational?logo=python&logoColor=white"></a>
  <a href="pyproject.toml"><img alt="Zero runtime dependencies" src="https://img.shields.io/badge/runtime%20deps-0-brightgreen"></a>
</p>

OpenSmartRoute is an open-source **AI decision control plane**. For each request it picks the best
**LLM, agent harness, skill, persona, tool, workflow or human**, honours hard constraints (privacy,
region, budget, latency), explains the choice, executes the resulting plan and **learns from every
outcome**. The core is pure Python with zero runtime dependencies.

```mermaid
flowchart LR
    R([request]) --> G["guard<br/>redact PII"]
    G --> S["signals<br/>&lt; 1 ms"]
    S --> P["policy<br/>hard constraints"]
    P --> ST["strategies + ensemble utility<br/>rules, capability, similarity, bandit, LLM judge"]
    ST --> D["decision<br/>trace + plan"]
    D --> X["execute<br/>persona -> skill -> model / agent / tool / human"]
    X -. outcomes .-> ST
```

## Why

- **Everything is a route target.** One `RouteTarget` contract for models, agent harnesses, SKILL.md
  packages, personas, MCP tools, workflows and human queues; one policy layer; one learning loop.
- **Plans, not just picks.** `route(plan=True)` composes persona, skill and model; `run()` executes
  the plan and records an `Outcome` per participant. An instructions-only skill or persona runs on
  the plan's model with its body disclosed in the system prompt.
- **Constraints are never traded off.** PII, data boundary, region, tenant, cost and latency SLOs are
  filtered before any score is computed, with the rejection reason in the trace. Redacted PII is
  restored only for targets allowed to hold it; every other target, and every log, sees placeholders.
- **Learns in production.** Thompson bandits, Item Response Theory, Bradley-Terry preferences,
  LinUCB, Markov lookahead, task-level credit assignment, drift detection and forgetting.
- **Honest evaluation.** Baselines, oracle, label-noise floor, paraphrase robustness, calibration
  (ECE, Brier, conformal sets), off-policy estimators, public benchmark presets.
- **Secure by design.** Learned and heuristic guards against rerouting gadgets and prompt injection,
  PII redaction, resource limits, signed MCP manifests, encrypted state, hash-chained audit.
- **Enterprise-ready.** Builder, middleware, telemetry / state / audit ports, Redis and SQL stores,
  shadow and A/B routing with SPRT, tenant fair share, async facade, container image and Helm chart.

## Install

The `osr` command line and the Python package ship together. On Linux and macOS:

```bash
curl -LsSf https://opensmartroute.ai/install.sh | sh
```

On Windows (PowerShell):

```powershell
powershell -ExecutionPolicy ByPass -c "irm https://opensmartroute.ai/install.ps1 | iex"
```

The installer puts `osr` in an isolated environment (uv, pipx or a private venv - whichever is
available, never the system Python) and adds it to your PATH. `OSR_VERSION=1.0.0` pins a release,
`OSR_EXTRAS=all` installs every optional dependency, `OSR_INSTALLER=uv|pipx|venv` forces a backend and
`OSR_NO_MODIFY_PATH=1` leaves your shell configuration alone. The scripts are
[install.sh](https://github.com/isathish/OpenSmartRoute/blob/main/install.sh) and [install.ps1](https://github.com/isathish/OpenSmartRoute/blob/main/install.ps1) in this repository and attached to every
GitHub release.

If you already manage Python tools yourself:

```bash
uv tool install 'opensmartroute[yaml,server]'   # or: pipx install 'opensmartroute[yaml,server]'
pip install opensmartroute                       # library only, zero runtime dependencies
pip install 'opensmartroute[yaml]'               # + YAML catalogues and rules
pip install 'opensmartroute[server]'             # + FastAPI server and OpenAI-compatible proxy
pip install 'opensmartroute[embeddings]'         # + sentence-transformers similarity
pip install 'opensmartroute[otel]'               # + OpenTelemetry telemetry
pip install 'opensmartroute[crypto]'             # + AES-GCM encrypted state
```

Container image: `ghcr.io/isathish/opensmartroute:<version>` ([deploy/README.md](https://opensmartroute.ai/docs/deploy.md)).
From a checkout: `pip install -e '.[dev]'`.

Then sign in. The hosted platform (community or enterprise edition) uses a browser hand-shake; a
self-hosted `osr serve` accepts a token you generate yourself:

```bash
osr login                                              # opens https://opensmartroute.ai/cli/authorize
osr login --url https://osr.example.com                # your own platform deployment
osr login --url http://router:8000 --token osr_local_...   # self-hosted server (see `osr serve --generate-token`)
osr whoami                                             # workspace, plan, edition, key
```

Credentials live in `~/.config/opensmartroute/credentials.json` (`%APPDATA%\opensmartroute` on
Windows), one profile per `--profile`; `OSR_API_URL` / `OSR_API_KEY` override them in CI.

## Quick start

```python
from opensmartroute import Router, TargetRegistry, RouteTarget, TargetKind, Capabilities, Outcome

registry = TargetRegistry([
    RouteTarget("llm-small", TargetKind.LLM,
                capabilities=Capabilities(max_complexity=0.45),
                cost={"usd_per_1k_tokens": 0.0002}, latency_ms=300, quality_prior=0.55,
                examples=["Hi, how are you?", "What is the capital of France?"]),
    RouteTarget("llm-frontier", TargetKind.LLM,
                capabilities=Capabilities(min_complexity=0.5, domains=["math", "coding"]),
                cost={"usd_per_1k_tokens": 0.015}, latency_ms=2500, quality_prior=0.93,
                examples=["Prove the theorem step by step."]),
    RouteTarget("human", TargetKind.HUMAN,
                capabilities=Capabilities(actions=["escalate"], tags=["safety"]),
                cost={"usd_per_1k_tokens": 0.5}, latency_ms=300_000),
])

router = Router(registry)
d = router.route("Prove that sqrt(2) is irrational, step by step.")
print(d.target.id, f"{d.confidence:.2f}")     # llm-frontier 0.97
print(d.trace.explain())                       # per-strategy scores and rationales

router.learn(Outcome(request_id=d.request_id, target_id=d.target.id, success=True,
                     quality=0.9, cost_usd=0.002, latency_ms=1800, domains=d.trace.signals.domains))
```

Hard constraints and a per-request objective:

```python
from opensmartroute import RouteRequest, RequestConstraints, Objective

req = RouteRequest("Summarize this patient intake note.",
                   constraints=RequestConstraints(region="eu", data_boundary="private", max_cost_per_1k=0.005),
                   objective=Objective(quality=1.0, cost=0.5, latency=0.1, quality_floor=0.6))
d = router.route(req)
print(d.trace.policy_rejections)   # {'llm-frontier': 'cost 0.015 > budget 0.005', ...}
```

Catalogue and rules from YAML, evaluated and served from the command line:

```bash
osr -t examples/targets.yaml -r examples/rules.yaml route "I want a refund for order #123" --plan
osr -t examples/targets.yaml -r examples/rules.yaml eval examples/eval_dataset.jsonl --frontier
osr -t examples/targets.yaml -r examples/rules.yaml serve        # http://127.0.0.1:8000/docs
```

Wiring real providers, executing plans with `router.run()`, agent harnesses, SKILL.md and MCP
catalogues, the decorator SDK and the enterprise builder are covered step by step in
[docs/GUIDE.md](https://opensmartroute.ai/docs/GUIDE.md).

## What it routes

| Kind | Examples | Executed by |
|---|---|---|
| `llm` | model endpoints, per reasoning effort or token budget | OpenAI-compatible client or any callable |
| `agent` | coding, research and support harnesses with tools and memory | callable, HTTP or subprocess harness |
| `skill` | deterministic capabilities, SKILL.md packages | your function; instructions disclosed to the model |
| `persona` | system-prompt layers composed on top of the primary target | `run()` prompt composition |
| `tool` | MCP or function tools, with schema-aware matching | tool call |
| `workflow` | fixed multi-step pipelines | workflow engine |
| `human` | queues and experts with Erlang-C capacity maths | ticketing or hand-off; also the abstention target |

Catalogues load from YAML or JSON, MCP `tools/list` payloads, A2A agent cards, SKILL.md and persona
directories and vLLM semantic-router configurations; the router plugs into LangGraph and Agent
Framework graphs as a node and into any chat loop as an OpenAI tool.

## How it decides

1. **Signals** (under 1 ms): task type, domains, complexity, reasoning need, PII, language, modality,
   history; optional verbalised difficulty, cheap-draft features and hidden-state probes.
2. **Policy**: hard constraints filter targets before any scoring.
3. **Strategies**: rules, capability fit, example similarity, task table, Thompson and LinUCB bandits,
   IRT, Bradley-Terry, Markov lookahead, multi-turn history embeddings, learning-to-defer, edge/cloud
   tiers, token budgets, auctions, user adaptation and an LLM judge consulted only below a confidence
   threshold; optionally a routing SLM (`RouterSLM`) distilled from the whole ensemble that keeps
   improving on its own from outcomes, public routing datasets and a live model catalogue
   ([guide](https://opensmartroute.ai/docs/GUIDE.md#the-routing-slm-and-the-self-improvement-loop)).
4. **Utility**: confidence-weighted ensemble, then
   `w_q * quality - w_c * norm(cost) - w_l * norm(latency)` with a hard quality floor; temperature
   scaling and conformal candidate sets calibrate the confidence; the router abstains when nothing is
   safe enough.
5. **After the answer**: cascades and self-escalation stop or reroute on response-side uncertainty
   (semantic entropy, P(True), streaming competence posterior); mixture-of-agents aggregation and
   permanent hand-off policies cover agentic trajectories.

Formulas and citations: [docs/MATH.md](https://opensmartroute.ai/docs/MATH.md), [docs/RESEARCH.md](https://opensmartroute.ai/docs/RESEARCH.md).

## Production

```python
from opensmartroute import RouteRequest, RequestConstraints
from opensmartroute.enterprise import RouterBuilder, MetricsTelemetry, FileAuditSink, TenantMiddleware
from opensmartroute.security import GuardMiddleware

app = (RouterBuilder(registry)
       .with_defaults().with_auto_learning(state_dir=".osr-state")
       .with_health(latency_slo_ms=3000)
       .with_middleware(GuardMiddleware(redact=True),
                        TenantMiddleware({"acme": {"deny_targets": ["llm-frontier"]},
                                          "globex": {"data_boundary": "on_prem"}}))
       .with_telemetry(MetricsTelemetry()).with_audit(FileAuditSink("audit.jsonl"))
       .build())

req = RouteRequest("Prove that sqrt(2) is irrational.", constraints=RequestConstraints(tenant="acme"))
d = app.route(req)                 # guard -> tenant -> policy -> strategies; audited and measured
result = app.run(req)              # ...then executes the plan once targets carry handlers, and learns
print(d.target.id, app.health_snapshot())
```

Routing overhead is 6 ms p50 with the default strategies and 9 ms with every learner enabled on a
16-target catalogue; retrieve-then-rank keeps it flat for thousands of tools
([measurements](https://opensmartroute.ai/docs/GUIDE.md#10-routing-latency)). `osr serve` (or `create_app(app)` with the
router above) exposes a FastAPI app and an OpenAI-compatible `/v1/chat/completions` proxy: point any
OpenAI client at it with `model="auto"`; guard and tenant violations return 400, no admissible target
422, an unreachable provider 503. Circuit breakers open on provider failures and recover on their own;
learner state survives restarts from `state_dir` or a Redis / SQL store. Deployment references:
[deploy/README.md](https://opensmartroute.ai/docs/deploy.md), [docs/ENTERPRISE.md](https://opensmartroute.ai/docs/ENTERPRISE.md).

## Hosted platform

[platform/](https://github.com/isathish/OpenSmartRoute/blob/main/platform/README.md) packages the router as a service in two containers: `osr-platform-api`
(FastAPI: self-serve signup with hashed API keys, plans and quotas, usage metering, the metered
`/api/v1/route` family with the full trace, per-tenant constraints and workspace policy, the audit
trail, per-request traces and a live event stream, and the OpenAI-compatible `/v1/chat/completions`
proxy) and `osr-platform-web` (Next.js: landing page, the rendered documentation, pricing, a live
playground and the account dashboard with activity, events, governance and health pages; it proxies
`/api` and `/v1` so the browser only talks to one origin). `OSR_PLATFORM_EDITION=community` runs the core
`Router`; `enterprise` runs the builder above with auto-learning, health, guard, metrics, a
hash-chained audit log and per-tenant constraints. `azd up` deploys both to Azure Container Apps with
Azure OpenAI from [infra/](https://github.com/isathish/OpenSmartRoute/blob/main/infra/main.bicep); the reference deployment is
https://osr-web.gentlepebble-235bed4c.swedencentral.azurecontainerapps.io. The end-user guide is
[docs/PLATFORM.md](https://opensmartroute.ai/docs/PLATFORM.md); the REST API reference is generated from
[platform/api/openapi.json](https://github.com/isathish/OpenSmartRoute/blob/main/platform/api/openapi.json).

```bash
curl -s -XPOST $OSR/api/v1/signup -H 'content-type: application/json' -d '{"email":"you@example.com"}'
curl -s $OSR/api/v1/route -H "authorization: Bearer $KEY" -H 'content-type: application/json' \
     -d '{"text":"Prove that sqrt(2) is irrational."}'
```

## Security

Routing is a control plane; its integrity is a security property. `InputGuard` combines a heuristic
and a learned detector for confounder gadgets that reroute queries, an injection-risk scorer, PII
redaction and prompt sanitisation for the LLM judge. Per-task resource limits, an origin policy for
sensitive tool parameters, signed MCP manifests, AES-GCM state, content-free logs and hash-chained
audit complete the model. CI runs `ruff -S`, `mypy`, `bandit`, CodeQL and the `osr safety` red-team
suite. Threat model: [docs/SECURITY.md](https://opensmartroute.ai/docs/SECURITY.md). Reporting: [SECURITY.md](https://opensmartroute.ai/docs/security-policy.md).

## Documentation

The user documentation is a searchable, versioned site at
https://osr-web.gentlepebble-235bed4c.swedencentral.azurecontainerapps.io/docs (built from these files
at release time, with a REST API reference generated from the platform's OpenAPI document). It covers
the hosted platform, the Python SDK and the pip package; the internal planning, go-to-market, brand
and sales documents below stay in the repository only.

| Document | Contents |
|---|---|
| [docs/GUIDE.md](https://opensmartroute.ai/docs/GUIDE.md) | User guide: targets, routing, execution, providers, learning, SDK, enterprise builder, research-track modules, CLI, latency, layout |
| [docs/PLATFORM.md](https://opensmartroute.ai/docs/PLATFORM.md) | Platform guide: authentication, `/api/v1/route`, execution, the OpenAI-compatible endpoint, feedback, plans and quotas, organizations, tenants, governance, observability (traces, events, readiness), the MCP server, dashboard |
| [docs/MARKETPLACE.md](https://opensmartroute.ai/docs/MARKETPLACE.md) | Marketplace: find, install, buy and publish agents, skills, personas, prompts and stack templates; ratings, review lifecycle, `osr stack` |
| [docs/MCP.md](https://opensmartroute.ai/docs/MCP.md) | Cost estimates (`estimate`, `POST /api/v1/estimate`), recommended models per use case and the MCP server for VS Code, Cursor, Claude, Windsurf and agents |
| [docs/SDK.md](https://opensmartroute.ai/docs/SDK.md) | API reference for the decorator SDK, components and settings |
| [docs/REFERENCE.md](https://opensmartroute.ai/docs/REFERENCE.md) | Generated API reference: every module and exported name (`python scripts/api_reference.py`) |
| [docs/ARCHITECTURE.md](https://opensmartroute.ai/docs/ARCHITECTURE.md) | Request path, module boundaries, performance envelope |
| [docs/ENTERPRISE.md](https://opensmartroute.ai/docs/ENTERPRISE.md) | Ports, stores, middleware, shadow and A/B, multi-replica operation |
| [docs/OBSERVABILITY.md](https://opensmartroute.ai/docs/OBSERVABILITY.md) | Tracing and observability: spans and events for every stage, sinks (memory, metrics, log, file, OpenTelemetry), `/events`, `/trace`, `/metrics`, `OSR_OBSERVABILITY_*`; the platform's per-workspace `/api/v1/trace`, `/api/v1/events`, `/api/v1/status` and dashboard pages |
| [docs/MATH.md](https://opensmartroute.ai/docs/MATH.md) | Every formula the router uses, with derivations |
| [docs/RESEARCH.md](https://opensmartroute.ai/docs/RESEARCH.md) | Literature survey and the idea-to-module map |
| [docs/SECURITY.md](https://opensmartroute.ai/docs/SECURITY.md) | Threat model and hardening checklist |
| [docs/SECURITY_REVIEW.md](https://opensmartroute.ai/docs/SECURITY_REVIEW.md) | External security review pack: scope, trust boundaries, evidence, reviewer questions, review log |
| [docs/ROADMAP.md](https://opensmartroute.ai/docs/ROADMAP.md) | Per-version exit criteria and status |
| [docs/PLATFORM_PLAN.md](https://github.com/isathish/OpenSmartRoute/blob/main/docs/PLATFORM_PLAN.md) | Platform strategy: agentic routing, Open Capability Manifest, marketplace, editions and pricing |
| [docs/GO_TO_MARKET.md](https://github.com/isathish/OpenSmartRoute/blob/main/docs/GO_TO_MARKET.md) | Go-to-market plan; the sales enablement kit lives in [docs/sales/](https://github.com/isathish/OpenSmartRoute/blob/main/docs/sales/README.md) |
| [spec/ocm/README.md](https://opensmartroute.ai/docs/ocm.md) | Open Capability Manifest specification and JSON Schema |
| [deploy/README.md](https://opensmartroute.ai/docs/deploy.md) | Container image, Helm chart, reference deployments |
| [platform/README.md](https://github.com/isathish/OpenSmartRoute/blob/main/platform/README.md) | Hosted platform: `api/` (FastAPI) and `web/` (Next.js), playground, API keys, plans, editions, `OSR_PLATFORM_*` settings, Azure deployment |
| [docs/BRAND.md](https://github.com/isathish/OpenSmartRoute/blob/main/docs/BRAND.md) | Logo system and naming conventions |

[.claude/skills/](https://github.com/isathish/OpenSmartRoute/blob/main/.claude/skills) ships Agent-Skills packages that teach coding assistants how to use
and extend the project; they are also valid routing targets (`osr skills` validates them) and each one is
published on the documentation site under `/docs/skills/`. Coding agents working in this repository
start from [AGENTS.md](https://github.com/isathish/OpenSmartRoute/blob/main/AGENTS.md).

## Roadmap

| Release | Theme | Status |
|---|---|---|
| 0.3 | Real integrations: OpenAI-compatible client, adapters, config loaders | Shipped |
| 0.4 | Learned signals and honest evaluation | Shipped |
| 0.5 | Target representations, effort and personalisation | Complete |
| 0.6 | Multi-step and agentic routing | Complete |
| 0.7 | Catalogue interop and discovery at scale | Complete |
| 0.8 | Operations, risk control and economics | Complete |
| 0.9 | Security hardening of the control plane | Complete |
| 1.0 | Stable API, reference deployments, security review pack | Complete |

Every item in the research track is implemented on `main` with tests; a version is **Complete** when
its measured exit criterion is published (`python scripts/exit_criteria.py`, 8/8 met) and **Shipped**
when it is in a tagged release. 1.0 means what the repository can prove: frozen public API with a
deprecation path, published performance envelope, reference deployments, security controls with a
red-team suite in CI and a published review pack ([docs/SECURITY_REVIEW.md](https://opensmartroute.ai/docs/SECURITY_REVIEW.md)),
the public leaderboard run ([examples/leaderboard/results](https://opensmartroute.ai/docs/leaderboard-results.md),
produced by the recipe in [examples/leaderboard](https://opensmartroute.ai/docs/leaderboard.md)) and two consecutive
releases without a breaking change. Adoption evidence - an independent security report, accepted
leaderboard listings, two production users ([ADOPTERS.md](https://github.com/isathish/OpenSmartRoute/blob/main/ADOPTERS.md)) - is tracked, never claimed:
`python scripts/release.py readiness` prints the whole table computed from the repository, and the
release tooling refuses any `1.x` version while a release row is open.
Details in [docs/ROADMAP.md](https://opensmartroute.ai/docs/ROADMAP.md) and [CHANGELOG.md](https://opensmartroute.ai/docs/changelog.md).

## Contributing

Issues and pull requests are welcome; see [CONTRIBUTING.md](https://opensmartroute.ai/docs/contributing.md). CI runs `ruff`, `mypy`,
`bandit` and `pytest` on CPython 3.10 to 3.13 (Linux and Windows), a routing-accuracy gate and the
safety suite on the example catalogue, builds the distribution and the container image, and scans with
CodeQL. The test suite has no mocks of the router itself: [tests/test_e2e_scenarios.py](https://github.com/isathish/OpenSmartRoute/blob/main/tests/test_e2e_scenarios.py)
drives the shipped example catalogue through an in-process OpenAI-compatible provider fleet over
HTTP, a real MCP server subprocess over stdio, the CLI, the FastAPI service, provider outages with
breaker recovery, agentic multi-round loops and concurrent traffic. Releases are automated: a release
pull request bumps the version and rolls the changelog; merging it tags, publishes to PyPI with
attestations and pushes the image.

## License

Apache-2.0

---

<!-- Page: User guide (https://opensmartroute.ai/docs/GUIDE) -->

<!-- OpenSmartRoute: User guide. Source https://github.com/isathish/OpenSmartRoute/blob/main/docs/GUIDE.md; HTML https://opensmartroute.ai/docs/GUIDE -->
# User guide

Everything the README leaves out: the full quick-start sequence, running plans end to end, wiring
real providers, the enterprise builder, the research-track modules, routing latency and the
repository layout. API reference material lives in [SDK.md](https://opensmartroute.ai/docs/SDK.md) (decorator SDK, components,
settings) and [REFERENCE.md](https://opensmartroute.ai/docs/REFERENCE.md) (generated; every module and exported name); design in
[ARCHITECTURE.md](https://opensmartroute.ai/docs/ARCHITECTURE.md).

Contents

1. [Targets](#1-targets)
2. [Routing](#2-routing)
3. [Executing a plan](#3-executing-a-plan)
4. [Real providers](#4-real-providers)
5. [Learning from outcomes](#5-learning-from-outcomes)
6. [Declaring components with decorators](#6-declaring-components-with-decorators)
7. [Enterprise builder](#7-enterprise-builder)
8. [Research-track modules](#8-research-track-modules)
9. [Command line](#9-command-line)
10. [Routing latency](#10-routing-latency)
11. [Repository layout](#11-repository-layout)

## 1. Targets

Every routable thing is a `RouteTarget` with the same contract: an `id`, a `kind`, declared
`capabilities`, `cost`, `latency_ms`, a `quality_prior`, optional `examples`, `instructions` and a
`handler`. Adding a target never requires retraining; it is routable from its declaration and learns
from its outcomes.

| Kind | What it is | Typical `capabilities` | Wins the primary route? | Executed by |
|---|---|---|---|---|
| `llm` | A model endpoint (optionally per reasoning effort) | `domains`, `min/max_complexity`, `languages`, `context_window` | yes | `chat_handler` / any callable |
| `skill` | A deterministic or narrow capability (SQL, translate, summarise, a SKILL.md package) | `actions`, `domains`, token footprint as `cost` | yes, or fills the `skill` plan slot | your function; `instructions` are disclosed to the model |
| `persona` | System-prompt / behaviour layer (`primary: false`) | `domains`, `actions` | no; fills the `persona` slot on top of a model or agent | `instructions` become the system prompt in `run()` |
| `agent` | An agent harness: coding, research or support runtime with tools, memory and its own loop | `supports_tools`, `tags: [autonomous]`, `actions` | yes | `CallableHarness` / `HTTPHarness` / `SubprocessHarness` |
| `tool` | A single MCP / function tool | `actions`, `input_schema` in metadata | yes | tool call |
| `workflow` | A fixed multi-step pipeline | `actions`, `domains` | yes | workflow engine |
| `human` | Queue, team or expert (Erlang-C capacity maths) | `actions: [escalate]`, `tags: [safety]` | yes; also the abstention target | ticketing / hand-off |

```yaml
# examples/targets.yaml (excerpt)
- id: coding-agent            # agent harness: repo access, tests, review loop
  kind: agent
  capabilities: { domains: [coding], actions: [code_generation, code_review, action],
                  min_complexity: 0.3, supports_tools: true, tags: [autonomous] }
  cost: { usd_per_1k_tokens: 0.008 }
  latency_ms: 6000
  examples: ["Refactor this class to remove the circular import and run the test suite."]

- id: skill-sql               # deterministic skill; `instructions` is its SKILL.md-style body
  kind: skill
  capabilities: { domains: [data_analysis, coding], actions: [code_generation, extract] }
  cost: { usd_per_1k_tokens: 0.001 }
  instructions: "Always qualify table names with the schema; never SELECT *."

- id: persona-legal-counsel   # never primary; layered on whichever model/agent wins
  kind: persona
  primary: false
  capabilities: { domains: [legal], actions: [reasoning, summarize, qa] }
  instructions: "You are cautious in-house counsel. Cite the clause, flag jurisdiction risk."
```

Catalogues load from JSON or YAML (`config.load_targets`), from MCP `tools/list` payloads
(`adapters.tools_from_mcp`), A2A agent cards (`adapters.agent_from_card`), SKILL.md directories
(`adapters.load_skills`), persona folders (`adapters.load_personas`) and vLLM semantic-router
configurations (`adapters.load_semantic_router_config`).

### Agent skills for coding assistants

[.claude/skills/](https://github.com/isathish/OpenSmartRoute/blob/main/.claude/skills) ships nine Agent-Skills packages (`SKILL.md`, agentskills.io
format) that teach a coding agent how to use and extend OpenSmartRoute: `osr-routing-catalogue`,
`osr-decorator-sdk`, `osr-enterprise-builder`, `osr-evaluation`, `osr-security-hardening`,
`osr-integrations`, `osr-deploy-serve`, `osr-platform` and `osr-contributing`. Claude Code discovers
them automatically in this repository; other agents can load the directory, and each package is
published on the documentation site under `/docs/skills/<name>`. They are also valid routing
targets: the same loader the SDK uses (`load_skills`) validates them in CI and the router fills the
plan's skill slot with the right one.

```bash
osr skills                                           # validate + list (default root .claude/skills)
osr -t examples/targets.yaml --skills .claude/skills route "Create a JSONL eval dataset and gate CI on accuracy" --plan
# plan: llm:llm-mid -> skill:osr-evaluation
```

## 2. Routing

```python
from opensmartroute import Router, RouteRequest, RequestConstraints, Objective

router = Router(registry)

# plain text
d = router.route("Prove that sqrt(2) is irrational, step by step.")
print(d.target.id, f"{d.confidence:.2f}")
print(d.trace.explain())                   # per-strategy scores and rationales

# hard constraints and a per-request objective
req = RouteRequest(
    text="Summarize this patient intake note.",
    constraints=RequestConstraints(region="eu", data_boundary="private", max_cost_per_1k=0.005),
    objective=Objective(quality=1.0, cost=0.5, latency=0.1, quality_floor=0.6),
)
d = router.route(req)
print(d.trace.policy_rejections)           # {'llm-frontier': 'cost 0.015 > budget 0.005', ...}

# a composed plan: persona -> skill -> model
d = router.route("Review this NDA clause for GDPR liability.", plan=True)
for slot in d.plan.slots:
    print(slot.role, slot.target.id, f"{slot.confidence:.2f}")
```

Policy runs before scoring and is never traded off: enable flags, allow / deny lists, kind, region,
data boundary, PII, tenant, cost / latency / token SLOs, modality, language and jailbreak risk.
Strategies then score the survivors and a confidence-weighted ensemble produces a quality estimate;
utility is `w_q * quality - w_c * norm(cost) - w_l * norm(latency)` with a hard `quality_floor`.
With a `ConformalCalibrator`, `RouteDecision.candidate_set` is the smallest set with the requested
coverage and `abstain` is set when no target is safe enough.

A generative judge can be consulted only when the ensemble is unsure:

```python
from opensmartroute import LLMJudgeStrategy
router = Router(registry,
                llm_judge=LLMJudgeStrategy(llm=lambda prompt: my_llm(prompt)),
                escalate_llm_judge_below=0.6)
```

## 3. Executing a plan

`route()` decides; `run()` decides and executes the whole plan: skill pre-processors run first,
persona and skill `instructions` are composed into the system prompt, the primary target is called,
and an `Outcome` is recorded for every participant so each one learns from the result.

```python
from opensmartroute.adapters import (CallableHarness, HTTPHarness, SubprocessHarness,
                                     chat_handler, harness_handler, load_skills)

registry.get("llm-frontier").handler = chat_handler(openai, model="gpt-4o")
registry.get("coding-agent").handler = harness_handler(
    SubprocessHarness(["my-coding-agent", "--repo", "."], system_flag="--system"))
registry.get("research-agent").handler = harness_handler(
    HTTPHarness("https://agents.internal/research/run", api_key_env="AGENT_TOKEN"))
registry.get("skill-sql").handler = lambda req: lookup_schema(req.text)   # runs BEFORE the model
for skill in load_skills("examples/skills"):                              # SKILL.md folders
    registry.add(skill)

res = router.run("Is this indemnification clause enforceable in California?", task_id="ticket-812")
res.text                 # the primary target's answer
res.steps                # [persona:persona-legal-counsel, llm:llm-frontier] with per-step latency
res.system_prompt        # composed persona + skill instructions
res.outcomes             # one Outcome per participant, role="persona"/"skill"/None, shared task_id
```

A skill handler may return a `RouteRequest` (rewrite the request), a `str` (attached as
`context["skill_output"]` and shown to the model) or `None` (instructions only). Async handlers work
through `AsyncRouter.run()`; `EnterpriseRouter.run()` adds health, budgets and telemetry. See
[examples/end_to_end.py](https://github.com/isathish/OpenSmartRoute/blob/main/examples/end_to_end.py) and [SDK.md](https://opensmartroute.ai/docs/SDK.md#executing-end-to-end).

## 4. Real providers

The bundled client is stdlib only and speaks the OpenAI chat and embeddings API, which covers OpenAI,
Azure OpenAI, vLLM, Ollama, LiteLLM, OpenRouter, Groq and Mistral.

```python
from opensmartroute import Router, LLMJudgeStrategy
from opensmartroute.adapters import OpenAICompatClient, chat_handler, embedder, judge_fn
from opensmartroute.strategies import CapabilityStrategy, SimilarityStrategy

openai = OpenAICompatClient()                                             # OPENAI_API_KEY from env
ollama = OpenAICompatClient("http://localhost:11434/v1", api_key="ollama")

registry.get("llm-frontier").handler = chat_handler(openai, model="gpt-4o")
registry.get("llm-small").handler   = chat_handler(ollama, model="llama3.1")

router = Router(registry,
                strategies=[CapabilityStrategy(),
                            SimilarityStrategy(embedder=embedder(openai, model="text-embedding-3-small"))],
                llm_judge=LLMJudgeStrategy(judge_fn(openai, model="gpt-4o-mini")),
                escalate_llm_judge_below=0.5)

d = router.route(req)
result = router.execute(d, req)          # ChatResult(text, prompt_tokens, completion_tokens, latency_ms)
res = router.run(req)                    # route + execute the full plan
```

`adapters.sentence_transformers_embedder()` and `adapters.OpenTelemetryTelemetry` are lazy imports
behind the `embeddings` and `otel` extras. `osr serve` exposes a FastAPI app with an OpenAI-compatible
`/v1/chat/completions` proxy that routes, executes and learns.

## 5. Learning from outcomes

```python
from opensmartroute import Outcome
router.learn(Outcome(request_id=d.request_id, target_id=d.target.id,
                     success=True, quality=0.9, cost_usd=0.002, latency_ms=1800,
                     domains=d.trace.signals.domains))
```

One `Outcome` fans out to every learner: Thompson bandits per domain, IRT ability and difficulty,
Bradley-Terry preferences, LinUCB on the signal vector, the Markov / MDP lookahead, the task table and
the personalisation posteriors. `Outcome.task_id` and `step` join delayed, task-level rewards to the
routing decisions that produced them (`learning.TaskCredit`, `learning.TaskPins`). Page-Hinkley
drift detection flags targets whose success rate degrades; every learner forgets with a configurable
`decay` and can be merged across replicas (`learning.merge_learners`). Each `learn()` also records a
`learn.outcome` event on the request's trace and, with `with_audit(sink, outcomes=True)`, an `outcome`
record in the audit chain, so a decision and its result stay joined by `request_id`
([OBSERVABILITY.md](https://opensmartroute.ai/docs/OBSERVABILITY.md); on the hosted platform `POST /api/v1/feedback` does the same).

Offline warm starts: `learning.warm_start_from_matrix()` replays a `(prompt, {target: reward})`
matrix, `learning.warm_start()` copies posteriors from the nearest declared neighbours and
`osr train --from rows.jsonl --out models.json` fits the hashed signal models.

### The routing SLM and the self-improvement loop

`learning.RouterSLM` is a small routing model in one JSON file: a hashed dual encoder
(`SLMSettings.dim` x `feature_dim`), a temperature calibrator and a snapshot of every target's price,
latency and quality prior. It has no dependencies, predicts in microseconds and can be trained from
any `EvalRow` source - labelled datasets, your own outcomes, synthetic ontology prompts or the full
router's decisions (`distill_router`, which compresses rules + capability fit + learners + judge into
one model).

```python
from opensmartroute import RouterSLM, SLMStrategy, Router, RouteRequest
from opensmartroute.eval import DatasetCollector, synthetic_rows
from opensmartroute.learning import distill_router

slm = RouterSLM(reg.all(), seed=0)
slm.fit(synthetic_rows(reg.all()))                         # no data yet: seed prompts scored by capability fit
slm.fit(DatasetCollector("data").corpus())                 # collected battle datasets (RouteLLM, LMArena)
student, report = distill_router(Router(reg), texts)       # or copy the whole ensemble
print(report.summary())                                    # accuracy, loss, ECE, Brier, cost/1k
slm.save("slm.json")

router = Router(reg, strategies=[SLMStrategy(RouterSLM.from_file("slm.json"))])
d = router.route(RouteRequest(text="prove that sqrt(2) is irrational"))
print(slm.predict(d.request.text, objective=Objective(cost=2.0))[:3])   # calibrated p, quality, $/1k, utility
```

`SLMStrategy` sits in the ensemble under weight `OSR_WEIGHTS_SLM` and, like the other learners, updates
the model online from every `router.learn(Outcome)`.

Data comes from three collectors that all degrade gracefully when offline:

- `adapters.ModelCatalogue` merges OpenRouter prices and Hugging Face model cards (downloads, likes,
  `model-index` benchmarks) into `ModelCard`s, turns them into `RouteTarget`s (`targets(min_quality=,
  max_usd_per_1k=, measured=True)`), prints a cost/quality benchmark and the Pareto frontier.
  `refresh(leaderboard=True)` adds Open LLM Leaderboard accuracies (IFEval, BBH, MATH, GPQA, MuSR,
  MMLU-Pro) as measured quality; `apply_quality()` takes the Bradley-Terry win rates that
  `eval.model_quality` derives from collected battles. Third-party descriptions are risk-scored and
  replaced when they look like injections.
- `eval.DatasetCollector` pulls routing datasets from the Hugging Face datasets-server into a JSONL cache.
  Ten public sources are on by default: the pairwise battles `routellm-battles` (GPT-4-judged), `arena-55k`,
  `arena-100k`, `arena-140k` (LMArena human votes, with the code / language / math / hard-prompt tags),
  `ppe-human`, `webdev-arena`, `mt-bench-human` and `reward-bench`, the per-model judge scores of
  `ultrafeedback`, and `routellm-gpt4` (RouteLLM's "was Mixtral good enough for this prompt" labels).
  RouterBench / RouterEval / the raw arena conversations are known but gated: download them with your own
  Hub token and load the file with `osr collect --file routerbench=routerbench.jsonl --preset routerbench`
  (`.jsonl` / `.json` / `.csv` in the preset's shape, `--tier` folds model columns onto your targets), and any
  dataset works through `DatasetSource`. Battles name a moving population of models, so `--tier frontier=llm-frontier --tier
  mid=llm-mid --tier small=llm-small` folds every side onto your tiers as it is parsed (`tier_of()`: the
  `ARENA_TIERS` table, name rules for the 2024-26 generations, then the parameter count in the name) - the
  RouteLLM question "was the cheaper tier good enough?". Collection is polite and resumable: pages are paced,
  429s are retried with backoff, and a run the server cuts short keeps its rows and continues from the
  recorded offset on the next `osr collect` (`<name>.meta.json`). `rows_from_feedback` turns your
  `FeedbackStore` into labelled rows.
- `adapters.WebKnowledge` searches Hugging Face (text-generation repos, per-keyword fallback) and
  DuckDuckGo (Brave with `BRAVE_API_KEY`) for new models and datasets; fetches are https-only,
  byte-capped and page text is risk-scored.

`learning.SelfImprover` closes the loop as a champion/challenger cycle: refresh the catalogue, discover,
gather evidence, split, train a challenger, calibrate it on the holdout and promote it only when holdout
accuracy improves by at least `OSR_SLM_MIN_GAIN` (or loss drops at equal accuracy). Every cycle is
appended to a JSONL history.

```bash
osr catalogue --cache data/catalogue.json --hf "llama instruct" --leaderboard --markdown   # live cost / quality
osr collect --cache-dir data --tier frontier=llm-frontier --tier mid=llm-mid --tier small=llm-small
osr -t targets.yaml slm train --out slm.json --cache-dir data --report          # every *-tiers cache, holdout metrics
osr slm predict slm.json "write a python function to parse csv" --cost-weight 1.0
osr -t targets.yaml improve --slm slm.json --cache-dir data --catalogue data/catalogue.json \
    --tier frontier=llm-frontier --tier mid=llm-mid --tier small=llm-small \
    --search "qwen2.5 instruct" --interval 3600                              # daemon; --offline for air-gapped
```

Serve the result: `osr -t targets.yaml --slm slm.json serve`, or in the container `OSR_SLM=/config/slm.json`
(Helm `config.slm`). With `OSR_LLM_BASE_URL` (and `OSR_LLM_API_KEY` / `OSR_LLM_MODEL`) pointing at any
OpenAI-compatible endpoint - OpenAI, Azure OpenAI, vLLM, Ollama, OpenRouter - every LLM target gets a
chat handler from `metadata.model`, so `/v1/chat/completions` with `model: "auto"` routes *and* answers.

#### Speed, and what the numbers mean

Training is plain SGD over hashed features; with numpy installed (extra `fast`, included in the serve
image) the same steps run on dense arrays - 18 000 rows train in under a minute, ten times the pure-Python
speed - and prediction projects through a dense mirror of the weights. `OSR_SLM_BACKEND=auto|numpy|python`
chooses; both paths seed identically and write the same model file, so a zero-dependency deployment reads a
numpy-trained model unchanged. The model carries a learned per-target bias (the base rate a unit-norm dot
product cannot express) and trains with inverse-time learning-rate decay (`OSR_SLM_LR_DECAY`), four epochs
and `l2` 1e-3 by default - on the 18 000-row mixed corpus the old constant-rate, eight-epoch defaults
overfit *below* the constant baseline.

Read `--report` against that baseline. Human preference between tiers is noisy: on the ten-source corpus
"always the mid tier" is acceptable for 64 % of holdout prompts and the SLM reaches 64-66 % with a holdout
cross-entropy of 0.98-1.00 (the constant predictor: 1.03), well calibrated (ECE 0.05-0.08). It is much
stronger where the label really depends on the prompt - 94 % on `routellm-gpt4` (baseline 86 %), +12 points
on `arena-100k` - and no better than the prior on sources where the prompt carries no signal. That is what a
learned routing prior is; the ensemble adds rules, capability, similarity and the bandit on top.

A corpus of model battles only knows the models it compared. Scored rows therefore train a softmax over the
*scored population* (every target any row scores, plus `OSR_SLM_NULL_TARGETS` learned "none of these"
logits that keep it from saturating), so a tool, skill or agent the data never mentions keeps its catalogue
embedding instead of being pushed to zero, and `SLMStrategy` abstains on such targets - its confidence is
scaled by the share of the request's candidates it has evidence about. Rows that name only an `expected`
target (your own feedback, `examples/eval_dataset.jsonl`) contrast against the whole catalogue, which is how
the model learns the non-LLM targets once outcomes arrive.

#### Where a transformer helps

The hashed encoder is a bag of words, bigrams and character grams: it cannot tell "explain this SQL"
from "SQL that explains itself". Two opt-in encoders add that context, both persisted in the same JSON
file and restored by `RouterSLM.from_file`:

- `OSR_SLM_ENCODER=attention` adds `learning.AttentionEncoder`, a pure-Python transformer block
  (hashed token embeddings, sinusoidal positions, `OSR_SLM_ATTENTION_HEADS` heads of
  `OSR_SLM_ATTENTION_HEAD_DIM`, residual, attention pooling) to the query encoder:
  $q = W\,\phi(x) + \operatorname{Attn}(x)$. No dependencies, deterministic, hand-derived gradients,
  but quadratic in `OSR_SLM_ATTENTION_MAX_TOKENS` - a few milliseconds per prompt and roughly four
  times the training time - so measure on your holdout (`osr slm train --report`) before switching.
  On the public arena corpora (six thousand pairwise rows, seventeen tiered targets) it gained two
  points at 1 500 rows but *lost* five at full size with the default budget: the hashed encoder
  stays the default, and attention earns its keep on order-sensitive traffic, not on generic chat.
- `OSR_SLM_EMBEDDER=sentence-transformers/all-MiniLM-L6-v2` (extra `embeddings`) or
  `RouterSLM(embedder=callable)` appends a *frozen* pretrained embedding as dense features
  (`learning.EmbeddingFeaturizer`): `W` then learns a linear head on top of a real transformer while
  the hashed features keep the surface cues (code fences, length, exact vocabulary). The file records
  the embedder's name; a custom callable is passed again to `from_file(embedder=...)`.

#### Self-operation

`osr serve --autopilot` runs the `SelfImprover` inside the server (`learning.Autopilot`): a cycle every
`OSR_SLM_AUTOPILOT_INTERVAL_S` seconds, an early one when `learning.DriftMonitor` (Page-Hinkley over
the success and quality of every outcome the router learns from, via `Router.observers`) sees the served
quality drop, never closer than `OSR_SLM_AUTOPILOT_MIN_GAP_S`. Feedback is joined back to prompts through
the router's request memory (`OSR_SLM_AUTOPILOT_REMEMBER` texts), an accepted challenger is hot-swapped
into the `SLMStrategy` and written back to disk, a failing cycle is logged and the server keeps serving.
`GET /stats` reports `autopilot` (cycles, promotions, drift, last report, next run) and
`POST /autopilot/cycle` schedules one now. In the container: `OSR_AUTOPILOT=1` with `OSR_STATE`
mounted (Helm `autopilot.enabled`); `--offline` keeps it off the network. Each cycle is traced as an
`autopilot.cycle` span with a `learn.improve` event (and `learn.promote` when the challenger wins), so
`GET /events?name=learn.*` shows the loop at work. The hosted platform runs the same loop from
`OSR_PLATFORM_SLM` / `OSR_PLATFORM_AUTOPILOT` and shows it at `/platform/dashboard/learning`
([PLATFORM.md](https://opensmartroute.ai/docs/PLATFORM.md), section 10).

```bash
osr -t targets.yaml --slm slm.json --state state serve --autopilot --autopilot-interval 1800 \
    --catalogue data/catalogue.json --source arena-55k --tier frontier=llm-frontier --tier small=llm-small
curl -s localhost:8000/stats | jq .autopilot
```

## 6. Declaring components with decorators

Targets, strategies, signal extractors, policy rules, middleware and telemetry can be declared where
they live and assembled by a `ComponentRegistry` (details in [SDK.md](https://opensmartroute.ai/docs/SDK.md)):

```python
import opensmartroute as osr

@osr.tool("weather", domains=["travel"], actions=["lookup"], cost=0.0, examples=["weather in paris"])
def weather(request, **kw):
    """Current weather for a city."""          # docstring -> target.description
    return lookup(request.text)

@osr.strategy(weight=0.8)                      # fn(request, signals, candidates) -> {id: score}
def geo_affinity(request, signals, candidates):
    region = request.constraints.region
    return {t.id: 1.0 if region in t.constraints.regions else 0.4 for t in candidates}

@osr.policy_rule                               # fn(target, request, signals) -> reason | None
def business_hours_only(target, request, signals):
    if target.kind is osr.TargetKind.HUMAN and request.context.get("after_hours"):
        return "human queues are closed"

router = osr.components.router(registry)
app = osr.components.builder(registry).with_auto_learning().build()
```

Every tunable (ensemble weights, thresholds, scales) lives in one typed `Settings` object and can be
overridden per process (`osr.configure(...)`) or through `OSR_<GROUP>_<FIELD>` environment variables;
`osr settings` prints the effective values.

## 7. Enterprise builder

```python
from opensmartroute.enterprise import (RouterBuilder, CacheMiddleware, TenantMiddleware,
                                       LoggingTelemetry, MetricsTelemetry, FileAuditSink)
from opensmartroute.security import GuardMiddleware

metrics = MetricsTelemetry()
app = (
    RouterBuilder(registry)
    .with_rules(rules)                              # declarative preferences
    .with_defaults()                                # capability + similarity + Thompson bandit
    .with_auto_learning(state_dir=".osr-state")     # IRT + Bradley-Terry + LinUCB + Markov lookahead
    .with_health(latency_slo_ms=3000)               # circuit breakers, rate limits, budgets
    .with_middleware(GuardMiddleware(redact=True),  # gadget defence + PII redaction
                     TenantMiddleware({"acme": {"max_cost_per_1k": 0.005, "data_boundary": "private"}}),
                     CacheMiddleware(ttl_s=30))
    .with_telemetry(LoggingTelemetry(), metrics)
    .with_audit(FileAuditSink("audit.jsonl"))
    .build()
)

d = app.route(RouteRequest("Summarize this contract", constraints=RequestConstraints(tenant="acme")))
app.learn(Outcome(d.request_id, d.target.id, success=True, quality=0.9, cost_usd=0.002, latency_ms=1200,
                  domains=d.trace.signals.domains, complexity=d.trace.signals.complexity))
print(metrics.snapshot()["route_latency_ms"], app.health_snapshot())
```

`with_retrieval(narrow_above=, narrow_to=)` switches to retrieve-then-rank for large catalogues,
`with_queue_awareness()` replaces static latency with a queue-aware estimate, `with_shadow(candidate,
mode="shadow" | "ab")` compares policies with SPRT, `with_calibration()` adds temperature scaling and
conformal candidate sets, `with_fair_share()` enforces tenant weights and `with_state_store()` accepts
Redis, SQL, encrypted, versioned or batched stores. Ports (state, telemetry, audit) are ABCs; see
[ENTERPRISE.md](https://opensmartroute.ai/docs/ENTERPRISE.md).

## 8. Research-track modules

Each module below implements one roadmap item with tests; papers are cited in
[RESEARCH.md](https://opensmartroute.ai/docs/RESEARCH.md) and formulas in [MATH.md](https://opensmartroute.ai/docs/MATH.md).

### Signals from a draft or a small model

```python
from opensmartroute.signals import (DEFAULT_EXTRACTORS, VerbalisedDifficultySignal,
                                    DraftResponseSignal, extract_signals)

extractors = [*DEFAULT_EXTRACTORS,
              VerbalisedDifficultySignal(source=lambda req: small_model_difficulty(req.text)),
              DraftResponseSignal(drafter=lambda req: cheap_model(req.text))]
router = Router(registry, extractors=extractors)
```

`VerbalisedDifficultySignal` accepts numbers, `7/10` or words (`trivial` ... `expert`) from
`context["difficulty"]` or a callable and blends them into `complexity`. `DraftResponseSignal`
measures hedging, self-corrections, query overlap and length ratio of a cheap draft.

### Response-side uncertainty

```python
from opensmartroute.signals import UncertaintyGate, EventTrigger, TriggerRule, response_uncertainty
from opensmartroute.strategies import Cascade

gate = UncertaintyGate(sampler=lambda req, n: [cheap_model(req.text) for _ in range(n)], n_samples=4)
cascade = Cascade(quality_gate=gate, threshold=0.6)                   # escalate when samples disagree
result = cascade.run(req, router.route(req).alternatives)

feats = response_uncertainty(samples, p_true=judge_p_true)
trigger = EventTrigger([TriggerRule("semantic_entropy", 0.5, "reroute"),
                        TriggerRule("agreement", 0.5, "ask_human", below=True)])
trigger.check(feats)                                                  # ["reroute", ...]
```

### Hidden-state Dirichlet probe

```python
from opensmartroute.strategies import HiddenStateStrategy

probe = HiddenStateStrategy(state_fn=lambda req: host_model.hidden_state(req.text),
                            targets=["small", "large"], dim=4096)
probe.fit(logged_states, logged_winners, epochs=20)                   # warm start
router = Router(registry, strategies=[*default_strategies(), probe])
```

The probe's confidence is `1 - K/S` (epistemic uncertainty), so unfamiliar states defer to the other
strategies. `math.DirichletProbe` is usable on its own with any feature vector.

### Elastic models and token budgets

```python
from opensmartroute.strategies import BudgetVariant, TokenBudgetStrategy, expand_elastic

for sib in expand_elastic(registry.get("reasoner"),
                          [BudgetVariant(1024, quality=0.7, cost_scale=0.4),
                           BudgetVariant(8192, quality=0.9)]):
    registry.add(sib)                                                 # reasoner@1024, reasoner@8192
router = Router(registry, strategies=[*default_strategies(), TokenBudgetStrategy()])
```

### Edge and cloud tiers

Tag targets with `metadata: {tier: edge}` or `{tier: cloud}` and add `EdgeCloudStrategy()`. Edge
success is a Thompson posterior per complexity bucket, penalised when decode time eats into the
latency SLO; the cloud tier is penalised by upload time and excluded when the request's
`data_boundary` is `private` or `on_prem` and the target is `public`.

### Auctions

`AuctionStrategy(bid_fn=None, value=0.01)` lets each target bid a claimed success probability and a
price (`default_bid` derives both from the catalogue). Claims are corrected by each bidder's observed
bias; the highest corrected surplus wins and pays the second price. Supply a `bid_fn` for
multi-vendor catalogues where providers self-report.

### Protocol selection, aggregation, self-escalation, hand-off

```python
from opensmartroute.strategies import ProtocolPolicy, MixtureOfAgents, SelfEscalation, wrap_stream
from opensmartroute.learning import HandoffPolicy, MixtureCureModel

policy = ProtocolPolicy()                                             # risk / budget -> protocol
choice = policy.choose(req, decision, decision.trace.signals, uncertainty=gate(req, draft))
if choice.protocol == "aggregate":
    res = MixtureOfAgents(k=3, budget_usd=0.05).run(decision, req)   # top-k answers, majority vote
    for o in res.outcomes(req):
        router.learn(o)

monitor = SelfEscalation.for_target(decision.target)                # streaming competence posterior
streamed = wrap_stream(provider.stream(req.text), monitor)
if streamed.escalated:
    ...                                                               # restart on a stronger target

handoff = HandoffPolicy(MixtureCureModel(), fallback_target="senior-agent", pins=task_pins)
if handoff.step(task_id, teacher_risk):                               # permanent hand-off
    target = handoff.target_for(task_id, decision.target.id)
handoff.finish(task_id, failed=False)                                 # censored observation
```

### Personalisation

`learning.UserAdaptiveStrategy` reads `context["user_id"]` (or `profile["user_id"]`) and keeps a
Beta posterior per user and target, shrunk toward similar users and the global posterior;
`observe_user()` warm-starts from logged interactions. `learning.SkillAffinity.relevance(profile,
skills)` feeds `retrieval.select_skill_set(relevance=)` so skill sets depend on who is asking.

### Multi-turn conversations

```python
from opensmartroute.learning import HistoryTargetStrategy

router = Router(registry, strategies=[*default_strategies(), HistoryTargetStrategy(turns=6)])
d = router.route(RouteRequest("can you explain more?", history=chat_history,
                              context={"last_target": "llm-small", "last_failed": True}))
```

The same follow-up message routes differently depending on what the conversation has been about:
the strategy learns a logistic model over the joint embedding of the recent history and each target's
catalogue embedding, so a target that keeps failing in legal threads but succeeds in coding threads
is scored accordingly, and an unseen target is still scored through its description and examples.
`last_target` gives the incumbent a small continuity bonus that turns into a penalty when the previous
turn failed.

### Discovery beyond text similarity

```python
from opensmartroute.discovery import SchemaAwareStrategy, CachePreservingSelector, SkillGraph

router = Router(registry, strategies=[*default_strategies(), SchemaAwareStrategy()])

selector = CachePreservingSelector(prefix_size=8, evict_after=5)
tools = selector.select(session_id, candidates, needed=[d.target.id])   # prefix-stable tool order

graph = SkillGraph(load_skills("skills"))                               # osr-requires / osr-conflicts
ordered, dropped = graph.compose(["report-builder"])
```

### MCP server recommendation and semantic-router import

```python
from opensmartroute.adapters import recommend_servers, load_semantic_router_config

for rec in recommend_servers("open a pull request", server_cards, k=2,
                             constraints=req.constraints, allowed_auth=["none", "token"]):
    print(rec.server.name, rec.score, rec.rationale)

imported = load_semantic_router_config("config.yaml")                  # vLLM semantic-router
router = Router(imported.registry, strategies=[RulesStrategy(imported.rules), *default_strategies()])
```

### Energy and carbon

`math.EnergyModel` fits Wh per target from measured samples; `math.hardware_profile("h100-sxm")`
gives priors when no meter exists. Both convert to gCO2 with a grid factor and feed
`wh_per_1k_tokens` / `gco2_per_1k_tokens` on the target's `cost`, which `Objective(energy=, carbon=)`
trades against quality.

## 9. Command line

Install `osr` with the platform installer (`curl -LsSf https://opensmartroute.ai/install.sh | sh` on
Linux and macOS, `irm https://opensmartroute.ai/install.ps1 | iex` in PowerShell) or with
`uv tool install 'opensmartroute[yaml,server]'` (add `fast` for numpy-accelerated SLM training); `osr --version`
confirms the install.

`osr` takes the catalogue with `-t targets.yaml`, optional rules with `-r rules.yaml` and an optional
routing SLM with `--slm slm.json`; `--help` on any subcommand lists its options. Grouped by task:

```bash
# Sign in and manage access tokens
osr login                                                            # browser sign-in to the hosted platform
osr login --url https://osr.example.com --profile work               # another deployment, kept as a profile
osr login --url http://router:8000 --token osr_local_...             # self-hosted osr serve token
echo "$TOKEN" | osr login --with-token                               # non-interactive (CI)
osr whoami                                                           # workspace, plan, edition, token source
osr token generate                                                   # mint an osr_local_ token for osr serve
osr token create --name ci                                           # new platform API key (shown once)
osr token list                                                       # keys of the signed-in workspace
osr token revoke <key-id>
osr logout                                                           # forget the profile (--all for every one)
# Route, quote and inspect
osr -t examples/targets.yaml -r examples/rules.yaml route "I want a refund for order #123" --plan
osr -t examples/targets.yaml route "Summarise this HR complaint" \
    --constraint data_boundary=on_prem --constraint region=eu     # hard constraints; --kinds agent,tool
osr -t examples/targets.yaml -r examples/rules.yaml route "" --event order.shipped --plan   # textless event
osr -t examples/targets.yaml estimate "Translate this contract" --monthly 100000   # cost per candidate + projection
osr -t examples/targets.yaml targets                                 # the catalogue as the router sees it
osr -t examples/targets.yaml stats                                   # feedback statistics per target
osr settings --json                                                  # effective tunables

# Evaluate and audit
osr -t examples/targets.yaml -r examples/rules.yaml eval examples/eval_dataset.jsonl --frontier --calibration
osr -t examples/targets.yaml audit traffic.jsonl --baseline llm-frontier --monthly 3000000 --markdown
                                                                     # Routing Audit: savings + violations
osr -t examples/targets.yaml ope decisions.jsonl                     # off-policy estimate (IPS/SNIPS/DR) of logged decisions
osr -t examples/targets.yaml safety --learned-guard                  # red-team suite

# Learn
osr train --from rows.jsonl --out models.json                        # fit signal models
osr catalogue --cache data/catalogue.json --hf "llama instruct"      # live model prices + quality
osr collect --cache-dir data --tier frontier=llm-frontier --tier mid=llm-mid --tier small=llm-small   # Hub datasets
osr collect --cache-dir data --file routerbench=routerbench.jsonl --preset routerbench             # a gated file you hold
osr -t examples/targets.yaml slm train --out slm.json --report       # routing SLM (see section 5)
osr slm info slm.json                                                # rows, sources, encoder, calibration, fit history
osr -t examples/targets.yaml --slm slm.json route "parse this csv"   # SLM inside the ensemble
osr slm eval slm.json rows.jsonl                                     # SLM accuracy on labelled rows
osr slm predict slm.json "parse this csv" --top 3                    # what the SLM alone would rank
osr slm info slm.json                                                # SLM metadata, targets and size
osr -t examples/targets.yaml improve --slm slm.json --cache-dir data # self-improvement cycle

# Serve and integrate
osr -t examples/targets.yaml -r examples/rules.yaml serve            # http://127.0.0.1:8000/docs (open, local dev)
osr -t examples/targets.yaml serve --generate-token                  # prints an osr_local_ token; all routes need it
osr -t examples/targets.yaml serve --require-auth --token "$TOKEN"   # production: refuse to start without a token
osr -t examples/targets.yaml mcp                                     # MCP server over stdio for an IDE
osr mcp --remote                                                     # stdio bridge to the platform you signed in to
osr mcp --url https://<platform>/mcp --api-key osr_...               # ...or an explicit endpoint and key
osr mcp-manifest tools.json --key-env OSR_MCP_KEY --out signed.json  # sign an MCP tools/list (--verify to check)

# Manifests, stacks and skills
osr validate examples/ocm                                            # Open Capability Manifests
osr -t examples/targets.yaml export-ocm --out manifests/             # catalogue -> OCM manifests
osr stack init --out stack.yaml                                      # starter declarative stack
osr stack validate examples/stack.yaml                               # check (imports resolved)
osr stack plan examples/stack.yaml --against deployed.yaml           # diff against what runs today
osr stack apply examples/stack.yaml --route "refund order 42?"       # build the router, route once
osr stack apply registry://support-desk@1.0.0 --registry $OSR        # marketplace template
osr skills                                                           # validate SKILL.md packages
```

`osr login` without `--token` runs the OAuth 2.0 device authorization grant against the hosted platform
(community or enterprise edition alike): the CLI prints a short code, opens `/platform/cli/authorize` in the
browser, and after you approve it there the platform mints a workspace API key (`osr_live_...`) for
that machine. Credentials are stored per profile in `~/.config/opensmartroute/credentials.json`
(`%APPDATA%\opensmartroute` on Windows, `OSR_CONFIG_DIR` overrides); flags, then `OSR_API_URL` /
`OSR_API_KEY`, then the saved profile decide which one a command uses. A self-hosted `osr serve` has
no accounts: `osr token generate` (or `serve --generate-token`) mints an `osr_local_...` token, the
server accepts it as `Authorization: Bearer` or `X-API-Key` on every route except the health,
metrics, `/whoami` and OpenAPI paths, and `OSR_SERVER_AUTH_TOKENS` / `OSR_SERVER_REQUIRE_AUTH`
configure the same thing from the environment (see :class:`ServerSettings` in
[SDK.md](https://opensmartroute.ai/docs/SDK.md#settings)).

The Routing Audit replays logged traffic (`{"text": ..., "model": ..., "cost_usd": ...}` per line)
through the catalogue and reports savings against a baseline target, policy violations the logged
route would have committed, and the route mix; `--min-savings` makes it a CI gate. In production the
same numbers come from `RouterBuilder.with_savings(baseline="max")` and
`ep.savings.report().to_markdown()`.

## 10. Routing latency

Routing is pure Python on the request path. Measured with `python scripts/bench.py --n 200 --scale`
on the 16-target example catalogue, CPython 3.11, x86-64 laptop:

| Configuration | p50 | p95 | p99 |
|---|---|---|---|
| signals + policy + rules + capability + similarity + bandit | 5.8 ms | 6.2 ms | 6.5 ms |
| + IRT + preference + LinUCB + Markov + health | 8.6 ms | 12.9 ms | 17.8 ms |
| + LLM judge (only when `confidence < escalate_llm_judge_below`) | + one provider round-trip | n/a | n/a |

Catalogue size: 64 targets route in 14.5 ms p50 and 256 in 16.7 ms with
`RouterBuilder.with_retrieval(narrow_above=32, narrow_to=24)` (versus 36 ms and 142 ms scoring every
target); 1024 targets route in 24 ms. Cost is linear in `scored targets x strategies`; signal
extraction is under 1 ms and independent of catalogue size. `RouteDecision.trace.elapsed_ms` records
the routing time per request, `MetricsTelemetry` exposes `route_latency_ms` percentiles and the
OpenTelemetry adapter emits the `osr.route_latency_ms` histogram. Full tables in
[ARCHITECTURE.md](https://opensmartroute.ai/docs/ARCHITECTURE.md#performance-envelope).

## 11. Repository layout

```
src/opensmartroute/
  core/        types (RouteTarget, RouteRequest, RouteDecision, Outcome, ...), registry
  signals/     lexicon extractors, task ontology, learned hashed models, verbalised difficulty,
               draft-response and response-uncertainty features
  policy/      hard-constraint filter
  strategies/  rules, capability, similarity, bandit, llm_judge, cascade, task_table, defer, progress,
               probe (Dirichlet), elastic, edge, auction, protocol, escalation, aggregate
  learning/    IRT, preference, LinUCB, Markov, AutoLearner, cold start, task credit,
               personal (user adaptation, skill affinity), handoff (mixture cure),
               multiturn (history-target joint embeddings)
  math/        bandits, irt, preference, markov, calibration, estimators, decision, dirichlet, energy
  realtime/    circuit breaker, rate limit, budget, health
  enterprise/  RouterBuilder, EnterpriseRouter, middleware, telemetry, audit, stores, ops
  security/    InputGuard, learned gadget detector, injection risk, redaction, limits, provenance
  adapters/    OpenAI-compatible client, harnesses, MCP (tools + server recommendation), A2A,
               frameworks, personas, SKILL.md, semantic-router import, embeddings, OpenTelemetry
  eval/        harness, baselines, robustness, frontiers, off-policy evaluation, dataset presets
  discovery.py schema-aware matching, cache-preserving selection, skill graph
  retrieval.py BM25 + dense retrieval, reciprocal-rank fusion, skill-set selection, meta-tools
  execution.py plan executor
  sdk.py       decorator SDK and plugin discovery
  settings.py  typed, environment-overridable tunables
  config.py    load_targets / load_rules (JSON, YAML)
  feedback/    outcome store and preference export
  router.py    orchestrator and plan builder
  aio.py       AsyncRouter
  server.py    FastAPI app and OpenAI-compatible proxy
  cli.py       osr command
deploy/        Dockerfile, entrypoint, Helm chart
examples/      targets.yaml, rules.yaml, skills/, eval_dataset.jsonl, demos
.claude/skills Agent-Skills packages for coding agents; also routable skill targets
tests/         unit and integration tests, public API snapshot, docs coverage gate (test_docs.py)
scripts/       bench.py, release.py, brand_build.py, api_reference.py, exit_criteria.py
docs/          GUIDE, REFERENCE (generated), RESEARCH, ARCHITECTURE, MATH, ENTERPRISE, SECURITY, SDK, ROADMAP, BRAND
```

<!-- Section: Hosted platform -->

---

<!-- Page: Platform guide (https://opensmartroute.ai/docs/PLATFORM) -->

<!-- OpenSmartRoute: Platform guide. Source https://github.com/isathish/OpenSmartRoute/blob/main/docs/PLATFORM.md; HTML https://opensmartroute.ai/docs/PLATFORM -->
# Platform guide

The hosted platform runs OpenSmartRoute as a service. A workspace has an API key and a plan; each
request to the API is routed to a target (model, agent, skill, persona or tool) and answered with the
decision, its trace and, on request, the target's output. An OpenAI-compatible endpoint accepts
existing chat-completion clients.

This page covers calling the platform: authentication, routing, execution, the OpenAI-compatible
endpoint, feedback, plans, organizations, tenants, governance (workspace policy and budgets),
observability (probes, metrics, per-request traces and the event stream), caching, the dashboard and
the MCP server. The
[marketplace](https://opensmartroute.ai/docs/MARKETPLACE.md) - shared agents, skills, personas, prompts and stack templates - has
its own page. To run the router on your own infrastructure see
[Deploy with Docker and Helm](https://opensmartroute.ai/docs/deploy.md); to use it as a Python library see the
[user guide](https://opensmartroute.ai/docs/GUIDE.md) and the [SDK guide](https://opensmartroute.ai/docs/SDK.md). Every endpoint below is listed in the REST API
reference (`/docs/api`) and in `/openapi.json`.

## 1. Authentication

1. Sign up at `/platform/signup` with GitHub, Google or an email address - with a password if you want to sign
   in by email later (`OSR_PLATFORM_PASSWORD_LOGIN`, on by default). Signup creates a personal workspace
   on the **free** plan and issues an API key.
2. The key is shown once. Store it in a secret manager. Keys are created, renamed, rotated and revoked in
   the dashboard (`/platform/dashboard/keys`) or with `POST /api/v1/keys`, `PATCH /api/v1/keys/{key_id}`
   `{name}`, `POST /api/v1/keys/{key_id}/rotate` (a new secret for the same key, shown once; the old one
   stops working immediately) and `DELETE /api/v1/keys/{key_id}`.
3. Send the key with every metered request in either header:

```http
Authorization: Bearer osr_...
X-API-Key: osr_...
```

Public endpoints (`/api/v1/info`, `/api/v1/models`, `/api/v1/llms`, `/api/v1/rankings`,
`/api/v1/catalogue`, `/api/v1/estimate`, `/api/v1/stats/public`, `/api/v1/registry`) need no key.

### Email and password

`POST /api/v1/auth/password/login` `{email, password}` returns the same browser session as a
single sign-on callback (`session_token`, `user`, `account`, `plan`); the dashboard stores it and
sends it as a bearer token. Passwords are at least 8 characters (no composition rules) and are stored
as salted scrypt hashes. A signed-in user sets or changes theirs with `POST /api/v1/auth/password`
`{password, current_password}` - the current one is required once set and every other browser is
signed out - and removes it with `DELETE /api/v1/auth/password` as long as a provider identity remains.
Sign-in attempts are limited per address and per email; `has_password` on the user object says which
users have one. `/platform/dashboard/account` has the form.

### Account lifecycle: confirmation, recovery, deletion

Anyone can create an account with any email address. What happens around it:

| Step | How | Endpoints |
|---|---|---|
| Email confirmation | Signup sends a link to `/platform/verify-email?token=...` (valid three days). Signing in through a provider that vouches for the address, accepting an invitation sent to it or completing a password reset also confirms it. `email_verified` on the user object; `/platform/dashboard/account` shows a banner with *Send again* until it is done. | `POST /api/v1/auth/verify` `{token}`, `POST /api/v1/auth/verify/send` (signed in, once a minute) |
| Forgot password | `/platform/forgot-password` emails a single-use link to `/platform/reset-password?token=...` (one hour). The answer is always `202` so it never reveals whether an address has an account; limited per address and per email. Resetting revokes every session, confirms the address, signs the person in and sends a notice. | `POST /api/v1/auth/password/forgot` `{email}`, `POST /api/v1/auth/password/reset` `{token, password}` |
| Invitations | `POST /api/v1/workspace/invites` emails the invitee a link to `/platform/invite/<token>` (fourteen days) and returns the same link once to the inviter. | see [Organizations](#7-organizations) |
| Account history | The person's own sign-ins, password and email changes and memberships, newest first. | `GET /api/v1/me/audit` |
| Download my data | One JSON document with everything held about the person and the current workspace: profile, identities, sessions, memberships and history; the workspace with plan, members, key prefixes, tenants, policy, SSO, usage, the request log and reported outcomes of up to a year. Prompt text is never stored, so none is included. *Download my data* on `/platform/dashboard/account`. | `GET /api/v1/me/export?days=` |
| Delete account | *Delete my account* on `/platform/dashboard/account`: type the email (and the password when one is set). Sessions, linked identities and memberships go, workspaces only this user belonged to are disabled and their keys revoked; an organization with other members needs another owner first. A notice is emailed. | `POST /api/v1/me/delete` `{confirm, password}` |
| Getting started | The checklist of a workspace: e-mail confirmed, API key created, first request routed, first outcome reported, a model provider connected and - for organizations - a teammate invited; each step carries the dashboard link and the API call that completes it, `next` is the first open one and `complete` the share done. The dashboard overview shows it until every step is done. | `GET /api/v1/onboarding` |

Signup, confirmation, password recovery, invitation links and this checklist form the `onboarding`
domain, which a deployment can run as its own service (see [Services](#services)).

Every one of these events - plus sign-ins, failed sign-ins, role changes, removals and operator actions
- is written to the account audit log. Workspace admins read their workspace's entries with
`GET /api/v1/workspace/audit`; operators see everything at `/platform/admin/audit` (`GET /api/v1/admin/audit-log`)
and per user on `/platform/admin/users/<id>`, where they can also issue a reset link, re-send or force the email
confirmation, disable, sign out everywhere or delete the user.

Email is delivered through the SMTP server in `OSR_PLATFORM_SMTP_URL`. Without one, nothing is lost:
every message is kept in the outbox and operators pick the links up at `/platform/admin/mail`
(`GET /api/v1/admin/mail`); `mail` in `GET /api/v1/info` tells the web app which case applies.

### Command line sign-in

Install the CLI (`curl -LsSf https://opensmartroute.ai/install.sh | sh`, or
`irm https://opensmartroute.ai/install.ps1 | iex` on Windows) and run `osr login`. The CLI shows an
eight-character code and opens `/platform/cli/authorize`; sign in there, check that the code and computer
name match, choose the key name and approve. The platform mints a new workspace API key for that
machine and the CLI stores it in `~/.config/opensmartroute/credentials.json`. This is the OAuth 2.0
device authorization grant (RFC 8628) and works the same for the community and enterprise editions:

| Step | Endpoint | Notes |
|---|---|---|
| CLI asks for a code | `POST /api/v1/auth/device/code` `{client_name}` | anonymous, rate limited per IP; returns `device_code`, `user_code`, `verification_uri[_complete]`, `expires_in` (900 s), `interval` (5 s) |
| Browser looks it up | `GET /api/v1/auth/device/{user_code}` | signed in; shows client name, status and expiry |
| Browser decides | `POST /api/v1/auth/device/approve` / `deny` `{user_code, name}` | approving needs the `admin` role and a free key slot in the plan |
| CLI polls | `POST /api/v1/auth/device/token` `{device_code}` | `authorization_pending`, `slow_down`, `access_denied`, `expired_token` as RFC 8628 errors; success returns `access_token` (the `osr_live_` key), `account`, `plan`, `edition` |

Only approve codes from a terminal you started yourself. Keys minted this way appear on
`/platform/dashboard/keys` with the chosen name and can be revoked like any other. Pasting a key works too
(`osr login --token osr_live_...`, or `--with-token` from stdin in CI); `osr whoami` shows the
workspace, plan and edition behind the stored credential, `osr token create|list|revoke` manage keys
and `osr mcp --remote` bridges an IDE to this workspace's MCP server without copying the key around.

## 2. Route a request

`POST /api/v1/route` returns a decision: the chosen target, the confidence, ranked alternatives and
the signals extracted from the request. The target is not called unless `execute` is set.

```bash
curl -s "$OSR_URL/api/v1/route" \
  -H "Authorization: Bearer $OSR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"text": "Prove that sqrt(2) is irrational.", "top_k": 3}'
```

```json
{
  "request_id": "7f3c...",
  "target": {"id": "llm-frontier", "kind": "llm", "name": "Frontier model"},
  "confidence": 0.81,
  "alternatives": [{"id": "llm-mid", "kind": "llm", "utility": 0.62}],
  "signals": {"complexity": 0.74, "domains": ["math"], "reasoning_need": 0.9, "contains_pii": false, "...": "..."},
  "ranked": [{"id": "llm-frontier", "utility": 0.79, "quality_estimate": 0.91, "breakdown": {"...": "..."}}],
  "policy_rejections": {},
  "explanation": "llm-frontier: high reasoning need, math domain, quality weight dominates ...",
  "executable": true,
  "elapsed_ms": 1.9
}
```

Request fields:

| Field | Type | Meaning |
|---|---|---|
| `text` | string, required | The user request. |
| `history` | list of `{role, content}` | Prior turns; the router reads them for intent, task type and session affinity. |
| `context` | object | Free-form facts (`app`, `session_id`, `user_locale`, ...). Keys starting with `_` are dropped. |
| `objective` | object | Relative weights: `quality` (default 1.0), `cost` (0.15), `latency` (0.05), plus `quality_floor`, `energy`, `carbon`. `cost_weight` is accepted as a synonym of `cost`. |
| `constraints` | object | Hard constraints, never traded off: see below. |
| `profile` | object | Opaque user/tenant features (locale, expertise, preferences). Hashed into the learners; never logged raw. |
| `kinds` | list | Restrict to target kinds: `llm`, `agent`, `skill`, `persona`, `tool`, `human`. |
| `top_k` | int, default 3 | How many alternatives to return. |
| `plan` | bool | Build a persona → skill → model plan instead of choosing a single target (**pro** plan and above). |
| `execute` | bool | Route and run the chosen target or plan; the output is returned in `result` (**pro** and above). |

Constraints (`constraints` object):

| Key | Effect |
|---|---|
| `max_cost_per_1k` | Drop targets whose price per 1k tokens is above this. |
| `max_latency_ms` | Drop targets slower than this. |
| `preferred_max_latency_ms` | Soft target: targets whose observed p90 latency is above it are penalised, not dropped. |
| `region` | Only targets served from this region. |
| `data_boundary` | Only targets at least this strict (`public` < `private` < `on_prem`). |
| `contains_pii` | Force the PII handling on or off (`null` lets the sensitivity signal decide). |
| `allow_targets` / `deny_targets` | Explicit allow / deny lists of target ids. |
| `allowed_kinds` | Same as `kinds`. |
| `require_tools` | Only targets that can call tools. |

### Quote before routing

`POST /api/v1/estimate` takes the same body and returns the price of each candidate without calling
anything: `targets[]` with `cost_usd`, `input_tokens`, `output_tokens`, vendor and model, plus the
router's `recommended` pick and the `cheapest`, `best_quality` and `fastest` candidates.
`output_tokens` overrides the assumed reply length; `monthly_requests` adds a `monthly` projection
(`recommended_usd`, `cheapest_usd`, `dearest_usd`, `savings_usd`). Without a key it is rate limited
per client and ignores tenant rules; with a key it is metered as `estimate` and applies your plan
and tenants. `GET /api/v1/models/recommended` returns the router's picks per use case from live
quotes. Details, the SDK form and the `/estimate` page: [MCP.md](https://opensmartroute.ai/docs/MCP.md).

## 3. Execute the chosen target

With `"execute": true` the platform runs the chosen target (or plan) with the configured providers and
returns the output, the steps that ran and the metered cost:

```bash
curl -s "$OSR_URL/api/v1/route" -H "Authorization: Bearer $OSR_API_KEY" -H "Content-Type: application/json" \
  -d '{"text": "Summarise this contract clause for a non-lawyer: ...", "execute": true, "objective": {"cost": 0.4}}'
```

The response adds `result` with `text`, `target_id`, `steps[]` (`role`, `target_id`, `ok`,
`latency_ms`), `tokens` and `cost_usd`.

### Model providers

A target executes when it is mapped to a model on an **OpenAI-compatible endpoint** - OpenAI, Azure OpenAI,
OpenRouter, Mistral, Groq, Together, Fireworks, DeepSeek, a local Ollama, vLLM or LiteLLM, anything that
speaks `/v1/chat/completions`. Two sources define the mapping and are merged (the console wins on the same
provider name or target):

* the **file** the deployment mounts, `OSR_PLATFORM_PROVIDERS` (YAML/JSON path or inline JSON) with
  `providers.<name>` (`base_url`, `api_key` or `api_key_env`, optional `api_key_header`, `extra_headers`,
  `timeout_s`, `max_retries`) and `models.<target_id>` (`provider`, `model`, optional `system_prompt` and
  request defaults). On first start it is imported into the console once, so it can be edited there.
* the **operator console**, `/platform/admin/providers`: add an endpoint from a preset, store the key inline or name
  the environment variable that holds it, *Check* it (`GET <base_url>/models` with its credentials records
  reachability, latency and the upstream model ids, which the mapping form then offers), map targets to
  upstream models, disable or delete. Every write re-binds the handlers in the API at once, and every other
  API replica re-reads the store within `OSR_PLATFORM_PROVIDERS_RELOAD_S` (30 s) - no restart, no redeploy.

`GET /api/v1/providers` is the public, OpenRouter-style view of what a deployment executes on, rendered
by the site at `/providers`: one row per
executable target with `provider`, `model`, the recognised reference model, `input_usd_per_1m` /
`output_usd_per_1m` (from the reference catalogue when the upstream model is known, else the target's
declared cost), context window, tool and reasoning support, circuit-breaker `health` and seven-day traffic;
plus each provider's host, kind, health and model count. Credentials never appear: the console shows only
whether a key is set and its last four characters (`api_key_set`, `api_key_hint`). Keys stored through the
console live in the platform database next to the organization SSO secrets; prefer `api_key_env` when the
host injects secrets. Endpoints: `GET|POST /api/v1/admin/providers`, `GET|PATCH|DELETE
/api/v1/admin/providers/{id}`, `POST /api/v1/admin/providers/{id}/check`, `PUT|DELETE
/api/v1/admin/providers/models/{target_id}`. Targets without a mapping still route (decisions, plans,
traces, quotes) and report `executable: false`.

### Response headers

| Header | On | Meaning |
|---|---|---|
| `X-Request-Id` | every response | The id to quote in feedback and support requests; echoed back when you send one. |
| `Server-Timing` | every response | `app;dur=<ms>` - time spent inside the platform. |
| `X-OSR-Target` | `/v1/chat/completions` | The target id that answered. |
| `X-OSR-Confidence` | `/v1/chat/completions` | The router's confidence in that choice. |
| `Retry-After` | `429` | Seconds until the per-minute window, the daily quota or the budget period resets. |
| `X-Quota-Limit` | `429` (daily) | Your plan's requests per day. |
| `X-Budget-Limit`, `X-Budget-Used`, `X-Budget-Period` | `429` (budget) | The exhausted workspace or tenant budget in USD and its period (`daily` / `monthly`). |
| `X-Upgrade: true` | `403` | The feature exists but is not in your plan. |
| `ETag`, `Cache-Control` | public catalogue reads | Conditional requests: send `If-None-Match` and receive `304` when nothing changed. |

## 4. OpenAI-compatible endpoint

Point an OpenAI SDK at `$OSR_URL/v1` and set `model` to `auto`. The router chooses the target per
request, runs it and returns a standard `chat.completion` whose `model` field is the target that
answered. Decision metadata is added in an `opensmartroute` object that other clients ignore.

```python
from openai import OpenAI

client = OpenAI(base_url=f"{OSR_URL}/v1", api_key=OSR_API_KEY)
reply = client.chat.completions.create(
    model="auto",
    messages=[{"role": "user", "content": "Write a haiku about routing."}],
)
print(reply.model)                 # e.g. "llm-small" - the target the router chose
print(reply.choices[0].message.content)
```

```json
"opensmartroute": {
  "request_id": "7f3c...", "target": "llm-small", "kind": "llm", "confidence": 0.88,
  "plan": [], "alternatives": ["llm-mid"], "cost_usd": 0.00012, "latency_ms": 412.5,
  "fallback_from": []
}
```

Options outside the OpenAI schema go in the request body and are ignored by other providers:

| Field | Meaning |
|---|---|
| `model` | `auto` (default) lets the router choose; a target id pins that target. |
| `models` | Candidate list: the router chooses among them and falls back down the list when a target fails. |
| `osr.objective` / `osr.constraints` | Same as on `/api/v1/route`. |
| `osr.tenant` | Tenant slug (or the `X-OSR-Tenant` header); applies the tenant's constraints (**enterprise**). |
| `osr.plan` | `true`/`false` to force or suppress plan composition; omitted = whatever the plan tier allows. |
| `osr.fallbacks` | `false` disables the automatic retry on the next best target (default `true`). |
| `stream` | `true` streams server-sent events in the OpenAI format: each provider token is relayed as it arrives; the last chunk carries `usage` and the `opensmartroute` metadata. Quota, routing and provider errors before the first token are ordinary HTTP errors; a fallback to another target happens only before the first token. |

`GET /v1/models` lists `auto` and every executable target.

## 5. Feedback

`POST /api/v1/feedback` reports how a routed answer turned out. The learners (Bradley-Terry, IRT,
LinUCB) update on the workspace's own traffic. Available on the **pro** plan and above.

```bash
curl -s "$OSR_URL/api/v1/feedback" -H "Authorization: Bearer $OSR_API_KEY" -H "Content-Type: application/json" \
  -d '{"request_id": "7f3c...", "target_id": "llm-frontier", "success": true, "quality": 0.9, "latency_ms": 2100}'
```

Fields: `request_id`, `target_id`, `success` (required); `quality` (0..1), `cost_usd`, `latency_ms`,
`domains`, `complexity`, `preferred_over` (the id of a target this one beat in a comparison).

Every report is kept against its request: `GET /api/v1/activity` summarises it per row (`outcome`),
`GET /api/v1/trace/{request_id}` lists the reports (`outcomes`) next to the spans, and in the enterprise
edition the audit chain records an `outcome` entry after the `decision` entry (section 10). The dashboard
has the same form without curl: expand a row on `/platform/dashboard/activity` (or open its trace) and
report good / poor, a quality score and, optionally, the target that would have done better - the latter
is recorded as a pairwise preference for the Bradley-Terry learner.

## 6. Plans and quotas

| Plan | Requests / day | Requests / min | Keys | Features |
|---|---|---|---|---|
| **free** | 500 | 30 | 2 | route + trace, OpenAI-compatible proxy |
| **pro** | 20,000 | 300 | 10 | + plans (`plan`/`execute`), feedback |
| **enterprise** | 500,000 | 3,000 | 100 | + tenants, `/stats`, hash-chained `/audit`, organization SSO, 50 tenants, 500 seats |

`GET /api/v1/info` (`plans`) returns the values for the deployment you are calling; `GET /api/v1/me`
returns your plan. Upgrades are made in the dashboard (`/platform/dashboard/billing`).

* `429 rate limit exceeded` - per-minute window; wait `Retry-After` seconds.
* `429 daily quota exhausted` - the day's budget; `X-Quota-Limit` shows the ceiling.
* `403` with `X-Upgrade: true` - the feature is not in your plan.
* `404 ... requires the enterprise edition` - the deployment runs the community image, which does not
  expose tenants, statistics or audit regardless of plan.

## 7. Organizations

Signup creates a personal workspace. An **organization** is a second kind of workspace with its own
members, roles, API keys, tenants, plan and usage; one user can belong to several and switches between
them in the dashboard (`/platform/dashboard/workspace`, `POST /api/v1/workspaces/{account_id}/switch`).
Creating one (`POST /api/v1/workspaces` with `name` and an optional `slug`) makes you its owner and
issues its first API key.

Roles are `member` < `admin` < `owner`:

| Action | Role |
|---|---|
| Route, read usage, savings and activity | `member` |
| Create and revoke API keys, change tenants, invite and remove members, rename the workspace | `admin` |
| Grant or remove `owner`, configure organization SSO, delete the connection | `owner` |

API keys belong to the workspace and act with its full rights; the role model applies to signed-in
browser sessions. Members are invited by email (`POST /api/v1/workspace/invites`): the invitee gets a
message with the link when the deployment has a mail server, and the platform returns the same link once
to the inviter so it can be shared directly. Seats and tenants
are limited per plan (`max_members`, `max_tenants` in `GET /api/v1/info`). An organization always
keeps at least one owner; owners transfer ownership before leaving.

**Organization SSO** (enterprise) lets people sign in with the company identity provider:
`PUT /api/v1/workspace/sso` with `kind` (`github`, `google`, `microsoft`, `gitlab` or `oidc` with an
`issuer`), `client_id`, `client_secret`, the allowed email `domains` and the `default_role` given to
anyone who signs in from one of those domains. The workspace needs a slug first; the sign-in page
shows the provider as `org-<slug>`.

## 8. Tenants

Tenants (enterprise) apply a fixed set of hard constraints per customer, team or region without the
caller repeating them. Create a tenant, then send its slug in `X-OSR-Tenant` (or `osr.tenant`).

```bash
curl -s -X PUT "$OSR_URL/api/v1/tenants" -H "Authorization: Bearer $OSR_API_KEY" -H "Content-Type: application/json" \
  -d '{"slug": "eu-health", "config": {"region": "eu", "data_boundary": "private", "max_cost_per_1k": 0.01, "deny_targets": ["llm-frontier"]}}'

curl -s "$OSR_URL/api/v1/route" -H "Authorization: Bearer $OSR_API_KEY" -H "X-OSR-Tenant: eu-health" \
  -H "Content-Type: application/json" -d '{"text": "Anonymise this discharge summary ..."}'
```

Tenant config keys: `region`, `data_boundary`, `max_cost_per_1k`, `max_latency_ms`, `preferred_max_latency_ms`,
`deny_targets`, `allow_targets`, `contains_pii`, `daily_budget_usd`, `monthly_budget_usd`. Explicit request constraints
win over the tenant defaults; the workspace policy (next section) applies on top of both. Executed spend
is attributed to the tenant, so `GET /api/v1/governance` shows spend and budgets per tenant.

## 9. Governance: workspace policy, budgets and controls

A **workspace policy** applies to every request of the workspace, whichever key or tenant sent it. It
takes the same keys as a tenant plus budgets, and needs the `admin` role:

```bash
curl -s -X PUT "$OSR_URL/api/v1/policy" -H "Authorization: Bearer $OSR_API_KEY" -H "Content-Type: application/json" \
  -d '{"config": {"data_boundary": "private", "deny_targets": ["llm-frontier"], "monthly_budget_usd": 250}}'
```

`GET /api/v1/policy` returns the policy, the accepted keys and the current budget lines; `DELETE`
removes it. Constraints merge deterministically: a boundary or region fills in when the request has
none, cost and latency caps take the stricter value, deny lists are merged and allow lists intersected.
Unknown keys, unknown target ids and an id present in both `allow_targets` and `deny_targets` are
rejected with `400`.

**Budgets** count executed spend (`execute: true`, `/v1/chat/completions`, MCP `ask`) - quotes and
route-only calls are free. `daily_budget_usd` resets at midnight UTC, `monthly_budget_usd` on the first
of the month. Once a workspace or tenant budget is exhausted, routed calls return `429 ... budget ...
exhausted` with `Retry-After` and the `X-Budget-*` headers until the period rolls over or the limit is
raised.

`GET /api/v1/governance` is the compliance view of the workspace in one payload:

| Block | Contents |
|---|---|
| `controls` | What the deployment enforces: `input_guard`, `pii_redaction`, `steering_strip`, `metrics`, `tracing`, `audit`, `health_breakers`, `decision_cache`, `learning` (`persisted` or `in-process`). |
| `audit` | Whether the hash-chained audit trail is on, its length and the result of the last verification. |
| `retention_days` | How long usage rows are kept before the background sweep purges them. |
| `policy`, `policy_keys`, `budgets`, `spend` | The workspace policy, its accepted keys, budget lines with used and remaining amounts, spend today and this month. |
| `quota` | Requests today against the plan's daily and per-minute limits. |
| `tenants` | Each tenant with its monthly spend and budget lines. |
| `catalogue` | Targets per `data_boundary`, targets that refuse PII, executable targets, providers, and which targets remain reachable under the policy. |

The dashboard renders the same data at `/platform/dashboard/governance`, with a policy editor for admins.

## 10. Observability and caching

The platform observes itself: every span, event, counter and alert below is produced, stored and served
by the platform - there is no external metrics system, tracing backend or alerting tool to run. Every
deployment exposes liveness and readiness probes and the counters without a key:

| Endpoint | Meaning |
|---|---|
| `GET /healthz` | Process is up: edition, version, target count, uptime. |
| `GET /readyz` | `200` when the database, catalogue, router and (enterprise) audit file all answer; `503` with the failing check otherwise. Also reports `database_dialect` (`sqlite` or `postgres`), `redis` and `events` (the event bus) - those two are informational and never fail readiness. |
| `GET /metrics` | The counters in text exposition format: `osr_platform_http_requests_total{method,route,status}`, `osr_platform_http_request_duration_ms_*`, `osr_platform_http_in_flight`, plus the router's `osr_route_decisions_total`, `osr_route_latency_ms_*`, `osr_outcome_*` and `osr_decision_cache_*`. |

Responses carry `X-Request-Id` (send your own to correlate with your logs) and `Server-Timing`; the
API writes one JSON access-log line per request (`osr.platform.access`) with the route template,
status, latency and account - never the prompt. `GET /api/v1/stats` (enterprise) adds `http`,
`controls`, `cache` and `observability` snapshots; `/platform/dashboard/health` shows them live.

### Time series and alerts

| Endpoint | Returns |
|---|---|
| `GET /api/v1/telemetry/series?window=24h` | The workspace's metered traffic bucketed over `1h` (per minute), `6h` (5 min), `24h` (15 min), `7d` (3 h) or `30d` (per day): requests, failures, `error_rate`, p50 / p95 latency, cost and tokens per bucket, totals per target and per endpoint, and the window total. Computed from the platform's own usage records, so it agrees with the activity log to the row. Plans with `stats` also get `http`: the deployment's requests, 4xx, 5xx and p50 / p95 / p99 latency per bucket from the per-minute counters the process keeps for 24 hours. |
| `GET /api/v1/alerts` | Active conditions, most severe first, each with the dashboard page that shows or fixes it. Workspace rules on every plan: a daily or monthly budget of the workspace or a tenant at 80 % (`warning`) or exhausted (`critical`), and 20 % or more of the last 15 minutes' metered requests failing (10+ requests). Deployment rules on plans with `stats`: a failing readiness check, 5 % or more 5xx over the last five minutes (20+ requests), p95 above 2 000 ms, an open or half-open circuit breaker per target, a routing-drift alarm in the last hour, a failed autopilot cycle, a routing SLM last fitted more than 30 days ago, and tracing being off. |

The overview shows the last 24 hours (traffic, latency) and the alerts; `/platform/dashboard/health` adds the
deployment series and the same alerts, refreshed with the page.

### Notifications: alert delivery

Alerts are evaluated by the API itself - no external scheduler. Every `OSR_PLATFORM_ALERTS_INTERVAL_S`
seconds (default 60; `0` turns the loop off and leaves *Evaluate now* to operators) one replica - the one
holding the `osr:alerts:leader` lease in Redis, or the only one without Redis - re-evaluates the
workspace rules for every workspace that was active in the last 15 minutes or has a channel or an open
alert, plus the deployment rules. Each condition becomes a **notification episode**: opened when the rule
starts firing, refreshed while it stays on (an escalation from `warning` to `critical` is delivered
again), resolved when it clears. Episodes land in an inbox with unread counts per severity, are published
to the `alerts` Kafka topic in cluster mode, and are delivered to the workspace's **channels**:

| Kind | Target | Delivery |
|---|---|---|
| `email` | An address | Through the platform mailer (`OSR_PLATFORM_SMTP_URL`, otherwise the outbox at `/platform/admin/mail`). |
| `webhook` | An `https://` URL | `POST` of `{event, alert, workspace, url}` with `X-OSR-Event` (`alert.firing`, `alert.escalated`, `alert.resolved`, `alert.test`), `X-OSR-Delivery` and, when the channel has a secret, `X-OSR-Signature: t=<unix>,v1=<hex>` = HMAC-SHA256 of `<t>.<body>`; verify like Stripe signatures. |
| `slack` | An incoming-webhook URL | Block Kit message with severity, rule, value against threshold and a link to the page that fixes it. |
| `teams` | An incoming-webhook URL | MessageCard with the same facts. |

A channel filters by minimum severity (`info`, `warning`, `critical`) and optionally by rule names, can be
disabled, records its last delivery and last error, and has *Send test*. Failed webhook deliveries are
retried on the following evaluation rounds up to three attempts. Up to 20 channels per workspace; creating
or changing them takes the `admin` role. Operators have the same inbox and channels for the **deployment**
scope (readiness, 5xx rate, p95, circuit breakers, drift, autopilot, SLM age, tracing) at
`/platform/admin/notifications`.

| Endpoint | Purpose |
|---|---|
| `GET /api/v1/notifications?state=firing&unread=true&limit=100` | The workspace inbox (`items`) and unread counts per severity (`unread`). |
| `GET /api/v1/notifications/unread` | Unread counts only - the dashboard bell polls this. |
| `POST /api/v1/notifications/read` | `{"ids": [...]}` marks those read, `{"ids": null}` marks everything read. |
| `GET|POST /api/v1/notifications/channels` | List channels (with the rule names you can filter on) or create one: `kind`, `name`, `target`, optional `secret`, `min_severity`, `rules`, `enabled`. |
| `PATCH|DELETE /api/v1/notifications/channels/{id}` | Change or remove a channel. |
| `POST /api/v1/notifications/channels/{id}/test` | Deliver a test notification and return `ok`, `status`, `error`. |
| `GET /api/v1/notifications/deliveries` | The last delivery attempts (channel, event, attempt, status, error). |
| `GET|POST /api/v1/admin/notifications` and `.../channels`, `.../deliveries` | The operator (deployment scope) equivalents; `POST /api/v1/admin/notifications/evaluate` runs one round now and returns what it opened, resolved, delivered and failed. |

### Shared state and replicas

A single API replica keeps everything on its data volume (SQLite, learner state, audit chain). A
deployment that runs several replicas moves the shared parts to services that are themselves
containers - no managed cloud database, cache or broker is required, and the same images run on a
laptop, a Kubernetes cluster or Azure Container Apps:

| State | Service | Setting |
|---|---|---|
| Accounts, users, keys, tenants, policies, usage, feedback, marketplace | PostgreSQL | `OSR_PLATFORM_DATABASE_URL` |
| Rate-limit windows, learner state (enterprise) | Redis | `OSR_PLATFORM_REDIS_URL` |
| `usage`, `feedback` and `admin` events as JSON topics `osr.<name>` | Kafka (any Kafka-protocol broker) | `OSR_PLATFORM_KAFKA_BOOTSTRAP`, `OSR_PLATFORM_KAFKA_TOPIC_PREFIX` |

`GET /api/v1/info` -> `storage` says which mode a deployment runs (`{database, clustered, redis,
events}`) and the operator console shows it on `/platform/admin`. Event payloads carry the same fields as the
activity log and never the prompt text. `platform/docker-compose.yml` in the repository starts the
complete stack; the Azure template provisions the three services as container apps with internal TCP
ingress next to the API and web app.

In that stack the routing SLM is its own service too: the API publishes each routed prompt to the
`training` topic (opt-in, `OSR_PLATFORM_TRAINING_EVENTS`), the SLM service pairs prompts with the
outcomes reported through `POST /api/v1/feedback`, trains a challenger on a schedule, keeps it only when
it beats the champion on a held-out split, and writes the promotion to the shared volume from which every
API replica reloads it. Operators see the model in service, the evidence ingested, every cycle and a
prompt probe on `/platform/admin/slm` (`GET /api/v1/admin/slm`, `/slm/reports`, `POST /slm/cycle`, `/slm/predict`).

### Services

The API is one image that can run as one process or as several: the documentation JSON (`docs`), the
public model catalogue and rankings (`rankings`), the marketplace (`marketplace`), provider management
(`providers`), signup and account links (`onboarding`), identity and workspaces (`accounts`: sign-in,
sessions, organizations, keys, tenants, policy, governance, usage), Stripe billing (`billing`), the
operator console API (`admin`), the MCP endpoint (`mcp`), the OpenAI-compatible proxy (`openai`), the
routing core with its trace, event and learning reads (`routing`) and the routing SLM (`slm`) each have
their own entry point and port. `/api/v1/info`, `/api/v1/status` and `/api/v1/estimate` always stay
with the API. The API remains the single public origin and forwards a domain's paths to its
service when `OSR_PLATFORM_<NAME>_URL` is set (streamed completions are relayed as they arrive); the
answer carries `X-OSR-Service` naming the process
that served it. `GET /api/v1/info` -> `services` lists the topology; operators see live health per
service on `/platform/admin/services` (`GET /api/v1/admin/services`). The Compose stack and the Azure template
deploy every service as its own container next to the gateway.

### Traces and events

The router records every request as a tree of **spans** (`request`, `route`, `plan`, `execute`) with
**events** inside them (`route.signals`, `route.policy`, `route.rank`, `route.fallback`, `guard.*`,
`execute.step`, `learn.outcome`, `cache.hit`, ...). The newest ones stay in an in-memory buffer (1000
by default, `OSR_OBSERVABILITY_MEMORY_EVENTS`) and every one of them is also written to the platform
database by the platform's own telemetry store (`OSR_PLATFORM_TELEMETRY_STORE`, on by default) - off
the request path, from a background writer, so tracing never slows routing. Events and traces therefore
survive restarts, can be read by time range and are purged after `OSR_PLATFORM_TELEMETRY_RETENTION_DAYS`
(14). They are readable per workspace:

| Endpoint | Returns |
|---|---|
| `GET /api/v1/trace/{request_id}` | The activity row plus every span and event of one request, in time order, with `trace_id`, `spans` and the root `duration_ms`; `outcomes` (the feedback reported for the request) and `audit` (its hash-chained records, plans with `audit`). `source` says whether the rows came from the buffer or the store. `404` when the request is not the workspace's; `events: []` once the request is older than the telemetry retention. |
| `GET /api/v1/events?name=route.*&kind=&level=&request_id=&since=&until=&limit=200` | Events of the workspace's requests (oldest first, `limit` up to 2000; `name` may end with `*`; `since` / `until` are epoch seconds, default the last 6 hours). Deployment-wide events without a request id (breaker transitions, autopilot) are included on plans with `stats`. `source`, `persisted`, `retention_start` and `retention_days` describe the store. `404` when tracing is off entirely. |
| `GET /api/v1/status` | Public readiness: the `/readyz` checks plus edition, versions, uptime, controls and tracing state; `503` while a check fails. `Cache-Control: no-store`. |

`POST /api/v1/route` responses include `trace_id` and the `X-OSR-Trace-Id` header when the request was
traced, and `GET /api/v1/activity` marks rows whose spans are kept (in memory or in the store) with
`traced: true` and summarises the feedback reported for each request in `outcome` (`POST /api/v1/feedback`
is stored per request, so the loop from decision to result is visible per row). Attributes hold ids,
numbers and short labels; the request text is represented only by its digest and length. The dashboard
shows the history at `/platform/dashboard/events` (window of 1 hour to 7 days, search by request or trace id),
opens a trace from any activity row (with a *How it was routed* summary of signals, policy, ranking and
decision, the outcome and the audit records) and adds a *Trace* tab to the playground.

Public catalogue reads (`/api/v1/info`, `/api/v1/models*`, `/api/v1/rankings`, `/api/v1/llms*`,
`/api/v1/catalogue`, `/api/v1/stats/public`) return an `ETag` and `Cache-Control: public, max-age=60,
stale-while-revalidate=60`; send `If-None-Match` to get `304` when nothing changed. Operators tune this
and the decision cache with:

| Setting | Default | Effect |
|---|---|---|
| `OSR_PLATFORM_METRICS_PUBLIC` | `true` | `false` restricts `/metrics` to requests carrying `X-Admin-Token`. |
| `OSR_PLATFORM_HTTP_CACHE_MAX_AGE_S` | `60` | `Cache-Control` lifetime of public reads; `0` sends `no-store`. |
| `OSR_PLATFORM_ROUTE_CACHE_TTL_S` | `0` | Reuse a routing decision for identical requests within this window (`0` = off). |
| `OSR_PLATFORM_RETENTION_DAYS` | `365` | Usage rows older than this are purged every six hours; `0` keeps everything. |
| `OSR_PLATFORM_TELEMETRY_STORE` | `true` | Persist every span and event to the platform database (the events page, traces and `since` / `until` queries outlive the in-memory buffer). |
| `OSR_PLATFORM_TELEMETRY_RETENTION_DAYS` | `14` | Persisted spans and events older than this are purged with the same sweep; `0` keeps them. |

### Learning: the routing SLM and the autopilot

Every deployment learns from `POST /api/v1/feedback`: the bandit, IRT, Bradley-Terry, LinUCB and Markov
strategies update on each outcome (persisted under the data volume in the enterprise edition). A
deployment can also serve a **routing SLM** - the small routing model `osr slm train` / `osr improve`
write to a file - as one more strategy in the ensemble, and run the **autopilot** in-process so the
platform's own feedback keeps retraining it:

| Setting | Default | Effect |
|---|---|---|
| `OSR_PLATFORM_SLM` | none | Path of the model file; it joins the ensemble as the `slm` strategy (weight `OSR_WEIGHTS_SLM`, default 1.0). |
| `OSR_PLATFORM_AUTOPILOT` | `false` | Run `SelfImprover` cycles on a thread: outcomes and remembered prompts become rows, a challenger is trained on a split and promoted only when it beats the champion on the holdout. The promoted model is written to `DATA_DIR/autopilot/slm.json` and outranks the mounted file after a restart. |
| `OSR_PLATFORM_AUTOPILOT_OFFLINE` | `true` | No catalogue refresh, dataset download or web search from the API process; `false` also refreshes `OSR_PLATFORM_AUTOPILOT_SOURCES` (public dataset presets, comma separated). |
| `OSR_PLATFORM_AUTOPILOT_INTERVAL_S` | `OSR_SLM_AUTOPILOT_INTERVAL_S` (3600) | Schedule; drift alarms from the outcome stream trigger a cycle early (`OSR_SLM_AUTOPILOT_DRIFT_*`, rate-limited by `OSR_SLM_AUTOPILOT_MIN_GAP_S`). |
| `OSR_PLATFORM_AUTOPILOT_MIN_ROWS` | `20` | A cycle needs at least this many rows before it trains a challenger. |

`GET /api/v1/learning` (any plan) returns the strategy ensemble with its weights, the learner state
(persistence, drift resets, quarantined state), per-target outcome statistics, the SLM in service
(rows, sources, encoder, calibration, fit history - never the weights) and the autopilot status with
its recent champion-vs-challenger reports. `controls.slm` / `controls.autopilot` appear in
`GET /api/v1/governance` and `GET /api/v1/info`. Operators trigger a cycle with
`POST /api/v1/admin/autopilot/cycle` (`X-Admin-Token`; 404 without an autopilot). Cycles are traced
as an `autopilot.cycle` span with `learn.improve` / `learn.promote` events, so they appear in
`GET /api/v1/events` and on `/platform/dashboard/events`. The dashboard renders all of it at
`/platform/dashboard/learning`.

## 11. Marketplace

The marketplace (`/marketplace`) is where agents, skills, personas, prompts, tools, model profiles and
stack templates are shared. Every listing is a manifest the SDK, CLI and this API understand; installing
records it on your account and returns the manifest and ready-made snippets, and `registry://<slug>`
references in a stack file resolve to `GET /api/v1/registry/{slug}/manifest`. Browsing is public;
installing, rating and publishing need an account. See the [marketplace guide](https://opensmartroute.ai/docs/MARKETPLACE.md) for
finding, installing, publishing and the review lifecycle; the endpoints are in the
[REST API reference](https://github.com/isathish/OpenSmartRoute/blob/main/platform/api/openapi.json) under *Marketplace*.

## 12. MCP server

The platform is also a [Model Context Protocol](https://modelcontextprotocol.io) server, so an IDE
or agent (VS Code, Cursor, Claude, Windsurf) can route, quote and explain through it. `GET /mcp`
describes the server without a key; `POST /mcp` accepts JSON-RPC 2.0 messages with the same
`Authorization` header as the REST API. Tools: `route`, `estimate`, `recommend`, `explain`,
`list_targets`, `feedback`, `ask` (pro plan and above; routes and executes) and `marketplace_search` /
`marketplace_get`. Routed calls are metered like `/api/v1/route`.

Clients that only speak stdio use the CLI as a bridge:

```json
{"mcpServers": {"opensmartroute": {"command": "osr", "args": ["mcp", "--url", "https://<platform>/mcp", "--api-key", "osr_..."]}}}
```

Per-client configuration, the tool reference and the SDK side are in [MCP.md](https://opensmartroute.ai/docs/MCP.md).

## 13. Dashboard

The site has two halves on one hostname. The **website** - landing page, `/docs`, `/models`, `/vendors`,
`/rankings`, `/pricing`, `/marketplace`, `/compare`, `/roi`, `/estimate` - needs no account. The **platform**
is everything under `/platform`: sign-in and sign-up (`/platform/login`, `/platform/signup`, password
recovery, invitations, `/platform/cli/authorize`), the dashboard (`/platform/dashboard/...`), the operator
console (`/platform/admin/...`), the playground (`/platform/playground`) and the marketplace publish flow
(`/platform/marketplace/publish`). On the hosted service the two are separate deployments; the older
paths without the prefix (for example /dashboard/keys, /login or /auth/callback) redirect permanently to their
`/platform` counterpart with the query string intact, so bookmarks, emailed links and OAuth redirect URIs
registered before the split keep working.

Signed-in users have a dashboard for the data the API exposes:

| Page | What it shows |
|---|---|
| `/platform/dashboard` | Requests, tokens and spend for the last 30 days; plan and quota usage; the last 24 hours as a traffic and latency series (`GET /api/v1/telemetry/series`) and the active alerts (`GET /api/v1/alerts`). |
| `/platform/dashboard/usage` | Per-day and per-target usage (`GET /api/v1/usage`); CSV export of the daily series and both breakdowns. |
| `/platform/dashboard/savings` | Routed cost against a baseline model (`GET /api/v1/savings?baseline=<id>`); CSV export per day and per target for finance. |
| `/platform/dashboard/activity` | Request-level activity log: endpoint, target, domain, complexity, tokens, reported outcome - never the prompt text. Rows still in the tracer buffer open a span waterfall with the decision story, outcome and audit records. CSV export of the loaded rows. |
| `/platform/dashboard/events` | The router's event history for the workspace (`GET /api/v1/events`, persisted by the platform): a window of 1 hour to 7 days, spans and events with level and stage filters, search by request id / trace id / name, most frequent names, click-through to the request trace, CSV export. |
| `/platform/dashboard/learning` | How the router learns (`GET /api/v1/learning`): strategy weights, outcomes per target, the routing SLM (rows, sources, the targets it ranks, accuracy history, calibration) and the autopilot (schedule, drift, cycles, a retraining-in-progress indicator, champion vs challenger; operators can run a cycle). Fits and cycles export as CSV. |
| `/platform/dashboard/keys` | Create, name and revoke API keys. |
| `/platform/dashboard/tenants` | Tenant constraints and budgets (enterprise). |
| `/platform/dashboard/governance` | Controls in force, workspace policy editor, budgets and spend, audit status, retention, catalogue data boundaries. |
| `/platform/dashboard/health` | Deployment readiness (`GET /api/v1/status`), the active alerts, HTTP traffic and latency percentiles over time (`GET /api/v1/telemetry/series`), live counters, routing decisions and latency, circuit breakers, decision cache (enterprise). |
| `/platform/dashboard/notifications` | The alert inbox (firing and resolved episodes, mark read), delivery channels (email, webhook, Slack, Teams) with severity and rule filters and *Send test*, and the delivery log. The bell in the header shows the unread count. |
| `/platform/dashboard/audit` | Hash-chained decision trail with chain verification (enterprise); export as CSV or as the JSONL the offline verifier reads. |
| `/platform/dashboard/billing` | Plan, invoices and upgrades. |
| `/platform/dashboard/workspace` | The current workspace: name, slug, plan; create or switch to an organization; organization SSO. |
| `/platform/dashboard/members` | Members, roles and pending invitations of an organization. |
| `/platform/dashboard/listings` | Marketplace listings you published and everything you installed. |
| `/platform/dashboard/integrations` | MCP configuration for VS Code, Cursor, Windsurf, Claude Code and Claude Desktop with your key. |
| `/platform/dashboard/account` | Your sign-in identities, password and browser sessions. |

Every table with an *Export CSV* button downloads exactly the rows shown (UTF-8 with BOM, RFC 4180 quoting,
cells that start with `=`, `+`, `-` or `@` are prefixed so spreadsheets do not run them as formulas). A failed
read shows what it means - session expired, feature not on the plan, budget exhausted with the amounts, rate
limit with the retry delay, API unreachable - with a *Retry* button; an expired session returns to sign-in and
back to the page afterwards.

The public pages `/platform/playground` (route a request in the browser and copy the equivalent `curl`),
`/models` (every routing target with request, token, cost, latency and health statistics; also
`GET /api/v1/models`, and the reference LLM catalogue with vendor prices and context windows at
`GET /api/v1/llms`), `/vendors` (one page per LLM vendor with prices, context windows and benchmarks),
`/rankings` (targets ordered by observed quality per domain), `/roi` (estimated
saving against always calling one model), `/compare` (alternatives side by side) and `/marketplace` need
no account. Machine-readable companions: `/llms.txt` (documentation map for AI assistants, llmstxt.org;
every documentation page is also served as Markdown at `/docs/<page>.md` and advertised with a
`text/markdown` alternate link), `/llms-full.txt` (the whole documentation as one Markdown file),
`/feed.xml` (Atom feed of releases), `/sitemap.xml` and `/openapi.json` (the complete OpenAPI 3.1
document of the API - the committed snapshot the REST reference renders, not the subset one gateway
process happens to serve).

### Operator console

The people who run a deployment sign in at `/platform/admin/login` with an operator **username and password**
(`POST /api/v1/admin/auth/login` returns an `osr_op_` session token valid for twelve hours; send it as
`Authorization: Bearer`). The first operator comes from the environment: set
`OSR_PLATFORM_ADMIN_USERNAME` and `OSR_PLATFORM_ADMIN_PASSWORD` and the API creates it - or resets its
password - at start-up as a `superadmin`. The static `OSR_PLATFORM_ADMIN_TOKEN` (`X-Admin-Token`)
keeps working for automation and counts as a superadmin. Operators are separate from workspace users.

| Page | What it does | Endpoints |
|---|---|---|
| `/platform/admin` | Deployment counters: users, workspaces, keys, tenants, operators, sessions, requests, errors and provider cost of the last seven days, plan mix, sign-in configuration. | `GET /api/v1/admin/overview`, `GET /api/v1/admin/plans` |
| `/platform/admin/analytics` | The whole deployment over 7, 30 or 90 days: requests and failures per day with mean latency, cost per day, endpoint share, workspaces per plan, signups per day, the busiest workspaces and the targets that carried the traffic. | `GET /api/v1/admin/analytics?days=` |
| `/platform/admin/health` | This replica of the routing process: readiness checks, HTTP traffic and latency series (1 h / 6 h / 24 h), the last five minutes, circuit breakers per target, the deployment alerts in force and the alert evaluator; auto-refresh. | `GET /api/v1/admin/health?window=` |
| `/platform/admin/activity` | Every metered request across workspaces, newest first: workspace, endpoint, target, status, latency, tokens, cost, request id; filter by endpoint, failures, workspace or request id; load older pages; CSV export. | `GET /api/v1/admin/activity?limit=&before=&account_id=&endpoint=&target=&failed=&request_id=` |
| `/platform/admin/users` | The user directory: search by email or name, filter by status, email confirmation and sign-in method (password or SSO / key only), sort by joined, last sign-in, email or name, export the matching rows as CSV. Select rows for a bulk action: disable, enable, mark the email verified, sign out everywhere, send a password reset, delete (each user is reported as done or failed). Create one with a personal workspace, plan, optional password and first key (shown once; without a password a reset link is issued so the person chooses one). Per user: workspaces and roles, linked identities, sessions, lifecycle history; rename, disable, reset password or send a reset link, re-send or force the email confirmation, sign out everywhere, delete (workspaces they alone belonged to are disabled and their keys revoked). | `GET|POST /api/v1/admin/users` (`q`, `disabled`, `verified`, `method`, `sort`, `order`, `format=csv`), `POST /api/v1/admin/users/bulk`, `GET|PATCH|DELETE /api/v1/admin/users/{id}`, `DELETE /api/v1/admin/users/{id}/sessions`, `POST /api/v1/admin/users/{id}/reset-link`, `POST /api/v1/admin/users/{id}/verify-link`, `GET /api/v1/admin/users/{id}/audit` |
| `/platform/admin/workspaces` | The workspace directory: search, filter by kind, plan and status, sort by name, plan or created, export CSV; create an organization for an owner email. Per workspace: plan, name, slug, disable, delete; members with roles (add by email, change role, remove - the last owner stays); API keys (mint, revoke); tenants (create or replace a validated configuration, delete); policy, SSO and 30-day usage. *Open dashboard* (superadmins) mints a one-hour browser session on the workspace as its owner and switches to the workspace portal, which shows an *Operator view* banner with the way back; the view sees every plan feature (statistics, audit, tenants) whatever the workspace's plan, while quotas stay the workspace's; the owner sees the session (provider `operator`) in their session list and the audit log records `workspace.open`. | `GET|POST /api/v1/admin/accounts` (`q`, `kind`, `plan`, `disabled`, `sort`, `order`, `format=csv`), `GET|PATCH|DELETE /api/v1/admin/accounts/{id}`, `.../usage`, `.../members[/{user_id}]`, `.../keys[/{key_id}]`, `.../tenants/{slug}`, `POST /api/v1/admin/accounts/{id}/session` |
| `/platform/admin/tenants` | Every tenant across workspaces with its configuration. | `GET /api/v1/admin/tenants` |
| `/platform/admin/leads` | Product-qualified leads: workspaces scored on volume, tenants, EU or private data boundaries, agentic traffic, several teams and a company email domain; CSV export. | `GET /api/v1/admin/pql?days=&qualified_only=` |
| `/platform/admin/plans` | The plan catalogue - limits, price, feature matrix on this edition - and the workspaces on each plan. | `GET /api/v1/admin/plans`, `GET /api/v1/admin/overview` |
| `/platform/admin/slm` | The routing SLM service: model in service, prompts and outcomes ingested, improvement cycles (champion vs challenger), run a cycle, probe the model with a prompt. | `GET /api/v1/admin/slm`, `GET /api/v1/admin/slm/reports`, `POST /api/v1/admin/slm/cycle`, `POST /api/v1/admin/slm/predict` |
| `/platform/admin/providers` | Model providers: add an OpenAI-compatible endpoint from a preset, check it (latency, upstream model list), map catalogue targets to upstream models with an optional system prompt, disable or delete; shows the mounted file, what is attached on this replica and the targets that cannot execute yet. | `GET|POST /api/v1/admin/providers`, `GET|PATCH|DELETE /api/v1/admin/providers/{id}`, `POST /api/v1/admin/providers/{id}/check`, `PUT|DELETE /api/v1/admin/providers/models/{target_id}` |
| `/platform/admin/services` | Where each domain runs (in the API or as its own service) with a live health probe of every remote service. | `GET /api/v1/admin/services` |
| `/platform/admin/marketplace` | Marketplace moderation: the review queue and every other status, approve, reject or unpublish with a note, featured and verified flags, the imported catalogue per source and the scheduled refresh (run it now). | `GET /api/v1/admin/registry?status=`, `POST /api/v1/admin/registry/{slug}/approve`, `POST /api/v1/admin/registry/{slug}/reject`, `POST /api/v1/admin/registry/{slug}/unpublish`, `POST /api/v1/admin/registry/{slug}/flags`, `GET /api/v1/admin/registry/import`, `POST /api/v1/admin/registry/import/refresh` |
| `/platform/admin/notifications` | Deployment alerts as an inbox (readiness, error rate, latency, breakers, drift, autopilot, SLM, tracing), operator channels (email, webhook, Slack, Teams), the delivery log and *Evaluate now*. | `GET /api/v1/admin/notifications`, `POST /api/v1/admin/notifications/read`, `POST /api/v1/admin/notifications/evaluate`, `GET|POST /api/v1/admin/notifications/channels`, `PATCH|DELETE /api/v1/admin/notifications/channels/{id}`, `POST /api/v1/admin/notifications/channels/{id}/test`, `GET /api/v1/admin/notifications/deliveries` |
| `/platform/admin/operators` | Operator accounts (superadmins create, disable, promote and delete them; the last enabled superadmin cannot be removed), your own password and sessions. Superadmins open any operator's console sessions and revoke one or all of them - a compromised console account is locked out at once. | `GET|POST /api/v1/admin/operators`, `PATCH|DELETE /api/v1/admin/operators/{id}`, `GET|DELETE /api/v1/admin/operators/{id}/sessions`, `POST /api/v1/admin/auth/password`, `GET /api/v1/admin/auth/sessions`, `POST /api/v1/admin/auth/logout` |
| `/platform/admin/audit` | Deployment-wide account-lifecycle log: signups, sign-ins and failed attempts, password and email changes, invitations, roles, removals, deletions and every operator action (actor `operator`). Filter by action (the menu lists every action with its count), actor type, time window, user, workspace or free text over actor, IP and details; page back with *Load older*; export the matching events as CSV. | `GET /api/v1/admin/audit-log` (`action`, `actor_type`, `actor_id`, `user_id`, `account_id`, `q`, `after`, `before`, `limit`, `format=csv`) |
| `/platform/admin/mail` | Every email the platform composed with its delivery state and counters (composed, awaiting delivery, failed); filter by recipient, kind or delivery; open a message to read it, copy its link, resend it (SMTP only) or delete it; send a test message to prove the SMTP settings. Without `OSR_PLATFORM_SMTP_URL` this is the delivery channel. | `GET /api/v1/admin/mail` (`to`, `kind`, `unsent`, `failed`, `limit`, `offset`), `GET|DELETE /api/v1/admin/mail/{id}`, `POST /api/v1/admin/mail/{id}/resend`, `POST /api/v1/admin/mail/test` |
| `/platform/admin/settings` | The effective configuration grouped (sign-in, storage, services, learning, marketplace, mail, billing, observability) with the variable that sets each value; secrets show only whether they are set. Retention: request rows, persisted telemetry, audit log and outbox sizes, and *Run the sweep now*. | `GET /api/v1/admin/settings`, `GET /api/v1/admin/retention`, `POST /api/v1/admin/retention/purge` |
| `/platform/admin/search` | Support search from the header box: users by email or name, workspaces by name, slug or email, API keys by prefix, operators, tenants by slug and request ids - each hit links to its page. | `GET /api/v1/admin/lookup?q=` |

Every `/api/v1/admin/*` call is written to the access log with `operator:<username>` (or
`operator:admin-token`) as the actor. The console is served by the web app but every action is an API
call, so the same administration works from scripts with the static token. Operators keep both portals
open at once: the console's account menu leads to the workspace dashboard (a superadmin opens any
workspace from `/platform/admin/workspaces`) and the dashboard's account menu leads back to the console
while an operator session exists.

## 14. Privacy and data handling

* Prompt text is routed in memory and, when execution is requested, forwarded to the configured
  provider. The platform stores request metadata (endpoint, target, domain, complexity, token and cost
  counts), not the prompt or the answer.
* `profile` values are hashed before they reach the learners.
* PII detection runs on every request; `contains_pii` together with a tenant `data_boundary` keeps
  sensitive requests on private or on-prem targets. Details: [security model](https://opensmartroute.ai/docs/SECURITY.md).
* The public web pages load Google Analytics 4 in consent mode: no analytics or advertising cookies
  until the visitor accepts the banner, IP addresses truncated, nothing on the dashboard. The choice is
  stored in `localStorage` (`osr-consent`) and can be changed on the privacy page. Besides page views the
  tag records Core Web Vitals, clicks on outbound links and calls to action, code-snippet copies and the
  sign-up / sign-in events. Operators of a
  self-hosted web image leave `NEXT_PUBLIC_GA_MEASUREMENT_ID` empty to ship no tag at all.

## 15. Errors

All errors are JSON: `{"detail": "..."}` with a conventional status code and an `X-Request-Id` to quote.

| Status | Typical cause |
|---|---|
| `400` | Invalid `objective`/`constraints`/policy keys, unknown target id in a policy, malformed email, an organization-only call on a personal workspace. |
| `401` | Missing, invalid or revoked API key (`WWW-Authenticate: Bearer`). |
| `403` | Feature not in plan (`X-Upgrade`), role too low, signup disabled. |
| `404` | Unknown target, tenant, model or listing; enterprise-only feature on a community deployment. |
| `409` | Duplicate signup, tenant or workspace slug; removing the last owner. |
| `422` | Body failed validation (the response lists the offending fields). |
| `429` | Rate limit, daily quota or an exhausted workspace/tenant budget (`Retry-After`, `X-Budget-*`). |
| `501` | `/v1/chat/completions` on a deployment with no model provider configured. |
| `502` | The web app could not reach the API. |
| `503` | `/readyz` when a dependency is not ready (the body names the failing check). |

## Related

* [REST API reference](https://github.com/isathish/OpenSmartRoute/blob/main/platform/api/openapi.json) - every endpoint, parameter and schema (rendered at `/docs/api`).
* [Marketplace](https://opensmartroute.ai/docs/MARKETPLACE.md) - find, install and publish agents, skills, personas, prompts and templates.
* [User guide](https://opensmartroute.ai/docs/GUIDE.md) - targets, rules, plans and learning.
* [Deploy](https://opensmartroute.ai/docs/deploy.md) - run the same router on your own infrastructure with `osr serve`.

---

<!-- Page: Marketplace (https://opensmartroute.ai/docs/MARKETPLACE) -->

<!-- OpenSmartRoute: Marketplace. Source https://github.com/isathish/OpenSmartRoute/blob/main/docs/MARKETPLACE.md; HTML https://opensmartroute.ai/docs/MARKETPLACE -->
# Marketplace

The marketplace is the part of the [hosted platform](https://opensmartroute.ai/docs/PLATFORM.md) where routing building blocks are
shared: **agents, skills, personas, prompts, tools, model profiles and stack templates**, with
versions, ratings and a short review lifecycle. Every listing is a plain YAML or JSON document that
the OpenSmartRoute SDK, CLI and hosted API understand, so what you install is what you run; there
is no marketplace-only format.

Browsing, installing, rating and publishing prompts and personas work from the website without
writing code. Templates, skills and agents come with copy-paste snippets for the CLI and the SDK.

## What is on it

| Kind | What it is | Manifest |
|---|---|---|
| **Template** (stack) | A complete routing setup - models, rules, weights - in one file. Apply it and you have a working router. | `kind: stack` document ([`opensmartroute.stack`](https://opensmartroute.ai/docs/REFERENCE.md#opensmartroutestack)) |
| **Agent** | A multi-step worker the router can hand a whole task to. | Open Capability Manifest ([spec](https://opensmartroute.ai/docs/ocm.md)) |
| **Skill** | An Agent-Skills package (`SKILL.md`) that is also a routable target. | OCM, generated from the `SKILL.md` |
| **Persona** | A system prompt with a voice, a domain and a complexity band. | OCM with `kind: persona` |
| **Tool** | A function or MCP tool with typed parameters. | OCM with `kind: tool` |
| **Model profile** (llm) | A model with cost, latency, context window and quality prior. | OCM with `kind: llm` |
| **Prompt** | A reusable prompt with `{{variables}}`. | `{"prompt": "...", "variables": [...]}` |

Every listing has a **slug** (its address, `/marketplace/<slug>`), a **version** (`1.2.3`), a
publisher, a licence, tags, a Markdown description, an install count and a star rating.

## Finding and installing

1. Open **Marketplace** in the top navigation. Filter by kind, tag, price or search; sort by
   popular, top rated, newest or name.
2. Open a listing. The **About** tab is the publisher's description; **Use it** shows ready-made
   snippets (CLI, Python, HTTP); **Manifest** is the raw document; **Reviews** lists ratings.
3. Click **Install** (free) or **Buy** (paid). You need an account - signing up is free. Free
   installs complete at once; paid ones go through the checkout and are recorded on your account
   when the payment completes.
4. Everything you installed is listed under **Dashboard -> Marketplace -> Installed by you**, together with
   the version you installed and whether a newer one exists.

Installing records the listing on your account and returns the manifest and snippets. Nothing is
executed on your behalf.

### Using an installed template

```bash
osr stack apply registry://support-desk@1.0.0 --registry https://<platform>   # build and summarise
osr stack apply registry://support-desk --registry https://<platform> --route "Refund for order 42?"
osr stack apply registry://support-desk --registry https://<platform> --out stack.yaml  # flatten & keep
```

Or reference it from your own stack and let imports do the work:

```yaml
osr: "1"
kind: stack
name: my-desk
imports:
  - registry://support-desk@1.0.0
targets:
  - id: llm-onprem
    kind: llm
```

`osr stack validate my-desk.yaml`, `osr stack plan my-desk.yaml --against deployed.yaml` and
`osr stack apply my-desk.yaml` are the Terraform-style verbs: check, diff, apply.

### Using an installed agent, skill, persona, tool or model

An OCM manifest becomes a target with one call:

```python
from opensmartroute import Router
from opensmartroute.ocm import target_from_capability

manifest = {...}  # GET /api/v1/registry/<slug>/manifest
router = Router([target_from_capability(manifest)])
```

Prompts are text: read `manifest["prompt"]` and fill in `manifest["variables"]`.

The HTTP endpoints are public for published listings:

```
GET /api/v1/registry?kind=template&q=support&sort=popular
GET /api/v1/registry/{slug}
GET /api/v1/registry/{slug}/manifest?version=1.0.0
GET /api/v1/registry/{slug}/reviews
```

## Ratings and reviews

Anyone with an account can rate a listing from 1 to 5 stars and leave a short review. One review per
account per listing (writing again replaces the previous one). Reviews from accounts that installed
the listing are marked as such. A listing shows its average rating, the number of ratings and a star
histogram.

## Publishing

**Marketplace -> Publish** walks through four steps: choose the kind, describe it, paste or write
the manifest, review and submit. Prompts and personas need no code at all; for templates the wizard
offers a starter you can edit. You can also post the same payload to `POST /api/v1/registry`.

### The lifecycle

```
draft  ->  review  ->  published  ->  archived
  ^          |
  +--- rejected
```

- **Draft** - only you can see it. Edit anything, including the manifest.
- **Submit** - the manifest is validated (OCM schema, stack schema or prompt shape). Free listings
  are published immediately; paid listings go to **review**. (Operators can require review for
  everything with `OSR_PLATFORM_REGISTRY_AUTO_PUBLISH=false`.)
- **Review** - a platform reviewer approves or rejects with a note. Rejected listings go back to
  you with the note; fix and submit again.
- **Published** - live. Name, summary, description, tags, price and licence stay editable. The
  manifest is frozen: ship a **new version** instead, with a changelog. Old versions remain
  downloadable (`?version=`).
- **Archived** - hidden from the marketplace; existing installs keep working. You can restore it as
  a draft.

Your listings, their status and any reviewer note are under **Dashboard -> Marketplace -> Published
by you**; each one has an edit page with the version form.

### Paid listings

Set a price in USD when publishing (Pro and Enterprise plans). Buyers pay through the platform's
checkout; the purchase is confirmed by the payment webhook and the listing is then recorded on their
account. Free plans can publish a limited number of free listings.

### Writing a good listing

- A one-line summary that says what it does and for whom.
- A description that covers what it needs (models, keys, data) and how to use it - Markdown, no
  raw HTML or images.
- Tags people would search for (`support`, `pii`, `sql`, `on-prem`).
- Test templates locally first: `osr stack validate stack.yaml`.
- Skills: keep `SKILL.md` front matter complete (`name`, `description`); `osr skills validate`
  checks it.

## Built-in content

Every platform ships seeded with the repository's own Agent-Skills packages, a starter template, the
`examples/stack.yaml` support-desk template, a persona and a couple of prompts, all published by
`OpenSmartRoute` and marked verified. Set `OSR_PLATFORM_REGISTRY_SEED=false` to run an empty
marketplace.

## Imported catalogue

The hosted marketplace also carries the public Agent Skills ecosystem and the official MCP server
registry, imported by `platform/api/scripts/harvest_skills.py` (module `osr_platform.harvest`):

| Source | What is imported | Listing kind |
|---|---|---|
| [skills.sh](https://skills.sh) | every `SKILL.md` in each repository on the leaderboard (about 2 400 repositories) | `skill` |
| GitHub code search | the long tail of `SKILL.md` files (needs a GitHub token; optional) | `skill` |
| [MCP registry](https://registry.modelcontextprotocol.io) | the latest active version of every server, with its remote endpoints and packages | `tool` |

Imported listings are published under the upstream owner's name, are not marked verified, and carry
`metadata.source` in their manifest (`provider`, `repository`, `path`, `ref`, `url`) so the listing page
links back to the original file; the readme starts with the upstream install command
(`npx skills add owner/repo --skill name`). Copyright and licence stay with the author (`license` is
taken from the frontmatter when present). Domains are inferred from the description with the router's
own lexicon; the `skill-md` / `mcp` tags and the source name are added to the author's tags.

```bash
# collect into a resumable JSONL cache (one file per source)
python -X utf8 platform/api/scripts/harvest_skills.py collect --work-dir .osr-harvest \
    --source skills.sh --source mcp-registry [--source github] [--limit N]

# publish into a running platform (batches of 200 through the admin import endpoint) ...
OSR_PLATFORM_ADMIN_TOKEN=... python -X utf8 platform/api/scripts/harvest_skills.py publish --work-dir .osr-harvest \
    --url https://api.example.com
# ... or straight into a local platform database
python -X utf8 platform/api/scripts/harvest_skills.py publish --work-dir .osr-harvest --data-dir .osr-platform
```

`collect` only fetches what the cache does not have yet, so re-running it picks up new skills;
`publish --refresh` overwrites listings that already exist (same version, manifest replaced in place),
without it existing slugs are skipped. Slugs are `owner-repo-skill` (or `mcp-<server name>`), truncated
with a short hash when longer than 63 characters.

For a platform whose SQLite file lives on a network share (the Azure Container Apps deployment mounts
Azure Files), publish tens of thousands of rows **offline**: build the database locally with
`publish --data-dir`, stop the API, replace `platform.sqlite3` on the share, start it again. Sustained
bulk writes through the HTTP endpoint over SMB have corrupted the listings table before; the endpoint is
meant for incremental updates of a few hundred rows.

The hosted marketplace stays current from inside the platform process - no CI job or cron is involved. Set
`OSR_PLATFORM_MARKETPLACE_REFRESH_S` (seconds between runs; `604800` = weekly, `0` = off, the Azure
deployment defaults to weekly) and the process that owns the marketplace harvests only rows that are new
upstream (`OSR_PLATFORM_MARKETPLACE_REFRESH_SOURCES`, default `skills.sh,mcp-registry`;
`OSR_PLATFORM_MARKETPLACE_REFRESH_LIMIT` new rows per source per run, default 2000) into
`<data dir>/harvest/` and publishes the ones the catalogue lacks in batches of 100. Set
`OSR_PLATFORM_GITHUB_TOKEN` so the GitHub API allows the repository listings. `GET /api/v1/admin/registry/import`
reports the last run and the next run time under `refresh`; `POST /api/v1/admin/registry/import/refresh` runs it now.
From a machine, `publish --url ... --only-new` does the same: it asks
`POST /api/v1/admin/registry/import/check {"slugs": [...]}` which slugs already exist and sends only the rest.

## For operators

| Setting | Default | Effect |
|---|---|---|
| `OSR_PLATFORM_REGISTRY_AUTO_PUBLISH` | `true` | Free listings go live on submit; `false` sends everything to review. |
| `OSR_PLATFORM_REGISTRY_SEED` | `true` | Seed built-in skills, templates, personas and prompts at startup. |

Moderation uses the admin token (`X-Admin-Token`):

```
GET  /api/v1/admin/registry?status=review
POST /api/v1/admin/registry/{slug}/approve      {"note": "..."}
POST /api/v1/admin/registry/{slug}/reject       {"note": "why"}
POST /api/v1/admin/registry/{slug}/flags        {"featured": true, "verified": true}
POST /api/v1/admin/registry/{slug}/unpublish    {"note": "..."}
GET  /api/v1/admin/registry/import              published listings per source
POST /api/v1/admin/registry/import              {"items": [...], "refresh": false}  (up to 500 harvested rows)
POST /api/v1/admin/registry/import/check        {"slugs": [...]} -> {"existing": [...]}  (up to 2000 slugs)
POST /api/v1/admin/registry/import/refresh      run the scheduled harvest now (202; 409 while running)
```

Paid purchases need billing to be configured (Stripe keys); without it `POST .../install` on a paid
listing answers `402`.

## Security notes

- Manifests are validated against the OCM / stack schemas and size-limited; descriptions are
  rendered without raw HTML, scripts or images and external links carry `rel="nofollow"`.
- Installing never runs anything: it returns a document. Review what a template or agent points at
  (endpoints, tools) before applying it, the same way you would review a Terraform module.
- Slugs are stable; a new version never changes the slug, so pin `@version` in production stacks.

## Related

* [Platform guide](https://opensmartroute.ai/docs/PLATFORM.md) - authentication, routing, plans, organizations and the dashboard.
* [REST API reference](https://github.com/isathish/OpenSmartRoute/blob/main/platform/api/openapi.json) - the *Marketplace* group lists every endpoint (rendered at `/docs/api/marketplace`).
* [User guide](https://opensmartroute.ai/docs/GUIDE.md) - stack files, `osr stack` and the Open Capability Manifest.

---

<!-- Page: Cost estimates and the MCP server (https://opensmartroute.ai/docs/MCP) -->

<!-- OpenSmartRoute: Cost estimates and the MCP server. Source https://github.com/isathish/OpenSmartRoute/blob/main/docs/MCP.md; HTML https://opensmartroute.ai/docs/MCP -->
# Cost estimates, recommended models and the MCP server

Three ways to ask the router "what will this cost and what should I use?" before spending tokens:
a quote API and page, a recommended-models digest, and a [Model Context Protocol](https://modelcontextprotocol.io)
server that puts all of it inside VS Code, Cursor, Claude, Windsurf or any agent framework. The
same pieces exist in the SDK (`opensmartroute.estimate`, `opensmartroute.mcp_server`, `osr estimate`,
`osr mcp`) and on the hosted platform (`POST /api/v1/estimate`, `GET /api/v1/models/recommended`,
`/mcp`, the `/estimate` page and *Dashboard -> Integrations*).

## 1. Quotes

A quote routes the request exactly as `route()` would - signals, policy, strategies, learned
quality - and then prices every ranked candidate instead of executing one:

- **Tokens.** Input tokens come from the text (word / digit / punctuation / CJK / code heuristic,
  `estimate_tokens`) plus the history (`estimate_messages_tokens`, 4 tokens of framing per message).
  Output tokens are the caller's `output_tokens`, else the signals' predicted answer length, else
  `DEFAULT_OUTPUT_TOKENS` (256); `output_tokens_source` says which.
- **Prices.** `usd_per_1k_input` / `usd_per_1k_output` on the target when declared, else the blended
  `usd_per_1k_tokens`; a fixed `usd_per_call` is added for agents, tools and humans. On the platform a
  target that resolves to a reference model is priced from the live catalogue (`priced_from:
  "catalogue"`), so quotes follow vendor list prices.
- **Picks.** `recommended` is the router's decision; `best_quality` the highest quality estimate;
  `cheapest` and `fastest` are chosen among candidates within `quality_tolerance` (0.1) of the best
  that fit their context window, so a free-but-useless target never wins. `savings_usd` is the gap
  between the dearest candidate and the recommended one.
- **Nothing is executed and the text is never stored.**

```python
from opensmartroute.estimate import estimate

q = estimate(router, "Summarise this contract in five bullets", output_tokens=300)
print(q.input_tokens, q.output_tokens, q.output_tokens_source)   # 9 300 caller
print(q.recommended.name, q.recommended.cost_usd)                # Small model 0.000062
for t in q.targets:                                              # every candidate, ranked
    print(t.rank, t.target_id, f"${t.cost_usd:.6f}", t.quality_estimate, t.latency_ms, t.fits_context)
print(q.cheapest.target_id, q.best_quality.target_id, q.fastest.target_id, q.savings_usd)
q.to_dict()                                                      # the JSON the API returns
```

`estimate(router, request, *, output_tokens=None, kinds=None, prices=None, quality_tolerance=0.1,
decision=None)` accepts a `RouteRequest` or a string, a `kinds` filter and a `prices(target) ->
(usd_per_1k_in, usd_per_1k_out) | None` hook for live price feeds; pass `decision=` to quote what an
earlier `route()` already ranked.

```bash
osr -t targets.yaml estimate "Translate this contract into German" --monthly 100000
osr -t targets.yaml estimate "..." --output-tokens 500 --cost-weight 1.0 --json
```

### `POST /api/v1/estimate`

Same body as `POST /api/v1/route` (`text`, `history`, `objective`, `constraints`, `kinds`) plus
`output_tokens` and `monthly_requests`. Without a key the endpoint is rate limited per client (20 per
minute, 8 000 characters) and ignores tenant rules; with a key it is metered as `estimate`, applies
your plan and tenants and returns `X-Request-Id`.

```bash
curl -s https://<platform>/api/v1/estimate -H "Authorization: Bearer $OSR_KEY" \
  -H "Content-Type: application/json" \
  -d '{"text": "Summarise this contract...", "monthly_requests": 50000}'
```

```json
{
  "request_id": "…", "input_tokens": 9, "output_tokens": 256, "output_tokens_source": "signals",
  "total_tokens": 265, "confidence": 0.71, "savings_usd": 0.00209,
  "recommended": {"target_id": "llm-small", "name": "Small model", "vendor": "openai", "model": "openai/gpt-4.1-nano",
                  "input_cost_usd": 0.0000009, "output_cost_usd": 0.0001, "cost_usd": 0.000103,
                  "latency_ms": 400, "quality_estimate": 0.62, "fits_context": true, "priced_from": "catalogue",
                  "rationale": {"semantic": "…", "cost": "…"}},
  "cheapest": {"…": "…"}, "best_quality": {"…": "…"}, "fastest": {"…": "…"},
  "targets": [{"…": "…"}],
  "signals": {"complexity": 0.31, "domains": ["legal"], "task_type": "summarise", "contains_pii": false, "…": "…"},
  "policy_rejections": {},
  "monthly": {"requests": 50000, "recommended_usd": 5.15, "cheapest_usd": 5.15, "best_quality_usd": 109.6,
              "dearest_usd": 109.6, "savings_usd": 104.45, "tokens": 13250000},
  "anonymous": false
}
```

The `/estimate` page is the same call with a form: paste a prompt, choose what to optimise for, set the
expected answer length and monthly volume, and read the four picks, the signals the router saw and the
full candidate table with input / output cost bars.

## 2. Recommended models

`GET /api/v1/models/recommended` (no key) quotes a representative prompt per use case - `chat`, `code`,
`reasoning`, `summarise`, `extraction`, `pii` - against the live catalogue and returns, for each, the
`recommended`, `cheapest`, `best_quality` and `fastest` pick with cost per request, quality estimate,
latency, 30-day traffic, the reference model's intelligence index and the reasons (`why`). A
`leaderboard` counts how often each target was the router's pick. The digest is recomputed at most
once a minute and follows the deployment's targets, prices and learned quality rather than a
hand-maintained list. It renders on `/models` ("Recommended per use case") and on `/estimate`.

## 3. MCP server

### What it exposes

| Tool | Purpose | Side effects |
|---|---|---|
| `route` | Pick the best target: decision, confidence, alternatives, signals, ranked candidates with reasons | none |
| `estimate` | The quote above for a request | none |
| `recommend` | One-paragraph recommendation for a task; `priority` = balanced, cost, quality or speed | none |
| `explain` | Human-readable trace: signals, policy filtering, per-strategy breakdown | none |
| `list_targets` | The catalogue (optionally one `kind`) with prices, latency, quality prior, boundary | none |
| `feedback` | Report an outcome (`request_id`, `target_id`, `success`, `quality`, `cost_usd`, `latency_ms`) | learners update |
| `ask` | Route **and** run the chosen target / plan; returns the answer, tokens and cost | executes |
| `marketplace_search`, `marketplace_get` | Search listings; fetch one with its manifest and snippets | none |

Resources: `osr://targets` (the catalogue as JSON) and `osr://stats` (feedback statistics). The
server speaks MCP `2025-06-18` (also accepts `2025-03-26` and `2024-11-05`) over JSON-RPC 2.0:
`initialize`, `ping`, `tools/list`, `tools/call`, `resources/list`, `resources/read`,
`prompts/list`, `logging/setLevel`; notifications get no reply; unknown methods return `-32601`,
unknown tools `-32602`, and tool failures come back as `isError` results, never as protocol errors.

`ask` and `marketplace_*` appear only when the server was given an `execute` / `marketplace` hook:
`osr serve` and the platform provide both; a bare `MCPServer(router)` exposes the read-only tools and
`feedback`.

### Hosted platform

`GET /mcp` describes the server (tools, resources, protocol version, connection snippets) without a
key. `POST /mcp` takes one JSON-RPC message or a batch with the same `Authorization: Bearer <api key>`
(or `X-API-Key`) header as the REST API; notifications return `202`. Every routed call carries the
workspace tenant and is metered as `mcp` against the plan's quotas; `feedback` needs the feedback
feature and `ask` the plan feature (Pro and above). Quota, plan and routing errors surface as tool
errors such as `429: rate limit exceeded`. *Dashboard -> Integrations* renders ready-to-paste
configuration for each client with the workspace's key.

HTTP transport (the client talks to `https://<platform>/mcp` directly):

```jsonc
// VS Code: .vscode/mcp.json
{"servers": {"opensmartroute": {"type": "http", "url": "https://<platform>/mcp",
                                "headers": {"Authorization": "Bearer osr_..."}}}}
// Cursor: .cursor/mcp.json
{"mcpServers": {"opensmartroute": {"url": "https://<platform>/mcp",
                                   "headers": {"Authorization": "Bearer osr_..."}}}}
// Windsurf: ~/.codeium/windsurf/mcp_config.json
{"mcpServers": {"opensmartroute": {"serverUrl": "https://<platform>/mcp",
                                   "headers": {"Authorization": "Bearer osr_..."}}}}
```

```bash
claude mcp add --transport http opensmartroute https://<platform>/mcp --header "Authorization: Bearer osr_..."
```

Stdio bridge for clients that only launch local processes (Claude Desktop and older clients): the
CLI forwards each line to the platform with your key.

```json
{"mcpServers": {"opensmartroute": {"command": "osr",
                                   "args": ["mcp", "--url", "https://<platform>/mcp", "--api-key", "osr_..."]}}}
```

### Self-hosted and local

`osr serve` mounts the same server at `POST /mcp` (and `GET /mcp` for the description) next to
`/route` and `/v1`, with `ask` backed by the configured providers. Without a service, the CLI runs the
server over stdio straight from a catalogue - useful for a laptop IDE setup:

```bash
osr -t targets.yaml -r rules.yaml mcp               # newline-delimited JSON-RPC on stdin/stdout
osr -t targets.yaml mcp --list-tools                 # what the IDE will see
osr mcp --url https://<platform>/mcp --api-key osr_... # bridge to a hosted platform
```

```json
{"servers": {"opensmartroute": {"type": "stdio", "command": "osr",
                                "args": ["-t", "targets.yaml", "-r", "rules.yaml", "mcp"]}}}
```

### In Python

```python
from opensmartroute.mcp_server import MCPServer, serve_stdio

server = MCPServer(router, execute=my_execute, marketplace=my_marketplace, prices=my_prices)
server.tools()                                   # tool specs (tools/list)
server.call("estimate", {"text": "…"})           # {"content": [...], "structuredContent": {...}}
reply = server.handle({"jsonrpc": "2.0", "id": 1, "method": "tools/call",
                       "params": {"name": "recommend", "arguments": {"task": "…", "priority": "cost"}}})
serve_stdio(server)                              # block on stdin/stdout
```

`execute(request) -> dict` enables `ask`; `marketplace(action, args)` with `action` in `search` /
`get` enables the marketplace tools; `prices` is the quote hook above. `RemoteMCP(url, api_key)` and
`bridge_stdio(url, api_key)` are the client side used by `osr mcp --url`. Everything is standard
library only; `osr serve` needs the `server` extra (FastAPI).

### Things to ask your assistant

- "Estimate what this prompt will cost and which model to use."
- "Recommend the cheapest model that can refactor this file."
- "Explain how OpenSmartRoute would route this request."
- "Search the marketplace for a SQL reporting skill and show me its manifest."
- "Route and answer this with the best model, then record that the answer was good."

<!-- Section: Python SDK -->

---

<!-- Page: SDK guide (https://opensmartroute.ai/docs/SDK) -->

<!-- OpenSmartRoute: SDK guide. Source https://github.com/isathish/OpenSmartRoute/blob/main/docs/SDK.md; HTML https://opensmartroute.ai/docs/SDK -->
# SDK Guide

## Stability & versioning

- **SemVer.** Everything exported from `opensmartroute` (top-level) is public API.
  `opensmartroute.enterprise`, `.security`, `.learning`, `.realtime`, `.math`, `.eval`, `.aio`,
  `.adapters`, `.config` are public too but may add fields between minor versions.
- Modules prefixed `_` and anything not in an `__all__` are private.
- The complete list of public modules and names, with one-line summaries, is generated into
  [REFERENCE.md](https://opensmartroute.ai/docs/REFERENCE.md) from the source (`python scripts/api_reference.py`).
- Deprecations are announced one minor version ahead with `DeprecationWarning`.

## Installation matrix

`curl -LsSf https://opensmartroute.ai/install.sh | sh` (Linux, macOS) and
`irm https://opensmartroute.ai/install.ps1 | iex` (Windows) install the `osr` CLI with the `yaml` and
`server` extras in an isolated tool environment (uv, pipx or a private venv). For the library, pick
the extras you need:

| Extra | Adds | Use when |
|---|---|---|
| *(none)* | pure stdlib | embedding in any Python 3.10+ service |
| `yaml` | PyYAML | loading `targets.yaml` / `rules.yaml` |
| `server` | FastAPI, uvicorn, pydantic | running `osr serve` |
| `embeddings` | sentence-transformers | semantic `SimilarityStrategy` |
| `otel` | opentelemetry-api/sdk | `adapters.OpenTelemetryTelemetry` |
| `crypto` | cryptography | AES-GCM encrypted state stores |
| `dev` | pytest, ruff, mypy, bandit | contributing |
| `all` | every extra above | the installer's `OSR_EXTRAS=all` |

## Connecting real providers

`adapters.OpenAICompatClient` is a stdlib HTTP client for any OpenAI-compatible API (OpenAI, Azure
OpenAI, vLLM, Ollama, LiteLLM, OpenRouter, Groq, Mistral). It has explicit timeouts, exponential
backoff on 408/429/5xx, and maps failures to SDK errors (`ConfigurationError` for 401/403,
`TargetUnavailableError` for connectivity/5xx, `ExecutionError` for the rest). The API key is read
from `OPENAI_API_KEY` (or `OPENAI_API_KEY_FILE`) and never appears in logs or exceptions.

```python
from opensmartroute.adapters import OpenAICompatClient, chat_handler, embedder, judge_fn

openai = OpenAICompatClient()                                   # https://api.openai.com/v1
local  = OpenAICompatClient("http://localhost:11434/v1", api_key="ollama")   # Ollama

registry = TargetRegistry([
    RouteTarget("gpt-4o-mini", TargetKind.LLM, cost={"usd_per_1k_tokens": 0.00015},
                handler=chat_handler(openai, model="gpt-4o-mini")),
    RouteTarget("llama-local", TargetKind.LLM, cost={"usd_per_1k_tokens": 0.0},
                constraints=TargetConstraints(data_boundary="on_prem"),
                handler=chat_handler(local, model="llama3.1")),
])

router = Router(
    registry,
    strategies=[CapabilityStrategy(),
                SimilarityStrategy(embedder=embedder(openai, model="text-embedding-3-small"))],
    llm_judge=LLMJudgeStrategy(judge_fn(openai, model="gpt-4o-mini")),
    escalate_llm_judge_below=0.5,
)

decision = router.route(request)
result = router.execute(decision, request)      # ChatResult(text, tokens, latency_ms); outcome auto-recorded
```

`chat_handler` builds the message list from `request.history`, and uses `request.context["system"]`
as the system prompt — which is how a **persona** slot in a `RoutePlan` changes model behaviour
without a separate deployment.

Optional adapters (lazy-imported):

```python
from opensmartroute.adapters import sentence_transformers_embedder, OpenTelemetryTelemetry
SimilarityStrategy(embedder=sentence_transformers_embedder("all-MiniLM-L6-v2"))
RouterBuilder(reg).with_telemetry(OpenTelemetryTelemetry())
```

## Loading configuration

```python
from opensmartroute.config import load_targets, load_rules
registry = load_targets("targets.yaml")      # or .json
rules    = load_rules("rules.yaml")
```

Both raise `ConfigurationError` with the file, the offending entry and the reason. Only
`yaml.safe_load` is used.

## Executing end to end

`Router.route()` returns a decision; `Router.run()` (also on `AsyncRouter` and `EnterpriseRouter`)
routes **and** executes the resulting plan, returning an `ExecutionResult`. `execute(decision, request)`
is still available and returns just the primary handler's raw response.

```python
res = router.run("Write SQL for monthly active users by region", task_id="job-17")
res.text              # best-effort text of the primary response
res.response          # the raw handler response (ChatResult, HarnessResult, dict, str, ...)
res.target_id         # primary target
res.steps             # [ExecutionStep(role, target_id, latency_ms, ok, note), ...]
res.system_prompt     # what was composed into context["system"] (None if nothing was added)
res.effective_request # the RouteRequest the primary handler actually received
res.outcomes          # one Outcome per participant, already passed to router.learn()
```

What `run()` does, in order:

1. `route(request, plan=True)` builds the plan: primary + optional `persona` and `skill` slots. A
   slot is only filled when its best candidate's quality estimate is at least
   `Router(slot_quality_floor=0.5)` and (for `run`) its confidence is at least `min_slot_confidence`.
2. **Skill pre-processing.** If the `skill` slot's target has a `handler`, it is called with the
   request and may return
   - a `RouteRequest` — replaces the request the primary sees (rewrite / enrich);
   - a `str` — attached as `context["skill_output"]` and appended to the system prompt;
   - `None` — the skill contributes instructions only.
   Exceptions are recorded as a failed step (and a failed `Outcome`) but do not abort the run.
3. **Prompt composition.** `context["system"]` is set to: any existing `system` + `# Persona: <name>`
   + `RouteTarget.instructions` of the persona + `# Skill: <name>` + skill instructions + the
   primary's own `instructions` (for `llm`/`agent` targets) + `# Skill output`. `chat_handler` and the
   harness adapters read this key.
4. **Primary call.** `target.handler(request, **kw)`. Missing handler → `ExecutionError`. In the sync
   path a coroutine result raises `ExecutionError("use aexecute()")`; `AsyncRouter.run()` awaits both
   skill and primary handlers.
5. **Outcomes.** Success is read from `response.success` / `response["success"]` (default `True`);
   cost from `response.cost_usd` or `total_tokens × unit_cost`; quality from `response.quality`.
   One `Outcome` is created for the primary (`role=None`) and one per slot (`role="persona"|"skill"`),
   all carrying `task_id` (argument or `request.context["task_id"]`) so delayed feedback can be
   joined later. Each is passed to `learn`; on a handler exception, failure outcomes are recorded
   and the exception re-raised.

### Targets carry `instructions`

```yaml
- id: persona-legal-counsel
  kind: persona
  primary: false
  instructions: |
    You are cautious in-house legal counsel. Cite the governing clause...
- id: skill-sql
  kind: skill
  instructions: "Always qualify table names with the schema; never SELECT *."
```

`instructions` is the persona's system prompt, a skill's body (SKILL.md style) or an agent's
standing instructions. It is disclosed to the model only when the target is part of the plan.

### Agent harnesses

An agent harness is any runtime that takes a task and drives its own loop (coding agent, research
agent, OpenHands, SWE-agent, Claude Code, LangGraph graph, ...). Adapters live in
`opensmartroute.adapters` and all produce a `HarnessResult(text, success, steps, prompt_tokens,
completion_tokens, cost_usd, latency_ms, quality, artifacts, raw)`:

```python
from opensmartroute.adapters import CallableHarness, HTTPHarness, SubprocessHarness, harness_handler

# in-process: fn(task, context, history) -> str | dict | HarnessResult
registry.get("research-agent").handler = harness_handler(CallableHarness(my_graph.invoke))

# HTTP: POST {"task", "context", "history"} -> JSON {"text"|"output", "success", "usage", "cost_usd"}
registry.get("support-agent").handler = harness_handler(
    HTTPHarness("https://agents.internal/support/run", api_key_env="AGENT_TOKEN", timeout_s=300))

# CLI: task on stdin, answer on stdout (or JSON with json_output=True); system prompt via a flag
registry.get("coding-agent").handler = harness_handler(
    SubprocessHarness(["my-agent", "--repo", "."], system_flag="--system", timeout_s=900))
```

Non-zero exit codes and `"success": false` become failed outcomes; HTTP 408/429/5xx, timeouts and
unreachable hosts raise `TargetUnavailableError` (so `HealthRegistry` breakers trip); other HTTP
errors raise `ExecutionError`; a missing binary or non-http URL raises `ConfigurationError`.
Implement the `AgentHarness` protocol (`run(task, *, context, history) -> HarnessResult`) for anything
else.

### Skills from `SKILL.md`

Folders following the [Agent Skills](https://agentskills.io/specification) layout load directly:

```python
from opensmartroute.adapters import load_skill, load_skills, skill_from_markdown
for skill in load_skills("skills/"):     # skills/<name>/SKILL.md, name must match the folder
    registry.add(skill)
```

Frontmatter `name`/`description` become `id`/`examples`; the Markdown body becomes `instructions`
(progressive disclosure: only the description is used for routing, the body is shown to the model
once selected). Routing metadata can be added under `metadata:` — `osr-domains`, `osr-actions`,
`osr-languages`, `osr-tags`, `osr-quality-prior`, `osr-primary` (`"false"` for slot-only skills),
`osr-latency-ms`. `license`, `compatibility`, `allowed-tools` and other metadata are preserved in
`RouteTarget.metadata`. Give the returned target a `handler` to make it an executable pre-processor
or primary skill.

## Learning from outcomes

Every learner (bandits, IRT, preference, LinUCB, Markov, task table, judge calibration) consumes the
same `Outcome`:

```python
from opensmartroute import Outcome
d = app.route(req)
resp = call_provider(d.target, req)
app.learn(Outcome(request_id=req.request_id, target_id=d.target.id, success=True,
                  quality=grade(resp), cost_usd=resp.cost, latency_ms=resp.ms))
```

- **Plans**: `Router.run()` records one `Outcome` per participant with `role="skill" | "persona" |
  "primary"`; learners keep a target's record as a skill separate from its record as the primary
  answerer (`signals.extra["role"]`).
- **Cascades**: `Cascade.run(request, d.trace.ranked)` returns a `CascadeResult`; feed every step to the
  learners with `app.learn_cascade(request, result)` (rejected steps are failures with the gate score as
  quality, the accepted step a success). `result.total_cost_usd` is the spend across steps.
- **Delayed / task-level rewards**: set `Outcome.task_id` (and `step`) so a reward observed at the end
  of a multi-step task credits every call in the trajectory (`Router(task_credit=...)`).
- **Offline warm start**: `learning.warm_start_from_matrix(rows, strategies, weight=0.5)` replays a
  full-information reward matrix `(prompt, {target_id: reward})` from an offline evaluation into every
  learner before serving traffic; `learning.warm_start()` seeds a brand-new target from its neighbours.
- **Replicas**: `learning.merge_learners(local, remote)` folds another replica's learners into yours
  (sufficient statistics only, no prompts); `AutoLearner.refresh()` re-loads a writer's snapshot from a
  shared `StateStore`. Corrupt state is quarantined on load (`AutoLearner.quarantined`) instead of
  raising.
- **Objective**: `Objective(quality, cost, latency, quality_floor, energy, carbon)`; `energy` and
  `carbon` weigh `RouteTarget.unit_energy` (`cost["wh_per_1k_tokens"]`) and `unit_carbon`
  (`cost["gco2_per_1k_tokens"]` or energy x `cost["gco2_per_wh"]`).
- **Cold-start exploration**: a target declared with only `id`, `kind` and `cost` is never ranked
  first and so never earns an outcome. `Router(explore_rate=0.1, explore_min_samples=200)` (or
  `RoutingSettings.explore_rate` / `explore_min_samples`) serves the least-seen viable candidate with
  probability `explore_rate` until it has that many outcomes; `decision.propensities` includes the
  exploration mass (so off-policy estimates stay unbiased) and `signals.extra["explored"]` marks the
  request. Off by default.
- **Roadmap exit criteria**: `opensmartroute.eval.criteria` turns each offline-measurable exit criterion
  in [ROADMAP.md](https://opensmartroute.ai/docs/ROADMAP.md) into a simulation of the real router (`cold_start_ratio()`,
  `effort_token_savings()`, `multi_round_vs_best_single()`, `match_at_1_at_scale()`,
  `conformal_coverage()`, `knapsack_never_exceeds_cap()`, `ope_within_live_ci()`) and
  `opensmartroute.eval.agentic` adds the tau-bench-style one (`AgentTask`, `load_agentic_tasks()`,
  `synthetic_agentic_tasks()`, `task_routing_frontier()`: per-step routing through `ProgressRouter`
  against every always-use-agent-X policy on the accuracy-latency frontier).
  `python scripts/exit_criteria.py [--full]` runs them all; on your own data, `osr eval DATASET
  --multi-round` runs the multi-round comparison on scored rows, `osr eval DATASET --effort` the
  effort-routing token comparison on rows that carry per-target `scores` and `tokens` (`EvalRow.tokens`),
  and `osr eval TASKS.jsonl --agentic` the task-routing frontier on tasks with per-agent per-step
  `success` and `latency_ms` tables.
- **Contrastive and policy-gradient learners**: `learning.ContrastiveRouter().fit(rows, targets)`
  trains on scored `EvalRow`s against each row's acceptable set (`objective="distilled"` uses
  reward-softened labels); wrap it in `learning.ContrastiveStrategy(model)` to keep learning online.
  `learning.PolicyGradientStrategy()` learns a routing policy directly from `decision_reward()`
  (quality minus weighted cost and latency) with no quality predictor in between;
  `learning.decision_regret(rows, targets, choose, predict)` compares decision regret with prediction
  error on the same rows.
- **Headroom before you learn**: `eval.headroom.routing_headroom(rows, targets)` tells you how much of
  the oracle gap is measurable on your data; `target_diversity`, `min_catalogue`, `scaling_curve` and
  `learnability_by_difficulty` answer whether, and where, a learned router can beat the best single
  target.
- **Other target kinds**: `strategies.SemanticCache.target()` turns a semantic cache into a
  `DESTINATION` target scored by `SemanticCacheStrategy`; `strategies.AnnotatorPool` +
  `HumanRoutingStrategy` route among `HUMAN` annotators with Dawid-Skene skill estimates and
  `select_quorum()`; `strategies.MemoryRouter` routes *what to remember* across memory tiers under a
  token budget; `strategies.ModalityStrategy` / `ModalityEscalation` handle text-first escalation to
  multimodal targets; `strategies.SpeculativeCascade` runs draft and strong models in parallel when the
  learned acceptance rate makes that cheaper.

## Error handling

All exceptions derive from `OpenSmartRouteError` and carry `.code` and `.details`:

```python
from opensmartroute import OpenSmartRouteError, NoRouteError, SecurityError

try:
    d = app.route(req)
except NoRouteError as e:
    # e.details["rejections"] -> {target_id: reason}
    fallback()
except SecurityError as e:
    reject_request(e.to_dict())
except OpenSmartRouteError:
    log.exception("routing failed")
```

The SDK never raises bare `Exception`/`RuntimeError`; strategy failures (e.g. a broken LLM judge)
degrade to neutral scores rather than failing the route.

## Threading & async

- `Router` and `EnterpriseRouter` are safe to share across threads. `Router.lock` (a
  re-entrant lock) guards strategy scoring, every learner update (`learn`, `learn_correction`,
  `credit_task`), task pins, the request memory, retrieval narrowing and the exploration RNG,
  so a strategy's `score` never observes a half-applied `update`. The optional LLM judge is
  scored *outside* the lock (network I/O) - put it behind `escalate_llm_judge_below` so it
  runs rarely, and set `TimeoutMiddleware`.
- `AutoLearner` takes the same lock (`EnterpriseRouter` / `RouterBuilder` pass `router.lock`;
  pass it yourself when wiring by hand) so `learn`, `reset_target`, `save`, `load` and
  `refresh` serialise against routing. `save()` deep-copies a sequence-numbered snapshot under
  the lock and writes it outside; a separate I/O lock orders writers and readers so a slow save
  never overwrites a newer snapshot and two saves never race on the state files.
- Per-module state elsewhere has its own lock: `SemanticCache`, `MemoryRouter`,
  `AnnotatorPool`, `ModalityEscalation` (posterior) and `SpeculativeCascade` (acceptance
  posteriors, history, failure counters). Embedding / hashing runs outside those locks.
- `SpeculativeCascade` runs `concurrency` speculative requests at once (pool of
  `2 * concurrency` workers); `draft_timeout_s` abandons a slow draft and serves the strong
  answer; a draft that raises escalates instead of failing (`details["draft_error"]`); a strong
  future that is no longer needed is cancelled or its late failure swallowed. `stats()`
  reports runs, modes, failure counts and per-bucket acceptance.
- `AsyncRouter` wraps any router for `await`-based apps and awaits coroutine handlers;
  `route` / `learn` run in the default executor under the same lock.
- Routing has no I/O of its own.

### Multi-round execution

`ProgressRouter.run_step` routes through `router.route` (middleware and telemetry apply) and
executes with `execution.execute`; a handler that raises is recorded as a failed step for its
target *before* the exception propagates, so the next step already denies it after
`failures_before_switch`. Step cost falls back to `target.unit_cost x estimated tokens` when the
handler reports none, so `budget_usd` is tracked even for plain-string handlers. `arun_step` is
the coroutine twin.

`MultiRoundExecutor` (route -> execute -> judge -> refine) learns **once per round** with the
judged quality (no provisional learn + correction double count); a round whose handler raised
becomes a `Round` with `error` set, learns a failure for that target and the loop continues on a
different target. `stop_reason` is one of `accepted`, `budget`, `deadline` (`deadline_ms`) or
`max_rounds`; `return_best=True` (default) returns the highest-quality round when nothing reached
the threshold; `credit_task=True` delivers the final quality as the task-level reward through
`router.credit_task`, which re-credits every step (learners then see one judged update per round
plus one credited update). `arun` awaits coroutine handlers and an `async` judge.

## Type checking

The package ships `py.typed`; `mypy --strict`-friendly signatures on the public surface.

## Configuration

```yaml
# targets.yaml
targets:
  - id: llm-frontier
    kind: llm
    capabilities: { domains: [math, coding], min_complexity: 0.5, supports_tools: true }
    constraints:  { data_boundary: public, pii_allowed: false, regions: [us, eu] }
    cost: { usd_per_1k_tokens: 0.015 }
    latency_ms: 2500
    quality_prior: 0.93
    examples: ["Prove that sqrt(2) is irrational."]
```

```yaml
# rules.yaml  (Arch-Router-style preferences)
rules:
  - name: escalation-intent
    when: { actions: [escalate] }
    prefer: [human-escalation]
    pin: true          # restrict candidates, not just boost
```

Secrets are **never** in YAML. Handlers read them at call time:

```python
from opensmartroute.security import load_secret
api_key = load_secret("OPENAI_API_KEY")   # env var or OPENAI_API_KEY_FILE
```

## Extending

### Decorator SDK

Every routing component - targets, strategies, signal extractors, policy rules, middleware and
telemetry sinks - can be declared with a decorator and assembled by a `ComponentRegistry`. The
decorators are transparent (they return the decorated object unchanged) and the registry stores
*blueprints* (factories), so one declaration can back many independent routers.

```python
import opensmartroute as osr

@osr.tool("weather", domains=["travel"], actions=["lookup"], cost=0.0, examples=["weather in paris"])
def weather(request, **kw):
    """Current weather for a city."""            # first paragraph -> target.description
    return lookup(request.text)

@osr.agent("planner", domains=["travel"], min_complexity=0.3, quality_prior=0.9)
def planner(request, **kw):
    """Plan a multi-day trip."""
    return plan(request.text)

@osr.strategy(weight=0.8)                        # fn(request, signals, candidates) -> {id: score}
def geo_affinity(request, signals, candidates):
    region = request.constraints.region
    return {t.id: 1.0 if region in t.constraints.regions else 0.4 for t in candidates}

@osr.strategy                                    # or a Strategy subclass; name inferred -> "recency"
class RecencyStrategy(osr.Strategy):
    def score(self, request, signals, candidates): ...

@osr.signal(order=-1)                            # runs before the built-in extractors
def urgency(request, signals):
    return {"urgent": "asap" in request.text.lower()}   # unknown keys land in signals.extra

@osr.policy_rule                                 # fn(target, request, signals) -> reason | None
def business_hours_only(target, request, signals):
    if target.kind is osr.TargetKind.HUMAN and request.context.get("after_hours"):
        return "human queues are closed"
    return None

@osr.middleware                                  # fn(request, next_) -> RouteDecision
def tag_tenant(request, next_):
    request.context.setdefault("tenant", "public")
    return next_(request)

router = osr.components.router()                 # Router: targets + strategies + signals + rules
app = osr.components.builder().with_auto_learning().build()   # EnterpriseRouter variant
```

| Decorator | Accepts | Materialised as |
|---|---|---|
| `@target(id, kind, ...)`, `@tool`, `@skill`, `@agent` | handler `fn(request, **kw)` | `RouteTarget(handler=fn)` |
| `@strategy(name=, weight=, order=)` | `Strategy` subclass or scorer function | `Strategy` / `FunctionStrategy` |
| `@signal(name=, order=)` | `SignalExtractor` subclass or `fn(request, signals)` | `SignalExtractor` / `FunctionSignal` |
| `@policy_rule(name=, order=)` | `fn(target, request, signals) -> str \| None` or `Policy` subclass | rule appended to `Policy.with_rules` |
| `@middleware(name=, order=)` | `Middleware` subclass or `fn(request, next_)` | `Middleware` / `FunctionMiddleware` |
| `@telemetry(name=, order=)` | `Telemetry` subclass | `Telemetry` |

- `osr.components` is the process-wide registry the top-level decorators bind to. Create your own
  `ComponentRegistry()` for isolation (tests, multi-tenant apps) - its methods are the same
  decorators.
- Duplicate names raise `ConfigurationError` (`register(..., replace=True)` to override);
  `unregister`, `clear`, `merge` and `add(*instances)` manage the catalogue.
- `RouterBuilder.with_components(registry, targets=, strategies=, signals=, policy=, middleware=,
  telemetry=)` wires a registry into an existing builder selectively.
- Plugins: `registry.include("pkg.module")` imports a module so its decorators run
  (`"pkg.module:setup"` calls `setup(registry)` afterwards); `registry.discover()` loads every
  installed distribution advertising an `opensmartroute.plugins` entry point.

### Subclassing

```python
from opensmartroute import Strategy, StrategyScore

class GeoAffinity(Strategy):
    name = "geo"
    def score(self, request, signals, candidates):
        region = request.context.get("region")
        return {t.id: StrategyScore(1.0 if region in t.constraints.regions else 0.5,
                                    f"region={region}") for t in candidates}

app = RouterBuilder(reg).with_defaults().with_strategy(GeoAffinity(), weight=0.8).build()
```

Custom `Policy`, `SignalExtractor`, `Middleware`, `Telemetry`, `StateStore`, `AuditSink` follow
the same pattern — subclass, implement, pass to the builder (or register with a decorator).

## Settings

Every tunable lives in one immutable `Settings` object, grouped by consumer
(`routing`, `policy`, `rules`, `capability`, `bandit`, `weights`, `slm`, `server`, `observability`). Components resolve the
process-wide settings unless given an explicit instance.

```python
import opensmartroute as osr
from opensmartroute.settings import RoutingSettings, WeightSettings

osr.configure(routing=RoutingSettings(narrow_above=200), weights=WeightSettings(rules=3.0))
router = osr.Router(reg)                                   # uses the configured settings
isolated = osr.Router(reg, settings=osr.Settings())        # library defaults, unaffected
print(osr.get_settings().env_keys())                       # every honoured OSR_* variable
```

Environment overlay: `OSR_<GROUP>_<FIELD>` (e.g. `OSR_ROUTING_SOFTMAX_TEMPERATURE=0.2`,
`OSR_WEIGHTS_CAPABILITY=1.5`, `OSR_POLICY_DATA_BOUNDARIES=public,internal,confidential`).
Values are parsed against the default's type; a bad value raises `ConfigurationError` with
`details={"env": key}`. `osr.configure()` with no arguments re-reads the environment.
`osr settings [--json]` prints the effective values, their `OSR_*` keys and which ones the
environment overrides.

## Naming conventions

`opensmartroute.branding` is the single source of truth for every brand-derived name:

| Convention | Value | Helper |
|---|---|---|
| Package / import | `opensmartroute` | `branding.PACKAGE` |
| CLI | `osr` | `branding.CLI` |
| Environment variables | `OSR_<GROUP>_<FIELD>` | `branding.env_key("routing", "narrow_to")` |
| Error codes | `OSR_<KIND>` (`OSR_NO_ROUTE`, `OSR_CONFIG`, ...) | `branding.error_code("no_route")` |
| Metadata / frontmatter keys | `osr-<field>` (`osr-quality-prior`) | `branding.metadata_key("quality_prior")` |
| State directory | `.osr-state` | `branding.STATE_DIR` |
| Loggers | `opensmartroute[.component]` | `branding.logger("enterprise")` |
| HTTP `User-Agent` | `opensmartroute/<version>` | `branding.user_agent()` |
| Plugin entry-point group | `opensmartroute.plugins` | `branding.ENTRY_POINT_GROUP` |
| Config directory | `~/.config/opensmartroute` (`%APPDATA%\opensmartroute`), `OSR_CONFIG_DIR` | `credentials.config_dir()` |
| Platform URL / key | `OSR_API_URL`, `OSR_API_KEY` | `branding.platform_url()`, `branding.API_KEY_ENV` |
| Access tokens | `osr_live_` (platform key), `osr_local_` (self-hosted server) | `credentials.token_kind()`, `generate_token()` |

Strategy names are snake_case nouns (`capability`, `llm_judge`); decorator-registered classes
drop the `Strategy`/`Signal`/`Middleware`/`Telemetry` suffix automatically (`GeoAffinityStrategy`
-> `geo_affinity`).

## HTTP API (`osr serve`)

| Method | Path | Body / response |
|---|---|---|
| POST | `/route` | `{text, context, objective, constraints, kinds, top_k, plan}` → `RouteDecision.to_dict()` |
| POST | `/feedback` | `Outcome` fields → `{status}` |
| GET | `/targets` | catalogue |
| GET | `/stats` | feedback aggregates (+ `health` snapshot when the router has one) |
| GET | `/healthz` | liveness |
| GET | `/whoami` | `{service, edition: "self-hosted", version, auth_required, authenticated, targets}` - what `osr whoami` shows |
| GET | `/v1/models` | OpenAI-compatible model list (`auto` + every primary target) |
| POST | `/v1/chat/completions` | OpenAI-compatible completion; `model: "auto"` routes, a target id pins |

`create_app(router)` takes a plain `Router` or an `EnterpriseRouter` (middleware, health, telemetry
and audit then apply to every request). Errors map to HTTP statuses: `SecurityError` /
`ValidationError` → 400, `NoRouteError` → 422, `TargetUnavailableError` → 503, `ExecutionError` → 502,
unknown pinned model → 404.

`create_app(router, auth_tokens=[...])` (CLI: `serve --token`, `--generate-token`, or
`OSR_SERVER_AUTH_TOKENS=a,b`) turns on access control: every path except `/healthz`, `/readyz`,
`/metrics`, `/whoami` and the OpenAPI documents needs `Authorization: Bearer <token>` or
`X-API-Key: <token>`, compared in constant time; anything else is `401` with `WWW-Authenticate:
Bearer`. `ServerSettings.require_auth` (`OSR_SERVER_REQUIRE_AUTH=1`, `serve --require-auth`) refuses to
start without a token. Tokens are opaque strings; `osr token generate` mints `osr_local_...` ones and
`osr login --url http://host:8000 --token ...` stores one for the CLI.

## CLI

```
osr login [--url URL] [--token T | --with-token] [--no-browser] [--profile P] [--json]
osr whoami [--json] | osr logout [--all] | osr token generate|create|list|revoke
osr -t targets.yaml -r rules.yaml route "text" [--plan] [--json] [--cost-weight 0.3]
osr -t targets.yaml -r rules.yaml eval dataset.jsonl [--frontier] [--multi-round --threshold 0.8 --max-rounds 3]
osr -t targets.yaml eval rows.jsonl --effort              # rows with per-target scores + tokens vs always-think
osr -t targets.yaml eval tasks.jsonl --agentic [--retries 1]   # per-step task routing vs best single agent
osr -t targets.yaml targets | stats | serve [--host --port] [--token T ...] [--generate-token] [--require-auth]
osr -t targets.yaml --skills .claude/skills route "text" --plan   # add SKILL.md packages as slot targets
osr skills [ROOT] [--json]                                       # validate + list SKILL.md packages
osr settings [--json]
```

`osr login` signs in to the hosted platform with the OAuth 2.0 device authorization grant (the
browser opens `/platform/cli/authorize`, you approve the code, the platform mints a workspace API key) or
stores a pasted token; `opensmartroute.credentials` (`CredentialStore`, `device_login`, `whoami`,
`PlatformClient`) is the library behind it and works with any injected transport.

## Interop roadmap

See [ROADMAP.md](https://opensmartroute.ai/docs/ROADMAP.md) — v0.7 covers MCP tool catalogues, A2A agent cards, the
OpenAI-compatible proxy mode, LangGraph / Microsoft Agent Framework nodes, and Redis/Postgres
`StateStore` adapters.

---

<!-- Page: Enterprise builder (https://opensmartroute.ai/docs/ENTERPRISE) -->

<!-- OpenSmartRoute: Enterprise builder. Source https://github.com/isathish/OpenSmartRoute/blob/main/docs/ENTERPRISE.md; HTML https://opensmartroute.ai/docs/ENTERPRISE -->
# Enterprise Architecture

## Design principles

| Principle | How it shows up |
|---|---|
| **Hexagonal / ports and adapters** | `StateStore`, `Telemetry`, `AuditSink`, `Policy`, `Strategy`, `SignalExtractor` are ports. In-memory and file adapters ship in `enterprise`; `enterprise.stores` adds Redis, SQL, versioned, encrypted, batched and namespaced stores; `adapters` provides the OpenAI-compatible client, agent harnesses, MCP / A2A / framework glue, sentence-transformers and OpenTelemetry. |
| **Separation of concerns** | Signals (what is asked), Policy (what is allowed), Strategies (what is good), Ensemble (decide), Execution, Learning. Each stage is a module with one job. |
| **Fail fast** | `RouterBuilder.build()` validates configuration (no targets, no strategies, duplicate names) before serving traffic. |
| **Fail safe** | Hard constraints are never traded off; if nothing is admissible you get `NoRouteError` with the rejection reasons, not a silent bad route. |
| **Explainability by default** | Every decision carries a `RouteTrace`; every strategy returns a rationale. |
| **Inductive learning** | Every strategy works for a brand-new target from its declared capabilities; learners start silent (confidence 0) and gain weight with evidence. |
| **Zero-dependency core** | Pure stdlib. Deterministic given a seed. Sub-millisecond per decision. |
| **Twelve-factor** | Config via YAML / env (`OSR_<GROUP>_<FIELD>` overrides every tunable in `opensmartroute.settings`), secrets via env or `*_FILE`, stateless router plus pluggable state store. |

## Design patterns used

- **Strategy**: `Strategy` subclasses are interchangeable scorers.
- **Chain of responsibility**: `Middleware` (guard, tenant, fair share, cache, timeout, shadow, router).
- **Builder**: `RouterBuilder` fluent assembly with validation.
- **Observer**: `Telemetry.on_decision / on_outcome / on_error`.
- **Registry**: `TargetRegistry`, `HealthRegistry`.
- **Circuit breaker / bulkhead / rate limiter**: `realtime`.
- **Template method**: `Policy.check` with `HealthPolicy` decorating the base policy.
- **Decorator**: `HealthPolicy(inner)`, `CostAwareBandit(inner)`, every `*StateStore(inner)` wrapper.
- **Facade**: `EnterpriseRouter`, `AsyncRouter`.
- **Memento**: `state()/load()` on every learner; `AutoLearner` atomic snapshots.

## Component view

```mermaid
flowchart TB
    subgraph Edge
        API[HTTP / gRPC / SDK call]
    end
    subgraph Middleware chain
        G[GuardMiddleware<br/>size, gadget, redaction]
        T[TenantMiddleware<br/>defaults, limits]
        F[FairShareMiddleware<br/>dominant-resource fairness]
        C[CacheMiddleware<br/>LRU + TTL]
        TO[TimeoutMiddleware]
        SH[ShadowMiddleware<br/>shadow / A-B + SPRT]
    end
    subgraph Core router
        S[Signals]
        P[Policy + HealthPolicy]
        RT[Retriever<br/>BM25 + dense, RRF]
        ST[Strategies<br/>rules, capability, similarity, task table, irt, preference, linucb, markov, health, queue, bandit]
        E[Ensemble + Objective + calibration]
        D[Decision + Trace + Plan + candidate set]
    end
    subgraph Learning loop
        O[Outcome]
        AL[AutoLearner<br/>fan-out, drift, persist]
        H[HealthRegistry<br/>breaker, rate, budget]
        IF[InflightTracker]
    end
    subgraph Ports
        SS[(StateStore<br/>memory, file, Redis, SQL<br/>versioned, encrypted, batched)]
        TEL[Telemetry]
        AU[AuditSink]
        FB[(FeedbackStore)]
    end
    API --> G --> T --> F --> C --> TO --> SH --> S --> P --> RT --> ST --> E --> D
    D --> TEL
    D --> AU
    O --> AL --> ST
    O --> H --> P
    IF --> ST
    AL --> SS
    O --> FB
```

## Builder surface

| Method | Wires |
|---|---|
| `with_rules(rules)` | `RulesStrategy` (pin / prefer / avoid) evaluated first |
| `with_defaults()` | `CapabilityStrategy`, `SimilarityStrategy`, `BanditStrategy` |
| `with_strategy(s, weight)` | any `Strategy`, optional ensemble weight |
| `with_auto_learning(state_dir)` | `IRTStrategy`, `PreferenceStrategy`, `LinUCBStrategy`, `MarkovStrategy` behind an `AutoLearner` with drift detection and atomic persistence |
| `with_state_store(store, save_every)` | persist learner state through any `StateStore` instead of files |
| `with_health(latency_slo_ms)` | `HealthRegistry`, `HealthPolicy`, `HealthStrategy` |
| `with_queue_awareness(slo_ms)` | `InflightTracker` + `QueueAwareStrategy`; `EnterpriseRouter.execute()` brackets each call with `acquire` / `release` |
| `with_calibration(conformal_alpha)` | `TemperatureScaler` for confidence, `ConformalCalibrator` for `RouteDecision.candidate_set`; `router.calibrate()` replays remembered outcomes |
| `with_retrieval(narrow_above, narrow_to, skill_set_size)` | `Retriever` (BM25 + dense, reciprocal-rank fusion) for large catalogues; submodular skill-set selection |
| `with_fair_share(weights, ...)` | `FairShareMiddleware` |
| `with_shadow(candidate, mode)` | `ShadowMiddleware` in `shadow` (log only) or `ab` (SPRT-judged) mode; `EnterpriseRouter.learn()` feeds outcomes to it |
| `with_middleware(...)`, `with_telemetry(...)`, `with_audit(sink)`, `with_feedback(store)` | chain, observers, audit trail, outcome log |
| `with_llm_judge(judge, escalate_below)`, `with_objective(obj)`, `with_policy(policy)`, `with_router_options(**kw)` | judge escalation, default objective, custom policy, any other `Router` argument |

`build()` validates the configuration (no targets, no strategies, duplicate strategy names) and returns
an `EnterpriseRouter` with `route()`, `execute()`, `run()`, `learn()` and `health_snapshot()`.

## State stores

All stores implement the three-method `StateStore` port (`get` / `put` / `delete` of JSON dicts) and
compose as decorators. External clients are duck-typed, so the core imports no driver.

| Store | Backend | Notes |
|---|---|---|
| `InMemoryStateStore` | process memory | tests, single replica |
| `FileStateStore` | one JSON file per key | keys hashed (no path traversal), atomic `replace` |
| `RedisStateStore(client, prefix, ttl_s)` | any client with `get` / `set(ex=)` / `delete` (redis-py, valkey, fakeredis) | shared state across replicas |
| `SQLStateStore(connection, table)` | any DB-API 2.0 connection (psycopg, sqlite3, pg8000, mysql-connector) | dialect detected; `ON CONFLICT` / `ON DUPLICATE KEY` upserts; table name validated |
| `VersionedStateStore(inner, current, migrations)` | wrapper | stamps `{"_schema": n}`, runs forward migrations on read, refuses newer state |
| `EncryptedStateStore(inner, key \| key_env)` | wrapper (`crypto` extra) | AES-256-GCM, per-record nonce, key id in the envelope for rotation, plaintext records still readable |
| `BatchedStateStore(inner, interval_s, max_pending)` | wrapper | write-behind; reads see pending writes; `flush()` on shutdown or as a context manager |
| `NamespacedStateStore(inner, namespace)` | wrapper | key prefix for several routers or tenants on one backend |

```python
import redis
from opensmartroute.enterprise import RouterBuilder
from opensmartroute.enterprise.stores import BatchedStateStore, EncryptedStateStore, RedisStateStore, VersionedStateStore

store = BatchedStateStore(
    VersionedStateStore(
        EncryptedStateStore(RedisStateStore(redis.Redis.from_url(url), prefix="osr:prod:"), key_env="OSR_STATE_KEY"),
        current=1,
    ),
    interval_s=2.0,
)
app = RouterBuilder(registry).with_defaults().with_auto_learning().with_state_store(store).build()
```

## Operational controls

- **Shadow and A/B** (`ShadowMiddleware`, `ABTest`, `SPRT`): a candidate router runs beside production.
  In shadow mode only agreement rate and cost delta are logged. In A/B mode a deterministic hash of the
  request id assigns a traffic share to the candidate and outcomes are compared with Wald's sequential
  probability ratio test, so a bad candidate is stopped early and a good one is promoted with a
  controlled error rate. `on_verdict` fires once with the summary.
- **Tenant fairness** (`FairShareMiddleware`): dominant-resource fairness across tenants over a sliding
  window; tenants above their fair share are steered to cheaper targets (cost weight boost) and, above
  the hard factor, capped out of premium targets. Nobody is starved.
- **Queue-aware latency** (`InflightTracker`, `QueueAwareStrategy`): live in-flight counts and
  observed service times feed Erlang-C (when `metadata.concurrency` is set) or Kingman G/G/1 wait
  estimates, so the latency in the utility is the current time-to-first-token, not the catalogue number.

## Deployment topologies

1. **Library**: import `Router` / `EnterpriseRouter` in-process. Lowest latency; state on local disk.
2. **Sidecar**: the container image (`ghcr.io/isathish/opensmartroute:<version>`, built from
   `deploy/Dockerfile`) next to each app; shared state via a `StateStore` adapter.
3. **Central control plane**: the Helm chart in `deploy/helm/opensmartroute` behind a gateway (LiteLLM,
   Envoy, aisix); the router returns a decision, the gateway executes. Scale horizontally; state in
   Redis / SQL. See [deploy/README.md](https://opensmartroute.ai/docs/deploy.md).

### Scalability notes

- Routing is CPU-bound and O(|targets| x |strategies|) with tiny constants; 10k decisions/s per
  core is typical for <= 50 targets. Above `narrow_above` admissible targets the retriever keeps only
  `narrow_to` candidates, so cost stays flat for catalogues with thousands of tools.
- All in-memory structures are bounded (LRU caches, capped lists, session maps cleared past a limit).
- Learner updates are O(1) (bandits, IRT, BT) or O(d^2) with d ~ 21 (LinUCB, Sherman-Morrison); value
  iteration is bounded to 30 sweeps over a small state space.
- Multi-replica consistency: learners are *eventually consistent*. Two supported patterns:
  1. **Writer / readers** - one replica learns and persists through a shared `StateStore`; the others
     call `AutoLearner.refresh()` on a timer and adopt the snapshot wholesale (no double counting).
  2. **Independent replicas** - every replica learns from the outcomes it sees and snapshots are folded
     together with `learning.merge_learners(local, remote)`; every learner merges sufficient statistics
     (`merge()` on `ThompsonBeta`, `LinUCB`, `IRTModel`, `BradleyTerry`, `MarkovChain` / `RoutingMDP`,
     `BanditStrategy`, `TaskTableStrategy`). Merge deltas, not the same snapshot twice.
- Corrupt or incompatible persisted state is quarantined on load (`*.corrupt-<ts>` file or
  `quarantine/...` store key) and listed in `AutoLearner.quarantined`; the learner starts blank and
  routing continues.

## Reliability

- Circuit breaker per target; half-open probes.
- Token-bucket rate limits and rolling budgets per target (and per tenant via `TenantMiddleware`).
  The policy only *checks* availability for each candidate; one token is consumed for the target that
  is actually chosen, so evaluating 16 candidates does not drain 16 tokens.
- `HealthStrategy` demotes slow/unreliable targets softly; `HealthPolicy` excludes them hard. Each target
  keeps a rolling window of its last 256 latencies (`LatencyWindow`; p50 / p90 / p99 in `health_snapshot()`
  and on the platform's Health page); once five calls are in the window the strategy judges the **p90**, so a
  target with a good mean and a bad tail is penalised. The SLO is the request's hard `max_latency_ms`, else
  its soft `preferred_max_latency_ms` (penalise, never exclude), else `with_health(latency_slo_ms=...)`.
- Drift detection flags targets whose quality degrades; wire `AutoLearner.drifted` to alerts.
- Atomic state writes (`os.replace`): a crash never leaves a half-written model.
- Per-task resource limits (`security.limits.ResourceLimitMiddleware`) cap steps, tool calls, depth,
  tokens, cost and wall-clock for agentic plans.

## Observability

- Tracing: every stage of every request (signals, policy, rank, plan, execute, learn, health, cache,
  guard, shadow, fair share, autopilot) emits spans and events into the `Tracer` -
  `RouterBuilder.with_tracing(...)`, `MemorySink` / `MetricsSink` / `LoggingSink` / `FileSink` /
  `OpenTelemetrySink`, `GET /events`, `GET /trace/{request_id}`, `X-OSR-Trace-Id`. The hosted
  platform exposes the same buffer per workspace (`GET /api/v1/trace/{request_id}` with the reported
  outcomes and the request's audit records, `GET /api/v1/events`, `/platform/dashboard/events`). See
  [OBSERVABILITY.md](https://opensmartroute.ai/docs/OBSERVABILITY.md).
- `LoggingTelemetry`: structured JSON (request hash, never raw text).
- `MetricsTelemetry`: counters, p50 / p95 / p99 routing latency, mean confidence; expose via `/metrics`.
- `OpenTelemetryTelemetry` (`otel` extra): spans and the `osr.route_latency_ms` histogram.
- `FileAuditSink`: hash-chained audit log (tamper-evident); the chain resumes across restarts and
  `FileAuditSink.verify(path)` re-walks it, naming the first edited, removed or unparseable line.
  Records carry the `request_id`, so a decision and the outcomes reported for it can be joined back to
  the request (`with_audit(sink, outcomes=True)`; the platform shows them on the trace).
- `RouteTrace.explain()`: human-readable decision explanation for support tooling.
- `ShadowMiddleware.summary()`, `FairShareMiddleware.snapshot()`, `InflightTracker.snapshot()` and
  `EnterpriseRouter.health_snapshot()` for dashboards.

## Multi-tenancy

`TenantMiddleware` enforces tenant presence, applies per-tenant cost ceilings, data boundaries and deny
lists. `FairShareMiddleware` keeps one tenant from crowding out the others. `TargetConstraints.tenants`
restricts a target to specific tenants. Learner state can be namespaced per tenant with
`NamespacedStateStore` or a per-tenant `RouterBuilder`.

## Compliance hooks

- Data boundary (`public` < `private` < `on_prem`) is a hard constraint.
- Region allow-lists per target.
- PII never reaches a `pii_allowed=False` target; `Redactor` masks before caching / logging.
- Audit trail with request id, target, confidence, tenant; no content.
- Learner state encrypted at rest with `EncryptedStateStore`; key from the environment or a `*_FILE`
  secret via `load_secret`.
- Signed MCP manifests (`adapters.mcp.verify_manifest`) and an origin policy for sensitive tool
  parameters (`security.provenance.OriginPolicy`) for plans that execute state-changing tools.

---

<!-- Page: Tracing and observability (https://opensmartroute.ai/docs/OBSERVABILITY) -->

<!-- OpenSmartRoute: Tracing and observability. Source https://github.com/isathish/OpenSmartRoute/blob/main/docs/OBSERVABILITY.md; HTML https://opensmartroute.ai/docs/OBSERVABILITY -->
# Tracing and observability

Every stage of a request - signal extraction, policy filtering, ranking, planning, execution, learning,
health, cache, guard, shadow rollout, fair share and the autopilot - emits **spans** and **events**
into a `Tracer`. Sinks decide what happens to them: keep the last N in memory for `/trace`, aggregate
them into Prometheus counters, log them as JSON lines, append them to a file or forward them to
OpenTelemetry. Instrumentation is always on and free when no sink is attached (a shared no-op span,
no allocation), so the SDK stays zero-dependency and the cost of capturing everything is opt-in.

```python
from opensmartroute import Router, configure_tracing
from opensmartroute.observability import MemorySink, MetricsSink

tracer = configure_tracing(MemorySink(), MetricsSink())      # process-wide; or configure_tracing() -> settings
router = Router(reg)                                          # picks up the global tracer
d = router.route("Prove that sqrt(2) is irrational")
for e in tracer.find(MemorySink).trace(d.request_id):        # spans + events of that request, in time order
    print(e["kind"], e["name"], e.get("duration_ms"), e["attributes"])
print(tracer.find(MetricsSink).prometheus())                 # osr_* exposition text
```

```
span  route            2.31ms {'text_sha256': '9f2c…', 'text_len': 35, 'target': 'large', 'confidence': 0.82, 'elapsed_ms': 2.1, ...}
event route.signals          {'duration_ms': 0.4, 'complexity': 0.71, 'domains': ['math'], 'reasoning_need': 0.9, 'contains_pii': False, ...}
event route.policy           {'pool': 2, 'admissible': 2, 'rejections': {}}
event route.rank             {'role': None, 'candidates': 2, 'strategies': {'capability': 0.05, ...}, 'top': [{'id': 'large', 'utility': 0.82, ...}]}
```

`osr route "text" --events` prints the same trace after the decision; `osr serve` exposes it over HTTP and
the hosted platform shows it per workspace in the dashboard (activity drawer, `/platform/dashboard/events`,
playground *Trace* tab - see the platform endpoints below).

## What is captured

| Span | Opened by | Attributes (end of span) |
|---|---|---|
| `request` | `EnterpriseRouter.route` | `middleware` (class names), `tenant`, `target`, `confidence`, `audited` |
| `http.request` | `osr serve` middleware | `method`, `path`, `app`, `status_code`, `target`, inbound `traceparent` |
| `route` | `Router.route` | `text_sha256`, `text_len`, `tenant`, `objective`, `kinds`, `candidates`, `exclude`, `plan`, `task_id`, `session_id`, `target`, `kind`, `confidence`, `elapsed_ms`, `alternatives`, `abstain`, `plan_slots` |
| `plan` | `Router._build_plan` (inside `route`) | `primary`, `kind`, `slots` |
| `execute` | `execution.execute` / `aexecute` | `target`, `kind`, `plan_slots`, `task_id`, `runner`, `ok`, `latency_ms`, `cost_usd`, `tokens`, `outcomes`, `system_prompt_len`, `pii_redacted`, `pii_restored` |
| `autopilot.cycle` | `Autopilot.run_once` | `reason`, `online`, `accepted`, `outcome`, `cycle` |

| Event | Emitted when |
|---|---|
| `route.signals` | signals extracted: `duration_ms`, complexity, domains, actions, language, token estimate, PII, jailbreak risk, tools, task type, reasoning need |
| `route.policy` | hard constraints applied: `pool`, `admissible`, `rejections` (target -> reason) |
| `route.no_route` | no admissible target (before `NoRouteError` is raised) |
| `route.pinned` | a rule pinned a target (`kept` tells whether it survived the policy) |
| `route.narrowed` | the retriever shortlisted a large catalogue |
| `route.shortlist` | a strategy shortlisted candidates |
| `route.rank` | strategies scored (`strategies` with per-strategy ms, `top` 3, `excluded_by_floor`) |
| `route.escalate` | the LLM judge was consulted because confidence was low |
| `route.explore` | the bandit chose to explore (`served` tells whether the exploration was taken) |
| `route.abstain` | the calibrated confidence was below the abstention threshold |
| `route.fallback` | a fallback target replaced an unavailable one |
| `plan.slot` | a plan slot was filled (or not: `filled=False`, `reason`) |
| `execute.step` | one execution step (persona, skill, primary) finished |
| `learn.outcome` | `Router.learn` / `learn_correction` recorded an outcome (`corrected`, `learner`) |
| `learn.task_credit` | task-level credit assigned to the plan's targets |
| `learn.calibrate` | the confidence calibrator was refit (temperature) |
| `learn.improve` | a `SelfImprover` cycle finished (`rows`, `champion_accuracy`, `challenger_accuracy`, `accepted`, `reason`; inside an `autopilot.cycle` span when the autopilot ran it) |
| `learn.promote` | a challenger SLM replaced the serving champion (`rows`, `trained_at`, `targets`, `saved`) |
| `health.breaker` | a circuit breaker changed state (`previous`, `state`; level `warning` when it opens) |
| `cache.hit` / `cache.miss` | `CacheMiddleware` lookup (`target`, `age_s` on a hit) |
| `tenant.rejected` | `TenantMiddleware` refused a request (`reason`, `tenant`) |
| `route.slow` | `TimeoutMiddleware` saw a decision exceed its budget |
| `guard.blocked` / `guard.flagged` / `guard.redacted` | `GuardMiddleware` verdicts (`reasons`, `gadget`, `entities`) |
| `shadow.compare` / `shadow.verdict` | `ShadowMiddleware` compared production and candidate; the A/B test reached a verdict |
| `fairshare.throttled` / `fairshare.capped` | `FairShareMiddleware` delayed or refused a tenant |
| `autopilot.drift` | the autopilot detected quality drift and scheduled a cycle |

`opensmartroute.observability.SPAN_NAMES` and `EVENT_NAMES` list the vocabulary; a test asserts every
emitted name belongs to it.

**Privacy.** Request text never enters a span or event: the `route` span carries `text_sha256` (16 hex
characters) and `text_len` (`text_digest()`); attribute strings are truncated to
`OSR_OBSERVABILITY_ATTRIBUTE_MAX_LEN` (500) and nested values are JSON-cleaned. Sinks receive ids,
numbers, target ids, reasons and scores.

## Spans, events, trace ids

An `Event` is a flat record: `name`, `kind` (`event` or `span`), `ts`, `trace_id`, `span_id`,
`parent_id`, `request_id`, `level`, `status` and `duration_ms` (spans only) and `attributes`. A span's
closing record is the same shape as an event so every sink handles one type.

The trace id of a request is derived once, at the root span: an inbound W3C `traceparent` wins,
otherwise a UUID request id is reused, otherwise the request id is hashed (so a `request_id` supplied
by the caller yields a stable trace id). Nested spans inherit trace id, request id and the sampling
decision through a `contextvars` context, so threads and `asyncio` tasks spawned by the router carry
the right parent. `Tracer.current()` returns the innermost open span; `current_tracer()` resolves the
tracer to report to (the open span's, else one bound with `use_tracer`, else the global one), which is
how middleware, the health registry and the executor emit without holding a reference.

## Sinks

| Sink | Purpose |
|---|---|
| `MemorySink(max_events=1000)` | ring buffer; `events(request_id=, trace_id=, name=, kind=, level=, limit=)` (`name` may end with `*`), `trace(request_id)`; backs `/events`, `/trace/{id}` and `osr route --events` |
| `MetricsSink()` | counters and p50/p95/p99 per span name; `snapshot()` for `/stats`, `prometheus()` for `/metrics` (`osr_events_total`, `osr_spans_total`, `osr_span_duration_ms`, `osr_span_decisions_total`, `osr_executions_total`, `osr_execution_cost_usd_total`, `osr_learn_outcomes_total`, `osr_breaker_transitions_total`, `osr_cache_events_total`, `osr_span_errors_total`) |
| `LoggingSink(log=None)` | one JSON line per event on the `opensmartroute.events` logger, level follows the event |
| `FileSink(path)` | append-only JSONL, one event per line (ship with any log forwarder) |
| `OpenTelemetrySink()` (`adapters.optional`, `otel` extra) | live bridge: each span becomes an OTel span (parented through the OTel context, joined to `traceparent`), each event a span event and an `osr.events` counter, durations an `osr.span_duration_ms` histogram; the host configures the SDK, exporter and resource |

Write your own by subclassing `EventSink` and implementing `emit(event)` (optionally `span_start`,
`span_end`, `snapshot`). A sink that raises is counted in `Tracer.sink_errors` and logged (first ten
occurrences); it never breaks routing.

## Configuration

`configure_tracing(*sinks, sample_rate=None)` installs sinks on the process-wide tracer that every
`Router` uses by default. With no sinks it reads `ObservabilitySettings` (`OSR_OBSERVABILITY_*`):

| Setting | Env | Default | Effect |
|---|---|---|---|
| `metrics` | `OSR_OBSERVABILITY_METRICS` | `true` | attach a `MetricsSink` |
| `memory_events` | `OSR_OBSERVABILITY_MEMORY_EVENTS` | `1000` | attach a `MemorySink` of that size (`0` disables) |
| `log_events` | `OSR_OBSERVABILITY_LOG_EVENTS` | `false` | attach a `LoggingSink` |
| `events_file` | `OSR_OBSERVABILITY_EVENTS_FILE` | `""` | attach a `FileSink` at that path |
| `otel` | `OSR_OBSERVABILITY_OTEL` | `false` | attach an `OpenTelemetrySink` (needs the `otel` extra) |
| `sample_rate` | `OSR_OBSERVABILITY_SAMPLE_RATE` | `1.0` | share of root spans that are recorded |
| `attribute_max_len` | `OSR_OBSERVABILITY_ATTRIBUTE_MAX_LEN` | `500` | longest attribute string kept |

`Router(reg, tracer=Tracer([...]))` gives one router a private tracer;
`RouterBuilder(...).with_tracing(*sinks_or_tracer, sample_rate=)` does the same for an
`EnterpriseRouter` (no arguments: build from settings). `osr`, `osr serve`, the Docker image and the
hosted platform call `configure_tracing()` / `Tracer.from_settings()` at start-up, so the environment
variables above are all that is needed in a container.

## HTTP endpoints (`osr serve`)

| Endpoint | Returns |
|---|---|
| `GET /metrics` | `MetricsTelemetry` exposition followed by the tracer's `MetricsSink` counters (names are disjoint) |
| `GET /events?request_id=&trace_id=&name=route.*&kind=&level=&limit=200` | recent events from the memory buffer (404 when `OSR_OBSERVABILITY_MEMORY_EVENTS=0`) |
| `GET /trace/{request_id}` | every span and event of one request, in time order, with its `trace_id` |
| `GET /stats` | adds an `observability` block: sinks, sample rate, sink errors and each sink's snapshot |

Every traced response carries `X-OSR-Trace-Id`; a client that sends `traceparent` gets its own trace id
back, so the routing spans line up with the caller's trace in Jaeger, Tempo, Azure Monitor or any OTel
backend. Probes (`/healthz`, `/readyz`), `/metrics`, `/events`, `/trace/*`, `/whoami` and the OpenAPI
pages are never traced. The endpoints follow the server's access control: with `OSR_SERVER_AUTH_TOKENS`
set, `/events`, `/trace/*` and `/stats` need a token (`/metrics` stays open for scrapers).

## HTTP endpoints (hosted platform)

The platform API (`platform/api`) exposes the same tracer per workspace, so a tenant only ever sees its
own requests - and it is the platform's own observability: it persists every span and event to its
database (`osr_platform.telemetry`, `OSR_PLATFORM_TELEMETRY_STORE`, on by default, off the request path
through a queued writer; `OSR_PLATFORM_TELEMETRY_RETENTION_DAYS`), keeps per-minute HTTP counters for a
day, computes time series from its usage records and evaluates alert rules in-process
(`osr_platform.alerts`). Nothing has to be exported to a metrics, tracing or alerting system.

| Endpoint | Returns |
|---|---|
| `GET /api/v1/trace/{request_id}` | the activity row of the request plus its spans and events (from the buffer, or from the store once evicted - `source` says which), the feedback reported for it (`outcomes`) and its audit records (`audit`); `404` for another workspace's request |
| `GET /api/v1/events?name=&kind=&level=&request_id=&since=&until=&limit=` | events joined to the workspace through its request ids (deployment-wide events on plans with `stats`); `since` / `until` select a time range from the store (default the last 6 hours) |
| `GET /api/v1/telemetry/series?window=` | the workspace's requests, failures, p50 / p95 latency, cost and tokens per bucket (`1h` .. `30d`) with per-target and per-endpoint totals; `http` (plans with `stats`) adds the deployment's requests, 4xx / 5xx and latency percentiles per bucket |
| `GET /api/v1/alerts` | active conditions - budgets, failing requests, and on plans with `stats` readiness, 5xx rate, latency SLO, breakers, drift, autopilot errors, a stale SLM, tracing off - each with the dashboard page that shows or fixes it |
| `GET /api/v1/status` | public readiness with the tracing state, for status pages (`503` while a check fails) |
| `GET /api/v1/activity` | request log with `traced: true` on rows whose spans are kept and `outcome` summarising the feedback per request |

`POST /api/v1/route` returns `trace_id` and `X-OSR-Trace-Id`. The dashboard renders the history at
`/platform/dashboard/events` (1 hour to 7 days), a span waterfall from every traced activity row and a *Trace* tab
in the playground; the overview shows the 24-hour series and the alerts, `/platform/dashboard/health` the
deployment series, the *Systems* card (`/api/v1/status`) and the same alerts.

## OpenTelemetry end to end

```python
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor, ConsoleSpanExporter

trace.set_tracer_provider(TracerProvider())
trace.get_tracer_provider().add_span_processor(BatchSpanProcessor(ConsoleSpanExporter()))

from opensmartroute import configure_tracing
from opensmartroute.adapters.optional import OpenTelemetrySink

configure_tracing(OpenTelemetrySink())      # or OSR_OBSERVABILITY_OTEL=true
```

OTel spans are named `osr.route`, `osr.plan`, `osr.execute`, ...; attributes carry the `osr.` prefix
(`osr.target`, `osr.confidence`, `osr.request_id`, `osr.trace_id`). `OpenTelemetryTelemetry` (the
older `Telemetry` port) still records the `osr.route_latency_ms` histogram per decision; the sink is the
richer, per-stage view and the two can run side by side.

## Relationship to telemetry and audit

`Telemetry` sinks (`LoggingTelemetry`, `MetricsTelemetry`, `OpenTelemetryTelemetry`) observe *decisions
and outcomes* on an `EnterpriseRouter`; `AuditSink` keeps a hash-chained, tamper-evident record of
*who got what*. The tracer observes *how* the decision was made - every stage, with timings, on plain
and enterprise routers alike. Use audit for compliance, telemetry for dashboards you already have and
the tracer for debugging, SLOs per stage and distributed tracing. `RouteTrace.explain()` remains the
human-readable explanation attached to a `RouteDecision`.

`MetricsTelemetry` is the dashboard store: `snapshot()` returns `decisions_total`, `decisions_by_target`,
`policy_rejections_total`, `errors_by_type`, `route_latency_ms` (`p50`, `p95`, `p99`, `mean`, `count`),
`route_latency_histogram_ms`, `mean_confidence`, `outcomes_by_target` (count, success rate, cost, quality,
latency) and `uptime_s`; `prometheus(namespace="osr", extra={...})` renders the same as Prometheus text
(`osr_route_decisions_total{target="..."}`, `osr_route_latency_ms_bucket{le="..."}`,
`osr_outcome_cost_usd_total`, `osr_decision_cache_hits_total`, extras as gauges). `osr serve` and the
hosted platform expose it at `GET /metrics`; `RouterBuilder.with_cache(ttl_s, max_size)` adds a decision
cache whose `CacheMiddleware.stats()` feed the `osr_decision_cache_*` series, and
`with_audit(sink, outcomes=True)` chains outcomes into the audit trail alongside decisions.

---

<!-- Page: Benchmarks and leaderboard recipe (https://opensmartroute.ai/docs/leaderboard) -->

<!-- OpenSmartRoute: Benchmarks and leaderboard recipe. Source https://github.com/isathish/OpenSmartRoute/blob/main/examples/leaderboard/README.md; HTML https://opensmartroute.ai/docs/leaderboard -->
# Reproducible leaderboard runs

Everything the public-suite numbers in [docs/ROADMAP.md](https://opensmartroute.ai/docs/ROADMAP.md) (v0.4 measurement) are
made of, as commands you can re-run. The same recipe is what an OpenSmartRoute entry on RouterArena,
LLMRouterBench or xRouteBench is built from; the runs themselves are tracked under v1.0 in the roadmap and
recorded, with their submission status, in [results/README.md](https://opensmartroute.ai/docs/leaderboard-results.md#listings).

Requirements: `pip install "opensmartroute[yaml]"`, network access to the Hugging Face datasets-server
(public suites only; RouterBench / RouterEval are gated and need a Hub token plus `--preset`).

## 1. Catalogue

[targets.yaml](https://github.com/isathish/OpenSmartRoute/blob/main/examples/leaderboard/targets.yaml): three tiers, `llm-small` / `llm-mid` / `llm-frontier`. Battle datasets name
2023-24 models; `osr collect --tier` maps each model to the tier it played in (`eval.collect.ARENA_TIERS`),
so every row's label is "which tier was good enough for this prompt" - the RouteLLM framing.

## 2. Collect the public suites

```bash
osr collect --cache-dir data --limit 4000 \
    --tier frontier=llm-frontier --tier mid=llm-mid --tier small=llm-small
```

Writes one `<source>-tiers.jsonl` (+ `.meta.json`) per public source: `arena-55k`, `arena-100k`,
`arena-140k`, `routellm-battles`, `ppe-human`, `webdev-arena`, `mt-bench-human`, `reward-bench`,
`ultrafeedback`, `routellm-gpt4`. `--source NAME` restricts the run; `--refresh` ignores the cache.

## 3. Train the routing SLM on the training suites only

```bash
osr -t examples/leaderboard/targets.yaml slm train --out slm.json --cache-dir data \
    --source arena-55k-tiers --source routellm-battles-tiers --report --seed 0
```

`--report` holds out 10 % and prints its accuracy. The suites in step 4 are **not** in `--source`, so
they are held out at the suite level, not just the row level.

## 4. Evaluate on the held-out suites

```bash
for suite in mt-bench-human ppe-human webdev-arena; do
  osr -t examples/leaderboard/targets.yaml --slm slm.json eval data/$suite-tiers.jsonl \
      --baselines --calibration --robustness --limit 1000 --json > results/$suite.json
done
# declarative router (no learned signals) for the ablation column
osr -t examples/leaderboard/targets.yaml eval data/mt-bench-human-tiers.jsonl --baselines --limit 1000
```

What each flag reports and where it lands in the table:

| Flag | Reports | Table column |
|---|---|---|
| (default) | accuracy, mean quality, cost, latency, mean confidence, raw ECE / Brier | Router, ECE raw |
| `--baselines` | random / cheapest / best-prior / static task table / oracle, gap to oracle, noise floor | Static task table, Random, Cheapest |
| `--robustness` | repeat flip rate, paraphrase stability, profile-swap dependence (first 200 rows) | Paraphrase |
| `--calibration` | ECE, Brier, reliability bins, and `held_out`: temperature and isotonic calibrators fitted on the even rows, scored on the odd rows | ECE held-out |

The published table used `--limit 1000` per suite and seed 0 throughout; the SLM bundle and the
`Router` are deterministic, so re-running gives the same numbers for the same cache.

## 5. Gated suites (RouterBench, RouterEval, xRouteBench, RouterXBench)

Download the benchmark file with your Hub token, then evaluate it directly through the layout preset:

```bash
osr -t examples/leaderboard/targets.yaml --slm slm.json eval routerbench_0shot.pkl.jsonl \
    --preset routerbench --baselines --calibration --limit 5000
```

These rows carry several sampled answers per model, so `--baselines` can also report the label-noise
floor (`noise_floor()`), which the pairwise suites above cannot.

## 6. Publish

Commit `results/*.json`, the exact `osr --version`, the dataset ids, splits and row counts, and the
SHA-256 of `slm.json` together; that triple is the reproducible config a leaderboard entry links to.
`python scripts/exit_criteria.py --full` prints the synthetic exit criteria alongside, so the two
tables can be published from one run.

The run published with 0.5.0 is in [results/README.md](https://opensmartroute.ai/docs/leaderboard-results.md) (one JSON per suite, learned
and declarative); it is the entry submitted to RouterArena, LLMRouterBench and xRouteBench.

---

<!-- Page: Published benchmark runs (https://opensmartroute.ai/docs/leaderboard-results) -->

<!-- OpenSmartRoute: Published benchmark runs. Source https://github.com/isathish/OpenSmartRoute/blob/main/examples/leaderboard/results/README.md; HTML https://opensmartroute.ai/docs/leaderboard-results -->
# Published runs - opensmartroute 0.5.0

Produced with the recipe in [../README.md](https://opensmartroute.ai/docs/leaderboard.md) on 2026-09-06, `osr 0.5.0`, seed 0,
`--limit 1000` per suite. One JSON per suite: `<suite>.json` is the learned router
(`--slm slm.json --baselines --calibration --robustness`), `<suite>.declarative.json` is the same
catalogue without the SLM (`--baselines` only). The per-row arrays `calibration.conformal_pairs` and
`calibration.temperature_samples` were removed from the committed files; every aggregate they feed
(`reliability`, `held_out`) is kept.

## Data

| Role | Cache name | Hugging Face dataset | Rows after `--tier` relabelling |
|---|---|---|---|
| train | `arena-55k-tiers` | `lmarena-ai/arena-human-preference-55k` | 1 986 |
| train | `routellm-battles-tiers` | `routellm/gpt4_judge_battles` | 4 000 |
| held out | `mt-bench-human-tiers` | `lmsys/mt_bench_human_judgments` (`human`) | 2 228 |
| held out | `ppe-human-tiers` | `lmarena-ai/PPE-Human-Preference-V1` (`test`) | 2 360 |
| held out | `webdev-arena-tiers` | `lmarena-ai/webdev-arena-preference-10k` (`test`) | 1 070 |

Catalogue: [../targets.yaml](https://github.com/isathish/OpenSmartRoute/blob/main/examples/leaderboard/targets.yaml) (`llm-small` / `llm-mid` / `llm-frontier`). SLM bundle:
`osr -t examples/leaderboard/targets.yaml slm train --out slm.json --cache-dir data --source
arena-55k-tiers --source routellm-battles-tiers --report --seed 0` -> 5 956 rows, 10 % holdout
accuracy 0.721, ECE 0.056, `sha256 8faf8466902b753a9f22382de24bfd0e1d5271c7fe5d6564dec8105310dcbb1e`
(4.0 MB, not committed; the command is deterministic for the same cache).

## Results

| Suite (n=1000) | Router (learned) | Declarative | Static task table | Random | Cheapest | Best prior | Paraphrase | Repeat flips | ECE raw | ECE held-out temp. | ECE held-out isotonic |
|---|---|---|---|---|---|---|---|---|---|---|---|
| MT-Bench human | **0.523** | 0.421 | 0.541 | 0.441 | 0.372 | 0.343 | 1.000 | 0.000 | 0.328 | 0.056 | 0.048 |
| PPE human | **0.552** | 0.418 | 0.651 | 0.445 | 0.174 | 0.651 | 0.898 | 0.000 | 0.298 | 0.121 | 0.035 |
| WebDev Arena | **0.573** | 0.192 | 0.670 | 0.391 | 0.000 | 0.670 | 0.883 | 0.000 | 0.283 | 0.032 | 0.047 |

Cost and latency per 1k tokens for the learned router: MT-Bench $0.00363 / 968 ms, PPE $0.00390 /
1005 ms, WebDev $0.00324 / 925 ms (static task table: $0.003 / 900 ms on MT-Bench, $0.015 / 2500 ms
on PPE and WebDev, since it degenerates to "always the best single tier").

Reading: the learned router beats the declarative router by +10 to +38 pp and random / cheapest
everywhere, at a fifth of the cost of "always frontier" on PPE and WebDev. It does **not** beat the
best single tier on any suite (-2 to -10 pp): a pairwise battle between two named models is a weak
label for a three-tier decision, and the label-noise floor cannot be measured on these rows (one
sample per prompt). Raw confidence is over-confident (ECE 0.28-0.33); a calibrator fitted on the even
rows and scored on the odd rows brings held-out ECE to 0.03-0.05 (isotonic). These are the numbers a
RouterArena / LLMRouterBench / xRouteBench entry should quote; the gated multi-sample suites (step 5 of
the recipe) are what would let the router be scored against the noise floor.

## Listings

Where this run has been submitted. `python scripts/release.py readiness` counts rows whose status is
`accepted` for the v1.0 adoption item "Leaderboard listings accepted" and reports it as open until one
listing is accepted. A submission links the exact commit, the SLM SHA-256 above and this page.

| Leaderboard | Entry | Status | Notes |
|---|---|---|---|
| RouterArena | - | not submitted | submission = a PR to the RouterArena repository with `runs/*.json` |
| LLMRouterBench | - | not submitted | needs the gated multi-sample suites (step 5) for the noise-floor comparison |
| xRouteBench | - | not submitted | component ablation report from `osr eval --ablation` |

---

<!-- Page: Python API reference (https://opensmartroute.ai/docs/REFERENCE) -->

<!-- OpenSmartRoute: Python API reference. Source https://github.com/isathish/OpenSmartRoute/blob/main/docs/REFERENCE.md; HTML https://opensmartroute.ai/docs/REFERENCE -->
# API reference

Every module under `src/opensmartroute` and every public name it exports, generated from the source
by `python scripts/api_reference.py` (checked by `tests/test_docs.py`; do not edit by hand). The
first line of each docstring is the summary; open the module for the full contract. Narrative
documentation: [GUIDE.md](https://opensmartroute.ai/docs/GUIDE.md) (usage), [SDK.md](https://opensmartroute.ai/docs/SDK.md) (stability, extension points),
[ARCHITECTURE.md](https://opensmartroute.ai/docs/ARCHITECTURE.md), [MATH.md](https://opensmartroute.ai/docs/MATH.md), [ENTERPRISE.md](https://opensmartroute.ai/docs/ENTERPRISE.md),
[SECURITY.md](https://opensmartroute.ai/docs/SECURITY.md), [RESEARCH.md](https://opensmartroute.ai/docs/RESEARCH.md).

Stability: names re-exported from `opensmartroute` (the top-level package) are frozen by
`tests/public_api.json` and follow SemVer. Sub-module names are public but may change in a minor
release with a CHANGELOG entry.

## Modules

- [`opensmartroute`](#opensmartroute) - OpenSmartRoute — an open, intelligent route to the right decision, solution, or destination. (69 names)
- [`opensmartroute.adapters`](#opensmartrouteadapters) - Adapters connect OpenSmartRoute to real providers and infrastructure. (70 names)
- [`opensmartroute.adapters.a2a`](#opensmartrouteadaptersa2a) - Import A2A (Agent-to-Agent protocol) **Agent Cards** as ``TargetKind.AGENT`` targets. (4 names)
- [`opensmartroute.adapters.catalogue`](#opensmartrouteadapterscatalogue) - Live model catalogue: collect model cards (price, context, modalities, benchmarks) from public sources. (11 names)
- [`opensmartroute.adapters.frameworks`](#opensmartrouteadaptersframeworks) - Drop-in nodes for agent frameworks. (7 names)
- [`opensmartroute.adapters.handlers`](#opensmartrouteadaptershandlers) - Executors for non-LLM targets: HTTP endpoints, MCP tools and asynchronous queues. (7 names)
- [`opensmartroute.adapters.harness`](#opensmartrouteadaptersharness) - Agent-harness adapters: route to a *runtime*, not just a model. (7 names)
- [`opensmartroute.adapters.mcp`](#opensmartrouteadaptersmcp) - Import MCP (Model Context Protocol) tools as ``TargetKind.TOOL`` targets. (11 names)
- [`opensmartroute.adapters.mcp_servers`](#opensmartrouteadaptersmcp_servers) - MCP *server* recommendation (MCP-Zero 2506.01056; ToolRet 2603.06467). (3 names)
- [`opensmartroute.adapters.openai_compat`](#opensmartrouteadaptersopenai_compat) - OpenAI-compatible HTTP client (stdlib only). (6 names)
- [`opensmartroute.adapters.optional`](#opensmartrouteadaptersoptional) - Optional adapters that need extra dependencies. Everything is lazily imported so the. (3 names)
- [`opensmartroute.adapters.personas`](#opensmartrouteadapterspersonas) - Import persona catalogues as ``TargetKind.PERSONA`` targets. (4 names)
- [`opensmartroute.adapters.semantic_router`](#opensmartrouteadapterssemantic_router) - Import a vLLM *semantic-router* configuration (vllm-project/semantic-router). (3 names)
- [`opensmartroute.adapters.skills`](#opensmartrouteadaptersskills) - Load Agent-Skills ``SKILL.md`` packages as ``TargetKind.SKILL`` targets. (4 names)
- [`opensmartroute.adapters.websearch`](#opensmartrouteadapterswebsearch) - Web knowledge for the self-improving router: stdlib HTTP fetch, search providers, page text. (14 names)
- [`opensmartroute.aio`](#opensmartrouteaio) - Async façade. Routing itself is CPU-bound and sub-millisecond, so we run it in. (1 names)
- [`opensmartroute.branding`](#opensmartroutebranding) - OpenSmartRoute naming conventions: one place for every brand-bound identifier. (27 names)
- [`opensmartroute.cli`](#opensmartroutecli) - ``osr`` command-line interface. (2 names)
- [`opensmartroute.config`](#opensmartrouteconfig) - Configuration loading: targets and rules from JSON or YAML. (3 names)
- [`opensmartroute.core`](#opensmartroutecore) - see module (16 names)
- [`opensmartroute.core.registry`](#opensmartroutecoreregistry) - Target registry: the catalogue of everything a request may be routed to. (1 names)
- [`opensmartroute.core.types`](#opensmartroutecoretypes) - Core data model for OpenSmartRoute. (16 names)
- [`opensmartroute.credentials`](#opensmartroutecredentials) - Credentials for the ``osr`` CLI: where the access token lives and how it is obtained. (18 names)
- [`opensmartroute.discovery`](#opensmartroutediscovery) - Tool discovery beyond text similarity. (5 names)
- [`opensmartroute.enterprise`](#opensmartrouteenterprise) - Enterprise integration layer: ports (hexagonal architecture), middleware and telemetry. (18 names)
- [`opensmartroute.enterprise.ops`](#opensmartrouteenterpriseops) - Operational controls: shadow / A-B routing, tenant fairness and queue-aware latency. (7 names)
- [`opensmartroute.enterprise.savings`](#opensmartrouteenterprisesavings) - Savings ledger - the always-on savings report that backs the ROI story and the dashboard. (3 names)
- [`opensmartroute.enterprise.stores`](#opensmartrouteenterprisestores) - Production state-store backends and wrappers. (7 names)
- [`opensmartroute.errors`](#opensmartrouteerrors) - Exception hierarchy. Every error raised by the SDK derives from :class:`OpenSmartRouteError`. (11 names)
- [`opensmartroute.estimate`](#opensmartrouteestimate) - Token, cost and latency estimates *before* a request is sent anywhere. (9 names)
- [`opensmartroute.eval`](#opensmartrouteeval) - RouterBench-style evaluation harness. (19 names)
- [`opensmartroute.eval.agentic`](#opensmartrouteevalagentic) - tau-bench-style agentic task evaluation: does per-step routing beat the best single agent?. (4 names)
- [`opensmartroute.eval.audit`](#opensmartrouteevalaudit) - Routing Audit - shadow-mode replay of logged LLM traffic to quantify what the router would change. (6 names)
- [`opensmartroute.eval.baselines`](#opensmartrouteevalbaselines) - Baselines every router must beat, plus oracle ceilings and the sampling noise floor. (13 names)
- [`opensmartroute.eval.collect`](#opensmartrouteevalcollect) - Collect routing datasets from the Hugging Face Hub, your own feedback log and synthetic seeds. (29 names)
- [`opensmartroute.eval.criteria`](#opensmartrouteevalcriteria) - Offline realisations of the ROADMAP exit criteria. (15 names)
- [`opensmartroute.eval.datasets`](#opensmartrouteevaldatasets) - Adapters from public routing benchmarks to :class:`EvalRow`. (7 names)
- [`opensmartroute.eval.frontier`](#opensmartrouteevalfrontier) - Three-objective frontier and ablations. (3 names)
- [`opensmartroute.eval.headroom`](#opensmartrouteevalheadroom) - Routing headroom: when does routing pay, and how much catalogue does it need?. (7 names)
- [`opensmartroute.eval.ope`](#opensmartrouteevalope) - Off-policy evaluation from logged routing decisions. (6 names)
- [`opensmartroute.eval.robustness`](#opensmartrouteevalrobustness) - Robustness and fairness checks for a router. (6 names)
- [`opensmartroute.execution`](#opensmartrouteexecution) - Plan-aware execution: turn a :class:`RouteDecision` into a real answer. (5 names)
- [`opensmartroute.feedback`](#opensmartroutefeedback) - Feedback store: append-only outcome log that closes the learning loop. (1 names)
- [`opensmartroute.learning`](#opensmartroutelearning) - Auto-learning strategies built on :mod:`opensmartroute.math`. (47 names)
- [`opensmartroute.learning.attention`](#opensmartroutelearningattention) - A pure-Python transformer block for the routing SLM's query encoder. (2 names)
- [`opensmartroute.learning.autopilot`](#opensmartroutelearningautopilot) - Self-operation: the routing SLM runs its own improvement loop inside the live process. (2 names)
- [`opensmartroute.learning.coldstart`](#opensmartroutelearningcoldstart) - Cold start for new targets and self-improving target descriptions. (8 names)
- [`opensmartroute.learning.contrastive`](#opensmartroutelearningcontrastive) - Contrastive and reward-distilled router training (RouterDC, NeurIPS 2024; Zooter 2311.08692). (4 names)
- [`opensmartroute.learning.credit`](#opensmartroutelearningcredit) - Delayed, task-level credit assignment for agentic trajectories. (2 names)
- [`opensmartroute.learning.embed`](#opensmartroutelearningembed) - Pretrained transformer embeddings as frozen features for the routing SLM. (2 names)
- [`opensmartroute.learning.handoff`](#opensmartroutelearninghandoff) - Permanent-handoff policy from censored teacher signals (TACIT-Switch 2608.27911). (3 names)
- [`opensmartroute.learning.multiturn`](#opensmartroutelearningmultiturn) - Multi-turn routing with history-target joint embeddings (MTRouter 2604.23530). (3 names)
- [`opensmartroute.learning.personal`](#opensmartroutelearningpersonal) - Few-shot personalisation (GMTRouter 2511.08590; SkillFeed 2608.28241). (4 names)
- [`opensmartroute.learning.policy_gradient`](#opensmartroutelearningpolicy_gradient) - End-to-end policy-gradient routing (Router-R1 2506.09033; RLCascadeRouter 2608.15817). (4 names)
- [`opensmartroute.learning.self_improve`](#opensmartroutelearningself_improve) - Self-improvement loop: refresh the catalogue, gather evidence, train a challenger, promote it only if better. (2 names)
- [`opensmartroute.learning.slm`](#opensmartroutelearningslm) - The OpenSmartRoute routing SLM: a small, self-contained model that picks the target for a prompt. (5 names)
- [`opensmartroute.math`](#opensmartroutemath) - Mathematical toolkit behind OpenSmartRoute's decisions. (46 names)
- [`opensmartroute.math.bandits`](#opensmartroutemathbandits) - Multi-armed and contextual bandits for online routing decisions. (8 names)
- [`opensmartroute.math.calibration`](#opensmartroutemathcalibration) - Calibration and distribution-free risk control for routing confidence. (6 names)
- [`opensmartroute.math.decision`](#opensmartroutemathdecision) - Multi-objective decision helpers and queueing theory for real-time routing. (9 names)
- [`opensmartroute.math.dirichlet`](#opensmartroutemathdirichlet) - Dirichlet probe over host hidden states (ProbeDirichlet, RouterXBench 2602.11877). (2 names)
- [`opensmartroute.math.energy`](#opensmartroutemathenergy) - Hardware-aware energy characterisation (HW-Router 2608.14575; 2608.28044). (3 names)
- [`opensmartroute.math.estimators`](#opensmartroutemathestimators) - Streaming estimators, drift detection and calibration. (11 names)
- [`opensmartroute.math.irt`](#opensmartroutemathirt) - Item Response Theory for routing (IRT-Router, ACL 2025). (3 names)
- [`opensmartroute.math.markov`](#opensmartroutemathmarkov) - Markov chains and MDPs for conversational / multi-step routing. (2 names)
- [`opensmartroute.math.preference`](#opensmartroutemathpreference) - Bradley–Terry pairwise preference model (RouteLLM, Prompt-to-Leaderboard). (2 names)
- [`opensmartroute.mcp_server`](#opensmartroutemcp_server) - Model Context Protocol server: the router as a set of tools for any IDE or agent. (8 names)
- [`opensmartroute.observability`](#opensmartrouteobservability) - Tracing and observability: every step of routing, execution and learning as a structured event. (18 names)
- [`opensmartroute.ocm`](#opensmartrouteocm) - Open Capability Manifest (OCM) - a vendor-neutral description of any routable capability. (12 names)
- [`opensmartroute.policy`](#opensmartroutepolicy) - Policy layer: hard constraints that are never traded off against utility. (20 names)
- [`opensmartroute.realtime`](#opensmartrouterealtime) - Real-time operational controls: health, circuit breaking, rate & budget limits. (9 names)
- [`opensmartroute.retrieval`](#opensmartrouteretrieval) - Retrieval-based candidate narrowing for very large target pools (ToolRet / Skill-RAG). (12 names)
- [`opensmartroute.router`](#opensmartrouterouter) - The Router: signals -> policy -> strategies -> ensemble -> decision (+plan). (3 names)
- [`opensmartroute.sdk`](#opensmartroutesdk) - OpenSmartRoute SDK: decorator-driven registration of routing components. (16 names)
- [`opensmartroute.security`](#opensmartroutesecurity) - Security controls for the routing control plane. (27 names)
- [`opensmartroute.security.gadget`](#opensmartroutesecuritygadget) - Learned confounder-gadget detector (Rerouting LLM Routers, Shafran et al. 2025). (6 names)
- [`opensmartroute.security.injection`](#opensmartroutesecurityinjection) - Instruction-injection detection for *text that is not the user's request*. (4 names)
- [`opensmartroute.security.limits`](#opensmartroutesecuritylimits) - Resource-amplification limits ("Beyond Max Tokens"). (8 names)
- [`opensmartroute.security.provenance`](#opensmartroutesecurityprovenance) - Origin (provenance) policy for tool parameters – ROPE-style control-flow integrity. (12 names)
- [`opensmartroute.security.safety`](#opensmartroutesecuritysafety) - Safety-routing regression suite ("When Safety Routing Breaks"). (4 names)
- [`opensmartroute.server`](#opensmartrouteserver) - Optional FastAPI server exposing the router over HTTP. (14 names)
- [`opensmartroute.settings`](#opensmartroutesettings) - Typed, environment-overridable settings: the only home for OpenSmartRoute's tunable constants. (13 names)
- [`opensmartroute.signals`](#opensmartroutesignals) - Signal extraction: cheap, deterministic features computed from the request. (44 names)
- [`opensmartroute.signals.events`](#opensmartroutesignalsevents) - Event- and workflow-driven signals for agentic inputs. (7 names)
- [`opensmartroute.signals.models`](#opensmartroutesignalsmodels) - Learned signal models: hashed n-gram linear models with no dependencies. (7 names)
- [`opensmartroute.signals.ontology`](#opensmartroutesignalsontology) - Task ontology: families -> types -> subtypes, with an orthogonal *domain* axis. (4 names)
- [`opensmartroute.signals.uncertainty`](#opensmartroutesignalsuncertainty) - Uncertainty signals that come from *outside the query text*. (12 names)
- [`opensmartroute.stack`](#opensmartroutestack) - Declarative *stacks*: one document that describes a whole routing setup. (12 names)
- [`opensmartroute.strategies`](#opensmartroutestrategies) - see module (66 names)
- [`opensmartroute.strategies.aggregate`](#opensmartroutestrategiesaggregate) - Routing / aggregation switch (Mixture-of-Agents, Wang et al. 2024; JiSi 2601.01330). (5 names)
- [`opensmartroute.strategies.auction`](#opensmartroutestrategiesauction) - Error-aware reverse auction across providers (EA-RAM 2608.12719). (5 names)
- [`opensmartroute.strategies.bandit`](#opensmartroutestrategiesbandit) - Online-learning strategy: contextual Thompson-sampling bandit. (1 names)
- [`opensmartroute.strategies.base`](#opensmartroutestrategiesbase) - Strategy interface. (3 names)
- [`opensmartroute.strategies.capability`](#opensmartroutestrategiescapability) - Capability-fit strategy: match request signals to a target's declared capabilities. (1 names)
- [`opensmartroute.strategies.cascade`](#opensmartroutestrategiescascade) - Cascade execution (FrugalGPT / Router-R1 multi-round / AutoMix POMDP). (5 names)
- [`opensmartroute.strategies.defer`](#opensmartroutestrategiesdefer) - Defer-to-human and effort ("think or not") strategies. (3 names)
- [`opensmartroute.strategies.edge`](#opensmartroutestrategiesedge) - Edge-cloud token-aware routing (Pro-Router 2608.28726; RelayLLM 2601.05167). (1 names)
- [`opensmartroute.strategies.elastic`](#opensmartroutestrategieselastic) - Token-budget-aware routing to elastic / many-in-one models (Nemotron Elastic 2511.16664; Star 2605.07182). (3 names)
- [`opensmartroute.strategies.escalation`](#opensmartroutestrategiesescalation) - Bayesian self-escalation during generation (2608.24087). (5 names)
- [`opensmartroute.strategies.human`](#opensmartroutestrategieshuman) - Routing among human annotators and experts (QUORUM 2608.27974; Dawid-Skene 1979). (5 names)
- [`opensmartroute.strategies.llm_judge`](#opensmartroutestrategiesllm_judge) - Generative routing: let an LLM act as the router (Router-R1 / LLM-as-judge). (2 names)
- [`opensmartroute.strategies.memory`](#opensmartroutestrategiesmemory) - Memory-tier routing for agents (BudgetMem 2602.06025; Gated-Memory Routing 2609.00237). (5 names)
- [`opensmartroute.strategies.modality`](#opensmartroutestrategiesmodality) - Multimodal routing and modality escalation (LatentRouter 2605.11301; modality escalation. (4 names)
- [`opensmartroute.strategies.probe`](#opensmartroutestrategiesprobe) - Hidden-state routing with a Dirichlet probe (ProbeDirichlet, RouterXBench 2602.11877). (2 names)
- [`opensmartroute.strategies.progress`](#opensmartroutestrategiesprogress) - Agentic trajectories: per-step routing and multi-round execution. (5 names)
- [`opensmartroute.strategies.protocol`](#opensmartroutestrategiesprotocol) - Collaboration-protocol selection (2608.14927). (7 names)
- [`opensmartroute.strategies.rules`](#opensmartroutestrategiesrules) - Declarative rule-based routing (Arch-Router-style domain/action preferences). (2 names)
- [`opensmartroute.strategies.semantic_cache`](#opensmartroutestrategiessemantic_cache) - Semantic caching as a routing target (GPTCache; vLLM semantic router 2603.04444). (4 names)
- [`opensmartroute.strategies.session`](#opensmartroutestrategiessession) - Session affinity: keep a conversation with the target that is already serving it. (2 names)
- [`opensmartroute.strategies.similarity`](#opensmartroutestrategiessimilarity) - Similarity-based routing (UniRoute / GraphRouter flavour). (3 names)
- [`opensmartroute.strategies.speculative`](#opensmartroutestrategiesspeculative) - Speculative (draft-based) cascades (speculative cascades, Narasimhan et al. 2024; Differential. (3 names)
- [`opensmartroute.strategies.task_table`](#opensmartroutestrategiestask_table) - Static task table strategy (SCX Router, 2609.02292). (1 names)

## `opensmartroute`

Source: [src/opensmartroute/__init__.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/__init__.py)

OpenSmartRoute — an open, intelligent route to the right decision, solution, or destination.

| Name | Kind | Summary |
|---|---|---|
| `AsyncRouter` | re-export of [`opensmartroute.aio.AsyncRouter`](#opensmartrouteaio) | ``await``-able wrapper around a :class:`Router`: route / learn / execute / run off the event loop. |
| `BanditStrategy` | re-export of [`opensmartroute.strategies.bandit.BanditStrategy`](#opensmartroutestrategiesbandit) | Thompson-sampling Beta bandit per (context, target); context = dominant domain (+ plan role). |
| `Capabilities` | re-export of [`opensmartroute.core.types.Capabilities`](#opensmartroutecoretypes) | Declarative description of what a target is good at. |
| `CapabilityStrategy` | re-export of [`opensmartroute.strategies.capability.CapabilityStrategy`](#opensmartroutestrategiescapability) | Declarative fit: domain / action overlap, complexity band, language, modality and quality prior. |
| `Cascade` | re-export of [`opensmartroute.strategies.cascade.Cascade`](#opensmartroutestrategiescascade) | Execute ranked targets in planner order (cheapest / MDP / POMDP), stopping when the quality gate passes. |
| `CascadePlanner` | re-export of [`opensmartroute.strategies.cascade.CascadePlanner`](#opensmartroutestrategiescascade) | Finite-horizon MDP over an ordered cascade with a stop action after each step. |
| `ComponentRegistry` | re-export of [`opensmartroute.sdk.ComponentRegistry`](#opensmartroutesdk) | Blueprints for every routing component, with decorators that register into it. |
| `ConfigurationError` | re-export of [`opensmartroute.errors.ConfigurationError`](#opensmartrouteerrors) | Invalid catalogue, rules, SKILL.md, settings or a missing optional dependency. |
| `DeferStrategy` | re-export of [`opensmartroute.strategies.defer.DeferStrategy`](#opensmartroutestrategiesdefer) | Learning-to-defer: scores human targets by risk, PII, escalation intent, frustration and model uncertainty. |
| `EffortStrategy` | re-export of [`opensmartroute.strategies.defer.EffortStrategy`](#opensmartroutestrategiesdefer) | Match a target's reasoning ``effort_level`` to ``signals.reasoning_need``; penalise over- and under-thinking. |
| `Event` | re-export of [`opensmartroute.observability.Event`](#opensmartrouteobservability) | One captured span or event: flat, JSON-friendly, never carries request text. |
| `EventSink` | re-export of [`opensmartroute.observability.EventSink`](#opensmartrouteobservability) | Receiver port: override :meth:`emit`; live bridges may also implement :meth:`span_start` / :meth:`span_end`. |
| `ExecutionError` | re-export of [`opensmartroute.errors.ExecutionError`](#opensmartrouteerrors) | A target handler failed while executing a plan. |
| `ExecutionResult` | re-export of [`opensmartroute.execution.ExecutionResult`](#opensmartrouteexecution) | What happened when a decision was executed. |
| `ExecutionStep` | re-export of [`opensmartroute.execution.ExecutionStep`](#opensmartrouteexecution) | One executed plan slot: role, target, latency and whether it succeeded. |
| `FeedbackStore` | re-export of [`opensmartroute.feedback.FeedbackStore`](#opensmartroutefeedback) | Append-only :class:`Outcome` log (in memory or JSONL file) with per-target statistics. |
| `LLMJudgeStrategy` | re-export of [`opensmartroute.strategies.llm_judge.LLMJudgeStrategy`](#opensmartroutestrategiesllm_judge) | LLM-as-router with optional **score calibration**. |
| `NoRouteError` | re-export of [`opensmartroute.errors.NoRouteError`](#opensmartrouteerrors) | No target satisfied the hard constraints. |
| `Objective` | re-export of [`opensmartroute.core.types.Objective`](#opensmartroutecoretypes) | What the caller wants to optimise. Weights are relative. |
| `OpenSmartRouteError` | re-export of [`opensmartroute.errors.OpenSmartRouteError`](#opensmartrouteerrors) | Base class for all SDK errors. |
| `Outcome` | re-export of [`opensmartroute.core.types.Outcome`](#opensmartroutecoretypes) | Feedback about how a routed request actually went. |
| `PlanSlot` | re-export of [`opensmartroute.core.types.PlanSlot`](#opensmartroutecoretypes) | One filled slot of a multi-target plan (persona -> skill -> model). |
| `Policy` | re-export of [`opensmartroute.policy.Policy`](#opensmartroutepolicy) | Ordered chain of :data:`PolicyRule`; returns the first rejection reason or ``None``. |
| `ProgressRouter` | re-export of [`opensmartroute.strategies.progress.ProgressRouter`](#opensmartroutestrategiesprogress) | Route each step of a task with trajectory context. |
| `RankedTarget` | re-export of [`opensmartroute.core.types.RankedTarget`](#opensmartroutecoretypes) | A scored candidate: utility, ensemble quality estimate and the per-strategy breakdown. |
| `RequestConstraints` | re-export of [`opensmartroute.core.types.RequestConstraints`](#opensmartroutecoretypes) | Hard constraints on the request (never traded off). |
| `RouteDecision` | re-export of [`opensmartroute.core.types.RouteDecision`](#opensmartroutecoretypes) | The answer to ``route()``: chosen target, confidence, alternatives, optional plan, trace and propensities. |
| `RoutePlan` | re-export of [`opensmartroute.core.types.RoutePlan`](#opensmartroutecoretypes) | A composed route (MasRouter-style): several targets working together. |
| `RouteRequest` | re-export of [`opensmartroute.core.types.RouteRequest`](#opensmartroutecoretypes) | The customer need. |
| `RouteTarget` | re-export of [`opensmartroute.core.types.RouteTarget`](#opensmartroutecoretypes) | A routable destination: an LLM, agent, skill, persona, tool, workflow or human. |
| `RouteTrace` | re-export of [`opensmartroute.core.types.RouteTrace`](#opensmartroutecoretypes) | Everything needed to explain a decision. |
| `Router` | re-export of [`opensmartroute.router.Router`](#opensmartrouterouter) | Signals -> policy -> strategies -> ensemble -> decision (+ optional plan). |
| `RouterSLM` | re-export of [`opensmartroute.learning.slm.RouterSLM`](#opensmartroutelearningslm) | Small routing model: dual encoder + target catalogue snapshot + calibration, in one JSON file. |
| `Rule` | re-export of [`opensmartroute.strategies.rules.Rule`](#opensmartroutestrategiesrules) | If all `when` conditions match, boost `prefer` targets and penalise `avoid`. |
| `RulesStrategy` | re-export of [`opensmartroute.strategies.rules.RulesStrategy`](#opensmartroutestrategiesrules) | Applies declarative :class:`Rule` preferences (prefer / avoid / pin) when their ``when`` conditions match. |
| `SLMStrategy` | re-export of [`opensmartroute.learning.slm.SLMStrategy`](#opensmartroutelearningslm) | Ensemble member backed by a :class:`RouterSLM`; scores are its probabilities, and it keeps learning online. |
| `SecurityError` | re-export of [`opensmartroute.errors.SecurityError`](#opensmartrouteerrors) | Request rejected by an input guard (prompt injection, oversize, etc.). |
| `SelfImprover` | re-export of [`opensmartroute.learning.self_improve.SelfImprover`](#opensmartroutelearningself_improve) | Closed loop that keeps a :class:`RouterSLM` current with the model market and its own traffic. |
| `Settings` | re-export of [`opensmartroute.settings.Settings`](#opensmartroutesettings) | All tunables, grouped by consumer. Immutable; derive variants with :meth:`replace`. |
| `Signals` | re-export of [`opensmartroute.core.types.Signals`](#opensmartroutecoretypes) | Cheap deterministic features extracted from a request. |
| `SimilarityStrategy` | re-export of [`opensmartroute.strategies.similarity.SimilarityStrategy`](#opensmartroutestrategiessimilarity) | Embed the request and each target's examples / description; score by best and top-k mean similarity. |
| `Span` | re-export of [`opensmartroute.observability.Span`](#opensmartrouteobservability) | An open unit of work; a context manager that records duration, status and nested events. |
| `StateStoreError` | re-export of [`opensmartroute.errors.StateStoreError`](#opensmartrouteerrors) | A learner-state store failed to load, save or migrate. |
| `Strategy` | re-export of [`opensmartroute.strategies.base.Strategy`](#opensmartroutestrategiesbase) | Scores each candidate target in [0, 1] and explains why. |
| `StrategyScore` | re-export of [`opensmartroute.core.types.StrategyScore`](#opensmartroutecoretypes) | One strategy's opinion about one target. |
| `TargetConstraints` | re-export of [`opensmartroute.core.types.TargetConstraints`](#opensmartroutecoretypes) | Where / for whom a target may be used. Checked by the policy layer. |
| `TargetKind` | re-export of [`opensmartroute.core.types.TargetKind`](#opensmartroutecoretypes) | Kinds of things a request can be routed to. |
| `TargetRegistry` | re-export of [`opensmartroute.core.registry.TargetRegistry`](#opensmartroutecoreregistry) | In-memory catalogue of :class:`RouteTarget` by id: add / upsert / remove, filtered listing, (de)serialisation. |
| `TargetUnavailableError` | re-export of [`opensmartroute.errors.TargetUnavailableError`](#opensmartrouteerrors) | A remote target or catalogue source could not be reached. |
| `TaskTableStrategy` | re-export of [`opensmartroute.strategies.task_table.TaskTableStrategy`](#opensmartroutestrategiestask_table) | Static ``task_type -> target -> quality`` table with family and prior fallbacks; learns from outcomes. |
| `Tracer` | re-export of [`opensmartroute.observability.Tracer`](#opensmartrouteobservability) | Opens spans, records events and fans them out to sinks; ``sample_rate`` < 1 traces a share of requests. |
| `ValidationError` | re-export of [`opensmartroute.errors.ValidationError`](#opensmartrouteerrors) | A request, outcome or target failed validation. |
| `__version__` | constant |  |
| `agent` | re-export of [`opensmartroute.sdk.agent`](#opensmartroutesdk) | @agent(id, ...): shorthand for an ``agent`` target. |
| `components` | re-export of [`opensmartroute.sdk.components`](#opensmartroutesdk) | : The process-wide registry that the top-level decorators (``opensmartroute.strategy`` ...) bind to. |
| `configure` | re-export of [`opensmartroute.settings.configure`](#opensmartroutesettings) | Install process-wide settings. ``configure()`` with no arguments re-reads the environment;. |
| `configure_tracing` | re-export of [`opensmartroute.observability.configure_tracing`](#opensmartrouteobservability) | Install sinks on the process-wide tracer. With no sinks, use the ones named by settings. |
| `get_settings` | re-export of [`opensmartroute.settings.get_settings`](#opensmartroutesettings) | The process-wide :class:`Settings` (environment overlay applied once, lazily). |
| `get_tracer` | re-export of [`opensmartroute.observability.get_tracer`](#opensmartrouteobservability) | The process-wide tracer (disabled until :func:`configure_tracing` adds sinks). |
| `load_rules` | re-export of [`opensmartroute.config.load_rules`](#opensmartrouteconfig) | Load a rules file (top-level list or ``rules:`` key) into a :class:`RulesStrategy`. |
| `load_targets` | re-export of [`opensmartroute.config.load_targets`](#opensmartrouteconfig) | Load a catalogue file (top-level list or ``targets:`` key) into a :class:`TargetRegistry`. |
| `middleware` | re-export of [`opensmartroute.sdk.middleware`](#opensmartroutesdk) | @middleware: register a Middleware class or ``fn(request, next_route)``. |
| `policy_rule` | re-export of [`opensmartroute.sdk.policy_rule`](#opensmartroutesdk) | @policy_rule: register ``fn(target, request, signals) -> reason \| None``. |
| `signal` | re-export of [`opensmartroute.sdk.signal`](#opensmartroutesdk) | @signal: register a SignalExtractor class or ``fn(request, signals) -> mapping``. |
| `skill` | re-export of [`opensmartroute.sdk.skill`](#opensmartroutesdk) | @skill(id, ...): shorthand for a ``skill`` target. |
| `strategy` | re-export of [`opensmartroute.sdk.strategy`](#opensmartroutesdk) | @strategy(weight=, name=): register a Strategy class or ``fn(request, signals, candidates)``. |
| `target` | re-export of [`opensmartroute.sdk.target`](#opensmartroutesdk) | @target(id, kind, ...): the decorated callable becomes a RouteTarget handler. |
| `telemetry` | re-export of [`opensmartroute.sdk.telemetry`](#opensmartroutesdk) | @telemetry: register a Telemetry sink class or factory. |
| `tool` | re-export of [`opensmartroute.sdk.tool`](#opensmartroutesdk) | @tool(id, ...): shorthand for a ``tool`` target. |

## `opensmartroute.adapters`

Source: [src/opensmartroute/adapters/__init__.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/adapters/__init__.py)

Adapters connect OpenSmartRoute to real providers and infrastructure.

| Name | Kind | Summary |
|---|---|---|
| `AgentHarness` | re-export of [`opensmartroute.adapters.harness.AgentHarness`](#opensmartrouteadaptersharness) | Structural interface of an agent runtime: ``run(task, context=, history=) -> HarnessResult``. |
| `CallableHarness` | re-export of [`opensmartroute.adapters.harness.CallableHarness`](#opensmartrouteadaptersharness) | Wrap an in-process agent: ``fn(task, context, history) -> str \| dict \| HarnessResult``. |
| `ChatResult` | re-export of [`opensmartroute.adapters.openai_compat.ChatResult`](#opensmartrouteadaptersopenai_compat) | Result of a chat completion: text, model, token counts, latency and the raw response. |
| `HTTPHarness` | re-export of [`opensmartroute.adapters.harness.HTTPHarness`](#opensmartrouteadaptersharness) | POST the task as JSON to an agent endpoint. |
| `HarnessResult` | re-export of [`opensmartroute.adapters.harness.HarnessResult`](#opensmartrouteadaptersharness) | What an agent harness returns: text, success, token / cost / latency usage and optional self-graded quality. |
| `InMemoryQueue` | re-export of [`opensmartroute.adapters.handlers.InMemoryQueue`](#opensmartrouteadaptershandlers) | Reference :class:`Queue`: FIFO in memory, resolves into :class:`~opensmartroute.Outcome`. |
| `ModelCard` | re-export of [`opensmartroute.adapters.catalogue.ModelCard`](#opensmartrouteadapterscatalogue) | What the catalogue knows about one model: identity, price, limits, evidence of quality, provenance. |
| `ModelCatalogue` | re-export of [`opensmartroute.adapters.catalogue.ModelCatalogue`](#opensmartrouteadapterscatalogue) | Merged, persisted model cards from every source; the SLM's view of the target universe. |
| `OpenAICompatClient` | re-export of [`opensmartroute.adapters.openai_compat.OpenAICompatClient`](#opensmartrouteadaptersopenai_compat) | Stdlib-only client for the OpenAI chat / embeddings API (OpenAI, Azure, vLLM, Ollama, LiteLLM) with retries. |
| `OpenTelemetrySink` | re-export of [`opensmartroute.adapters.optional.OpenTelemetrySink`](#opensmartrouteadaptersoptional) | Live bridge from the tracer to OpenTelemetry: every OpenSmartRoute span becomes an OTel span. |
| `OpenTelemetryTelemetry` | re-export of [`opensmartroute.adapters.optional.OpenTelemetryTelemetry`](#opensmartrouteadaptersoptional) | Emits one span per decision and counters/histograms via the OTel API. |
| `PendingResult` | re-export of [`opensmartroute.adapters.handlers.PendingResult`](#opensmartrouteadaptershandlers) | Immediate answer of a queued target: the request was accepted and will be answered later. |
| `Queue` | re-export of [`opensmartroute.adapters.handlers.Queue`](#opensmartrouteadaptershandlers) | Structural interface of an asynchronous queue target (ticketing, human tier, workflow run). |
| `QueuedItem` | re-export of [`opensmartroute.adapters.handlers.QueuedItem`](#opensmartrouteadaptershandlers) | One request waiting in (or resolved from) a queue. |
| `SearchHit` | re-export of [`opensmartroute.adapters.websearch.SearchHit`](#opensmartrouteadapterswebsearch) | One search result: where it came from, what it says, and when it was seen. |
| `SemanticRouterImport` | re-export of [`opensmartroute.adapters.semantic_router.SemanticRouterImport`](#opensmartrouteadapterssemantic_router) | Result of importing a vLLM semantic-router config: registry, rules, default model, categories, warnings. |
| `ServerCard` | re-export of [`opensmartroute.adapters.mcp_servers.ServerCard`](#opensmartrouteadaptersmcp_servers) | Description of an MCP server (tools, tags, auth, latency, cost, region, data boundary) for recommendation. |
| `ServerRecommendation` | re-export of [`opensmartroute.adapters.mcp_servers.ServerRecommendation`](#opensmartrouteadaptersmcp_servers) | A ranked server from ``recommend_servers`` with its fused score, rationale and matched tool names. |
| `StdioMCPClient` | re-export of [`opensmartroute.adapters.mcp.StdioMCPClient`](#opensmartrouteadaptersmcp) | Tiny JSON-RPC-over-stdio MCP client (newline-delimited). Thread-safe, blocking. |
| `SubprocessHarness` | re-export of [`opensmartroute.adapters.harness.SubprocessHarness`](#opensmartrouteadaptersharness) | Run a CLI agent: task on stdin, answer on stdout, exit code 0 = success. |
| `WebKnowledge` | re-export of [`opensmartroute.adapters.websearch.WebKnowledge`](#opensmartrouteadapterswebsearch) | Fan a query out to search providers, de-duplicate by URL and cache the hits as JSON. |
| `a2a_handler` | re-export of [`opensmartroute.adapters.a2a.a2a_handler`](#opensmartrouteadaptersa2a) | Return a handler that sends the request text to an A2A agent and returns its text. |
| `agent_from_card` | re-export of [`opensmartroute.adapters.a2a.agent_from_card`](#opensmartrouteadaptersa2a) | Build an ``agent`` RouteTarget from an A2A agent card (name, skills, tags, input modes). |
| `brave_search` | re-export of [`opensmartroute.adapters.websearch.brave_search`](#opensmartrouteadapterswebsearch) | Brave Search API web results; the key is read from ``BRAVE_API_KEY`` (or ``BRAVE_API_KEY_FILE``). |
| `card_to_target` | re-export of [`opensmartroute.adapters.catalogue.card_to_target`](#opensmartrouteadapterscatalogue) | A :class:`RouteTarget` for a model card; risky third-party descriptions are replaced by the name. |
| `attach_chat_handlers` | re-export of [`opensmartroute.adapters.openai_compat.attach_chat_handlers`](#opensmartrouteadaptersopenai_compat) | Give every LLM target a :func:`chat_handler` on ``client``; returns the ids that got one. |
| `chat_handler` | re-export of [`opensmartroute.adapters.openai_compat.chat_handler`](#opensmartrouteadaptersopenai_compat) | Adapter for ``RouteTarget.handler``: turns a RouteRequest into a chat call. |
| `connect_mcp` | re-export of [`opensmartroute.adapters.mcp.connect_mcp`](#opensmartrouteadaptersmcp) | Spawn a stdio MCP server, list its tools and return ``(client, targets)``. |
| `duckduckgo_search` | re-export of [`opensmartroute.adapters.websearch.duckduckgo_search`](#opensmartrouteadapterswebsearch) | DuckDuckGo instant-answer API (abstract + related topics). Keyless; shallow but good for definitions. |
| `embedder` | re-export of [`opensmartroute.adapters.openai_compat.embedder`](#opensmartrouteadaptersopenai_compat) | Adapter for ``SimilarityStrategy(embedder=...)``. |
| `enrich_description` | re-export of [`opensmartroute.adapters.mcp.enrich_description`](#opensmartrouteadaptersmcp) | Return ``(routing_description, examples, capabilities)`` for a tool. |
| `fetch_agent_card` | re-export of [`opensmartroute.adapters.a2a.fetch_agent_card`](#opensmartrouteadaptersa2a) | Download an agent card over HTTPS (plain HTTP is refused); raises TargetUnavailableError on failure. |
| `fetch_bytes` | re-export of [`opensmartroute.adapters.websearch.fetch_bytes`](#opensmartrouteadapterswebsearch) | GET ``url`` over https with a timeout and a body cap; transport errors become ``TargetUnavailableError``. |
| `fetch_huggingface_models` | re-export of [`opensmartroute.adapters.catalogue.fetch_huggingface_models`](#opensmartrouteadapterscatalogue) | Model cards from the Hugging Face Hub search (downloads, likes, tags, ``model-index`` benchmarks). |
| `fetch_json` | re-export of [`opensmartroute.adapters.websearch.fetch_json`](#opensmartrouteadapterswebsearch) | GET a JSON document (see :func:`fetch_bytes`); malformed bodies raise ``TargetUnavailableError``. |
| `fetch_leaderboard_quality` | re-export of [`opensmartroute.adapters.catalogue.fetch_leaderboard_quality`](#opensmartrouteadapterscatalogue) | ``{hub model id: {benchmark: accuracy}}`` from the Open LLM Leaderboard table (official rows, unflagged). |
| `fetch_openrouter_models` | re-export of [`opensmartroute.adapters.catalogue.fetch_openrouter_models`](#opensmartrouteadapterscatalogue) | Model cards from OpenRouter's public listing (prices per token, context, modalities, tool support). |
| `fetch_page_text` | re-export of [`opensmartroute.adapters.websearch.fetch_page_text`](#opensmartrouteadapterswebsearch) | Fetch a page and return ``{url, title, text, risk}``; ``risk`` is the injection/gadget risk of the text. |
| `harness_handler` | re-export of [`opensmartroute.adapters.harness.harness_handler`](#opensmartrouteadaptersharness) | Adapter for ``RouteTarget.handler``: a RouteRequest becomes a harness task. |
| `html_to_text` | re-export of [`opensmartroute.adapters.websearch.html_to_text`](#opensmartrouteadapterswebsearch) | ``(title, text)`` of an HTML document with scripts/styles removed and whitespace collapsed. |
| `http_handler` | re-export of [`opensmartroute.adapters.handlers.http_handler`](#opensmartrouteadaptershandlers) | ``RouteTarget.handler`` that POSTs the request to ``url`` and returns a :class:`HarnessResult`. |
| `huggingface_search` | re-export of [`opensmartroute.adapters.websearch.huggingface_search`](#opensmartrouteadapterswebsearch) | Search the Hugging Face Hub (``what`` = ``models`` or ``datasets``), ranked by downloads. No key needed. |
| `judge_fn` | re-export of [`opensmartroute.adapters.openai_compat.judge_fn`](#opensmartrouteadaptersopenai_compat) | Adapter for ``LLMJudgeStrategy(llm=...)``: prompt in, completion text out. |
| `langgraph_condition` | re-export of [`opensmartroute.adapters.frameworks.langgraph_condition`](#opensmartrouteadaptersframeworks) | Edge selector for ``add_conditional_edges``: routes on target id (``by='target'``) or kind. |
| `langgraph_node` | re-export of [`opensmartroute.adapters.frameworks.langgraph_node`](#opensmartrouteadaptersframeworks) | Return a LangGraph-compatible node ``state -> dict`` (partial state update). |
| `load_personas` | re-export of [`opensmartroute.adapters.personas.load_personas`](#opensmartrouteadapterspersonas) | Load personas from a directory of markdown files or a JSON/JSONL/CSV catalogue. |
| `load_semantic_router_config` | re-export of [`opensmartroute.adapters.semantic_router.load_semantic_router_config`](#opensmartrouteadapterssemantic_router) | Convert a vLLM semantic-router ``model_config`` / ``categories`` document into targets and rules. |
| `load_skill` | re-export of [`opensmartroute.adapters.skills.load_skill`](#opensmartrouteadaptersskills) | Load one skill directory (must contain ``SKILL.md``). |
| `load_skills` | re-export of [`opensmartroute.adapters.skills.load_skills`](#opensmartrouteadaptersskills) | Load every ``*/SKILL.md`` under ``root`` (one level deep, sorted by name). |
| `maf_router_executor` | re-export of [`opensmartroute.adapters.frameworks.maf_router_executor`](#opensmartrouteadaptersframeworks) | Agent Framework style: ``executor(message, ctx) -> target id \| response``; ``handoffs`` maps. |
| `manifest_from_targets` | re-export of [`opensmartroute.adapters.mcp.manifest_from_targets`](#opensmartrouteadaptersmcp) | Reverse: dump MCP-shaped tool dicts (for signing / publishing a catalogue). |
| `model_key` | re-export of [`opensmartroute.adapters.catalogue.model_key`](#opensmartrouteadapterscatalogue) | Vendor-agnostic key for matching model names across sources: lowercase, no vendor prefix, no punctuation. |
| `mcp_tool_handler` | re-export of [`opensmartroute.adapters.handlers.mcp_tool_handler`](#opensmartrouteadaptershandlers) | ``RouteTarget.handler`` that invokes one MCP tool via ``call(name, arguments)``. |
| `openai_tool_handler` | re-export of [`opensmartroute.adapters.frameworks.openai_tool_handler`](#opensmartrouteadaptersframeworks) | Callable behind ``openai_tool_spec``: ``(text, objective?, kinds?) -> RouteDecision.to_dict()``. |
| `openai_tool_spec` | re-export of [`opensmartroute.adapters.frameworks.openai_tool_spec`](#opensmartrouteadaptersframeworks) | OpenAI function-calling tool definition that lets a model ask the router for a target. |
| `persona_from_markdown` | re-export of [`opensmartroute.adapters.personas.persona_from_markdown`](#opensmartrouteadapterspersonas) | Parse a ``*.agent.md`` / ``*.chatmode.md`` / front-matter markdown file into a persona target. |
| `persona_target` | re-export of [`opensmartroute.adapters.personas.persona_target`](#opensmartrouteadapterspersonas) | Build a ``persona`` RouteTarget whose ``instructions`` is the system prompt (non-primary by default). |
| `personas_from_records` | re-export of [`opensmartroute.adapters.personas.personas_from_records`](#opensmartrouteadapterspersonas) | Persona targets from JSON / CSV-style records (``name`` + ``prompt``/``system`` keys); others skipped. |
| `quality_from_benchmarks` | re-export of [`opensmartroute.adapters.catalogue.quality_from_benchmarks`](#opensmartrouteadapterscatalogue) | Mean of normalised benchmark scores (percentages are divided by 100); None when there are none. |
| `quality_from_popularity` | re-export of [`opensmartroute.adapters.catalogue.quality_from_popularity`](#opensmartrouteadapterscatalogue) | Weak prior in [0.35, 0.75] from log-scaled downloads and likes (popularity is not quality; it is a hint). |
| `queue_handler` | re-export of [`opensmartroute.adapters.handlers.queue_handler`](#opensmartrouteadaptershandlers) | ``RouteTarget.handler`` that enqueues the request and returns a :class:`PendingResult`. |
| `recommend_servers` | re-export of [`opensmartroute.adapters.mcp_servers.recommend_servers`](#opensmartrouteadaptersmcp_servers) | Rank MCP servers for a task; constraint violations are excluded, not down-weighted. |
| `route_and_execute` | re-export of [`opensmartroute.adapters.frameworks.route_and_execute`](#opensmartrouteadaptersframeworks) | Route ``text`` and optionally execute the plan; returns ``(decision, ExecutionResult \| None)``. |
| `sentence_transformers_embedder` | re-export of [`opensmartroute.adapters.optional.sentence_transformers_embedder`](#opensmartrouteadaptersoptional) | Semantic embedder for ``SimilarityStrategy``. Requires ``opensmartroute[embeddings]``. |
| `sign_manifest` | re-export of [`opensmartroute.adapters.mcp.sign_manifest`](#opensmartrouteadaptersmcp) | Wrap a tool list in a signed manifest. ``algorithm`` = ``hmac-sha256`` (key = shared. |
| `skill_from_markdown` | re-export of [`opensmartroute.adapters.skills.skill_from_markdown`](#opensmartrouteadaptersskills) | Parse one SKILL.md (Agent-Skills front matter + body) into a ``skill`` RouteTarget; strict validation. |
| `skills_from_card` | re-export of [`opensmartroute.adapters.a2a.skills_from_card`](#opensmartrouteadaptersa2a) | One non-primary ``skill`` target per skill declared on an A2A agent card (``<agent>/<skill>``). |
| `tools_from_manifest` | re-export of [`opensmartroute.adapters.mcp.tools_from_manifest`](#opensmartrouteadaptersmcp) | Import tools from a (signed) manifest; ``kw`` goes to :func:`tools_from_mcp`. |
| `tools_from_mcp` | re-export of [`opensmartroute.adapters.mcp.tools_from_mcp`](#opensmartrouteadaptersmcp) | Convert ``tools/list`` output into targets. ``call(name, arguments)`` becomes the handler. |
| `verify_manifest` | re-export of [`opensmartroute.adapters.mcp.verify_manifest`](#opensmartrouteadaptersmcp) | Constant-time verification. ``key`` = shared secret (HMAC) or 32-byte Ed25519 public key. |

## `opensmartroute.adapters.a2a`

Source: [src/opensmartroute/adapters/a2a.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/adapters/a2a.py)

Import A2A (Agent-to-Agent protocol) **Agent Cards** as ``TargetKind.AGENT`` targets.

| Name | Kind | Summary |
|---|---|---|
| `a2a_handler` | function `(url: str, token: str \| None=None, timeout_s: float=60.0, method: str='message/send')` | Return a handler that sends the request text to an A2A agent and returns its text. |
| `agent_from_card` | function `(card: dict[str, Any], *, call: Callable[[str, RouteRequest], Any] \| None=None, cost_per_1k_tokens: float=0.0, latency_ms: float=2000.0, quality_prior: float=0.65, languages: list[str] \| None=None)` | Build an ``agent`` RouteTarget from an A2A agent card (name, skills, tags, input modes). |
| `fetch_agent_card` | function `(base_url: str, timeout_s: float=10.0, path: str='/.well-known/agent.json')` | Download an agent card over HTTPS (plain HTTP is refused); raises TargetUnavailableError on failure. |
| `skills_from_card` | function `(card: dict[str, Any], agent_id: str \| None=None)` | One non-primary ``skill`` target per skill declared on an A2A agent card (``<agent>/<skill>``). |

## `opensmartroute.adapters.catalogue`

Source: [src/opensmartroute/adapters/catalogue.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/adapters/catalogue.py)

Live model catalogue: collect model cards (price, context, modalities, benchmarks) from public sources.

| Name | Kind | Summary |
|---|---|---|
| `LEADERBOARD_DATASET` | constant | Open LLM Leaderboard v2 results table on the Hub. |
| `OPENROUTER_MODELS` | constant | public model + pricing listing (no key). |
| `ModelCard` | class | What the catalogue knows about one model: identity, price, limits, evidence of quality, provenance. |
| `ModelCatalogue` | class | Merged, persisted model cards from every source; the SLM's view of the target universe. |
| `card_to_target` | function `(card: ModelCard, *, default_latency_ms: float=1500.0, settings: Settings \| None=None)` | A :class:`RouteTarget` for a model card; risky third-party descriptions are replaced by the name. |
| `fetch_huggingface_models` | function `(query: str='', *, limit: int=50, pipeline: str='text-generation', timeout_s: float \| None=None, settings: Settings \| None=None)` | Model cards from the Hugging Face Hub search (downloads, likes, tags, ``model-index`` benchmarks). |
| `fetch_leaderboard_quality` | function `(*, limit: int=5000, dataset: str=LEADERBOARD_DATASET, timeout_s: float \| None=None, settings: Settings \| None=None)` | ``{hub model id: {benchmark: accuracy}}`` from the Open LLM Leaderboard table (official rows, unflagged). |
| `fetch_openrouter_models` | function `(*, url: str=OPENROUTER_MODELS, timeout_s: float \| None=None, settings: Settings \| None=None)` | Model cards from OpenRouter's public listing (prices per token, context, modalities, tool support). |
| `model_key` | function `(model_id: str)` | Vendor-agnostic key for matching model names across sources: lowercase, no vendor prefix, no punctuation. |
| `quality_from_benchmarks` | function `(benchmarks: dict[str, float])` | Mean of normalised benchmark scores (percentages are divided by 100); None when there are none. |
| `quality_from_popularity` | function `(downloads: int, likes: int)` | Weak prior in [0.35, 0.75] from log-scaled downloads and likes (popularity is not quality; it is a hint). |

## `opensmartroute.adapters.frameworks`

Source: [src/opensmartroute/adapters/frameworks.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/adapters/frameworks.py)

Drop-in nodes for agent frameworks.

| Name | Kind | Summary |
|---|---|---|
| `langgraph_condition` | function `(by: str='target', default: str='__end__')` | Edge selector for ``add_conditional_edges``: routes on target id (``by='target'``) or kind. |
| `langgraph_node` | function `(router: Router, *, execute: bool=False, plan: bool=False, context_key: str='route_context', on_no_route: str='__no_route__')` | Return a LangGraph-compatible node ``state -> dict`` (partial state update). |
| `last_user_text` | function `(state: dict[str, Any])` | (last user utterance, prior history as role/content dicts). |
| `maf_router_executor` | function `(router: Router, *, execute: bool=False)` | Agent Framework style: ``executor(message, ctx) -> target id \| response``; ``handoffs`` maps. |
| `openai_tool_handler` | function `(router: Router)` | Callable behind ``openai_tool_spec``: ``(text, objective?, kinds?) -> RouteDecision.to_dict()``. |
| `openai_tool_spec` | function `(name: str='route_request')` | OpenAI function-calling tool definition that lets a model ask the router for a target. |
| `route_and_execute` | function `(router: Router, text: str, *, history: list[dict[str, str]] \| None=None, context: dict[str, Any] \| None=None, objective: Objective \| None=None, execute: bool=False, plan: bool=False, learn: bool=True)` | Route ``text`` and optionally execute the plan; returns ``(decision, ExecutionResult \| None)``. |

## `opensmartroute.adapters.handlers`

Source: [src/opensmartroute/adapters/handlers.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/adapters/handlers.py)

Executors for non-LLM targets: HTTP endpoints, MCP tools and asynchronous queues.

| Name | Kind | Summary |
|---|---|---|
| `InMemoryQueue` | class | Reference :class:`Queue`: FIFO in memory, resolves into :class:`~opensmartroute.Outcome`. |
| `PendingResult` | class | Immediate answer of a queued target: the request was accepted and will be answered later. |
| `Queue` | class | Structural interface of an asynchronous queue target (ticketing, human tier, workflow run). |
| `QueuedItem` | class | One request waiting in (or resolved from) a queue. |
| `http_handler` | function `(url: str, *, api_key: str \| None=None, api_key_env: str \| None=None, timeout_s: float=300.0, headers: Mapping[str, str] \| None=None, payload: Callable[[str, Mapping[str, Any], list[dict[str, str]]], dict[str, Any]] \| None=None, body: Mapping[str, Any] \| None=None)` | ``RouteTarget.handler`` that POSTs the request to ``url`` and returns a :class:`HarnessResult`. |
| `mcp_tool_handler` | function `(call: Callable[[str, dict[str, Any]], Any], tool: str, input_schema: dict[str, Any] \| None=None, arguments: Callable[[RouteRequest], dict[str, Any]] \| None=None)` | ``RouteTarget.handler`` that invokes one MCP tool via ``call(name, arguments)``. |
| `queue_handler` | function `(queue: Queue, target_id: str \| None=None)` | ``RouteTarget.handler`` that enqueues the request and returns a :class:`PendingResult`. |

## `opensmartroute.adapters.harness`

Source: [src/opensmartroute/adapters/harness.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/adapters/harness.py)

Agent-harness adapters: route to a *runtime*, not just a model.

| Name | Kind | Summary |
|---|---|---|
| `AgentHarness` | class | Structural interface of an agent runtime: ``run(task, context=, history=) -> HarnessResult``. |
| `CallableHarness` | class | Wrap an in-process agent: ``fn(task, context, history) -> str \| dict \| HarnessResult``. |
| `HTTPHarness` | class | POST the task as JSON to an agent endpoint. |
| `HarnessResult` | class | What an agent harness returns: text, success, token / cost / latency usage and optional self-graded quality. |
| `SubprocessHarness` | class | Run a CLI agent: task on stdin, answer on stdout, exit code 0 = success. |
| `coerce_result` | function `(out: Any)` | Normalise ``str \| dict \| HarnessResult \| object-with-text`` into a HarnessResult. |
| `harness_handler` | function `(harness: AgentHarness)` | Adapter for ``RouteTarget.handler``: a RouteRequest becomes a harness task. |

## `opensmartroute.adapters.mcp`

Source: [src/opensmartroute/adapters/mcp.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/adapters/mcp.py)

Import MCP (Model Context Protocol) tools as ``TargetKind.TOOL`` targets.

| Name | Kind | Summary |
|---|---|---|
| `MCPTool` | class | A tool as listed by an MCP server: name, description, JSON input schema and annotations. |
| `StdioMCPClient` | class | Tiny JSON-RPC-over-stdio MCP client (newline-delimited). Thread-safe, blocking. |
| `canonical_json` | function `(obj: Any)` | Deterministic JSON encoding (sorted keys, no whitespace) used for manifest digests and signatures. |
| `connect_mcp` | function `(command: list[str], server: str='', **kw: Any)` | Spawn a stdio MCP server, list its tools and return ``(client, targets)``. |
| `enrich_description` | function `(name: str, description: str, input_schema: dict[str, Any] \| None, server: str='', enricher: Callable[[str], list[str]] \| None=None)` | Return ``(routing_description, examples, capabilities)`` for a tool. |
| `manifest_digest` | function `(tools: list[dict[str, Any]])` | SHA-256 hex digest of a tool list in canonical JSON. |
| `manifest_from_targets` | function `(targets: Iterable[RouteTarget])` | Reverse: dump MCP-shaped tool dicts (for signing / publishing a catalogue). |
| `sign_manifest` | function `(tools: list[dict[str, Any]], key: bytes, *, server: str='', algorithm: str='hmac-sha256', key_id: str='')` | Wrap a tool list in a signed manifest. ``algorithm`` = ``hmac-sha256`` (key = shared. |
| `tools_from_manifest` | function `(manifest: dict[str, Any], key: bytes \| None, *, require_signature: bool=True, max_age_s: float \| None=None, **kw: Any)` | Import tools from a (signed) manifest; ``kw`` goes to :func:`tools_from_mcp`. |
| `tools_from_mcp` | function `(payload: Any, server: str='', call: Callable[[str, dict[str, Any]], Any] \| None=None, enricher: Callable[[str], list[str]] \| None=None, cost_per_call_usd: float=0.0, latency_ms: float=300.0, quality_prior: float=0.6, max_description_risk: float=0.6, guard: InputGuard \| None=None)` | Convert ``tools/list`` output into targets. ``call(name, arguments)`` becomes the handler. |
| `verify_manifest` | function `(manifest: dict[str, Any], key: bytes, max_age_s: float \| None=None)` | Constant-time verification. ``key`` = shared secret (HMAC) or 32-byte Ed25519 public key. |

## `opensmartroute.adapters.mcp_servers`

Source: [src/opensmartroute/adapters/mcp_servers.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/adapters/mcp_servers.py)

MCP *server* recommendation (MCP-Zero 2506.01056; ToolRet 2603.06467).

| Name | Kind | Summary |
|---|---|---|
| `ServerCard` | class | Description of an MCP server (tools, tags, auth, latency, cost, region, data boundary) for recommendation. |
| `ServerRecommendation` | class | A ranked server from ``recommend_servers`` with its fused score, rationale and matched tool names. |
| `recommend_servers` | function `(request: RouteRequest \| str, servers: Iterable[ServerCard \| dict[str, Any]], k: int=3, constraints: RequestConstraints \| None=None, allowed_auth: Iterable[str] \| None=None, min_score: float=0.0)` | Rank MCP servers for a task; constraint violations are excluded, not down-weighted. |

## `opensmartroute.adapters.openai_compat`

Source: [src/opensmartroute/adapters/openai_compat.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/adapters/openai_compat.py)

OpenAI-compatible HTTP client (stdlib only).

| Name | Kind | Summary |
|---|---|---|
| `ChatResult` | class | Result of a chat completion: text, model, token counts, latency and the raw response. |
| `OpenAICompatClient` | class | Stdlib-only client for the OpenAI chat / embeddings API (OpenAI, Azure, vLLM, Ollama, LiteLLM) with retries. |
| `attach_chat_handlers` | function `(targets: Iterable[Any], client: OpenAICompatClient, *, model_key: str='model', default_model: str \| None=None, overwrite: bool=False)` | Give every LLM target a :func:`chat_handler` on ``client``; returns the ids that got one. |
| `chat_handler` | function `(client: OpenAICompatClient, model: str, system_prompt: str \| None=None, **defaults: Any)` | Adapter for ``RouteTarget.handler``: turns a RouteRequest into a chat call. |
| `embedder` | function `(client: OpenAICompatClient, model: str, batch: int=64)` | Adapter for ``SimilarityStrategy(embedder=...)``. |
| `judge_fn` | function `(client: OpenAICompatClient, model: str, max_tokens: int=400)` | Adapter for ``LLMJudgeStrategy(llm=...)``: prompt in, completion text out. |

## `opensmartroute.adapters.optional`

Source: [src/opensmartroute/adapters/optional.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/adapters/optional.py)

Optional adapters that need extra dependencies. Everything is lazily imported so the.

| Name | Kind | Summary |
|---|---|---|
| `OpenTelemetrySink` | class | Live bridge from the tracer to OpenTelemetry: every OpenSmartRoute span becomes an OTel span. |
| `OpenTelemetryTelemetry` | class | Emits one span per decision and counters/histograms via the OTel API. |
| `sentence_transformers_embedder` | function `(model_name: str='sentence-transformers/all-MiniLM-L6-v2', device: str \| None=None, normalize: bool=True)` | Semantic embedder for ``SimilarityStrategy``. Requires ``opensmartroute[embeddings]``. |

## `opensmartroute.adapters.personas`

Source: [src/opensmartroute/adapters/personas.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/adapters/personas.py)

Import persona catalogues as ``TargetKind.PERSONA`` targets.

| Name | Kind | Summary |
|---|---|---|
| `load_personas` | function `(path: str \| Path, **kw: Any)` | Load personas from a directory of markdown files or a JSON/JSONL/CSV catalogue. |
| `persona_from_markdown` | function `(text: str, *, path: Path \| None=None, **kw: Any)` | Parse a ``*.agent.md`` / ``*.chatmode.md`` / front-matter markdown file into a persona target. |
| `persona_target` | function `(name: str, prompt: str, description: str='', *, id_prefix: str='persona:', domains: list[str] \| None=None, tags: list[str] \| None=None, languages: list[str] \| None=None, primary: bool=False, quality_prior: float=0.6, source: str='', extra: dict[str, Any] \| None=None)` | Build a ``persona`` RouteTarget whose ``instructions`` is the system prompt (non-primary by default). |
| `personas_from_records` | function `(records: Iterable[dict[str, Any]], **kw: Any)` | Persona targets from JSON / CSV-style records (``name`` + ``prompt``/``system`` keys); others skipped. |

## `opensmartroute.adapters.semantic_router`

Source: [src/opensmartroute/adapters/semantic_router.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/adapters/semantic_router.py)

Import a vLLM *semantic-router* configuration (vllm-project/semantic-router).

| Name | Kind | Summary |
|---|---|---|
| `DEFAULT_DOMAIN_MAP` | constant | semantic-router's MMLU-style categories -> OpenSmartRoute ontology domains. |
| `SemanticRouterImport` | class | Result of importing a vLLM semantic-router config: registry, rules, default model, categories, warnings. |
| `load_semantic_router_config` | function `(source: str \| Path \| dict[str, Any], domain_map: dict[str, list[str]] \| None=None, rule_weight: float=0.8)` | Convert a vLLM semantic-router ``model_config`` / ``categories`` document into targets and rules. |

## `opensmartroute.adapters.skills`

Source: [src/opensmartroute/adapters/skills.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/adapters/skills.py)

Load Agent-Skills ``SKILL.md`` packages as ``TargetKind.SKILL`` targets.

| Name | Kind | Summary |
|---|---|---|
| `load_skill` | function `(skill_dir: str \| Path, **kw: Any)` | Load one skill directory (must contain ``SKILL.md``). |
| `load_skills` | function `(root: str \| Path, **kw: Any)` | Load every ``*/SKILL.md`` under ``root`` (one level deep, sorted by name). |
| `parse_frontmatter` | function `(text: str)` | Split ``---`` frontmatter from the body. Returns ``(frontmatter, body)``. |
| `skill_from_markdown` | function `(text: str, *, path: Path \| None=None, cost_per_1k_tokens: float=0.0)` | Parse one SKILL.md (Agent-Skills front matter + body) into a ``skill`` RouteTarget; strict validation. |

## `opensmartroute.adapters.websearch`

Source: [src/opensmartroute/adapters/websearch.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/adapters/websearch.py)

Web knowledge for the self-improving router: stdlib HTTP fetch, search providers, page text.

| Name | Kind | Summary |
|---|---|---|
| `BRAVE_API` | constant | Brave web search (needs BRAVE_API_KEY). |
| `DATASETS_SERVER` | constant | Hugging Face datasets-server REST root (/rows). |
| `DDG_API` | constant | DuckDuckGo instant-answer JSON endpoint (no key). |
| `HF_API` | constant | Hugging Face Hub REST root (models, datasets, search). |
| `SearchHit` | class | One search result: where it came from, what it says, and when it was seen. |
| `SearchProvider` | constant | (query, limit) -> hits. |
| `WebKnowledge` | class | Fan a query out to search providers, de-duplicate by URL and cache the hits as JSON. |
| `brave_search` | function `(query: str, limit: int=10, *, api_key_env: str='BRAVE_API_KEY', timeout_s: float \| None=None, settings: Settings \| None=None)` | Brave Search API web results; the key is read from ``BRAVE_API_KEY`` (or ``BRAVE_API_KEY_FILE``). |
| `duckduckgo_search` | function `(query: str, limit: int=10, *, timeout_s: float \| None=None, settings: Settings \| None=None)` | DuckDuckGo instant-answer API (abstract + related topics). Keyless; shallow but good for definitions. |
| `fetch_bytes` | function `(url: str, *, timeout_s: float \| None=None, max_bytes: int \| None=None, headers: Mapping[str, str] \| None=None, settings: Settings \| None=None)` | GET ``url`` over https with a timeout and a body cap; transport errors become ``TargetUnavailableError``. |
| `fetch_json` | function `(url: str, *, timeout_s: float \| None=None, headers: Mapping[str, str] \| None=None, settings: Settings \| None=None)` | GET a JSON document (see :func:`fetch_bytes`); malformed bodies raise ``TargetUnavailableError``. |
| `fetch_page_text` | function `(url: str, *, max_chars: int=20000, timeout_s: float \| None=None, settings: Settings \| None=None)` | Fetch a page and return ``{url, title, text, risk}``; ``risk`` is the injection/gadget risk of the text. |
| `html_to_text` | function `(html: str)` | ``(title, text)`` of an HTML document with scripts/styles removed and whitespace collapsed. |
| `huggingface_search` | function `(query: str, limit: int=10, *, what: str='models', pipeline_tag: str \| None='text-generation', timeout_s: float \| None=None, settings: Settings \| None=None)` | Search the Hugging Face Hub (``what`` = ``models`` or ``datasets``), ranked by downloads. No key needed. |

## `opensmartroute.aio`

Source: [src/opensmartroute/aio.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/aio.py)

Async façade. Routing itself is CPU-bound and sub-millisecond, so we run it in.

| Name | Kind | Summary |
|---|---|---|
| `AsyncRouter` | class | ``await``-able wrapper around a :class:`Router`: route / learn / execute / run off the event loop. |

## `opensmartroute.branding`

Source: [src/opensmartroute/branding.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/branding.py)

OpenSmartRoute naming conventions: one place for every brand-bound identifier.

| Name | Kind | Summary |
|---|---|---|
| `API_KEY_ENV` | constant | ``OSR_API_KEY`` - access token the CLI sends (overrides the credentials file). |
| `API_URL_ENV` | constant | ``OSR_API_URL`` - platform / server URL the CLI talks to. |
| `BRAND` | constant | product name, one word. |
| `CLI` | constant | console script name. |
| `CONFIG_DIR_ENV` | constant | ``OSR_CONFIG_DIR`` - overrides the per-user config folder. |
| `CONFIG_DIR_NAME` | constant | per-user config folder name (``~/.config/opensmartroute``, ``%APPDATA%\opensmartroute``). |
| `ENTRY_POINT_GROUP` | constant | importlib.metadata entry-point group for plugins. |
| `ENV_PREFIX` | constant | ``OSR_`` - every settings environment variable starts with this. |
| `ERROR_CODE_PREFIX` | constant | ``OSR_`` - machine-readable error codes (``OSR_NO_ROUTE``). |
| `INSTALL_SCRIPT_PS1` | constant | Windows installer: ``irm .../install.ps1 \| iex``. |
| `INSTALL_SCRIPT_SH` | constant | Linux / macOS installer: ``curl -fsSL .../install.sh \| sh``. |
| `LOCAL_TOKEN_PREFIX` | constant | self-hosted ``osr serve`` access tokens (``osr_local_...``). |
| `METADATA_PREFIX` | constant | ``osr-`` - SKILL.md / persona front-matter keys (``osr-domains``). |
| `PACKAGE` | constant | Python package / distribution / logger root. |
| `REPOSITORY` | constant | source repository (installer fallback URLs). |
| `SHORT_NAME` | constant | short form used for env-var and error-code prefixes. |
| `SKILLS_DIR` | constant | : Default Agent-Skills root (``*/SKILL.md``); the location Claude Code discovers project skills in. |
| `STATE_DIR` | constant | default learner-state directory. |
| `TOKEN_PREFIX` | constant | hosted-platform API keys (``osr_live_...``). |
| `WEBSITE` | constant | public website; also the default hosted-platform URL of ``osr login``. |
| `env_key` | function `(*parts: str)` | ``env_key("routing", "softmax_temperature") -> "OSR_ROUTING_SOFTMAX_TEMPERATURE"``. |
| `error_code` | function `(kind: str)` | ``error_code("no_route") -> "OSR_NO_ROUTE"``. |
| `logger` | function `(component: str \| None=None)` | ``logger() -> "opensmartroute"``; ``logger("enterprise") -> "opensmartroute.enterprise"``. |
| `metadata_key` | function `(field: str)` | ``metadata_key("quality_prior") -> "osr-quality-prior"`` (SKILL.md / persona frontmatter). |
| `platform_url` | function `(environ: Mapping[str, str] \| None=None)` | The platform / server URL the CLI talks to: ``OSR_API_URL`` or the hosted platform (:data:`WEBSITE`). |
| `user_agent` | function `()` | ``"opensmartroute/<version>"`` for outbound HTTP clients. |
| `version` | function `()` | The package version (``opensmartroute.__version__``), resolved at call time. |

## `opensmartroute.cli`

Source: [src/opensmartroute/cli.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/cli.py)

``osr`` command-line interface.

| Name | Kind | Summary |
|---|---|---|
| `build_router` | function `(targets: str, rules: str \| None=None, state: str \| None=None, models: str \| None=None, skills: str \| None=None, slm: str \| None=None, remember_requests: int=0)` | Assemble the CLI's router from targets / rules / state / models / SKILL.md / SLM paths (every command). |
| `main` | function `(argv: list[str] \| None=None)` | Entry point of the ``osr`` console script; returns the process exit status. |

## `opensmartroute.config`

Source: [src/opensmartroute/config.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/config.py)

Configuration loading: targets and rules from JSON or YAML.

| Name | Kind | Summary |
|---|---|---|
| `load_document` | function `(path: str \| Path)` | Read a JSON or YAML file (YAML needs the ``yaml`` extra); raises ConfigurationError on any problem. |
| `load_rules` | function `(path: str \| Path)` | Load a rules file (top-level list or ``rules:`` key) into a :class:`RulesStrategy`. |
| `load_targets` | function `(path: str \| Path)` | Load a catalogue file (top-level list or ``targets:`` key) into a :class:`TargetRegistry`. |

## `opensmartroute.core`

Source: [src/opensmartroute/core/__init__.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/core/__init__.py)

| Name | Kind | Summary |
|---|---|---|
| `Capabilities` | re-export of [`opensmartroute.core.types.Capabilities`](#opensmartroutecoretypes) | Declarative description of what a target is good at. |
| `Objective` | re-export of [`opensmartroute.core.types.Objective`](#opensmartroutecoretypes) | What the caller wants to optimise. Weights are relative. |
| `Outcome` | re-export of [`opensmartroute.core.types.Outcome`](#opensmartroutecoretypes) | Feedback about how a routed request actually went. |
| `PlanSlot` | re-export of [`opensmartroute.core.types.PlanSlot`](#opensmartroutecoretypes) | One filled slot of a multi-target plan (persona -> skill -> model). |
| `RankedTarget` | re-export of [`opensmartroute.core.types.RankedTarget`](#opensmartroutecoretypes) | A scored candidate: utility, ensemble quality estimate and the per-strategy breakdown. |
| `RequestConstraints` | re-export of [`opensmartroute.core.types.RequestConstraints`](#opensmartroutecoretypes) | Hard constraints on the request (never traded off). |
| `RouteDecision` | re-export of [`opensmartroute.core.types.RouteDecision`](#opensmartroutecoretypes) | The answer to ``route()``: chosen target, confidence, alternatives, optional plan, trace and propensities. |
| `RoutePlan` | re-export of [`opensmartroute.core.types.RoutePlan`](#opensmartroutecoretypes) | A composed route (MasRouter-style): several targets working together. |
| `RouteRequest` | re-export of [`opensmartroute.core.types.RouteRequest`](#opensmartroutecoretypes) | The customer need. |
| `RouteTarget` | re-export of [`opensmartroute.core.types.RouteTarget`](#opensmartroutecoretypes) | A routable destination: an LLM, agent, skill, persona, tool, workflow or human. |
| `RouteTrace` | re-export of [`opensmartroute.core.types.RouteTrace`](#opensmartroutecoretypes) | Everything needed to explain a decision. |
| `Signals` | re-export of [`opensmartroute.core.types.Signals`](#opensmartroutecoretypes) | Cheap deterministic features extracted from a request. |
| `StrategyScore` | re-export of [`opensmartroute.core.types.StrategyScore`](#opensmartroutecoretypes) | One strategy's opinion about one target. |
| `TargetConstraints` | re-export of [`opensmartroute.core.types.TargetConstraints`](#opensmartroutecoretypes) | Where / for whom a target may be used. Checked by the policy layer. |
| `TargetKind` | re-export of [`opensmartroute.core.types.TargetKind`](#opensmartroutecoretypes) | Kinds of things a request can be routed to. |
| `TargetRegistry` | re-export of [`opensmartroute.core.registry.TargetRegistry`](#opensmartroutecoreregistry) | In-memory catalogue of :class:`RouteTarget` by id: add / upsert / remove, filtered listing, (de)serialisation. |

## `opensmartroute.core.registry`

Source: [src/opensmartroute/core/registry.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/core/registry.py)

Target registry: the catalogue of everything a request may be routed to.

| Name | Kind | Summary |
|---|---|---|
| `TargetRegistry` | class | In-memory catalogue of :class:`RouteTarget` by id: add / upsert / remove, filtered listing, (de)serialisation. |

## `opensmartroute.core.types`

Source: [src/opensmartroute/core/types.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/core/types.py)

Core data model for OpenSmartRoute.

| Name | Kind | Summary |
|---|---|---|
| `TargetKind` | class | Kinds of things a request can be routed to. |
| `Capabilities` | class | Declarative description of what a target is good at. |
| `TargetConstraints` | class | Where / for whom a target may be used. Checked by the policy layer. |
| `RouteTarget` | class | A routable destination: an LLM, agent, skill, persona, tool, workflow or human. |
| `EFFORT_LEVELS` | constant | Named reasoning-effort levels -> numeric effort in [0, 1] (``RouteTarget.effort``). |
| `Objective` | class | What the caller wants to optimise. Weights are relative. |
| `RequestConstraints` | class | Hard constraints on the request (never traded off). |
| `RouteRequest` | class | The customer need. |
| `Signals` | class | Cheap deterministic features extracted from a request. |
| `StrategyScore` | class | One strategy's opinion about one target. |
| `RankedTarget` | class | A scored candidate: utility, ensemble quality estimate and the per-strategy breakdown. |
| `RouteTrace` | class | Everything needed to explain a decision. |
| `PlanSlot` | class | One filled slot of a multi-target plan (persona -> skill -> model). |
| `RoutePlan` | class | A composed route (MasRouter-style): several targets working together. |
| `RouteDecision` | class | The answer to ``route()``: chosen target, confidence, alternatives, optional plan, trace and propensities. |
| `Outcome` | class | Feedback about how a routed request actually went. |

## `opensmartroute.credentials`

Source: [src/opensmartroute/credentials.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/credentials.py)

Credentials for the ``osr`` CLI: where the access token lives and how it is obtained.

| Name | Kind | Summary |
|---|---|---|
| `CREDENTIALS_FILE` | constant | file name inside :func:`config_dir`. |
| `DEFAULT_PROFILE` | constant | profile used when ``--profile`` is not given. |
| `DEVICE_CODE_PATH` | constant | RFC 8628 device-authorization endpoint of the platform. |
| `DEVICE_TOKEN_PATH` | constant | noqa: S105  # nosec B105 - RFC 8628 token endpoint (a URL path). |
| `ME_PATH` | constant | platform: who am I (workspace, plan, edition). |
| `TOKEN_BYTES` | constant | entropy of :func:`generate_token`. |
| `WHOAMI_PATH` | constant | self-hosted ``osr serve``: is this token accepted. |
| `Credential` | class | One saved sign-in: where (``url``), what (``token``) and what the platform said about it. |
| `CredentialStore` | class | Profiles in ``<config dir>/credentials.json`` (owner-only permissions on POSIX). |
| `PlatformClient` | class | Minimal JSON client for the platform / server API used by the CLI (stdlib only, injectable transport). |
| `Transport` | constant | : ``(method, url, headers, body, timeout) -> (status, json)``; tests inject a fake instead of urllib. |
| `apply_identity` | function `(cred: Credential, info: Mapping[str, Any])` | Copy what :func:`whoami` learned (kind, workspace, plan, edition) onto ``cred``. |
| `config_dir` | function `(environ: Mapping[str, str] \| None=None)` | Per-user configuration directory (``OSR_CONFIG_DIR`` > ``%APPDATA%`` > ``$XDG_CONFIG_HOME`` > ``~/.config``). |
| `device_login` | function `(url: str \| None=None, *, client_name: str \| None=None, open_browser: bool=True, out: Callable[[str], None]=print, transport: Transport \| None=None, sleep: Callable[[float], None]=time.sleep, timeout_s: float \| None=None)` | Sign in to the hosted platform with the device authorization grant and return the credential. |
| `generate_token` | function `(prefix: str=LOCAL_TOKEN_PREFIX)` | A fresh random access token (``osr_local_<43 chars>``) for a self-hosted ``osr serve``. |
| `redact` | function `(token: str, keep: int=6)` | ``osr_live_abc123...`` - the prefix plus a few characters, never the whole token. |
| `token_kind` | function `(token: str)` | ``"platform"`` for ``osr_live_`` keys, ``"server"`` for ``osr_local_`` tokens, else ``"unknown"``. |
| `whoami` | function `(cred: Credential, *, transport: Transport \| None=None, timeout: float=30.0)` | Validate ``cred`` against its server and describe the identity behind it. |

## `opensmartroute.discovery`

Source: [src/opensmartroute/discovery.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/discovery.py)

Tool discovery beyond text similarity.

| Name | Kind | Summary |
|---|---|---|
| `CachePreservingSelector` | class | Keep the serialized tool list prefix-stable across turns of a session. |
| `SchemaAwareStrategy` | class | Scores tools by how many of their ``input_schema`` parameters the request can fill (see ``schema_match``). |
| `SkillGraph` | class | Dependency / conflict / composition graph over skill targets. |
| `extract_entities` | function `(text: str)` | Typed entity mentions found in ``text`` (kind -> values). |
| `schema_match` | function `(request: RouteRequest \| str, target: RouteTarget)` | Return ``(coverage, filled, missing)`` for the target's input schema. |

## `opensmartroute.enterprise`

Source: [src/opensmartroute/enterprise/__init__.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/enterprise/__init__.py)

Enterprise integration layer: ports (hexagonal architecture), middleware and telemetry.

| Name | Kind | Summary |
|---|---|---|
| `AuditSink` | class | Tamper-evident audit trail port (hash-chained). |
| `CacheMiddleware` | class | LRU decision cache keyed on (text, constraints, objective, route options). TTL in seconds. |
| `EnterpriseRouter` | class | Router + middleware chain + telemetry + auto-learning + audit. Thread-safe façade. |
| `FileAuditSink` | class | Append-only JSONL, each line ``{prev, hash, record}`` with. |
| `FileStateStore` | class | One JSON file per key under ``root``; keys are hashed so they can't traverse paths. |
| `InMemoryStateStore` | class | Thread-safe dict-backed :class:`StateStore`; values are deep-copied through JSON on read and write. |
| `LoggingTelemetry` | class | Structured JSON logs; never logs raw request text (only a hash + length). |
| `MetricsTelemetry` | class | In-process counters, outcome tallies and a latency histogram. |
| `Middleware` | class | Chain-of-responsibility hook around routing: ``__call__(request, next_) -> RouteDecision``. |
| `RouteFn` | constant | the "next" callable a middleware wraps. |
| `RouterBuilder` | class | Fluent builder that validates configuration and wires all enterprise pieces. |
| `SavingsEntry` | re-export of [`opensmartroute.enterprise.savings.SavingsEntry`](#opensmartrouteenterprisesavings) | One routed request in the ledger. |
| `SavingsLedger` | re-export of [`opensmartroute.enterprise.savings.SavingsLedger`](#opensmartrouteenterprisesavings) | Telemetry sink that keeps a per-request baseline-vs-routed cost ledger (see module docs). |
| `SavingsReport` | re-export of [`opensmartroute.enterprise.savings.SavingsReport`](#opensmartrouteenterprisesavings) | Aggregate savings and the quality they were bought at. |
| `StateStore` | class | Key/value persistence port for learner state, caches, breaker state. |
| `Telemetry` | class | Observer port with no-op defaults. Implement for OpenTelemetry, Prometheus, Datadog…. |
| `TenantMiddleware` | class | Enforces that a tenant is present and applies per-tenant defaults/limits. |
| `TimeoutMiddleware` | class | Soft deadline: raise if routing itself exceeded ``budget_ms`` (should never happen. |

## `opensmartroute.enterprise.ops`

Source: [src/opensmartroute/enterprise/ops.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/enterprise/ops.py)

Operational controls: shadow / A-B routing, tenant fairness and queue-aware latency.

| Name | Kind | Summary |
|---|---|---|
| `SPRT` | class | Wald SPRT for two Bernoulli success rates: H0 p=p0 vs H1 p=p0+delta. |
| `ABTest` | class | Traffic split + SPRT comparison of candidate vs control outcomes. |
| `FairShareMiddleware` | class | Dominant Resource Fairness across tenants over a sliding window. |
| `InflightTracker` | class | Per-target in-flight counters + arrival/service statistics for queueing estimates. |
| `QueueAwareStrategy` | class | Scores targets by *current* time-to-first-token = catalogue latency + queueing wait vs. |
| `ShadowMiddleware` | class | Run ``candidate`` beside production. ``mode='shadow'``: log only. ``mode='ab'``: serve. |
| `TenantUsage` | class | Sliding-window ledger of one tenant's requests, cost and tokens for :class:`FairShareMiddleware`. |

## `opensmartroute.enterprise.savings`

Source: [src/opensmartroute/enterprise/savings.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/enterprise/savings.py)

Savings ledger - the always-on savings report that backs the ROI story and the dashboard.

| Name | Kind | Summary |
|---|---|---|
| `SavingsEntry` | class | One routed request in the ledger. |
| `SavingsLedger` | class | Telemetry sink that keeps a per-request baseline-vs-routed cost ledger (see module docs). |
| `SavingsReport` | class | Aggregate savings and the quality they were bought at. |

## `opensmartroute.enterprise.stores`

Source: [src/opensmartroute/enterprise/stores.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/enterprise/stores.py)

Production state-store backends and wrappers.

| Name | Kind | Summary |
|---|---|---|
| `BatchedStateStore` | class | Write-behind buffer. Reads are served from the pending map first. |
| `EncryptedStateStore` | class | AES-256-GCM envelope encryption. ``key`` is 32 raw bytes, or read (base64/hex/raw). |
| `Migration` | constant | state-document transform applied by VersionedStateStore. |
| `NamespacedStateStore` | class | Prefix every key with ``<namespace>/`` so several routers or tenants can share one backing store. |
| `RedisStateStore` | class | StateStore on any redis-py-compatible client (``get`` / ``set`` / ``delete``), JSON values, optional TTL. |
| `SQLStateStore` | class | DB-API 2.0 key/value store (PostgreSQL, SQLite, MySQL). |
| `VersionedStateStore` | class | Schema-versioned envelope with forward migrations. |

## `opensmartroute.errors`

Source: [src/opensmartroute/errors.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/errors.py)

Exception hierarchy. Every error raised by the SDK derives from :class:`OpenSmartRouteError`.

| Name | Kind | Summary |
|---|---|---|
| `OpenSmartRouteError` | class | Base class for all SDK errors. |
| `ConfigurationError` | class | Invalid catalogue, rules, SKILL.md, settings or a missing optional dependency. |
| `ValidationError` | class | A request, outcome or target failed validation. |
| `NoRouteError` | class | No target satisfied the hard constraints. |
| `TargetUnavailableError` | class | A remote target or catalogue source could not be reached. |
| `ExecutionError` | class | A target handler failed while executing a plan. |
| `SecurityError` | class | Request rejected by an input guard (prompt injection, oversize, etc.). |
| `StateStoreError` | class | A learner-state store failed to load, save or migrate. |
| `AuthenticationError` | class | The CLI has no valid access token for the platform / server, or a sign-in was denied or timed out. |
| `OpenSmartRouteDeprecationWarning` | class | Emitted by :func:`deprecated`; filter with ``warnings.simplefilter`` on this class. |
| `deprecated` | function `(name: str, *, since: str, removal: str, replacement: str \| None=None, stacklevel: int=3)` | Announce a deprecation according to the policy in CONTRIBUTING.md. |

## `opensmartroute.estimate`

Source: [src/opensmartroute/estimate.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/estimate.py)

Token, cost and latency estimates *before* a request is sent anywhere.

| Name | Kind | Summary |
|---|---|---|
| `DEFAULT_OUTPUT_TOKENS` | constant | : Output length assumed when neither the caller nor the signals say (a short answer). |
| `MESSAGE_OVERHEAD_TOKENS` | constant | : Tokens a chat API adds per message for role / separators (OpenAI-style framing). |
| `PriceHook` | constant | : ``(usd_per_1k_input, usd_per_1k_output)`` for a target, or ``None`` to fall back to its declared cost. |
| `RequestEstimate` | class | A quote for one request across every candidate, with named picks. |
| `TargetEstimate` | class | The quote for one candidate target. |
| `estimate` | function `(router: Router, request: RouteRequest \| str, *, output_tokens: int \| None=None, kinds: list[str] \| None=None, prices: PriceHook \| None=None, quality_tolerance: float=0.1, decision: RouteDecision \| None=None)` | Quote ``request`` against every candidate the router would consider - nothing is executed. |
| `estimate_messages_tokens` | function `(messages: list[dict[str, Any]])` | Token estimate for an OpenAI-style message list (content plus per-message framing). |
| `estimate_tokens` | function `(text: str)` | Approximate the tokenizer count of ``text`` without any tokenizer library. |
| `target_prices` | function `(target: RouteTarget)` | ``(usd per 1k input tokens, usd per 1k output tokens)`` from the target's declared cost. |

## `opensmartroute.eval`

Source: [src/opensmartroute/eval/__init__.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/eval/__init__.py)

RouterBench-style evaluation harness.

| Name | Kind | Summary |
|---|---|---|
| `AuditReport` | re-export of [`opensmartroute.eval.audit.AuditReport`](#opensmartrouteevalaudit) | Aggregate of a shadow replay; ``to_markdown`` renders the Routing Audit deliverable. |
| `AuditRow` | re-export of [`opensmartroute.eval.audit.AuditRow`](#opensmartrouteevalaudit) | One logged request: what happened, and what the router would have done. |
| `DatasetCollector` | re-export of [`opensmartroute.eval.collect.DatasetCollector`](#opensmartrouteevalcollect) | Cache-backed corpus builder: collect sources, add feedback / synthetic rows, dedupe, split. |
| `DatasetSource` | re-export of [`opensmartroute.eval.collect.DatasetSource`](#opensmartrouteevalcollect) | One Hub dataset split to collect: repo id, config, split, the preset that parses it, an optional model map. |
| `EvalResult` | class | Aggregate routing metrics over a dataset: accuracy, cost, latency, confidence, calibration, coverage. |
| `EvalRow` | class | One labelled prompt: expected / acceptable targets or per-target quality scores (RouterBench style). |
| `area_under_frontier` | function `(points: list[dict[str, float]])` | Trapezoidal area under accuracy(cost) — analogous to RouterBench's AIQ. |
| `calibration_report` | function `(router: Router, rows: list[EvalRow], objective: Objective \| None=None)` | Confidence calibration of the router on labelled rows: ECE, Brier, reliability bins and. |
| `collect_dataset` | re-export of [`opensmartroute.eval.collect.collect_dataset`](#opensmartrouteevalcollect) | Download ``source`` from the Hub and parse it into rows. |
| `cost_quality_frontier` | function `(router: Router, rows: list[EvalRow], cost_weights: list[float] \| None=None)` | Sweep the cost weight to trace the accuracy-vs-cost curve (RouterBench Fig. 1 style). |
| `evaluate` | function `(router: Router, rows: list[EvalRow], objective: Objective \| None=None)` | Route every row and score accuracy, realised quality, cost, latency, ECE / Brier and conformal coverage. |
| `fetch_hf_rows` | re-export of [`opensmartroute.eval.collect.fetch_hf_rows`](#opensmartrouteevalcollect) | One page (max 100) of records from the datasets-server ``/rows`` endpoint, flattened to ``{column: value}``. |
| `load_audit_log` | re-export of [`opensmartroute.eval.audit.load_audit_log`](#opensmartrouteevalaudit) | Read a JSONL traffic log (``text`` or ``prompt``/``messages`` per line). |
| `load_dataset` | function `(path: str \| Path)` | Read a JSONL evaluation dataset (``text`` or ``prompt`` plus the optional EvalRow keys); a line that is. |
| `model_quality` | re-export of [`opensmartroute.eval.collect.model_quality`](#opensmartrouteevalcollect) | Data-derived quality prior per model in [0, 1]: pairwise rows (two scored models) are fitted with. |
| `routing_audit` | re-export of [`opensmartroute.eval.audit.routing_audit`](#opensmartrouteevalaudit) | Replay ``rows`` through ``router`` in shadow mode and aggregate an :class:`AuditReport`. |
| `rows_from_feedback` | re-export of [`opensmartroute.eval.collect.rows_from_feedback`](#opensmartrouteevalcollect) | Rows from :class:`Outcome` records. ``texts`` maps ``request_id`` to the prompt (outcomes carry no text);. |
| `summarize_audit` | re-export of [`opensmartroute.eval.audit.summarize_audit`](#opensmartrouteevalaudit) | Aggregate :class:`AuditRow` records into an :class:`AuditReport`. |
| `synthetic_rows` | re-export of [`opensmartroute.eval.collect.synthetic_rows`](#opensmartrouteevalcollect) | Ontology seed prompts labelled with the best capability-fit target (cold-start supervision). |

## `opensmartroute.eval.agentic`

Source: [src/opensmartroute/eval/agentic.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/eval/agentic.py)

tau-bench-style agentic task evaluation: does per-step routing beat the best single agent?.

| Name | Kind | Summary |
|---|---|---|
| `AgentTask` | class | A multi-step task with the measured per-step success probability and latency of each agent. |
| `load_agentic_tasks` | function `(path: str \| Path)` | Read tasks from JSONL: ``{"task_id", "steps", "success", "latency_ms", "cost_usd"?, "domain"?}``. |
| `synthetic_agentic_tasks` | function `(n: int=200, *, seed: int=0, hard_share: float=0.3)` | Three agents (fast / balanced / strong) and ``n`` tasks of 3-5 steps whose steps are mostly. |
| `task_routing_frontier` | function `(registry: TargetRegistry, tasks: Sequence[AgentTask], *, retries: int=1, accuracy_tolerance: float=0.02, latency_ratio_target: float=0.9, objective: Objective \| None=None, router_factory: Callable[[TargetRegistry], Router] \| None=None, seed: int=0)` | Replay ``tasks`` step by step through :class:`ProgressRouter` (each agent's handler draws the. |

## `opensmartroute.eval.audit`

Source: [src/opensmartroute/eval/audit.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/eval/audit.py)

Routing Audit - shadow-mode replay of logged LLM traffic to quantify what the router would change.

| Name | Kind | Summary |
|---|---|---|
| `DEFAULT_COMPLETION_TOKENS` | constant | : Default assumed output length when a log row carries no ``completion_tokens``. |
| `AuditReport` | class | Aggregate of a shadow replay; ``to_markdown`` renders the Routing Audit deliverable. |
| `AuditRow` | class | One logged request: what happened, and what the router would have done. |
| `load_audit_log` | function `(path: str \| Path)` | Read a JSONL traffic log (``text`` or ``prompt``/``messages`` per line). |
| `routing_audit` | function `(router: Router, rows: list[dict[str, Any]], *, baseline: str \| None=None, monthly_requests: int \| None=None, plan: bool=False, keep_rows: bool=True)` | Replay ``rows`` through ``router`` in shadow mode and aggregate an :class:`AuditReport`. |
| `summarize_audit` | function `(rows: list[AuditRow], *, monthly_requests: int \| None=None, keep_rows: bool=True)` | Aggregate :class:`AuditRow` records into an :class:`AuditReport`. |

## `opensmartroute.eval.baselines`

Source: [src/opensmartroute/eval/baselines.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/eval/baselines.py)

Baselines every router must beat, plus oracle ceilings and the sampling noise floor.

| Name | Kind | Summary |
|---|---|---|
| `Policy` | constant | a baseline: (row, targets) -> chosen target id. |
| `PolicyResult` | class | Metrics of one baseline policy on a dataset (same fields as the router's EvalResult headline). |
| `baseline_suite` | function `(rows: list[EvalRow], targets: Sequence[RouteTarget], train_rows: list[EvalRow] \| None=None, quality_floor: float=0.7, seed: int=0)` | Run every baseline and both oracles; ``train_rows`` (default: ``rows``) fit the task table. |
| `best_prior_policy` | function `(row: EvalRow, targets: Sequence[RouteTarget])` | Always the target with the highest declared ``quality_prior``. |
| `cheapest_policy` | function `(row: EvalRow, targets: Sequence[RouteTarget])` | Always the lowest unit cost (ties broken by latency). |
| `evaluate_policy` | function `(name: str, policy: Policy, rows: list[EvalRow], targets: Sequence[RouteTarget])` | Run a baseline :data:`Policy` over the rows and aggregate accuracy, quality, cost and latency. |
| `fit_task_table` | function `(rows: list[EvalRow], targets: Sequence[RouteTarget])` | ``task_type -> target`` with the highest mean quality on the training rows. |
| `most_expensive_policy` | function `(row: EvalRow, targets: Sequence[RouteTarget])` | Always the highest unit cost - the "just use the frontier model" baseline. |
| `multi_sample_oracle` | function `(rows: list[EvalRow], targets: Sequence[RouteTarget])` | Mean quality of the oracle that picks by the *mean over all samples* per target. |
| `noise_floor` | function `(rows: list[EvalRow], targets: Sequence[RouteTarget], seed: int=0)` | How much accuracy is lost to sampling noise alone. |
| `oracle_policy` | function `(quality_floor: float \| None=None)` | Per-row best target. With ``quality_floor`` -> cheapest target reaching the floor. |
| `random_policy` | function `(seed: int=0)` | Uniformly random target (seeded). |
| `static_task_table_policy` | function `(table: dict[str, str])` | Look the row's ``task_type`` up in a table from :func:`fit_task_table` (``"*"`` = default). |

## `opensmartroute.eval.collect`

Source: [src/opensmartroute/eval/collect.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/eval/collect.py)

Collect routing datasets from the Hugging Face Hub, your own feedback log and synthetic seeds.

| Name | Kind | Summary |
|---|---|---|
| `ARENA_TIERS` | constant | everything else in those datasets is a 7B-14B chat model -> "small". |
| `DATASETS_SERVER` | re-export of [`opensmartroute.adapters.websearch.DATASETS_SERVER`](#opensmartrouteadapterswebsearch) | Hugging Face datasets-server REST root (/rows). |
| `DEFAULT_SOURCES` | constant | public ones. |
| `HISTORY_FILE` | constant | the self-improver's report log; lives in the cache dir but is not a dataset. |
| `KNOWN_SOURCES` | constant | pairwise human / judge preference battles - the winner is the label. |
| `PAIRWISE` | constant | preset name for battle datasets: prompt / conversation, model_a, model_b, winner. |
| `PARSERS` | constant | collector presets that are not wide / nested tables. |
| `REWARD_BENCH` | constant | preset name for RewardBench: prompt, chosen_model, rejected_model, subset. |
| `ROUTELLM_GOOD_ENOUGH` | constant | RouteLLM's threshold: a weak-model score >= 4 means the cheap model was good enough. |
| `ROUTELLM_GPT4` | constant | preset name for RouteLLM's gpt4_dataset: prompt + GPT-4-judged Mixtral score 1-5. |
| `ROUTELLM_STRONG` | constant | the strong model of RouteLLM's gpt4_dataset (its answers are the reference). |
| `ROUTELLM_WEAK` | constant | the weak model whose answer GPT-4 scores 1-5 against the reference. |
| `ULTRAFEEDBACK` | constant | preset name for UltraFeedback: instruction + N completions with a model and score. |
| `DatasetCollector` | class | Cache-backed corpus builder: collect sources, add feedback / synthetic rows, dedupe, split. |
| `DatasetSource` | class | One Hub dataset split to collect: repo id, config, split, the preset that parses it, an optional model map. |
| `Parser` | constant | record -> row (or None to skip). |
| `collect_dataset` | function `(source: DatasetSource, *, timeout_s: float \| None=None, settings: Settings \| None=None)` | Download ``source`` from the Hub and parse it into rows. |
| `fetch_hf_rows` | function `(dataset: str, *, config: str='default', split: str='train', offset: int=0, length: int=100, timeout_s: float \| None=None, settings: Settings \| None=None)` | One page (max 100) of records from the datasets-server ``/rows`` endpoint, flattened to ``{column: value}``. |
| `from_pairwise_row` | function `(rec: dict[str, Any], model_map: dict[str, str] \| None=None, source: DatasetSource \| None=None)` | A battle record -> row with ``scores`` 1 / 0 for winner / loser (0.5 each on a tie). Understands the. |
| `from_reward_bench_row` | function `(rec: dict[str, Any], source: DatasetSource \| None=None)` | A RewardBench record (``prompt``, ``chosen_model``, ``rejected_model``, ``subset``) -> a pairwise row. |
| `from_routellm_gpt4_row` | function `(rec: dict[str, Any], source: DatasetSource \| None=None)` | A ``routellm/gpt4_dataset`` record (``prompt`` + ``mixtral_score`` 1-5) -> the strong model scores 1.0 and. |
| `from_ultrafeedback_row` | function `(rec: dict[str, Any], source: DatasetSource \| None=None)` | An UltraFeedback record (``instruction`` + ``completions[{model, overall_score}]``) -> a scored row with. |
| `iter_hf_rows` | function `(source: DatasetSource, *, offset: int=0, timeout_s: float \| None=None, settings: Settings \| None=None)` | Page through ``source`` from raw record ``offset`` until ``source.limit`` records or the split is exhausted. |
| `model_quality` | function `(rows: Iterable[EvalRow], *, min_n: int=20, epochs: int=5, seed: int=0)` | Data-derived quality prior per model in [0, 1]: pairwise rows (two scored models) are fitted with. |
| `rows_from_feedback` | function `(outcomes: Iterable[Outcome] \| FeedbackStore, texts: dict[str, str] \| None=None, *, min_quality: float=0.5)` | Rows from :class:`Outcome` records. ``texts`` maps ``request_id`` to the prompt (outcomes carry no text);. |
| `rows_from_records` | function `(records: Iterable[dict[str, Any]], source: DatasetSource)` | Parse raw records with the source's preset (``text_key`` is tried first when set). |
| `synthetic_rows` | function `(targets: Sequence[RouteTarget], *, per_template: int=2, seed: int=0)` | Ontology seed prompts labelled with the best capability-fit target (cold-start supervision). |
| `tier_model_map` | function `(tiers: Mapping[str, str], models: Iterable[str]=())` | ``model_map`` for a :class:`DatasetSource`: every known battle model (plus ``models``) -> the target id. |
| `tier_of` | function `(model: str)` | Tier of a battle-dataset model name: the :data:`ARENA_TIERS` entry, else the name rules, else the parameter. |

## `opensmartroute.eval.criteria`

Source: [src/opensmartroute/eval/criteria.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/eval/criteria.py)

Offline realisations of the ROADMAP exit criteria.

| Name | Kind | Summary |
|---|---|---|
| `CriterionResult` | class | One measured exit criterion. |
| `bootstrap_ci` | function `(values: Sequence[float], *, n_boot: int=1000, level: float=0.95, seed: int=0)` | Percentile bootstrap confidence interval for the mean of ``values``. |
| `cold_start_ratio` | function `(n_outcomes: int=200, *, domain: str='legal', n_eval: int=200, seed: int=0, target_ratio: float=0.9, generalist_quality: float=0.6, explore_rate: float=0.1, max_requests: int=20000)` | Accuracy on ``domain`` prompts of a target declared with only ``id`` / ``kind`` / ``cost``. |
| `conformal_coverage` | function `(alpha: float=0.1, *, n_cal: int=1000, n_test: int=1000, label_noise: float=0.5, seed: int=0, tolerance: float=0.02)` | Fit :class:`ConformalCalibrator` on the real router's propensities over labelled prompts and. |
| `domain_expert_registry` | function `(domains: Sequence[str]=_DOMAINS, *, strip: str \| None=None, generalist: bool=False)` | One hand-configured expert per domain. ``strip=<domain>`` replaces that expert with a. |
| `effort_token_savings` | function `(registry: TargetRegistry, rows: Sequence[EvalRow], *, quality_tolerance: float=0.02, token_ratio_target: float=0.7, objective: Objective \| None=None, router_factory: Callable[[TargetRegistry], Router] \| None=None)` | Route rows that carry per-target ``scores`` *and* ``tokens`` and compare the tokens spent. |
| `knapsack_never_exceeds_cap` | function `(steps: int=1000000, *, window: int=1000, drift_every: int=100000, seed: int=0)` | Drive :class:`MultiKnapsackBandit` (``on_capped="abstain"``) for ``steps`` pulls with. |
| `match_at_1` | function `(router: Router \| EnterpriseRouter, prompts: Sequence[tuple[str, str]])` | Fraction of ``(text, expected_target_id)`` pairs the router gets right at rank 1. |
| `match_at_1_at_scale` | function `(small: int=50, large: int=5000, *, n_prompts: int=200, seed: int=0, max_drop: float=0.05, narrow_above: int=32, narrow_to: int=24)` | Match@1 on a ``small`` catalogue versus a ``large`` one with retrieve-then-rank narrowing. |
| `multi_round_vs_best_single` | function `(registry: TargetRegistry, rows: Sequence[EvalRow], *, threshold: float=0.8, max_rounds: int=3, failures_before_switch: int=1, objective: Objective \| None=None, cost_ratio_target: float=0.6, router_factory: Callable[[TargetRegistry], Router] \| None=None)` | Run :class:`MultiRoundExecutor` over rows that carry per-target ``scores`` (RouterBench. |
| `ope_within_live_ci` | function `(n_log: int=2000, *, seed: int=0, logging_objective: Objective \| None=None, target_objective: Objective \| None=None, logging_temperature: float=0.3)` | Log decisions from a cost-seeking router (actions *sampled* from its propensities), estimate. |
| `run_all` | function `(*, quick: bool=True, seed: int=0)` | Every offline criterion. ``quick`` shrinks the expensive simulations (10^5 steps, 1 000 tools,. |
| `synthetic_effort_rows` | function `(n: int=400, *, seed: int=0)` | One reasoning model exposed as two effort siblings (``reasoner@fast`` with ``effort="low"``. |
| `synthetic_scored_rows` | function `(n: int=300, *, seed: int=0)` | A three-tier catalogue (small / medium / large) and RouterBench-style rows with a quality. |
| `synthetic_tool_catalogue` | function `(n: int, *, seed: int=0)` | ``n`` distinct tools (verb x object x qualifier, up to 5 000 unique combinations) and one. |

## `opensmartroute.eval.datasets`

Source: [src/opensmartroute/eval/datasets.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/eval/datasets.py)

Adapters from public routing benchmarks to :class:`EvalRow`.

| Name | Kind | Summary |
|---|---|---|
| `PRESETS` | constant | Named layouts accepted by ``load_benchmark`` / ``osr eval --preset``. |
| `NestedPreset` | class | Layout of a "nested" benchmark: a per-model mapping under ``container_keys`` holding score / cost / samples. |
| `WidePreset` | class | Column layout of a "wide" benchmark: one row per prompt, ``<model><suffix>`` columns for scores / costs. |
| `from_nested_row` | function `(row: dict[str, Any], preset: NestedPreset, model_map: dict[str, str] \| None=None)` | Convert one nested-format record to an :class:`EvalRow`, keeping repeated samples when present. |
| `from_wide_row` | function `(row: dict[str, Any], preset: WidePreset, model_map: dict[str, str] \| None=None)` | Convert one wide-format record to an :class:`EvalRow` (None when it has no prompt or scores). |
| `load_benchmark` | function `(path: str \| Path, preset: str \| WidePreset \| NestedPreset='routerbench', model_map: dict[str, str] \| None=None, limit: int \| None=None)` | Load a benchmark file (.jsonl / .json / .csv) into :class:`EvalRow` objects. |
| `models_in` | function `(rows: Iterable[EvalRow])` | Distinct target ids that carry scores in the rows, in first-seen order. |

## `opensmartroute.eval.frontier`

Source: [src/opensmartroute/eval/frontier.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/eval/frontier.py)

Three-objective frontier and ablations.

| Name | Kind | Summary |
|---|---|---|
| `ablation_report` | function `(router: Router, rows: list[EvalRow], objective: Objective \| None=None)` | Leave-one-out over strategies (and extractors, when explicitly set on the router). |
| `frontier3` | function `(router: Router, rows: list[EvalRow], cost_weights: list[float] \| None=None, latency_weights: list[float] \| None=None)` | Sweep cost / latency objective weights and return the Pareto-optimal (quality, cost, latency) points. |
| `hypervolume` | function `(points: list[dict[str, float]], ref: tuple[float, float, float] \| None=None)` | Dominated hypervolume of the Pareto points in normalised (quality, cost, latency) space. |

## `opensmartroute.eval.headroom`

Source: [src/opensmartroute/eval/headroom.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/eval/headroom.py)

Routing headroom: when does routing pay, and how much catalogue does it need?.

| Name | Kind | Summary |
|---|---|---|
| `DiversityReport` | class | How differently the targets score rows: pairwise disagreement, winner entropy and winner share. |
| `HeadroomReport` | class | Oracle vs best-single quality on a dataset, with the label-noise floor that says whether the gap is real. |
| `learnability_by_difficulty` | function `(rows: Sequence[EvalRow], targets: Sequence[RouteTarget], buckets: int=4)` | Routing gain (oracle - best single) per difficulty bucket. Difficulty of a row is. |
| `min_catalogue` | function `(rows: Sequence[EvalRow], targets: Sequence[RouteTarget], fraction: float=0.95)` | Greedy forward selection: the smallest ordered subset whose oracle reaches ``fraction`` of the. |
| `routing_headroom` | function `(rows: Sequence[EvalRow], targets: Sequence[RouteTarget], seed: int=0)` | Oracle minus best-single quality with the label-noise floor for context. |
| `scaling_curve` | function `(rows: Sequence[EvalRow], targets: Sequence[RouteTarget], sizes: Sequence[int] \| None=None, *, trials: int=20, seed: int=0)` | Mean oracle / best-single quality and headroom on random subsets of each size. |
| `target_diversity` | function `(rows: Sequence[EvalRow], targets: Sequence[RouteTarget])` | How different the targets are on this data: pairwise score disagreement and winner spread. |

## `opensmartroute.eval.ope`

Source: [src/opensmartroute/eval/ope.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/eval/ope.py)

Off-policy evaluation from logged routing decisions.

| Name | Kind | Summary |
|---|---|---|
| `LoggedDecision` | class | One logged routing event for off-policy evaluation: text, action taken, its propensity and observed reward. |
| `OPEResult` | class | IPS, self-normalised IPS and doubly-robust value estimates with effective sample size and clip count. |
| `ips_confidence_interval` | function `(logs: list[LoggedDecision], weights: list[float], z: float=1.96)` | Normal-approximation CI for the IPS estimate given per-sample weights. |
| `mean_reward_model` | function `(logs: list[LoggedDecision])` | Simplest DR reward model: mean logged reward per action, global mean for unseen actions. |
| `off_policy_evaluate` | function `(router: Router, logs: list[LoggedDecision], objective: Objective \| None=None, max_weight: float=20.0, reward_model: Callable[[LoggedDecision, str], float] \| None=None, deterministic: bool=False)` | Estimate the value of ``router`` on logged traffic. |
| `target_propensities` | function `(router: Router, lg: LoggedDecision, objective: Objective \| None=None)` | The evaluated router's ``P(target \| x)`` for a logged context (empty when it has no route). |

## `opensmartroute.eval.robustness`

Source: [src/opensmartroute/eval/robustness.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/eval/robustness.py)

Robustness and fairness checks for a router.

| Name | Kind | Summary |
|---|---|---|
| `coreset` | function `(rows: list[EvalRow], k: int, embedder: Callable[[list[str]], list[list[float]]] \| None=None, seed: int=0)` | Greedy k-center (farthest-first) subset of ``rows`` for a diverse evaluation set. |
| `diversity` | function `(rows: list[EvalRow], embedder: Callable[[list[str]], list[list[float]]] \| None=None, sample: int=300)` | Mean pairwise cosine distance of the row texts (sampled), 0 = all identical. |
| `paraphrase_robustness` | function `(router: Router, rows: list[EvalRow], n: int=4, objective: Objective \| None=None, seed: int=0)` | Share of rule-based paraphrases that route to the same target as the original (``robustness``). |
| `paraphrases` | function `(text: str, n: int=4, seed: int=0)` | ``n`` deterministic surface-level paraphrases of ``text`` (never returns the original). |
| `profile_swap_fairness` | function `(router: Router, rows: list[EvalRow], profiles: Sequence[dict[str, Any]] \| None=None, objective: Objective \| None=None)` | Decision agreement across user profiles. ``dependence`` = fraction of rows whose. |
| `repeat_flip_rate` | function `(router: Router, rows: list[EvalRow], k: int=5, objective: Objective \| None=None)` | Route each row ``k`` times; ``flip_rate`` = share of rows whose decision is not identical every time. |

## `opensmartroute.execution`

Source: [src/opensmartroute/execution.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/execution.py)

Plan-aware execution: turn a :class:`RouteDecision` into a real answer.

| Name | Kind | Summary |
|---|---|---|
| `PRELUDE_ROLES` | constant | Roles executed *before* the primary target, in this order. |
| `ExecutionResult` | class | What happened when a decision was executed. |
| `ExecutionStep` | class | One executed plan slot: role, target, latency and whether it succeeded. |
| `aexecute` | async function `(decision: RouteDecision, request: RouteRequest, learn: LearnFn \| None=None, *, min_slot_confidence: float=0.0, task_id: str \| None=None, **kw: Any)` | Async twin of :func:`execute`; awaits coroutine handlers. |
| `execute` | function `(decision: RouteDecision, request: RouteRequest, learn: LearnFn \| None=None, *, min_slot_confidence: float=0.0, task_id: str \| None=None, **kw: Any)` | Run the plan (persona -> skill -> primary) and record outcomes via ``learn``. |

## `opensmartroute.feedback`

Source: [src/opensmartroute/feedback/__init__.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/feedback/__init__.py)

Feedback store: append-only outcome log that closes the learning loop.

| Name | Kind | Summary |
|---|---|---|
| `FeedbackStore` | class | Append-only :class:`Outcome` log (in memory or JSONL file) with per-target statistics. |

## `opensmartroute.learning`

Source: [src/opensmartroute/learning/__init__.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/learning/__init__.py)

Auto-learning strategies built on :mod:`opensmartroute.math`.

| Name | Kind | Summary |
|---|---|---|
| `DEFAULT_DOMAINS` | constant | Domain one-hot block of the LinUCB context vector (ontology domains + the "general" fallback). |
| `AttentionEncoder` | re-export of [`opensmartroute.learning.attention.AttentionEncoder`](#opensmartroutelearningattention) | One self-attention block with attention pooling; encodes text into a ``dim``-vector (unnormalised). |
| `AutoLearner` | class | Single entry point for closing the loop. |
| `Autopilot` | re-export of [`opensmartroute.learning.autopilot.Autopilot`](#opensmartroutelearningautopilot) | Runs a :class:`SelfImprover` on a schedule and on drift, inside a live process. |
| `ContrastiveRouter` | re-export of [`opensmartroute.learning.contrastive.ContrastiveRouter`](#opensmartroutelearningcontrastive) | Dual-encoder router trained with a contrastive or a distillation objective. |
| `ContrastiveStrategy` | re-export of [`opensmartroute.learning.contrastive.ContrastiveStrategy`](#opensmartroutelearningcontrastive) | Scores candidates with a trained :class:`ContrastiveRouter`. |
| `DriftMonitor` | re-export of [`opensmartroute.learning.autopilot.DriftMonitor`](#opensmartroutelearningautopilot) | Page-Hinkley over outcomes: alarms when the served success rate (or quality) drops for real. |
| `EmbeddingFeaturizer` | re-export of [`opensmartroute.learning.embed.EmbeddingFeaturizer`](#opensmartroutelearningembed) | Hashed features plus a frozen dense embedding, indexed above the hashed space (``dim + i``). |
| `ExampleMiner` | re-export of [`opensmartroute.learning.coldstart.ExampleMiner`](#opensmartroutelearningcoldstart) | Promote prompts a target handled well into that target's ``examples``. |
| `HandoffPolicy` | re-export of [`opensmartroute.learning.handoff.HandoffPolicy`](#opensmartroutelearninghandoff) | Permanent handoff of a task to ``fallback_target`` once the failure risk is too high. |
| `HistoryTargetModel` | re-export of [`opensmartroute.learning.multiturn.HistoryTargetModel`](#opensmartroutelearningmultiturn) | Logistic model over ``h * e_t`` with a shared weight vector and per-target biases. |
| `HistoryTargetStrategy` | re-export of [`opensmartroute.learning.multiturn.HistoryTargetStrategy`](#opensmartroutelearningmultiturn) | Scores each target by the learned success probability given the conversation so far. |
| `IRTStrategy` | class | 2PL Item Response Theory: target ability vs (domain, complexity-bucket) item difficulty, learned online. |
| `ImprovementReport` | re-export of [`opensmartroute.learning.self_improve.ImprovementReport`](#opensmartroutelearningself_improve) | What one cycle did: evidence gathered, catalogue changes, champion vs challenger, and the verdict. |
| `LinUCBStrategy` | class | Contextual bandit (LinUCB) on the signal vector; keeps the last context per request for the update. |
| `MarkovStrategy` | class | Prefers targets that do well on the *predicted next* conversation state too. |
| `MixtureCureModel` | re-export of [`opensmartroute.learning.handoff.MixtureCureModel`](#opensmartroutelearninghandoff) | Weibull mixture-cure model on cumulative risk with censoring. |
| `PolicyGradientStrategy` | re-export of [`opensmartroute.learning.policy_gradient.PolicyGradientStrategy`](#opensmartroutelearningpolicy_gradient) | REINFORCE-trained softmax routing policy over hashed request features. |
| `PreferenceStrategy` | class | Bradley-Terry strengths per domain, fed by ``Outcome.preferred_over`` pairwise comparisons. |
| `RegretReport` | re-export of [`opensmartroute.learning.policy_gradient.RegretReport`](#opensmartroutelearningpolicy_gradient) | Decision regret versus prediction error on a scored dataset. |
| `RequestMemory` | re-export of [`opensmartroute.learning.coldstart.RequestMemory`](#opensmartroutelearningcoldstart) | Bounded LRU of request texts keyed by request id. |
| `RouterSLM` | re-export of [`opensmartroute.learning.slm.RouterSLM`](#opensmartroutelearningslm) | Small routing model: dual encoder + target catalogue snapshot + calibration, in one JSON file. |
| `SLMReport` | re-export of [`opensmartroute.learning.slm.SLMReport`](#opensmartroutelearningslm) | How an SLM did on a set of rows: accuracy, realised quality, cost, calibration and the training loss. |
| `SLMStrategy` | re-export of [`opensmartroute.learning.slm.SLMStrategy`](#opensmartroutelearningslm) | Ensemble member backed by a :class:`RouterSLM`; scores are its probabilities, and it keeps learning online. |
| `SelfImprover` | re-export of [`opensmartroute.learning.self_improve.SelfImprover`](#opensmartroutelearningself_improve) | Closed loop that keeps a :class:`RouterSLM` current with the model market and its own traffic. |
| `SimilarityFallback` | re-export of [`opensmartroute.learning.coldstart.SimilarityFallback`](#opensmartroutelearningcoldstart) | For targets with no observations yet, rank by request<->description similarity. |
| `SkillAffinity` | re-export of [`opensmartroute.learning.personal.SkillAffinity`](#opensmartroutelearningpersonal) | Profile-conditioned skill relevance: Beta posterior per (profile bucket, skill). |
| `TaskCredit` | re-export of [`opensmartroute.learning.credit.TaskCredit`](#opensmartroutelearningcredit) | Buffers per-step outcomes of a task and redistributes the final reward (uniform / discounted / last / blend). |
| `TaskPins` | re-export of [`opensmartroute.learning.credit.TaskPins`](#opensmartroutelearningcredit) | Admission-time pinning: ``task_id -> target_id`` while the target keeps succeeding. |
| `UserAdaptiveStrategy` | re-export of [`opensmartroute.learning.personal.UserAdaptiveStrategy`](#opensmartroutelearningpersonal) | Per-user Beta posteriors per target, shrunk toward similar users and the global posterior. |
| `acceptable_set` | re-export of [`opensmartroute.learning.contrastive.acceptable_set`](#opensmartroutelearningcontrastive) | Targets that are 'fine' for a row: ``expected`` + ``acceptable`` when labelled, otherwise every. |
| `decision_regret` | re-export of [`opensmartroute.learning.policy_gradient.decision_regret`](#opensmartroutelearningpolicy_gradient) | Mean decision regret of ``choose`` against the per-row oracle, next to the prediction error of. |
| `decision_reward` | re-export of [`opensmartroute.learning.policy_gradient.decision_reward`](#opensmartroutelearningpolicy_gradient) | Scalar decision reward of an outcome under ``objective`` (quality minus normalised cost/latency). |
| `distill_router` | re-export of [`opensmartroute.learning.slm.distill_router`](#opensmartroutelearningslm) | Compress the full router into an SLM: route every text, take the ensemble's ranked utilities as soft. |
| `history_vector` | re-export of [`opensmartroute.learning.multiturn.history_vector`](#opensmartroutelearningmultiturn) | Recency-weighted joint embedding of the last ``turns`` messages and the current text. |
| `load_embedder` | re-export of [`opensmartroute.learning.embed.load_embedder`](#opensmartroutelearningembed) | Build the embedder a model file names: ``sentence-transformers/...`` (or any Hugging Face id) via. |
| `merge_learners` | function `(local: Iterable[Strategy], remote: Iterable[Strategy])` | Federated merge: fold the evidence of ``remote`` strategies into the same-named ``local``. |
| `nearest_targets` | re-export of [`opensmartroute.learning.coldstart.nearest_targets`](#opensmartroutelearningcoldstart) | The ``k`` most similar existing targets (cosine + same-kind and domain-overlap bonuses) for warm starts. |
| `outcome_counts` | re-export of [`opensmartroute.learning.coldstart.outcome_counts`](#opensmartroutelearningcoldstart) | Number of recorded outcomes per target id. |
| `profile_bucket` | re-export of [`opensmartroute.learning.personal.profile_bucket`](#opensmartroutelearningpersonal) | Coarse profile bucket used to pool users with the same declared attributes. |
| `profile_vector` | re-export of [`opensmartroute.learning.personal.profile_vector`](#opensmartroutelearningpersonal) | Hashed one-hot encoding of ``key=value`` pairs (lists expand to one pair per element). |
| `signal_vector` | function `(signals: Signals, domains: list[str] \| None=None)` | Fixed-length numeric context for contextual bandits (dim = 8 + len(domains)). |
| `soft_labels` | re-export of [`opensmartroute.learning.contrastive.soft_labels`](#opensmartroutelearningcontrastive) | Zooter soft labels: ``softmax(score / temperature)`` over the scored targets. |
| `target_document` | re-export of [`opensmartroute.learning.coldstart.target_document`](#opensmartroutelearningcoldstart) | Text used to embed a target: name, description, domains, actions, tags and up to 12 examples. |
| `target_embedding` | re-export of [`opensmartroute.learning.coldstart.target_embedding`](#opensmartroutelearningcoldstart) | Unit-norm centroid of the target document and its examples (hashing embedder by default). |
| `warm_start` | re-export of [`opensmartroute.learning.coldstart.warm_start`](#opensmartroutelearningcoldstart) | Seed every learner with shrunk knowledge from the new target's nearest neighbours. |
| `warm_start_from_matrix` | re-export of [`opensmartroute.learning.coldstart.warm_start_from_matrix`](#opensmartroutelearningcoldstart) | Offline **full-information reward-matrix** warm start (OrcaRouter 2605.30736). |

## `opensmartroute.learning.attention`

Source: [src/opensmartroute/learning/attention.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/learning/attention.py)

A pure-Python transformer block for the routing SLM's query encoder.

| Name | Kind | Summary |
|---|---|---|
| `AttentionContext` | class | Everything :meth:`AttentionEncoder.backward` needs from one forward pass. |
| `AttentionEncoder` | class | One self-attention block with attention pooling; encodes text into a ``dim``-vector (unnormalised). |

## `opensmartroute.learning.autopilot`

Source: [src/opensmartroute/learning/autopilot.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/learning/autopilot.py)

Self-operation: the routing SLM runs its own improvement loop inside the live process.

| Name | Kind | Summary |
|---|---|---|
| `Autopilot` | class | Runs a :class:`SelfImprover` on a schedule and on drift, inside a live process. |
| `DriftMonitor` | class | Page-Hinkley over outcomes: alarms when the served success rate (or quality) drops for real. |

## `opensmartroute.learning.coldstart`

Source: [src/opensmartroute/learning/coldstart.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/learning/coldstart.py)

Cold start for new targets and self-improving target descriptions.

| Name | Kind | Summary |
|---|---|---|
| `ExampleMiner` | class | Promote prompts a target handled well into that target's ``examples``. |
| `RequestMemory` | class | Bounded LRU of request texts keyed by request id. |
| `SimilarityFallback` | class | For targets with no observations yet, rank by request<->description similarity. |
| `nearest_targets` | function `(new: RouteTarget, pool: Iterable[RouteTarget], k: int=3, embedder: Embedder \| None=None)` | The ``k`` most similar existing targets (cosine + same-kind and domain-overlap bonuses) for warm starts. |
| `outcome_counts` | function `(feedback: FeedbackStore)` | Number of recorded outcomes per target id. |
| `target_document` | function `(t: RouteTarget)` | Text used to embed a target: name, description, domains, actions, tags and up to 12 examples. |
| `target_embedding` | function `(t: RouteTarget, embedder: Embedder \| None=None)` | Unit-norm centroid of the target document and its examples (hashing embedder by default). |
| `warm_start` | function `(new: RouteTarget, registry: TargetRegistry, strategies: list[Strategy], k: int=3, shrink: float=0.5, embedder: Embedder \| None=None)` | Seed every learner with shrunk knowledge from the new target's nearest neighbours. |

## `opensmartroute.learning.contrastive`

Source: [src/opensmartroute/learning/contrastive.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/learning/contrastive.py)

Contrastive and reward-distilled router training (RouterDC, NeurIPS 2024; Zooter 2311.08692).

| Name | Kind | Summary |
|---|---|---|
| `ContrastiveRouter` | class | Dual-encoder router trained with a contrastive or a distillation objective. |
| `ContrastiveStrategy` | class | Scores candidates with a trained :class:`ContrastiveRouter`. |
| `acceptable_set` | function `(row: EvalRow, slack: float=0.05)` | Targets that are 'fine' for a row: ``expected`` + ``acceptable`` when labelled, otherwise every. |
| `soft_labels` | function `(scores: dict[str, float], temperature: float=0.1)` | Zooter soft labels: ``softmax(score / temperature)`` over the scored targets. |

## `opensmartroute.learning.credit`

Source: [src/opensmartroute/learning/credit.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/learning/credit.py)

Delayed, task-level credit assignment for agentic trajectories.

| Name | Kind | Summary |
|---|---|---|
| `TaskCredit` | class | Buffers per-step outcomes of a task and redistributes the final reward (uniform / discounted / last / blend). |
| `TaskPins` | class | Admission-time pinning: ``task_id -> target_id`` while the target keeps succeeding. |

## `opensmartroute.learning.embed`

Source: [src/opensmartroute/learning/embed.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/learning/embed.py)

Pretrained transformer embeddings as frozen features for the routing SLM.

| Name | Kind | Summary |
|---|---|---|
| `EmbeddingFeaturizer` | class | Hashed features plus a frozen dense embedding, indexed above the hashed space (``dim + i``). |
| `load_embedder` | function `(name: str)` | Build the embedder a model file names: ``sentence-transformers/...`` (or any Hugging Face id) via. |

## `opensmartroute.learning.handoff`

Source: [src/opensmartroute/learning/handoff.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/learning/handoff.py)

Permanent-handoff policy from censored teacher signals (TACIT-Switch 2608.27911).

| Name | Kind | Summary |
|---|---|---|
| `HandoffPolicy` | class | Permanent handoff of a task to ``fallback_target`` once the failure risk is too high. |
| `MixtureCureModel` | class | Weibull mixture-cure model on cumulative risk with censoring. |
| `Trajectory` | class | Per-task state tracked by :class:`HandoffPolicy`: cumulative risk, steps, failure and hand-off flags. |

## `opensmartroute.learning.multiturn`

Source: [src/opensmartroute/learning/multiturn.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/learning/multiturn.py)

Multi-turn routing with history-target joint embeddings (MTRouter 2604.23530).

| Name | Kind | Summary |
|---|---|---|
| `HistoryTargetModel` | class | Logistic model over ``h * e_t`` with a shared weight vector and per-target biases. |
| `HistoryTargetStrategy` | class | Scores each target by the learned success probability given the conversation so far. |
| `history_vector` | function `(request: RouteRequest, embedder: Embedder, turns: int=6, decay: float=0.7)` | Recency-weighted joint embedding of the last ``turns`` messages and the current text. |

## `opensmartroute.learning.personal`

Source: [src/opensmartroute/learning/personal.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/learning/personal.py)

Few-shot personalisation (GMTRouter 2511.08590; SkillFeed 2608.28241).

| Name | Kind | Summary |
|---|---|---|
| `SkillAffinity` | class | Profile-conditioned skill relevance: Beta posterior per (profile bucket, skill). |
| `UserAdaptiveStrategy` | class | Per-user Beta posteriors per target, shrunk toward similar users and the global posterior. |
| `profile_bucket` | function `(profile: dict[str, Any], keys: tuple[str, ...]=('tier', 'expertise', 'role', 'language'))` | Coarse profile bucket used to pool users with the same declared attributes. |
| `profile_vector` | function `(profile: dict[str, Any], dim: int=_PROFILE_DIM)` | Hashed one-hot encoding of ``key=value`` pairs (lists expand to one pair per element). |

## `opensmartroute.learning.policy_gradient`

Source: [src/opensmartroute/learning/policy_gradient.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/learning/policy_gradient.py)

End-to-end policy-gradient routing (Router-R1 2506.09033; RLCascadeRouter 2608.15817).

| Name | Kind | Summary |
|---|---|---|
| `PolicyGradientStrategy` | class | REINFORCE-trained softmax routing policy over hashed request features. |
| `RegretReport` | class | Decision regret versus prediction error on a scored dataset. |
| `decision_regret` | function `(rows: Sequence[EvalRow], targets: Sequence[RouteTarget], choose: Callable[[EvalRow, Sequence[RouteTarget]], str], predict: Callable[[EvalRow, RouteTarget], float] \| None=None)` | Mean decision regret of ``choose`` against the per-row oracle, next to the prediction error of. |
| `decision_reward` | function `(outcome: Outcome, target: RouteTarget \| None, objective: Objective, *, cost_scale: float=0.01, latency_scale: float=2000.0)` | Scalar decision reward of an outcome under ``objective`` (quality minus normalised cost/latency). |

## `opensmartroute.learning.self_improve`

Source: [src/opensmartroute/learning/self_improve.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/learning/self_improve.py)

Self-improvement loop: refresh the catalogue, gather evidence, train a challenger, promote it only if better.

| Name | Kind | Summary |
|---|---|---|
| `ImprovementReport` | class | What one cycle did: evidence gathered, catalogue changes, champion vs challenger, and the verdict. |
| `SelfImprover` | class | Closed loop that keeps a :class:`RouterSLM` current with the model market and its own traffic. |

## `opensmartroute.learning.slm`

Source: [src/opensmartroute/learning/slm.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/learning/slm.py)

The OpenSmartRoute routing SLM: a small, self-contained model that picks the target for a prompt.

| Name | Kind | Summary |
|---|---|---|
| `SLM_FORMAT` | constant | on-disk format version of ``RouterSLM.state()``. |
| `RouterSLM` | class | Small routing model: dual encoder + target catalogue snapshot + calibration, in one JSON file. |
| `SLMReport` | class | How an SLM did on a set of rows: accuracy, realised quality, cost, calibration and the training loss. |
| `SLMStrategy` | class | Ensemble member backed by a :class:`RouterSLM`; scores are its probabilities, and it keeps learning online. |
| `distill_router` | function `(router: Router, texts: Iterable[str], *, targets: Sequence[RouteTarget] \| None=None, settings: Settings \| None=None, seed: int=0)` | Compress the full router into an SLM: route every text, take the ensemble's ranked utilities as soft. |

## `opensmartroute.math`

Source: [src/opensmartroute/math/__init__.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/math/__init__.py)

Mathematical toolkit behind OpenSmartRoute's decisions.

| Name | Kind | Summary |
|---|---|---|
| `EWMA` | re-export of [`opensmartroute.math.estimators.EWMA`](#opensmartroutemathestimators) | Exponentially weighted moving mean and variance with a z-score helper. |
| `Bandit` | re-export of [`opensmartroute.math.bandits.Bandit`](#opensmartroutemathbandits) | Common interface. ``context`` is optional; non-contextual learners ignore it. |
| `BradleyTerry` | re-export of [`opensmartroute.math.preference.BradleyTerry`](#opensmartroutemathpreference) | Online Bradley-Terry: per-context target strengths from pairwise wins, with L2 and forgetting. |
| `ConformalCalibrator` | re-export of [`opensmartroute.math.calibration.ConformalCalibrator`](#opensmartroutemathcalibration) | Split conformal prediction over routing candidates. |
| `CostAwareBandit` | re-export of [`opensmartroute.math.bandits.CostAwareBandit`](#opensmartroutemathbandits) | Lagrangian budget wrapper (C2MAB-V flavour). |
| `DelayedFeedback` | re-export of [`opensmartroute.math.bandits.DelayedFeedback`](#opensmartroutemathbandits) | Delayed-feedback wrapper (Joulani, György & Szepesvári, 2013). |
| `DirichletProbe` | re-export of [`opensmartroute.math.dirichlet.DirichletProbe`](#opensmartroutemathdirichlet) | Linear Dirichlet head over a fixed target list. |
| `Elo` | re-export of [`opensmartroute.math.preference.Elo`](#opensmartroutemathpreference) | Elo rating — a fixed-step Bradley–Terry with K-factor; handy for fast warm-up. |
| `EnergyModel` | re-export of [`opensmartroute.math.energy.EnergyModel`](#opensmartroutemathenergy) | Per-target linear energy model ``wh = e0 + e_in * prompt + e_out * output`` fit by ridge. |
| `EpsilonGreedy` | re-export of [`opensmartroute.math.bandits.EpsilonGreedy`](#opensmartroutemathbandits) | Explore uniformly with probability ``epsilon``, otherwise exploit the empirical means. |
| `HardwareProfile` | re-export of [`opensmartroute.math.energy.HardwareProfile`](#opensmartroutemathenergy) | Static device characterisation used before measurements exist. |
| `IRTModel` | re-export of [`opensmartroute.math.irt.IRTModel`](#opensmartroutemathirt) | Online 2PL IRT: target ability, item difficulty / discrimination via SGD, with forgetting and merge support. |
| `IsotonicCalibrator` | re-export of [`opensmartroute.math.calibration.IsotonicCalibrator`](#opensmartroutemathcalibration) | Monotone non-decreasing map score -> P(correct), fitted with PAV. |
| `LinUCB` | re-export of [`opensmartroute.math.bandits.LinUCB`](#opensmartroutemathbandits) | Disjoint LinUCB (Li et al., WWW 2010) with Sherman–Morrison updates and optional. |
| `MarkovChain` | re-export of [`opensmartroute.math.markov.MarkovChain`](#opensmartroutemathmarkov) | Dirichlet-smoothed transition counts. ``decay`` < 1 multiplies a row's counts by ``decay``. |
| `MultiKnapsackBandit` | re-export of [`opensmartroute.math.bandits.MultiKnapsackBandit`](#opensmartroutemathbandits) | Bandits with several knapsack constraints (Badanidiyuru et al., 2013) with the. |
| `PageHinkley` | re-export of [`opensmartroute.math.estimators.PageHinkley`](#opensmartroutemathestimators) | Detects a *decrease* in the monitored mean (e.g. quality dropping). |
| `RoutingMDP` | re-export of [`opensmartroute.math.markov.RoutingMDP`](#opensmartroutemathmarkov) | Finite-horizon / discounted MDP over conversation states and route actions. |
| `TemperatureScaler` | re-export of [`opensmartroute.math.calibration.TemperatureScaler`](#opensmartroutemathcalibration) | Fits :math:`\tau` for ``confidence = softmax(u/\tau)[argmax]`` by 1-D golden-section. |
| `ThompsonBeta` | re-export of [`opensmartroute.math.bandits.ThompsonBeta`](#opensmartroutemathbandits) | Beta–Bernoulli Thompson sampling. |
| `UCB1` | re-export of [`opensmartroute.math.bandits.UCB1`](#opensmartroutemathbandits) | UCB1: :math:`\hat\mu_a + c\sqrt{\frac{2\ln t}{n_a}}`; untried arms get +inf. |
| `Welford` | re-export of [`opensmartroute.math.estimators.Welford`](#opensmartroutemathestimators) | Numerically stable running mean / variance (Welford's algorithm). |
| `WindowDrift` | re-export of [`opensmartroute.math.estimators.WindowDrift`](#opensmartroutemathestimators) | ADWIN-lite: compare the first and second half of a sliding window with a. |
| `bayesian_average` | re-export of [`opensmartroute.math.estimators.bayesian_average`](#opensmartroutemathestimators) | Shrink a small-sample mean toward a prior: (n·m + k·μ₀)/(n + k). |
| `brier_score` | re-export of [`opensmartroute.math.calibration.brier_score`](#opensmartroutemathcalibration) | Mean squared error between confidence and the 0/1 correctness label. |
| `conformal_quantile` | re-export of [`opensmartroute.math.calibration.conformal_quantile`](#opensmartroutemathcalibration) | Finite-sample corrected :math:`\lceil (n+1)(1-\alpha)\rceil / n` empirical quantile. |
| `digamma` | re-export of [`opensmartroute.math.dirichlet.digamma`](#opensmartroutemathdirichlet) | Digamma :math:`\psi(x)` for x > 0 via recurrence to x >= 6 and the asymptotic series. |
| `dominates` | re-export of [`opensmartroute.math.decision.dominates`](#opensmartroutemathdecision) | a dominates b if it is ≥ quality, ≤ cost, ≤ latency and strictly better in one. |
| `entropy` | re-export of [`opensmartroute.math.estimators.entropy`](#opensmartroutemathestimators) | Shannon entropy (nats) of a probability vector. |
| `erlang_c` | re-export of [`opensmartroute.math.decision.erlang_c`](#opensmartroutemathdecision) | P(an arriving request must wait) for M/M/c. Returns 1.0 if unstable. |
| `expected_calibration_error` | re-export of [`opensmartroute.math.estimators.expected_calibration_error`](#opensmartroutemathestimators) | ECE: how well does router confidence predict routing correctness?. |
| `expected_wait` | re-export of [`opensmartroute.math.decision.expected_wait`](#opensmartroutemathdecision) | Mean time in queue (Wq) for M/M/c, in the same time unit as the rates. |
| `gini` | re-export of [`opensmartroute.math.estimators.gini`](#opensmartroutemathestimators) | Gini impurity ``1 - sum(p^2)``; 0 = certain. |
| `hardware_profile` | re-export of [`opensmartroute.math.energy.hardware_profile`](#opensmartroutemathenergy) | Built-in profile by name (``a100-80gb``, ``h100-sxm``, ``l4``, ``rtx-4090``, ``cpu-16c``, ``npu-edge``). |
| `item_key` | re-export of [`opensmartroute.math.irt.item_key`](#opensmartroutemathirt) | Default item id: domain × difficulty bucket, e.g. ``legal/3``. |
| `kingman_wait` | re-export of [`opensmartroute.math.decision.kingman_wait`](#opensmartroutemathdecision) | Kingman's G/G/1 approximation. |
| `littles_law` | re-export of [`opensmartroute.math.decision.littles_law`](#opensmartroutemathdecision) | L = λ·W — average number of in-flight requests. |
| `normalized_entropy` | re-export of [`opensmartroute.math.estimators.normalized_entropy`](#opensmartroutemathestimators) | 0 = certain, 1 = uniform. Useful as an 'ask the LLM judge' trigger. |
| `pareto_front` | re-export of [`opensmartroute.math.decision.pareto_front`](#opensmartroutemathdecision) | Keys of the non-dominated (quality, cost, latency) points. |
| `reliability_diagram` | re-export of [`opensmartroute.math.calibration.reliability_diagram`](#opensmartroutemathcalibration) | Per-bin (mean confidence, empirical accuracy, count) — plot or print it. |
| `servers_for_sla` | re-export of [`opensmartroute.math.decision.servers_for_sla`](#opensmartroutemathdecision) | Smallest c such that E[Wq] ≤ max_wait and P(wait) ≤ max_p_wait. |
| `sigmoid` | re-export of [`opensmartroute.math.irt.sigmoid`](#opensmartroutemathirt) | Overflow-safe logistic function. |
| `softmax` | re-export of [`opensmartroute.math.estimators.softmax`](#opensmartroutemathestimators) | Numerically stable softmax; lower ``temperature`` sharpens the distribution. |
| `topsis` | re-export of [`opensmartroute.math.decision.topsis`](#opensmartroutemathdecision) | TOPSIS closeness coefficient in [0,1]; quality is a benefit, cost/latency are costs. |
| `weighted_sum` | re-export of [`opensmartroute.math.decision.weighted_sum`](#opensmartroutemathdecision) | Scalarise ``w_q * quality - w_c * norm(cost) - w_l * norm(latency)`` with min-max normalised cost / latency. |
| `wilson_interval` | re-export of [`opensmartroute.math.estimators.wilson_interval`](#opensmartroutemathestimators) | Wilson score interval for a binomial proportion (robust at small n). |

## `opensmartroute.math.bandits`

Source: [src/opensmartroute/math/bandits.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/math/bandits.py)

Multi-armed and contextual bandits for online routing decisions.

| Name | Kind | Summary |
|---|---|---|
| `Bandit` | class | Common interface. ``context`` is optional; non-contextual learners ignore it. |
| `ThompsonBeta` | class | Beta–Bernoulli Thompson sampling. |
| `UCB1` | class | UCB1: :math:`\hat\mu_a + c\sqrt{\frac{2\ln t}{n_a}}`; untried arms get +inf. |
| `LinUCB` | class | Disjoint LinUCB (Li et al., WWW 2010) with Sherman–Morrison updates and optional. |
| `EpsilonGreedy` | class | Explore uniformly with probability ``epsilon``, otherwise exploit the empirical means. |
| `CostAwareBandit` | class | Lagrangian budget wrapper (C2MAB-V flavour). |
| `MultiKnapsackBandit` | class | Bandits with several knapsack constraints (Badanidiyuru et al., 2013) with the. |
| `DelayedFeedback` | class | Delayed-feedback wrapper (Joulani, György & Szepesvári, 2013). |

## `opensmartroute.math.calibration`

Source: [src/opensmartroute/math/calibration.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/math/calibration.py)

Calibration and distribution-free risk control for routing confidence.

| Name | Kind | Summary |
|---|---|---|
| `ConformalCalibrator` | class | Split conformal prediction over routing candidates. |
| `IsotonicCalibrator` | class | Monotone non-decreasing map score -> P(correct), fitted with PAV. |
| `TemperatureScaler` | class | Fits :math:`\tau` for ``confidence = softmax(u/\tau)[argmax]`` by 1-D golden-section. |
| `brier_score` | function `(confidences: list[float], hits: list[bool])` | Mean squared error between confidence and the 0/1 correctness label. |
| `conformal_quantile` | function `(scores: list[float], alpha: float)` | Finite-sample corrected :math:`\lceil (n+1)(1-\alpha)\rceil / n` empirical quantile. |
| `reliability_diagram` | function `(confidences: list[float], hits: list[bool], bins: int=10)` | Per-bin (mean confidence, empirical accuracy, count) — plot or print it. |

## `opensmartroute.math.decision`

Source: [src/opensmartroute/math/decision.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/math/decision.py)

Multi-objective decision helpers and queueing theory for real-time routing.

| Name | Kind | Summary |
|---|---|---|
| `dominates` | function `(a: Point, b: Point)` | a dominates b if it is ≥ quality, ≤ cost, ≤ latency and strictly better in one. |
| `pareto_front` | function `(points: dict[str, Point])` | Keys of the non-dominated (quality, cost, latency) points. |
| `weighted_sum` | function `(points: dict[str, Point], w_q: float, w_c: float, w_l: float)` | Scalarise ``w_q * quality - w_c * norm(cost) - w_l * norm(latency)`` with min-max normalised cost / latency. |
| `topsis` | function `(points: dict[str, Point], weights: Sequence[float]=(0.6, 0.25, 0.15))` | TOPSIS closeness coefficient in [0,1]; quality is a benefit, cost/latency are costs. |
| `erlang_c` | function `(arrival_rate: float, service_rate: float, servers: int)` | P(an arriving request must wait) for M/M/c. Returns 1.0 if unstable. |
| `expected_wait` | function `(arrival_rate: float, service_rate: float, servers: int)` | Mean time in queue (Wq) for M/M/c, in the same time unit as the rates. |
| `servers_for_sla` | function `(arrival_rate: float, service_rate: float, max_wait: float, max_p_wait: float=0.2)` | Smallest c such that E[Wq] ≤ max_wait and P(wait) ≤ max_p_wait. |
| `littles_law` | function `(arrival_rate: float, mean_time_in_system: float)` | L = λ·W — average number of in-flight requests. |
| `kingman_wait` | function `(utilization: float, ca2: float, cs2: float, mean_service: float)` | Kingman's G/G/1 approximation. |

## `opensmartroute.math.dirichlet`

Source: [src/opensmartroute/math/dirichlet.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/math/dirichlet.py)

Dirichlet probe over host hidden states (ProbeDirichlet, RouterXBench 2602.11877).

| Name | Kind | Summary |
|---|---|---|
| `DirichletProbe` | class | Linear Dirichlet head over a fixed target list. |
| `digamma` | function `(x: float)` | Digamma :math:`\psi(x)` for x > 0 via recurrence to x >= 6 and the asymptotic series. |

## `opensmartroute.math.energy`

Source: [src/opensmartroute/math/energy.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/math/energy.py)

Hardware-aware energy characterisation (HW-Router 2608.14575; 2608.28044).

| Name | Kind | Summary |
|---|---|---|
| `EnergyModel` | class | Per-target linear energy model ``wh = e0 + e_in * prompt + e_out * output`` fit by ridge. |
| `HardwareProfile` | class | Static device characterisation used before measurements exist. |
| `hardware_profile` | function `(name: str)` | Built-in profile by name (``a100-80gb``, ``h100-sxm``, ``l4``, ``rtx-4090``, ``cpu-16c``, ``npu-edge``). |

## `opensmartroute.math.estimators`

Source: [src/opensmartroute/math/estimators.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/math/estimators.py)

Streaming estimators, drift detection and calibration.

| Name | Kind | Summary |
|---|---|---|
| `EWMA` | class | Exponentially weighted moving mean and variance with a z-score helper. |
| `Welford` | class | Numerically stable running mean / variance (Welford's algorithm). |
| `PageHinkley` | class | Detects a *decrease* in the monitored mean (e.g. quality dropping). |
| `WindowDrift` | class | ADWIN-lite: compare the first and second half of a sliding window with a. |
| `wilson_interval` | function `(successes: float, n: int, z: float=1.96)` | Wilson score interval for a binomial proportion (robust at small n). |
| `bayesian_average` | function `(mean: float, n: int, prior_mean: float, prior_n: float=5.0)` | Shrink a small-sample mean toward a prior: (n·m + k·μ₀)/(n + k). |
| `softmax` | function `(xs: list[float], temperature: float=1.0)` | Numerically stable softmax; lower ``temperature`` sharpens the distribution. |
| `entropy` | function `(ps: list[float])` | Shannon entropy (nats) of a probability vector. |
| `normalized_entropy` | function `(ps: list[float])` | 0 = certain, 1 = uniform. Useful as an 'ask the LLM judge' trigger. |
| `gini` | function `(ps: list[float])` | Gini impurity ``1 - sum(p^2)``; 0 = certain. |
| `expected_calibration_error` | function `(confidences: list[float], hits: list[bool], bins: int=10)` | ECE: how well does router confidence predict routing correctness?. |

## `opensmartroute.math.irt`

Source: [src/opensmartroute/math/irt.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/math/irt.py)

Item Response Theory for routing (IRT-Router, ACL 2025).

| Name | Kind | Summary |
|---|---|---|
| `sigmoid` | function `(z: float)` | Overflow-safe logistic function. |
| `IRTModel` | class | Online 2PL IRT: target ability, item difficulty / discrimination via SGD, with forgetting and merge support. |
| `item_key` | function `(domain: str, complexity: float, buckets: int=4)` | Default item id: domain × difficulty bucket, e.g. ``legal/3``. |

## `opensmartroute.math.markov`

Source: [src/opensmartroute/math/markov.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/math/markov.py)

Markov chains and MDPs for conversational / multi-step routing.

| Name | Kind | Summary |
|---|---|---|
| `MarkovChain` | class | Dirichlet-smoothed transition counts. ``decay`` < 1 multiplies a row's counts by ``decay``. |
| `RoutingMDP` | class | Finite-horizon / discounted MDP over conversation states and route actions. |

## `opensmartroute.math.preference`

Source: [src/opensmartroute/math/preference.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/math/preference.py)

Bradley–Terry pairwise preference model (RouteLLM, Prompt-to-Leaderboard).

| Name | Kind | Summary |
|---|---|---|
| `BradleyTerry` | class | Online Bradley-Terry: per-context target strengths from pairwise wins, with L2 and forgetting. |
| `Elo` | class | Elo rating — a fixed-step Bradley–Terry with K-factor; handy for fast warm-up. |

## `opensmartroute.mcp_server`

Source: [src/opensmartroute/mcp_server.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/mcp_server.py)

Model Context Protocol server: the router as a set of tools for any IDE or agent.

| Name | Kind | Summary |
|---|---|---|
| `PROTOCOL_VERSION` | constant | : Protocol revision this server speaks by default. |
| `SUPPORTED_PROTOCOL_VERSIONS` | constant | : Revisions accepted from clients (the client's choice is echoed when it is one of these). |
| `MCPServer` | class | Serve a :class:`Router` (or an enterprise router wrapping one) over MCP. |
| `RemoteMCP` | class | Forward JSON-RPC messages to an HTTP MCP endpoint (``POST /mcp``) - the bridge's counterpart of. |
| `ToolError` | class | Raised inside a tool: reported to the client as a tool result with ``isError`` (not a protocol error). |
| `bridge_stdio` | function `(url: str, api_key: str \| None=None)` | Expose a remote HTTP MCP endpoint as a local stdio server (for clients that only spawn processes). |
| `decision_dict` | function `(d: RouteDecision)` | ``RouteDecision.to_dict()`` plus signals and the ranked candidates - what IDE clients want to show. |
| `serve_stdio` | function `(server: Any, stdin: IO[bytes] \| None=None, stdout: IO[str] \| None=None)` | Run ``server`` (anything with ``handle_json``) over newline-delimited JSON on stdin/stdout until EOF. |

## `opensmartroute.observability`

Source: [src/opensmartroute/observability.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/observability.py)

Tracing and observability: every step of routing, execution and learning as a structured event.

| Name | Kind | Summary |
|---|---|---|
| `EVENT_NAMES` | constant | Router.route. |
| `METRIC_PREFIX` | constant | Prometheus metric name prefix (``osr_``). |
| `SPAN_NAMES` | constant | : Every span name the SDK opens (an :class:`Event` with ``kind="span"``). |
| `TRACEPARENT_KEY` | constant | request.context key (and HTTP header) carrying an inbound W3C trace context. |
| `TRACE_ID_HEADER` | constant | response header ``osr serve`` sets to the trace id of the request. |
| `Event` | class | One captured span or event: flat, JSON-friendly, never carries request text. |
| `EventSink` | class | Receiver port: override :meth:`emit`; live bridges may also implement :meth:`span_start` / :meth:`span_end`. |
| `FileSink` | class | Append-only JSONL file, one event per line. |
| `LoggingSink` | class | One JSON line per event on the ``opensmartroute.events`` logger (level follows the event). |
| `MemorySink` | class | Thread-safe ring buffer of the newest ``max_events`` events; queryable by request, trace, name or level. |
| `MetricsSink` | class | Counters and latency percentiles derived from events; :meth:`prometheus` renders the exposition text. |
| `Span` | class | An open unit of work; a context manager that records duration, status and nested events. |
| `Tracer` | class | Opens spans, records events and fans them out to sinks; ``sample_rate`` < 1 traces a share of requests. |
| `configure_tracing` | function `(*sinks: EventSink, sample_rate: float \| None=None, settings: Settings \| None=None)` | Install sinks on the process-wide tracer. With no sinks, use the ones named by settings. |
| `current_tracer` | function `()` | The tracer of the innermost open span, else the one bound with :func:`use_tracer`, else the global one. |
| `get_tracer` | function `()` | The process-wide tracer (disabled until :func:`configure_tracing` adds sinks). |
| `text_digest` | function `(text: str)` | Loggable stand-in for request text: ``{"text_sha256": <16 hex>, "text_len": n}``. |
| `use_tracer` | class | ``with use_tracer(t):`` makes ``t`` the tracer that :func:`current_tracer` returns in this context. |

## `opensmartroute.ocm`

Source: [src/opensmartroute/ocm.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/ocm.py)

Open Capability Manifest (OCM) - a vendor-neutral description of any routable capability.

| Name | Kind | Summary |
|---|---|---|
| `OCM_KINDS` | constant | : Manifest kinds (identical to :class:`~opensmartroute.TargetKind` values). |
| `OCM_PROTOCOLS` | constant | : Endpoint protocols a manifest may declare; ``http`` and ``a2a`` can be bound to executors directly. |
| `OCM_VERSION` | constant | : Current manifest version (the ``ocm`` field). |
| `bind_endpoint` | function `(target: RouteTarget, region: str \| None=None, **kw: Any)` | Attach an executor for the selected endpoint (``http`` -> :func:`~opensmartroute.adapters.http_handler`,. |
| `capability_from_target` | function `(target: RouteTarget)` | Reverse mapping: publish a target as an OCM v1 manifest (``dict``; dump as YAML or JSON). |
| `dump_capability` | function `(doc: dict[str, Any])` | Serialise a manifest to YAML when PyYAML is available, else to pretty JSON (both are valid OCM). |
| `is_capability` | function `(doc: Any)` | True when ``doc`` looks like an OCM manifest (has an ``ocm`` version field). |
| `load_capabilities` | function `(paths: Iterable[str \| Path] \| str \| Path, *, strict: bool=True)` | Load manifests from files and/or directories (``**/capability.{yaml,yml,json}``). |
| `load_capability` | function `(path: str \| Path, *, strict: bool=True)` | Load one ``capability.yaml`` / ``.json`` manifest as a target. |
| `select_endpoint` | function `(target: RouteTarget, region: str \| None=None)` | The manifest endpoint to use: a region-matching one first, then the first region-less one. |
| `target_from_capability` | function `(doc: dict[str, Any], *, strict: bool=True)` | Convert a manifest into a :class:`~opensmartroute.RouteTarget` (validating first when ``strict``). |
| `validate_capability` | function `(doc: Any)` | Validate a manifest; returns a list of human-readable problems (empty = valid). |

## `opensmartroute.policy`

Source: [src/opensmartroute/policy/__init__.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/policy/__init__.py)

Policy layer: hard constraints that are never traded off against utility.

| Name | Kind | Summary |
|---|---|---|
| `DataBoundaryRule` | class | The target's boundary must be at least as strict as the request's (public < private < on_prem). |
| `JailbreakRule` | class | Risky prompts may only reach humans or targets tagged as safe. |
| `LanguageRule` | class | The detected language must be declared by the target, unless it declares the wildcard language. |
| `Policy` | class | Ordered chain of :data:`PolicyRule`; returns the first rejection reason or ``None``. |
| `PolicyRule` | constant | A hard-constraint check: return a rejection reason, or None to let the target through. |
| `allow_list` | function `(target: RouteTarget, request: RouteRequest, signals: Signals)` | When the request names ``allow_targets``, reject everything else. |
| `allowed_kinds` | function `(target: RouteTarget, request: RouteRequest, signals: Signals)` | Reject kinds outside the request's ``allowed_kinds``. |
| `context_window` | function `(target: RouteTarget, request: RouteRequest, signals: Signals)` | Reject when the estimated input exceeds the target's ``context_window``. |
| `cost_budget` | function `(target: RouteTarget, request: RouteRequest, signals: Signals)` | Reject targets whose unit cost exceeds the request's ``max_cost_per_1k``. |
| `default_rules` | function `(settings: PolicySettings \| None=None)` | The built-in chain, in evaluation order (cheap identity checks first). |
| `deny_list` | function `(target: RouteTarget, request: RouteRequest, signals: Signals)` | Reject targets listed in the request's ``deny_targets``. |
| `enabled` | function `(target: RouteTarget, request: RouteRequest, signals: Signals)` | Reject disabled targets. |
| `input_tokens` | function `(target: RouteTarget, request: RouteRequest, signals: Signals)` | Reject when the estimated input exceeds the target's ``max_tokens_in``. |
| `latency_slo` | function `(target: RouteTarget, request: RouteRequest, signals: Signals)` | Reject targets whose declared latency exceeds the request's ``max_latency_ms``. |
| `modalities` | function `(target: RouteTarget, request: RouteRequest, signals: Signals)` | Every modality detected in the request must be supported by the target. |
| `pii` | function `(target: RouteTarget, request: RouteRequest, signals: Signals)` | Requests flagged as containing PII may only reach targets with ``pii_allowed``. |
| `region` | function `(target: RouteTarget, request: RouteRequest, signals: Signals)` | The request's ``region`` must be one the target serves (targets with no regions serve all). |
| `rule_name` | function `(rule: PolicyRule)` | Display name of a rule: its ``name`` attribute, else ``__name__``, else the class name. |
| `tenant` | function `(target: RouteTarget, request: RouteRequest, signals: Signals)` | The request's ``tenant`` must be permitted by targets that restrict tenants. |
| `tools` | function `(target: RouteTarget, request: RouteRequest, signals: Signals)` | When tools are required (constraint or ``context['tools']``), LLM targets must ``supports_tools``. |

## `opensmartroute.realtime`

Source: [src/opensmartroute/realtime/__init__.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/realtime/__init__.py)

Real-time operational controls: health, circuit breaking, rate & budget limits.

| Name | Kind | Summary |
|---|---|---|
| `BreakerState` | class | Circuit-breaker states: closed (healthy), open (tripped), half_open (probing recovery). |
| `CircuitBreaker` | class | Classic three-state breaker with a sliding failure window. |
| `TokenBucket` | class | Rate limiter: ``rate`` tokens/sec, burst up to ``capacity``. Thread-safe. |
| `Budget` | class | Rolling spend cap (e.g. USD per hour) per target or tenant. |
| `LatencyWindow` | class | Rolling window of the last ``size`` latencies with nearest-rank percentiles (p50 / p90 / p99). |
| `TargetHealth` | class | Live health of one target: breaker, latency EWMA + percentiles, success counts, optional rate limit / budget. |
| `HealthRegistry` | class | Tracks health per target id. Shared by policy + strategy + executor. |
| `HealthPolicy` | class | Policy that additionally rejects targets whose breaker is open, whose rate. |
| `HealthStrategy` | class | Soft signal: observed reliability x latency-SLO fit from live health data. |

## `opensmartroute.retrieval`

Source: [src/opensmartroute/retrieval.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/retrieval.py)

Retrieval-based candidate narrowing for very large target pools (ToolRet / Skill-RAG).

| Name | Kind | Summary |
|---|---|---|
| `BM25Index` | class | In-memory BM25 (Okapi) index over target documents. |
| `DenseIndex` | class | Cosine index. Default = sparse signed-hashing features with an inverted index (pure. |
| `NarrowingStats` | class | Counters kept by the retrieval middleware: calls, how often it narrowed, last pool and kept sizes. |
| `RetrievalResult` | class | Narrowed candidate ids with fused scores and how many came from the lexical / dense legs. |
| `Retriever` | class | Hybrid lexical + dense narrowing that tracks a registry (or an explicit pool). |
| `execute_tool_target` | function `(registry: TargetRegistry, allow: Callable[[RouteTarget, dict[str, Any]], bool] \| None=None, target_id: str='execute_tool')` | A *tool that runs a tool by id* (with an optional allow-policy hook). |
| `narrow_signals_hint` | function `(signals: Signals)` | Extra lexical hints from signals (domains / actions) appended to the retrieval query. |
| `rrf` | function `(rankings: list[list[tuple[str, float]]], k: int=60, weights: list[float] \| None=None)` | Reciprocal Rank Fusion over several ranked lists of ``(id, score)``. |
| `select_skill_set` | function `(request: RouteRequest \| str, pool: list[RouteTarget], k: int=3, embedder: Embedder \| None=None, redundancy: float=0.7, min_gain: float=0.02, relevance: dict[str, float] \| None=None)` | Greedy facility-location selection of complementary targets. |
| `target_text` | function `(t: RouteTarget)` | Lexical document for a target: id, name, description, capabilities and up to 8 examples. |
| `tokenize` | function `(text: str)` | Lower-case alphanumeric tokens. |
| `tool_search_target` | function `(retriever: Retriever, registry: TargetRegistry, kinds: tuple[str, ...]=('tool', 'skill'), k: int=8, target_id: str='tool_search')` | A *tool that finds tools*: returns compact descriptors for the ``k`` best matches. |

## `opensmartroute.router`

Source: [src/opensmartroute/router.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/router.py)

The Router: signals -> policy -> strategies -> ensemble -> decision (+plan).

| Name | Kind | Summary |
|---|---|---|
| `Router` | class | Signals -> policy -> strategies -> ensemble -> decision (+ optional plan). |
| `NoRouteError` | re-export of [`opensmartroute.errors.NoRouteError`](#opensmartrouteerrors) | No target satisfied the hard constraints. |
| `DEFAULT_WEIGHTS` | constant | : Library-default ensemble weights (``Settings().weights``); kept for backwards compatibility. |

## `opensmartroute.sdk`

Source: [src/opensmartroute/sdk.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/sdk.py)

OpenSmartRoute SDK: decorator-driven registration of routing components.

| Name | Kind | Summary |
|---|---|---|
| `ComponentKind` | class | The component families a :class:`ComponentRegistry` can hold. |
| `ComponentRegistry` | class | Blueprints for every routing component, with decorators that register into it. |
| `FunctionMiddleware` | class | Adapts ``fn(request, next_) -> RouteDecision``. |
| `FunctionSignal` | class | Adapts ``fn(request, signals) -> None \| {field: value}``. |
| `FunctionStrategy` | class | Adapts ``fn(request, signals, candidates) -> {target_id: score \| StrategyScore}``. |
| `Registration` | class | A registered blueprint. ``factory()`` returns a fresh component instance. |
| `agent` | constant | @agent(id, ...): shorthand for an ``agent`` target. |
| `components` | constant | : The process-wide registry that the top-level decorators (``opensmartroute.strategy`` ...) bind to. |
| `middleware` | constant | @middleware: register a Middleware class or ``fn(request, next_route)``. |
| `policy_rule` | constant | @policy_rule: register ``fn(target, request, signals) -> reason \| None``. |
| `signal` | constant | @signal: register a SignalExtractor class or ``fn(request, signals) -> mapping``. |
| `skill` | constant | @skill(id, ...): shorthand for a ``skill`` target. |
| `strategy` | constant | @strategy(weight=, name=): register a Strategy class or ``fn(request, signals, candidates)``. |
| `target` | constant | @target(id, kind, ...): the decorated callable becomes a RouteTarget handler. |
| `telemetry` | constant | @telemetry: register a Telemetry sink class or factory. |
| `tool` | constant | @tool(id, ...): shorthand for a ``tool`` target. |

## `opensmartroute.security`

Source: [src/opensmartroute/security/__init__.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/security/__init__.py)

Security controls for the routing control plane.

| Name | Kind | Summary |
|---|---|---|
| `GadgetDetector` | re-export of [`opensmartroute.security.gadget.GadgetDetector`](#opensmartroutesecuritygadget) | Learned tail classifier. ``score(text)`` in [0, 1]; ``split(text)`` finds the gadget tail. |
| `GuardMiddleware` | class | Applies :class:`InputGuard` (+ optional redaction) before routing. |
| `GuardReport` | class | Outcome of :meth:`InputGuard.inspect`: pass / fail, reasons, gadget suspicion score and the cleaned text. |
| `InputGuard` | class | Validates and normalises request text. Cheap: O(len(text)). |
| `OriginPolicy` | re-export of [`opensmartroute.security.provenance.OriginPolicy`](#opensmartroutesecurityprovenance) | Decides which tool arguments must be user-originated and verifies them. |
| `OriginRule` | re-export of [`opensmartroute.security.provenance.OriginRule`](#opensmartroutesecurityprovenance) | Per-target override. ``sensitive=None`` = infer from parameter names. |
| `OriginViolation` | re-export of [`opensmartroute.security.provenance.OriginViolation`](#opensmartroutesecurityprovenance) | Raised by :class:`OriginPolicy` when a sensitive argument of a state-changing tool comes from untrusted text. |
| `Redactor` | class | Replace PII with typed placeholders, e.g. ``<EMAIL_1>``. Reversible per-request. |
| `ResourceLimitExceeded` | re-export of [`opensmartroute.security.limits.ResourceLimitExceeded`](#opensmartroutesecuritylimits) | A task went over one of its :class:`ResourceLimits` (steps, tool calls, depth, tokens, cost, wall time). |
| `ResourceLimitMiddleware` | re-export of [`opensmartroute.security.limits.ResourceLimitMiddleware`](#opensmartroutesecuritylimits) | Charge one step (+ the request's token estimate) per ``route()`` call. |
| `ResourceLimiter` | re-export of [`opensmartroute.security.limits.ResourceLimiter`](#opensmartroutesecuritylimits) | Thread-safe per-task accounting against :class:`ResourceLimits`. |
| `ResourceLimits` | re-export of [`opensmartroute.security.limits.ResourceLimits`](#opensmartroutesecuritylimits) | Per-task ceilings that stop runaway agents; enforced by :class:`ResourceLimiter`. |
| `SafetyCase` | re-export of [`opensmartroute.security.safety.SafetyCase`](#opensmartroutesecuritysafety) | One red-team scenario: an adversarial request, its clean baseline and the routing invariants to check. |
| `apply_limits` | re-export of [`opensmartroute.security.limits.apply_limits`](#opensmartroutesecuritylimits) | Wrap handlers of the given kinds. Returns the number wrapped. |
| `apply_origin_policy` | re-export of [`opensmartroute.security.provenance.apply_origin_policy`](#opensmartroutesecurityprovenance) | Wrap every state-changing target in ``registry``. Returns the number wrapped. |
| `description_risk` | re-export of [`opensmartroute.security.injection.description_risk`](#opensmartroutesecurityinjection) | Risk of a *tool/agent description*: max of injection lexicon and learned gadget score. |
| `guard_handler` | re-export of [`opensmartroute.security.provenance.guard_handler`](#opensmartroutesecurityprovenance) | Return a copy of ``target`` whose handler enforces ``policy`` before invoking the tool. |
| `injection_risk` | re-export of [`opensmartroute.security.injection.injection_risk`](#opensmartroutesecurityinjection) | Shortcut for ``inspect_injection(text).score``. |
| `inspect_injection` | re-export of [`opensmartroute.security.injection.inspect_injection`](#opensmartroutesecurityinjection) | Score instruction-injection likelihood with the matched lexicon entries for the trace. |
| `limit_handler` | re-export of [`opensmartroute.security.limits.limit_handler`](#opensmartroutesecuritylimits) | Copy of ``target`` whose handler charges one tool call (and nesting depth) per invocation. |
| `load_secret` | function `(name: str, *, file_env_suffix: str='_FILE', default: str \| None=None)` | 12-factor secret loading: ``NAME`` env var, else the file at ``NAME_FILE`` (k8s/Docker secrets). |
| `mark_untrusted` | re-export of [`opensmartroute.security.provenance.mark_untrusted`](#opensmartroutesecurityprovenance) | Record content that entered the request from a non-user source. |
| `run_safety_suite` | re-export of [`opensmartroute.security.safety.run_safety_suite`](#opensmartroutesecuritysafety) | Run every case through ``route`` and return pass/fail details grouped by category. |
| `sanitize_for_prompt` | function `(text: str, max_len: int=4000)` | Make user text safe to embed in an LLM-judge prompt. |
| `shannon_entropy` | function `(s: str)` | Character-level Shannon entropy in bits (high values flag encoded / random payloads). |
| `strip_steering` | re-export of [`opensmartroute.security.injection.strip_steering`](#opensmartroutesecurityinjection) | Remove sentences that instruct the *router* (self-declared complexity, "use the best. |
| `synthesize_gadget_corpus` | re-export of [`opensmartroute.security.gadget.synthesize_gadget_corpus`](#opensmartroutesecuritygadget) | Labelled ``(text, label)`` rows: clean prompts + prompts with an appended gadget. |

## `opensmartroute.security.gadget`

Source: [src/opensmartroute/security/gadget.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/security/gadget.py)

Learned confounder-gadget detector (Rerouting LLM Routers, Shafran et al. 2025).

| Name | Kind | Summary |
|---|---|---|
| `CLEAN` | constant | class labels used by the detector and the synthetic corpus. |
| `GADGET` | constant | class labels used by the detector and the synthetic corpus. |
| `GadgetDetector` | class | Learned tail classifier. ``score(text)`` in [0, 1]; ``split(text)`` finds the gadget tail. |
| `synthesize_gadget_corpus` | function `(per_template: int=2, n_gadgets: int=260, seed: int=0)` | Labelled ``(text, label)`` rows: clean prompts + prompts with an appended gadget. |
| `synthesize_gadgets` | function `(n: int, seed: int=0)` | Adversarial suffixes drawn from the gadget families described in the paper. |
| `tail_features` | function `(tokens: Sequence[str])` | Hand-crafted statistics of a token window (all in [0, 1]) – complements hashed n-grams. |

## `opensmartroute.security.injection`

Source: [src/opensmartroute/security/injection.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/security/injection.py)

Instruction-injection detection for *text that is not the user's request*.

| Name | Kind | Summary |
|---|---|---|
| `InjectionReport` | class | Instruction-injection score in [0, 1] plus the lexicon hits that produced it. |
| `description_risk` | function `(text: str, gadget_score: float \| None=None)` | Risk of a *tool/agent description*: max of injection lexicon and learned gadget score. |
| `injection_risk` | function `(text: str)` | Shortcut for ``inspect_injection(text).score``. |
| `inspect_injection` | function `(text: str)` | Score instruction-injection likelihood with the matched lexicon entries for the trace. |

## `opensmartroute.security.limits`

Source: [src/opensmartroute/security/limits.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/security/limits.py)

Resource-amplification limits ("Beyond Max Tokens").

| Name | Kind | Summary |
|---|---|---|
| `ResourceLimitExceeded` | class | A task went over one of its :class:`ResourceLimits` (steps, tool calls, depth, tokens, cost, wall time). |
| `ResourceLimitMiddleware` | class | Charge one step (+ the request's token estimate) per ``route()`` call. |
| `ResourceLimiter` | class | Thread-safe per-task accounting against :class:`ResourceLimits`. |
| `ResourceLimits` | class | Per-task ceilings that stop runaway agents; enforced by :class:`ResourceLimiter`. |
| `TaskUsage` | class | Running consumption of one task (steps, tool calls, depth, tokens, cost, timestamps). |
| `apply_limits` | function `(registry: TargetRegistry, limiter: ResourceLimiter, kinds: tuple[str, ...]=('tool', 'agent', 'workflow'))` | Wrap handlers of the given kinds. Returns the number wrapped. |
| `limit_handler` | function `(target: RouteTarget, limiter: ResourceLimiter)` | Copy of ``target`` whose handler charges one tool call (and nesting depth) per invocation. |
| `task_id_of` | function `(request: RouteRequest, key: str='task_id')` | ``context[key]`` when present, otherwise the request id (each request is its own task). |

## `opensmartroute.security.provenance`

Source: [src/opensmartroute/security/provenance.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/security/provenance.py)

Origin (provenance) policy for tool parameters – ROPE-style control-flow integrity.

| Name | Kind | Summary |
|---|---|---|
| `OriginFinding` | class | Where one tool argument's value came from (user, context, untrusted content or unknown). |
| `OriginPolicy` | class | Decides which tool arguments must be user-originated and verifies them. |
| `OriginReport` | class | Provenance check of a tool call: all findings plus the sensitive parameters with a disallowed origin. |
| `OriginRule` | class | Per-target override. ``sensitive=None`` = infer from parameter names. |
| `OriginViolation` | class | Raised by :class:`OriginPolicy` when a sensitive argument of a state-changing tool comes from untrusted text. |
| `SENSITIVE_PARAM_HINTS` | constant | Parameter-name fragments treated as sensitive when a target has no explicit OriginRule. |
| `STATE_CHANGING_ACTIONS` | constant | Verbs in a tool's id / description / actions that mark it as state-changing (side effects). |
| `apply_origin_policy` | function `(registry: TargetRegistry, policy: OriginPolicy)` | Wrap every state-changing target in ``registry``. Returns the number wrapped. |
| `guard_handler` | function `(target: RouteTarget, policy: OriginPolicy, arguments_key: str='tool_arguments')` | Return a copy of ``target`` whose handler enforces ``policy`` before invoking the tool. |
| `mark_untrusted` | function `(request: RouteRequest, text: str, source: str='retrieved')` | Record content that entered the request from a non-user source. |
| `untrusted_texts` | function `(request: RouteRequest)` | ``(source, text)`` pairs recorded by :func:`mark_untrusted` on this request. |
| `user_texts` | function `(request: RouteRequest)` | Everything the user actually typed: current text, prior user turns and the pre-guard original text. |

## `opensmartroute.security.safety`

Source: [src/opensmartroute/security/safety.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/security/safety.py)

Safety-routing regression suite ("When Safety Routing Breaks").

| Name | Kind | Summary |
|---|---|---|
| `SafetyCase` | class | One red-team scenario: an adversarial request, its clean baseline and the routing invariants to check. |
| `SafetyResult` | class | Verdict for one :class:`SafetyCase`: chosen vs baseline target and the invariants that failed. |
| `default_cases` | function `(seed: int=0)` | Deployment-agnostic red-team cases; extend with catalogue-specific ones. |
| `run_safety_suite` | function `(route: RouteCall, cases: Sequence[SafetyCase] \| None=None, *, seed: int=0)` | Run every case through ``route`` and return pass/fail details grouped by category. |

## `opensmartroute.server`

Source: [src/opensmartroute/server.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/server.py)

Optional FastAPI server exposing the router over HTTP.

| Name | Kind | Summary |
|---|---|---|
| `AUTO_SLUG` | constant | : Slug prefix of the router's own pseudo-models on the OpenAI-compatible proxy (``osr/auto``). |
| `DEFAULT_PROXY_ALIASES` | constant | : Model names that mean "let the router choose" on the OpenAI-compatible proxy. |
| `AUTO_VARIANTS` | constant | : ``osr/auto:<variant>`` presets: objective weights, hard constraints and allowed kinds. |
| `APP_HEADER` | constant | : Request header naming the calling application (OpenRouter-style attribution); echoed in metadata. |
| `TARGET_HEADER` | constant | : Response headers carrying the decision (target id, request id) beside the OpenAI-shaped body. |
| `REQUEST_ID_HEADER` | constant | : Response headers carrying the decision (target id, request id) beside the OpenAI-shaped body. |
| `OPEN_PATHS` | constant | : Paths that never require a token when access control is on (probes, metrics, token check, OpenAPI pages). |
| `UNTRACED_PATHS` | constant | : Paths that never open an ``http.request`` span (probes and the observability endpoints themselves). |
| `bearer_token` | function `(headers: Any)` | The access token of a request: ``X-API-Key`` first, else ``Authorization: Bearer <token>``. |
| `token_accepted` | function `(token: str \| None, tokens: Iterable[str])` | Constant-time membership test of ``token`` in the configured access tokens. |
| `create_app` | function `(router: Any, proxy_aliases: frozenset[str]=DEFAULT_PROXY_ALIASES, autopilot: Any=None, auth_tokens: Iterable[str]=())` | FastAPI app: ``/route``, ``/feedback``, ``/targets``, ``/stats``, ``/healthz`` and the OpenAI-compatible ``/v1``. |
| `resolve_model` | function `(model: str, aliases: frozenset[str]=DEFAULT_PROXY_ALIASES)` | Split a proxy ``model`` into ``(pinned target id \| None, route options)``. |
| `proxy_request` | function `(body: dict[str, Any], aliases: frozenset[str]=frozenset({'auto'}), app: str \| None=None)` | Turn an OpenAI chat-completion body into a :class:`RouteRequest`. |
| `proxy_response` | function `(decision: RouteDecision, result: Any, request: RouteRequest \| None=None)` | Shape an execution result as an OpenAI ``chat.completion`` object (``model`` = chosen target id). |

## `opensmartroute.settings`

Source: [src/opensmartroute/settings.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/settings.py)

Typed, environment-overridable settings: the only home for OpenSmartRoute's tunable constants.

| Name | Kind | Summary |
|---|---|---|
| `BanditSettings` | class | Thompson-sampling strategy (:class:`~opensmartroute.BanditStrategy`). |
| `CapabilitySettings` | class | Capability-fit strategy (:class:`~opensmartroute.CapabilityStrategy`). |
| `ObservabilitySettings` | class | Tracing and event capture (:mod:`opensmartroute.observability`); read by ``Tracer.from_settings``. |
| `PolicySettings` | class | Hard-constraint thresholds (read by :class:`~opensmartroute.Policy`). |
| `RoutingSettings` | class | Ensemble, confidence and plan composition (read by :class:`~opensmartroute.Router`). |
| `RulesSettings` | class | Declarative rules strategy (:class:`~opensmartroute.RulesStrategy`). |
| `SLMSettings` | class | Routing SLM, dataset collection and the self-improvement loop (:mod:`opensmartroute.learning.slm`). |
| `ServerSettings` | class | ``osr serve`` (:mod:`opensmartroute.server`): access control of the self-hosted HTTP API. |
| `Settings` | class | All tunables, grouped by consumer. Immutable; derive variants with :meth:`replace`. |
| `WeightSettings` | class | Ensemble weight per strategy name (looked up as ``weights[strategy.name]``). |
| `configure` | function `(settings: Settings \| None=None, **groups: Any)` | Install process-wide settings. ``configure()`` with no arguments re-reads the environment;. |
| `get_settings` | function `()` | The process-wide :class:`Settings` (environment overlay applied once, lazily). |
| `resolve` | function `(settings: Settings \| None)` | ``settings`` if given, else the process-wide settings (component constructors use this). |

## `opensmartroute.signals`

Source: [src/opensmartroute/signals/__init__.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/signals/__init__.py)

Signal extraction: cheap, deterministic features computed from the request.

| Name | Kind | Summary |
|---|---|---|
| `ACTION_LEXICON` | constant | Action label -> trigger phrases; the keys are the ontology action names (summarize, translate, ... escalate). |
| `DEFAULT_EXTRACTORS` | constant | after domain detection (uses domain bonus). |
| `DEFAULT_ONTOLOGY` | re-export of [`opensmartroute.signals.ontology.DEFAULT_ONTOLOGY`](#opensmartroutesignalsontology) | the built-in TASK_TYPES ontology used by TaskTypeSignal. |
| `DOMAIN_LEXICON` | constant | Domain label -> trigger phrases; the keys are the ontology domain names targets and skills may declare. |
| `EVENT_LEXICON` | re-export of [`opensmartroute.signals.events.EVENT_LEXICON`](#opensmartroutesignalsevents) | Event name -> (domains, actions). Keys are matched exactly, then by ``subject.*`` prefix. |
| `EVENT_SUBJECTS` | re-export of [`opensmartroute.signals.events.EVENT_SUBJECTS`](#opensmartroutesignalsevents) | Generic fallbacks when the exact event / prefix is unknown: subject -> domain, verb -> action. |
| `EVENT_VERBS` | re-export of [`opensmartroute.signals.events.EVENT_VERBS`](#opensmartroutesignalsevents) | event verb -> action when the exact event / prefix is unknown. |
| `TASK_PRIORS` | re-export of [`opensmartroute.signals.models.TASK_PRIORS`](#opensmartroutesignalsmodels) | name: (difficulty, reasoning_need, expected_output_tokens). |
| `TASK_TYPES` | re-export of [`opensmartroute.signals.ontology.TASK_TYPES`](#opensmartroutesignalsontology) | ---------------------------------------------------------------- transformation. |
| `ComplexitySignal` | class | Hybrid-LLM-style difficulty estimate in [0, 1]. |
| `DomainActionSignal` | class | Arch-Router-style domain / action detection via lexicon matching. |
| `DraftResponseSignal` | re-export of [`opensmartroute.signals.uncertainty.DraftResponseSignal`](#opensmartroutesignalsuncertainty) | Query-response mixed representation: run a cheap drafter and expose draft features. |
| `EventInfo` | re-export of [`opensmartroute.signals.events.EventInfo`](#opensmartroutesignalsevents) | Parsed event: raw name, subject, verb and the domains / actions it implies. |
| `EventSignal` | re-export of [`opensmartroute.signals.events.EventSignal`](#opensmartroutesignalsevents) | Domains / actions from ``context["event"]`` and ``context["intent"]`` (event-driven requests). |
| `EventTrigger` | re-export of [`opensmartroute.signals.uncertainty.EventTrigger`](#opensmartroutesignalsuncertainty) | Event-triggered invocation: evaluate uncertainty features against rules and return the. |
| `HashedClassifier` | re-export of [`opensmartroute.signals.models.HashedClassifier`](#opensmartroutesignalsmodels) | Multinomial logistic regression over hashed features (sparse weights per class). |
| `HashedFeaturizer` | re-export of [`opensmartroute.signals.models.HashedFeaturizer`](#opensmartroutesignalsmodels) | Hashing-trick sparse features: words, bigrams, character n-grams and a few numeric text statistics. |
| `HashedRegressor` | re-export of [`opensmartroute.signals.models.HashedRegressor`](#opensmartroutesignalsmodels) | Squared-loss linear regressor on hashed features; output clipped to [lo, hi]. |
| `HistorySignal` | class | Conversation-state features from ``request.history`` (RCRouter-style):. |
| `LanguageSignal` | class | ``language`` from script / stop-word hints (``LANGUAGE_HINTS``); defaults to ``en``. |
| `LearnedDifficultySignal` | class | Blend the heuristic complexity with a trained regressor (Hybrid-LLM style difficulty). |
| `LengthSignal` | class | ``token_estimate`` from text + history length (about 4 characters per token). |
| `ModalitySignal` | class | ``modalities`` from context keys (``images`` / ``image_url``, ``audio``, ``files``) on top of ``text``. |
| `OutputLengthSignal` | class | Expected output tokens (drives cost estimates and effort/max_tokens choices). |
| `ProfileSignal` | class | User-profile features (``request.profile``): tier, expertise, preferences. |
| `ReasoningNeedSignal` | class | How much a request benefits from extended thinking (ThinkSwitcher / Sketch-of-Thought). |
| `SensitivitySignal` | class | PII and prompt-injection / jailbreak heuristics (vLLM-semantic-router-style). |
| `SignalExtractor` | class | Fills in part of a :class:`Signals` object. |
| `SignalModelBundle` | re-export of [`opensmartroute.signals.models.SignalModelBundle`](#opensmartroutesignalsmodels) | All learned signal models together, persisted as one JSON document. |
| `TaskOntology` | re-export of [`opensmartroute.signals.ontology.TaskOntology`](#opensmartroutesignalsontology) | Lookup helpers over :data:`TASK_TYPES`. |
| `TaskType` | re-export of [`opensmartroute.signals.ontology.TaskType`](#opensmartroutesignalsontology) | One leaf of the task ontology: family, description, the actions it maps to and template phrasings. |
| `TaskTypeSignal` | class | Ontology task type. Uses the learned classifier when one is loaded, else maps the. |
| `TriggerRule` | re-export of [`opensmartroute.signals.uncertainty.TriggerRule`](#opensmartroutesignalsuncertainty) | ``feature >= threshold`` (or ``<=`` when ``below=True``) fires ``action``. |
| `UncertaintyGate` | re-export of [`opensmartroute.signals.uncertainty.UncertaintyGate`](#opensmartroutesignalsuncertainty) | A cascade quality gate built from response uncertainty. |
| `VerbalisedDifficultySignal` | re-export of [`opensmartroute.signals.uncertainty.VerbalisedDifficultySignal`](#opensmartroutesignalsuncertainty) | Blend a verbalised difficulty into ``complexity`` and ``reasoning_need``. |
| `WorkflowSignal` | re-export of [`opensmartroute.signals.events.WorkflowSignal`](#opensmartroutesignalsevents) | Detect a workflow trigger (``context["workflow"]`` or "run the <name> workflow\|process\|..."). |
| `extract_signals` | function `(request: RouteRequest, extractors: list[SignalExtractor] \| None=None)` | Run the extractor chain (``DEFAULT_EXTRACTORS`` unless given) and return the populated :class:`Signals`. |
| `learned_extractors` | function `(bundle: SignalModelBundle)` | Extractor pipeline with the trained :class:`SignalModelBundle` plugged in. |
| `load_training_rows` | re-export of [`opensmartroute.signals.models.load_training_rows`](#opensmartroutesignalsmodels) | Read JSONL training rows. Accepts eval-dataset rows too (``prompt``/``text``,. |
| `parse_difficulty` | re-export of [`opensmartroute.signals.uncertainty.parse_difficulty`](#opensmartroutesignalsuncertainty) | Map a verbalised difficulty (number, ``"7/10"``, ``"hard"``) to [0, 1]; ``None`` if unparseable. |
| `parse_event` | re-export of [`opensmartroute.signals.events.parse_event`](#opensmartroutesignalsevents) | Map an event name (``subject.verb``, ``subject:verb`` or ``subject_verb``) onto domains / actions. |
| `response_uncertainty` | re-export of [`opensmartroute.signals.uncertainty.response_uncertainty`](#opensmartroutesignalsuncertainty) | Post-hoc uncertainty features for a set of sampled answers. |
| `semantic_entropy` | re-export of [`opensmartroute.signals.uncertainty.semantic_entropy`](#opensmartroutesignalsuncertainty) | Entropy (nats) over meaning clusters of sampled answers; 0 when every sample agrees. |
| `synthesize_dataset` | re-export of [`opensmartroute.signals.models.synthesize_dataset`](#opensmartroutesignalsmodels) | Generate labelled prompts from ontology templates x topics with light noise. |

## `opensmartroute.signals.events`

Source: [src/opensmartroute/signals/events.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/signals/events.py)

Event- and workflow-driven signals for agentic inputs.

| Name | Kind | Summary |
|---|---|---|
| `EVENT_LEXICON` | constant | Event name -> (domains, actions). Keys are matched exactly, then by ``subject.*`` prefix. |
| `EVENT_SUBJECTS` | constant | Generic fallbacks when the exact event / prefix is unknown: subject -> domain, verb -> action. |
| `EVENT_VERBS` | constant | event verb -> action when the exact event / prefix is unknown. |
| `EventInfo` | class | Parsed event: raw name, subject, verb and the domains / actions it implies. |
| `EventSignal` | class | Domains / actions from ``context["event"]`` and ``context["intent"]`` (event-driven requests). |
| `WorkflowSignal` | class | Detect a workflow trigger (``context["workflow"]`` or "run the <name> workflow\|process\|..."). |
| `parse_event` | function `(name: str)` | Map an event name (``subject.verb``, ``subject:verb`` or ``subject_verb``) onto domains / actions. |

## `opensmartroute.signals.models`

Source: [src/opensmartroute/signals/models.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/signals/models.py)

Learned signal models: hashed n-gram linear models with no dependencies.

| Name | Kind | Summary |
|---|---|---|
| `TASK_PRIORS` | constant | name: (difficulty, reasoning_need, expected_output_tokens). |
| `HashedClassifier` | class | Multinomial logistic regression over hashed features (sparse weights per class). |
| `HashedFeaturizer` | class | Hashing-trick sparse features: words, bigrams, character n-grams and a few numeric text statistics. |
| `HashedRegressor` | class | Squared-loss linear regressor on hashed features; output clipped to [lo, hi]. |
| `SignalModelBundle` | class | All learned signal models together, persisted as one JSON document. |
| `load_training_rows` | function `(path: str \| Path)` | Read JSONL training rows. Accepts eval-dataset rows too (``prompt``/``text``,. |
| `synthesize_dataset` | function `(ontology: TaskOntology=DEFAULT_ONTOLOGY, topics: Sequence[str] \| None=None, per_template: int=6, seed: int=0)` | Generate labelled prompts from ontology templates x topics with light noise. |

## `opensmartroute.signals.ontology`

Source: [src/opensmartroute/signals/ontology.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/signals/ontology.py)

Task ontology: families -> types -> subtypes, with an orthogonal *domain* axis.

| Name | Kind | Summary |
|---|---|---|
| `DEFAULT_ONTOLOGY` | constant | the built-in TASK_TYPES ontology used by TaskTypeSignal. |
| `TASK_TYPES` | constant | ---------------------------------------------------------------- transformation. |
| `TaskOntology` | class | Lookup helpers over :data:`TASK_TYPES`. |
| `TaskType` | class | One leaf of the task ontology: family, description, the actions it maps to and template phrasings. |

## `opensmartroute.signals.uncertainty`

Source: [src/opensmartroute/signals/uncertainty.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/signals/uncertainty.py)

Uncertainty signals that come from *outside the query text*.

| Name | Kind | Summary |
|---|---|---|
| `HEDGES` | constant | Hedging phrases counted in a draft response (uncertainty evidence). |
| `SELF_CORRECTIONS` | constant | Self-correction markers counted in a draft response. |
| `DraftResponseSignal` | class | Query-response mixed representation: run a cheap drafter and expose draft features. |
| `EventTrigger` | class | Event-triggered invocation: evaluate uncertainty features against rules and return the. |
| `TriggerRule` | class | ``feature >= threshold`` (or ``<=`` when ``below=True``) fires ``action``. |
| `UncertaintyGate` | class | A cascade quality gate built from response uncertainty. |
| `VerbalisedDifficultySignal` | class | Blend a verbalised difficulty into ``complexity`` and ``reasoning_need``. |
| `draft_features` | function `(text: str, draft: str, expected_tokens: int=0)` | Features of a cheap draft answer relative to the query: hedging, refusal, self-correction,. |
| `parse_difficulty` | function `(value: Any)` | Map a verbalised difficulty (number, ``"7/10"``, ``"hard"``) to [0, 1]; ``None`` if unparseable. |
| `response_uncertainty` | function `(samples: list[str], embedder: Embedder \| None=None, threshold: float=0.85, p_true: float \| None=None)` | Post-hoc uncertainty features for a set of sampled answers. |
| `semantic_clusters` | function `(samples: Iterable[str], embedder: Embedder \| None=None, threshold: float=0.85)` | Greedy single-link clustering of answers into meaning classes; returns a cluster id per sample. |
| `semantic_entropy` | function `(samples: Iterable[str], embedder: Embedder \| None=None, threshold: float=0.85)` | Entropy (nats) over meaning clusters of sampled answers; 0 when every sample agrees. |

## `opensmartroute.stack`

Source: [src/opensmartroute/stack.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/stack.py)

Declarative *stacks*: one document that describes a whole routing setup.

| Name | Kind | Summary |
|---|---|---|
| `REGISTRY_SCHEME` | constant | : Scheme of marketplace references inside ``imports`` (``registry://<slug>[@<version>]``). |
| `STACK_VERSION` | constant | : Current stack document version (the ``osr`` field). |
| `Resolver` | constant | : Resolver for ``registry://`` imports: takes the reference (without the scheme) and returns the stack document. |
| `Stack` | class | A resolved stack: every import merged, ready to build a router. |
| `StackChange` | class | One line of a plan: ``add`` / ``change`` / ``remove`` of a target, rule, setting or objective weight. |
| `StackPlan` | class | What applying ``desired`` would change compared with ``current`` (``osr stack plan``). |
| `dump_stack` | function `(doc: dict[str, Any], fmt: str \| None=None)` | Serialise a stack document: ``fmt`` = ``"yaml"`` \| ``"json"``; default YAML when PyYAML is installed. |
| `is_stack` | function `(doc: Any)` | True when ``doc`` looks like a stack document (``kind: stack`` or an ``osr`` version with stack sections). |
| `load_stack` | function `(source: str \| Path \| dict[str, Any], *, resolver: Resolver \| None=None, settings: Settings \| None=None, _seen: frozenset[str]=frozenset())` | Load and resolve a stack (file path, ``registry://`` reference or inline mapping), imports first. |
| `plan_stack` | function `(desired: Stack, current: Stack \| None=None)` | Diff ``desired`` against ``current`` (``None`` = nothing deployed: everything is an ``add``). |
| `starter_stack` | function `(name: str='starter', description: str='')` | A minimal, valid stack document to start from (what the marketplace publish wizard pre-fills). |
| `validate_stack` | function `(doc: Any)` | Validate a stack document; returns human-readable problems (empty list = valid). Imports are not resolved. |

## `opensmartroute.strategies`

Source: [src/opensmartroute/strategies/__init__.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/strategies/__init__.py)

| Name | Kind | Summary |
|---|---|---|
| `DEFAULT_RULES` | re-export of [`opensmartroute.strategies.protocol.DEFAULT_RULES`](#opensmartroutestrategiesprotocol) | Built-in risk / budget ladder: handoff > debate > aggregate > cascade > single. |
| `AggregateResult` | re-export of [`opensmartroute.strategies.aggregate.AggregateResult`](#opensmartroutestrategiesaggregate) | Outcome of :class:`MixtureOfAgents`: the final response, participants, winner and agreement ratio. |
| `AnnotatorPool` | re-export of [`opensmartroute.strategies.human.AnnotatorPool`](#opensmartroutestrategieshuman) | Skill estimation from agreement (Dawid-Skene EM) and quorum selection. Labels usually arrive from. |
| `AnnotatorSkill` | re-export of [`opensmartroute.strategies.human.AnnotatorSkill`](#opensmartroutestrategieshuman) | Per-domain Beta accuracy estimate, cost and latency of one human annotator. |
| `AuctionResult` | re-export of [`opensmartroute.strategies.auction.AuctionResult`](#opensmartroutestrategiesauction) | Winner, second-price payment and the per-bidder surplus / corrected claims of one auction. |
| `AuctionStrategy` | re-export of [`opensmartroute.strategies.auction.AuctionStrategy`](#opensmartroutestrategiesauction) | Error-aware reverse auction: bias-corrected bids, highest surplus wins, second-price payment. |
| `BanditStrategy` | re-export of [`opensmartroute.strategies.bandit.BanditStrategy`](#opensmartroutestrategiesbandit) | Thompson-sampling Beta bandit per (context, target); context = dominant domain (+ plan role). |
| `BeliefTracker` | re-export of [`opensmartroute.strategies.cascade.BeliefTracker`](#opensmartroutestrategiescascade) | AutoMix-style POMDP belief over the hidden state "current answer is correct". |
| `BudgetVariant` | re-export of [`opensmartroute.strategies.elastic.BudgetVariant`](#opensmartroutestrategieselastic) | One budget of an elastic model. ``quality`` and ``cost_scale`` are relative to the parent. |
| `CacheEntry` | re-export of [`opensmartroute.strategies.semantic_cache.CacheEntry`](#opensmartroutestrategiessemantic_cache) | A cached response keyed by the embedding of its prompt, with source target, quality, timestamp and hits. |
| `CacheHit` | re-export of [`opensmartroute.strategies.semantic_cache.CacheHit`](#opensmartroutestrategiessemantic_cache) | The matched :class:`CacheEntry` and its similarity to the query. |
| `CapabilityStrategy` | re-export of [`opensmartroute.strategies.capability.CapabilityStrategy`](#opensmartroutestrategiescapability) | Declarative fit: domain / action overlap, complexity band, language, modality and quality prior. |
| `Cascade` | re-export of [`opensmartroute.strategies.cascade.Cascade`](#opensmartroutestrategiescascade) | Execute ranked targets in planner order (cheapest / MDP / POMDP), stopping when the quality gate passes. |
| `CascadePlanner` | re-export of [`opensmartroute.strategies.cascade.CascadePlanner`](#opensmartroutestrategiescascade) | Finite-horizon MDP over an ordered cascade with a stop action after each step. |
| `CascadeResult` | re-export of [`opensmartroute.strategies.cascade.CascadeResult`](#opensmartroutestrategiescascade) | Final response of a cascade with every step, the planned order and the planner's expected value. |
| `CascadeStep` | re-export of [`opensmartroute.strategies.cascade.CascadeStep`](#opensmartroutestrategiescascade) | One executed rung of a cascade: gate quality, latency, cost, acceptance and POMDP belief. |
| `DeferStrategy` | re-export of [`opensmartroute.strategies.defer.DeferStrategy`](#opensmartroutestrategiesdefer) | Learning-to-defer: scores human targets by risk, PII, escalation intent, frustration and model uncertainty. |
| `EdgeCloudStrategy` | re-export of [`opensmartroute.strategies.edge.EdgeCloudStrategy`](#opensmartroutestrategiesedge) | Edge vs cloud tier choice: learned edge competence per complexity bucket vs upload / decode penalties. |
| `EffortStrategy` | re-export of [`opensmartroute.strategies.defer.EffortStrategy`](#opensmartroutestrategiesdefer) | Match a target's reasoning ``effort_level`` to ``signals.reasoning_need``; penalise over- and under-thinking. |
| `EscalationDecision` | re-export of [`opensmartroute.strategies.escalation.EscalationDecision`](#opensmartroutestrategiesescalation) | Verdict after a streamed chunk: continue / escalate / done with the competence estimate and reason. |
| `EscalationResult` | re-export of [`opensmartroute.strategies.modality.EscalationResult`](#opensmartroutestrategiesmodality) | Outcome of :class:`ModalityEscalation`: which target answered, whether it escalated, confidence, cost. |
| `HiddenStateStrategy` | re-export of [`opensmartroute.strategies.probe.HiddenStateStrategy`](#opensmartroutestrategiesprobe) | Dirichlet probe over a dense request representation; confidence drops with epistemic uncertainty. |
| `HumanRoutingStrategy` | re-export of [`opensmartroute.strategies.human.HumanRoutingStrategy`](#opensmartroutestrategieshuman) | Scores HUMAN candidates by estimated accuracy in the request's domain. |
| `ImportanceGate` | re-export of [`opensmartroute.strategies.memory.ImportanceGate`](#opensmartroutestrategiesmemory) | Hashed logistic gate: P(item will be used later \| text, hint). |
| `LLMJudgeStrategy` | re-export of [`opensmartroute.strategies.llm_judge.LLMJudgeStrategy`](#opensmartroutestrategiesllm_judge) | LLM-as-router with optional **score calibration**. |
| `MemoryItem` | re-export of [`opensmartroute.strategies.memory.MemoryItem`](#opensmartroutestrategiesmemory) | One stored memory: text, size, turn written, learned importance gate, tier and usage counters. |
| `MemoryRouter` | re-export of [`opensmartroute.strategies.memory.MemoryRouter`](#opensmartroutestrategiesmemory) | Routes memory writes to tiers under budgets and recalls the most valuable items per token. |
| `MemoryTier` | re-export of [`opensmartroute.strategies.memory.MemoryTier`](#opensmartroutestrategiesmemory) | A storage tier: token capacity, read / write cost per token, latency and the minimum item value it accepts. |
| `MixtureOfAgents` | re-export of [`opensmartroute.strategies.aggregate.MixtureOfAgents`](#opensmartroutestrategiesaggregate) | Route-or-aggregate switch: below a confidence threshold call the top-k alternatives and aggregate. |
| `ModalityEscalation` | re-export of [`opensmartroute.strategies.modality.ModalityEscalation`](#opensmartroutestrategiesmodality) | Try the text-only target on the surrogate first; escalate to the multimodal target on low confidence. |
| `ModalityStrategy` | re-export of [`opensmartroute.strategies.modality.ModalityStrategy`](#opensmartroutestrategiesmodality) | Coverage of the request's modalities by each candidate, with surrogate discounts. |
| `MultiRoundExecutor` | re-export of [`opensmartroute.strategies.progress.MultiRoundExecutor`](#opensmartroutestrategiesprogress) | Router-R1 style: keep routing/executing until the judge accepts or rounds run out. |
| `ProgressRouter` | re-export of [`opensmartroute.strategies.progress.ProgressRouter`](#opensmartroutestrategiesprogress) | Route each step of a task with trajectory context. |
| `ProtocolChoice` | re-export of [`opensmartroute.strategies.protocol.ProtocolChoice`](#opensmartroutestrategiesprotocol) | The protocol picked for a request, the risk that drove it, the reason and the matching rule. |
| `ProtocolPolicy` | re-export of [`opensmartroute.strategies.protocol.ProtocolPolicy`](#opensmartroutestrategiesprotocol) | Ordered rule table from (risk, budget, task type) to a protocol, with per-protocol ledgers. |
| `ProtocolRule` | re-export of [`opensmartroute.strategies.protocol.ProtocolRule`](#opensmartroutestrategiesprotocol) | First matching rule wins. ``None`` bounds are open. |
| `QuorumPlan` | re-export of [`opensmartroute.strategies.human.QuorumPlan`](#opensmartroutestrategieshuman) | A chosen set of annotators with the quorum's majority accuracy, total cost and latency. |
| `RecallResult` | re-export of [`opensmartroute.strategies.memory.RecallResult`](#opensmartroutestrategiesmemory) | Items recalled for a turn with their total tokens, read cost and latency. |
| `Round` | re-export of [`opensmartroute.strategies.progress.Round`](#opensmartroutestrategiesprogress) | One route -> execute -> judge iteration of :class:`MultiRoundExecutor`. |
| `RoundResult` | re-export of [`opensmartroute.strategies.progress.RoundResult`](#opensmartroutestrategiesprogress) | Final response of a multi-round run with every round and the shared task id. |
| `Rule` | re-export of [`opensmartroute.strategies.rules.Rule`](#opensmartroutestrategiesrules) | If all `when` conditions match, boost `prefer` targets and penalise `avoid`. |
| `RulesStrategy` | re-export of [`opensmartroute.strategies.rules.RulesStrategy`](#opensmartroutestrategiesrules) | Applies declarative :class:`Rule` preferences (prefer / avoid / pin) when their ``when`` conditions match. |
| `SelfEscalation` | re-export of [`opensmartroute.strategies.escalation.SelfEscalation`](#opensmartroutestrategiesescalation) | Streaming competence monitor with Bayesian optimal stopping. |
| `SemanticCache` | re-export of [`opensmartroute.strategies.semantic_cache.SemanticCache`](#opensmartroutestrategiessemantic_cache) | Embedding-keyed LRU cache with TTL and a similarity threshold. Safe to share across threads:. |
| `SemanticCacheStrategy` | re-export of [`opensmartroute.strategies.semantic_cache.SemanticCacheStrategy`](#opensmartroutestrategiessemantic_cache) | Scores the cache target by calibrated hit quality; leaves real targets to the other strategies. |
| `SessionAffinityStrategy` | re-export of [`opensmartroute.strategies.session.SessionAffinityStrategy`](#opensmartroutestrategiessession) | Prefer the target already serving ``context["session_id"]`` unless the intent shifted or it failed. |
| `SessionState` | re-export of [`opensmartroute.strategies.session.SessionState`](#opensmartroutestrategiessession) | What the strategy remembers about one session. |
| `SimilarityStrategy` | re-export of [`opensmartroute.strategies.similarity.SimilarityStrategy`](#opensmartroutestrategiessimilarity) | Embed the request and each target's examples / description; score by best and top-k mean similarity. |
| `SpeculativeCascade` | re-export of [`opensmartroute.strategies.speculative.SpeculativeCascade`](#opensmartroutestrategiesspeculative) | Two-target cascade that overlaps the draft and the strong call when it pays. |
| `SpeculativeResult` | re-export of [`opensmartroute.strategies.speculative.SpeculativeResult`](#opensmartroutestrategiesspeculative) | Outcome of a speculative run: chosen mode, whether the draft was accepted, latency and cost. |
| `Strategy` | re-export of [`opensmartroute.strategies.base.Strategy`](#opensmartroutestrategiesbase) | Scores each candidate target in [0, 1] and explains why. |
| `StreamResult` | re-export of [`opensmartroute.strategies.escalation.StreamResult`](#opensmartroutestrategiesescalation) | Text consumed by :func:`wrap_stream`, whether it escalated and the final decision. |
| `TaskProgress` | re-export of [`opensmartroute.strategies.progress.TaskProgress`](#opensmartroutestrategiesprogress) | Where a multi-step task stands: step index, budget spent, failures in a row, last target, history. |
| `TaskTableStrategy` | re-export of [`opensmartroute.strategies.task_table.TaskTableStrategy`](#opensmartroutestrategiestask_table) | Static ``task_type -> target -> quality`` table with family and prior fallbacks; learns from outcomes. |
| `TokenBudgetStrategy` | re-export of [`opensmartroute.strategies.elastic.TokenBudgetStrategy`](#opensmartroutestrategieselastic) | Score budgeted siblings by fit between the budget and the tokens the answer needs. |
| `decide_effort` | re-export of [`opensmartroute.strategies.defer.decide_effort`](#opensmartroutestrategiesdefer) | Pick the effort level whose numeric value is closest to the reasoning need. |
| `default_bid` | re-export of [`opensmartroute.strategies.auction.default_bid`](#opensmartroutestrategiesauction) | Catalogue-derived bid: quality prior as the claim, unit cost per 1k tokens x tokens as the price. |
| `default_strategies` | function `(seed: int \| None=None, settings: Settings \| None=None, state_dir: str \| Path \| None=None)` | The zero-configuration ensemble: capability fit + example similarity + Thompson bandit. |
| `expand_elastic` | re-export of [`opensmartroute.strategies.elastic.expand_elastic`](#opensmartroutestrategieselastic) | Create one sibling target per budget variant, sharing the parent's family, handler and metadata. |
| `expected_mode_costs` | re-export of [`opensmartroute.strategies.speculative.expected_mode_costs`](#opensmartroutestrategiesspeculative) | Expected weighted cost (``objective.cost`` x USD + ``objective.latency`` x seconds) per mode. |
| `failure_risk` | re-export of [`opensmartroute.strategies.protocol.failure_risk`](#opensmartroutestrategiesprotocol) | Risk in [0, 1] that a single call fails: low confidence, high complexity and reasoning need,. |
| `hashing_embedder` | re-export of [`opensmartroute.strategies.similarity.hashing_embedder`](#opensmartroutestrategiessimilarity) | Word + bigram hashing embedder. Deterministic, zero deps, decent for routing. |
| `majority_vote` | re-export of [`opensmartroute.strategies.aggregate.majority_vote`](#opensmartroutestrategiesaggregate) | Largest meaning cluster wins; returns ``(answer, winner_target_id, agreement)``. |
| `quorum_accuracy` | re-export of [`opensmartroute.strategies.human.quorum_accuracy`](#opensmartroutestrategieshuman) | P(weighted majority is correct) for independent annotators with the given accuracies. |
| `request_modalities` | re-export of [`opensmartroute.strategies.modality.request_modalities`](#opensmartroutestrategiesmodality) | Modalities a request carries and which of them have a text surrogate in ``context``. |
| `wrap_stream` | re-export of [`opensmartroute.strategies.escalation.wrap_stream`](#opensmartroutestrategiesescalation) | Consume ``chunks`` until the monitor escalates or the stream ends. |

## `opensmartroute.strategies.aggregate`

Source: [src/opensmartroute/strategies/aggregate.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/strategies/aggregate.py)

Routing / aggregation switch (Mixture-of-Agents, Wang et al. 2024; JiSi 2601.01330).

| Name | Kind | Summary |
|---|---|---|
| `AggregateResult` | class | Outcome of :class:`MixtureOfAgents`: the final response, participants, winner and agreement ratio. |
| `Aggregator` | constant | (request, [(target_id, answer)]) -> answer. |
| `MixtureOfAgents` | class | Route-or-aggregate switch: below a confidence threshold call the top-k alternatives and aggregate. |
| `Participant` | class | One target's contribution to an aggregate: response, cost, latency, error and whether it agreed. |
| `majority_vote` | function `(request: RouteRequest, answers: list[tuple[str, Any]], embedder: Embedder \| None=None, threshold: float=0.85)` | Largest meaning cluster wins; returns ``(answer, winner_target_id, agreement)``. |

## `opensmartroute.strategies.auction`

Source: [src/opensmartroute/strategies/auction.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/strategies/auction.py)

Error-aware reverse auction across providers (EA-RAM 2608.12719).

| Name | Kind | Summary |
|---|---|---|
| `AuctionResult` | class | Winner, second-price payment and the per-bidder surplus / corrected claims of one auction. |
| `AuctionStrategy` | class | Error-aware reverse auction: bias-corrected bids, highest surplus wins, second-price payment. |
| `BidFn` | constant | -> (claimed P(success), price). |
| `BidderRecord` | class | Calibration ledger of one bidder: signed claim bias and a Beta record of realised success. |
| `default_bid` | function `(target: RouteTarget, request: RouteRequest, signals: Signals)` | Catalogue-derived bid: quality prior as the claim, unit cost per 1k tokens x tokens as the price. |

## `opensmartroute.strategies.bandit`

Source: [src/opensmartroute/strategies/bandit.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/strategies/bandit.py)

Online-learning strategy: contextual Thompson-sampling bandit.

| Name | Kind | Summary |
|---|---|---|
| `BanditStrategy` | class | Thompson-sampling Beta bandit per (context, target); context = dominant domain (+ plan role). |

## `opensmartroute.strategies.base`

Source: [src/opensmartroute/strategies/base.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/strategies/base.py)

Strategy interface.

| Name | Kind | Summary |
|---|---|---|
| `Strategy` | class | Scores each candidate target in [0, 1] and explains why. |
| `role_of` | function `(signals: Signals)` | Plan-slot role (``persona`` / ``skill`` / ``llm``) the router is currently filling, or. |
| `memory_key` | function `(request_id: str, role: str \| None)` | Key for per-request strategy memory: the same request is scored once per plan role, and. |

## `opensmartroute.strategies.capability`

Source: [src/opensmartroute/strategies/capability.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/strategies/capability.py)

Capability-fit strategy: match request signals to a target's declared capabilities.

| Name | Kind | Summary |
|---|---|---|
| `CapabilityStrategy` | class | Declarative fit: domain / action overlap, complexity band, language, modality and quality prior. |

## `opensmartroute.strategies.cascade`

Source: [src/opensmartroute/strategies/cascade.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/strategies/cascade.py)

Cascade execution (FrugalGPT / Router-R1 multi-round / AutoMix POMDP).

| Name | Kind | Summary |
|---|---|---|
| `CascadeStep` | class | One executed rung of a cascade: gate quality, latency, cost, acceptance and POMDP belief. |
| `CascadeResult` | class | Final response of a cascade with every step, the planned order and the planner's expected value. |
| `CascadePlanner` | class | Finite-horizon MDP over an ordered cascade with a stop action after each step. |
| `BeliefTracker` | class | AutoMix-style POMDP belief over the hidden state "current answer is correct". |
| `Cascade` | class | Execute ranked targets in planner order (cheapest / MDP / POMDP), stopping when the quality gate passes. |

## `opensmartroute.strategies.defer`

Source: [src/opensmartroute/strategies/defer.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/strategies/defer.py)

Defer-to-human and effort ("think or not") strategies.

| Name | Kind | Summary |
|---|---|---|
| `DeferStrategy` | class | Learning-to-defer: scores human targets by risk, PII, escalation intent, frustration and model uncertainty. |
| `EffortStrategy` | class | Match a target's reasoning ``effort_level`` to ``signals.reasoning_need``; penalise over- and under-thinking. |
| `decide_effort` | function `(signals: Signals, levels: tuple[str, ...]=('none', 'low', 'medium', 'high'))` | Pick the effort level whose numeric value is closest to the reasoning need. |

## `opensmartroute.strategies.edge`

Source: [src/opensmartroute/strategies/edge.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/strategies/edge.py)

Edge-cloud token-aware routing (Pro-Router 2608.28726; RelayLLM 2601.05167).

| Name | Kind | Summary |
|---|---|---|
| `EdgeCloudStrategy` | class | Edge vs cloud tier choice: learned edge competence per complexity bucket vs upload / decode penalties. |

## `opensmartroute.strategies.elastic`

Source: [src/opensmartroute/strategies/elastic.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/strategies/elastic.py)

Token-budget-aware routing to elastic / many-in-one models (Nemotron Elastic 2511.16664; Star 2605.07182).

| Name | Kind | Summary |
|---|---|---|
| `BudgetVariant` | class | One budget of an elastic model. ``quality`` and ``cost_scale`` are relative to the parent. |
| `TokenBudgetStrategy` | class | Score budgeted siblings by fit between the budget and the tokens the answer needs. |
| `expand_elastic` | function `(parent: RouteTarget, variants: list[BudgetVariant \| dict[str, Any]])` | Create one sibling target per budget variant, sharing the parent's family, handler and metadata. |

## `opensmartroute.strategies.escalation`

Source: [src/opensmartroute/strategies/escalation.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/strategies/escalation.py)

Bayesian self-escalation during generation (2608.24087).

| Name | Kind | Summary |
|---|---|---|
| `REFUSALS` | constant | Refusal phrases that lower the streaming competence posterior. |
| `EscalationDecision` | class | Verdict after a streamed chunk: continue / escalate / done with the competence estimate and reason. |
| `SelfEscalation` | class | Streaming competence monitor with Bayesian optimal stopping. |
| `StreamResult` | class | Text consumed by :func:`wrap_stream`, whether it escalated and the final decision. |
| `wrap_stream` | function `(chunks: Iterable[str], monitor: SelfEscalation \| None=None)` | Consume ``chunks`` until the monitor escalates or the stream ends. |

## `opensmartroute.strategies.human`

Source: [src/opensmartroute/strategies/human.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/strategies/human.py)

Routing among human annotators and experts (QUORUM 2608.27974; Dawid-Skene 1979).

| Name | Kind | Summary |
|---|---|---|
| `AnnotatorPool` | class | Skill estimation from agreement (Dawid-Skene EM) and quorum selection. Labels usually arrive from. |
| `AnnotatorSkill` | class | Per-domain Beta accuracy estimate, cost and latency of one human annotator. |
| `HumanRoutingStrategy` | class | Scores HUMAN candidates by estimated accuracy in the request's domain. |
| `QuorumPlan` | class | A chosen set of annotators with the quorum's majority accuracy, total cost and latency. |
| `quorum_accuracy` | function `(accuracies: Sequence[float])` | P(weighted majority is correct) for independent annotators with the given accuracies. |

## `opensmartroute.strategies.llm_judge`

Source: [src/opensmartroute/strategies/llm_judge.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/strategies/llm_judge.py)

Generative routing: let an LLM act as the router (Router-R1 / LLM-as-judge).

| Name | Kind | Summary |
|---|---|---|
| `PROMPT` | constant | Prompt template for the judge; formatted with complexity, domains, actions, the catalogue and the request. |
| `LLMJudgeStrategy` | class | LLM-as-router with optional **score calibration**. |

## `opensmartroute.strategies.memory`

Source: [src/opensmartroute/strategies/memory.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/strategies/memory.py)

Memory-tier routing for agents (BudgetMem 2602.06025; Gated-Memory Routing 2609.00237).

| Name | Kind | Summary |
|---|---|---|
| `ImportanceGate` | class | Hashed logistic gate: P(item will be used later \| text, hint). |
| `MemoryItem` | class | One stored memory: text, size, turn written, learned importance gate, tier and usage counters. |
| `MemoryRouter` | class | Routes memory writes to tiers under budgets and recalls the most valuable items per token. |
| `MemoryTier` | class | A storage tier: token capacity, read / write cost per token, latency and the minimum item value it accepts. |
| `RecallResult` | class | Items recalled for a turn with their total tokens, read cost and latency. |

## `opensmartroute.strategies.modality`

Source: [src/opensmartroute/strategies/modality.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/strategies/modality.py)

Multimodal routing and modality escalation (LatentRouter 2605.11301; modality escalation.

| Name | Kind | Summary |
|---|---|---|
| `EscalationResult` | class | Outcome of :class:`ModalityEscalation`: which target answered, whether it escalated, confidence, cost. |
| `ModalityEscalation` | class | Try the text-only target on the surrogate first; escalate to the multimodal target on low confidence. |
| `ModalityStrategy` | class | Coverage of the request's modalities by each candidate, with surrogate discounts. |
| `request_modalities` | function `(request: RouteRequest)` | Modalities a request carries and which of them have a text surrogate in ``context``. |

## `opensmartroute.strategies.probe`

Source: [src/opensmartroute/strategies/probe.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/strategies/probe.py)

Hidden-state routing with a Dirichlet probe (ProbeDirichlet, RouterXBench 2602.11877).

| Name | Kind | Summary |
|---|---|---|
| `HiddenStateStrategy` | class | Dirichlet probe over a dense request representation; confidence drops with epistemic uncertainty. |
| `StateFn` | constant | request -> dense feature / hidden-state vector. |

## `opensmartroute.strategies.progress`

Source: [src/opensmartroute/strategies/progress.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/strategies/progress.py)

Agentic trajectories: per-step routing and multi-round execution.

| Name | Kind | Summary |
|---|---|---|
| `MultiRoundExecutor` | class | Router-R1 style: keep routing/executing until the judge accepts or rounds run out. |
| `ProgressRouter` | class | Route each step of a task with trajectory context. |
| `Round` | class | One route -> execute -> judge iteration of :class:`MultiRoundExecutor`. |
| `RoundResult` | class | Final response of a multi-round run with every round and the shared task id. |
| `TaskProgress` | class | Where a multi-step task stands: step index, budget spent, failures in a row, last target, history. |

## `opensmartroute.strategies.protocol`

Source: [src/opensmartroute/strategies/protocol.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/strategies/protocol.py)

Collaboration-protocol selection (2608.14927).

| Name | Kind | Summary |
|---|---|---|
| `DEFAULT_RULES` | constant | Built-in risk / budget ladder: handoff > debate > aggregate > cascade > single. |
| `PROTOCOLS` | constant | in escalation order. |
| `Protocol` | constant | execution protocols. |
| `ProtocolChoice` | class | The protocol picked for a request, the risk that drove it, the reason and the matching rule. |
| `ProtocolPolicy` | class | Ordered rule table from (risk, budget, task type) to a protocol, with per-protocol ledgers. |
| `ProtocolRule` | class | First matching rule wins. ``None`` bounds are open. |
| `failure_risk` | function `(decision: RouteDecision \| None, signals: Signals \| None=None, uncertainty: float \| None=None)` | Risk in [0, 1] that a single call fails: low confidence, high complexity and reasoning need,. |

## `opensmartroute.strategies.rules`

Source: [src/opensmartroute/strategies/rules.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/strategies/rules.py)

Declarative rule-based routing (Arch-Router-style domain/action preferences).

| Name | Kind | Summary |
|---|---|---|
| `Rule` | class | If all `when` conditions match, boost `prefer` targets and penalise `avoid`. |
| `RulesStrategy` | class | Applies declarative :class:`Rule` preferences (prefer / avoid / pin) when their ``when`` conditions match. |

## `opensmartroute.strategies.semantic_cache`

Source: [src/opensmartroute/strategies/semantic_cache.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/strategies/semantic_cache.py)

Semantic caching as a routing target (GPTCache; vLLM semantic router 2603.04444).

| Name | Kind | Summary |
|---|---|---|
| `CacheEntry` | class | A cached response keyed by the embedding of its prompt, with source target, quality, timestamp and hits. |
| `CacheHit` | class | The matched :class:`CacheEntry` and its similarity to the query. |
| `SemanticCache` | class | Embedding-keyed LRU cache with TTL and a similarity threshold. Safe to share across threads:. |
| `SemanticCacheStrategy` | class | Scores the cache target by calibrated hit quality; leaves real targets to the other strategies. |

## `opensmartroute.strategies.session`

Source: [src/opensmartroute/strategies/session.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/strategies/session.py)

Session affinity: keep a conversation with the target that is already serving it.

| Name | Kind | Summary |
|---|---|---|
| `SessionAffinityStrategy` | class | Prefer the target already serving ``context["session_id"]`` unless the intent shifted or it failed. |
| `SessionState` | class | What the strategy remembers about one session. |

## `opensmartroute.strategies.similarity`

Source: [src/opensmartroute/strategies/similarity.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/strategies/similarity.py)

Similarity-based routing (UniRoute / GraphRouter flavour).

| Name | Kind | Summary |
|---|---|---|
| `hashing_embedder` | function `(dim: int=512)` | Word + bigram hashing embedder. Deterministic, zero deps, decent for routing. |
| `cosine` | function `(a: list[float], b: list[float])` | Cosine similarity; zero vectors are treated as unit norm (no division by zero). |
| `SimilarityStrategy` | class | Embed the request and each target's examples / description; score by best and top-k mean similarity. |

## `opensmartroute.strategies.speculative`

Source: [src/opensmartroute/strategies/speculative.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/strategies/speculative.py)

Speculative (draft-based) cascades (speculative cascades, Narasimhan et al. 2024; Differential.

| Name | Kind | Summary |
|---|---|---|
| `SpeculativeCascade` | class | Two-target cascade that overlaps the draft and the strong call when it pays. |
| `SpeculativeResult` | class | Outcome of a speculative run: chosen mode, whether the draft was accepted, latency and cost. |
| `expected_mode_costs` | function `(p_accept: float, draft: RouteTarget, strong: RouteTarget, objective: Objective, *, tokens: int=500, cancellable: bool=False, verify_ms: float=0.0)` | Expected weighted cost (``objective.cost`` x USD + ``objective.latency`` x seconds) per mode. |

## `opensmartroute.strategies.task_table`

Source: [src/opensmartroute/strategies/task_table.py](https://github.com/isathish/OpenSmartRoute/blob/main/src/opensmartroute/strategies/task_table.py)

Static task table strategy (SCX Router, 2609.02292).

| Name | Kind | Summary |
|---|---|---|
| `TaskTableStrategy` | class | Static ``task_type -> target -> quality`` table with family and prior fallbacks; learns from outcomes. |

<!-- Section: Self-hosting -->

---

<!-- Page: Deploy with Docker and Helm (https://opensmartroute.ai/docs/deploy) -->

<!-- OpenSmartRoute: Deploy with Docker and Helm. Source https://github.com/isathish/OpenSmartRoute/blob/main/deploy/README.md; HTML https://opensmartroute.ai/docs/deploy -->
# Reference deployments

OpenSmartRoute runs in three shapes. All three use the same `Router`; only where it lives differs.

| Shape | When | How |
|---|---|---|
| **Library** | one Python service, in-process decisions (~0.3 ms / route) | `pip install opensmartroute` and call `Router.route()`; see [docs/SDK.md](https://opensmartroute.ai/docs/SDK.md) |
| **Sidecar** | polyglot services, per-pod learner state | run the image next to your app: `docker run -p 8000:8000 -v ./examples:/config ghcr.io/isathish/opensmartroute:1.0.0` and call `POST /route` |
| **Control plane** | many callers, shared catalogue, shared learning | the Helm chart in [helm/opensmartroute](https://github.com/isathish/OpenSmartRoute/blob/main/deploy/helm/opensmartroute) behind a `Service`/`Ingress`; state in Redis/SQL via `RouterBuilder.with_state_store` |

## Image

[Dockerfile](https://github.com/isathish/OpenSmartRoute/blob/main/deploy/Dockerfile) builds a non-root, read-only-rootfs image with the `yaml` and `server` extras:

```sh
docker build -t opensmartroute:1.0.0 -f deploy/Dockerfile .
docker run --rm -p 8000:8000 -v "$PWD/examples:/config:ro" opensmartroute:1.0.0
curl -s localhost:8000/route -H 'content-type: application/json' -d '{"text":"summarise this contract"}'
```

Configuration is by environment (`OSR_TARGETS`, `OSR_RULES`, `OSR_MODELS`, `OSR_SLM`, `OSR_STATE`,
`OSR_HOST`, `OSR_PORT`) so the same image serves both sidecar and control-plane roles;
[entrypoint.sh](https://github.com/isathish/OpenSmartRoute/blob/main/deploy/entrypoint.sh) translates them into `osr serve` arguments. `OSR_SLM` points at a
routing SLM file from `osr slm train`; it joins the strategy ensemble and is reported as the `slm`
strategy in every decision trace.

The OpenAI-compatible proxy (`/v1/chat/completions`) executes LLM targets when the container knows an
upstream: set `OSR_LLM_BASE_URL` (OpenAI, Azure OpenAI, vLLM, Ollama, OpenRouter, LiteLLM ...) and
`OSR_LLM_API_KEY` (or `OSR_LLM_API_KEY_ENV` naming another variable / `<NAME>_FILE` mount). Each LLM
target's `metadata.model` is the upstream model name; `OSR_LLM_MODEL` is the default for targets
without one. Without a base URL the proxy still routes but answers 502 for LLM targets.

Access control: the image starts open (every endpoint reachable) so it can act as a sidecar on a
private network. To require a token set `OSR_SERVER_AUTH_TOKENS=<t1>,<t2>` (or mount the list and
point `OSR_SERVER_AUTH_TOKENS_FILE` at it) and, for a hard guarantee, `OSR_SERVER_REQUIRE_AUTH=1` so
the container refuses to start without one. Mint tokens with `osr token generate`; clients send
`Authorization: Bearer <t>` or `X-API-Key: <t>`, and `osr login --url https://<host> --token <t>`
stores one for the CLI (`osr whoami` then shows `edition self-hosted`). `/healthz`, `/readyz`,
`/metrics`, `/whoami` and the OpenAPI documents stay open for probes and discovery.

Self-operation: `OSR_AUTOPILOT=1` (with `OSR_SLM` and a writable `OSR_STATE`) runs the SLM's
improvement loop inside the container - every `OSR_SLM_AUTOPILOT_INTERVAL_S` seconds (default one
hour) and early when the drift monitor sees the served success rate drop, it gathers evidence
(feedback joined to prompts, cached datasets, seeds), trains a challenger, and hot-swaps it into the
ensemble when it beats the champion on the holdout. The promoted model is written to
`$OSR_STATE/autopilot/slm.json` (or `OSR_CACHE_DIR`), which the entrypoint prefers over `OSR_SLM` on
the next start. `OSR_AUTOPILOT_ARGS` passes extra flags (`--offline`, `--source NAME`,
`--tier TIER=TARGET`, `--search QUERY`); `OSR_CATALOGUE` names a catalogue JSON to refresh. `GET /stats`
shows the `autopilot` status and `POST /autopilot/cycle` schedules a cycle now.

Observability: every request is traced (spans and events per stage, see
[docs/OBSERVABILITY.md](https://opensmartroute.ai/docs/OBSERVABILITY.md)). `GET /healthz` is the liveness probe and `GET /readyz`
the readiness probe (503 until the router can score). `GET /metrics` serves the Prometheus exposition
(`osr_route_decisions_total{target}`, `osr_route_latency_ms_bucket`, `osr_route_policy_rejections_total`,
`osr_route_errors_total`, `osr_outcome_*`, `osr_decision_cache_*`, a `targets` gauge, plus per-stage
`osr_span_duration_ms`), `GET /trace/{request_id}` the full trace of one
request (each response carries `X-OSR-Trace-Id`; send `traceparent` to join your own trace) and
`GET /events?name=route.*` the recent buffer. Tune with `OSR_OBSERVABILITY_MEMORY_EVENTS` (buffer size, `0`
off), `OSR_OBSERVABILITY_LOG_EVENTS=true` (JSON lines on stdout), `OSR_OBSERVABILITY_EVENTS_FILE`
(JSONL under the state volume), `OSR_OBSERVABILITY_OTEL=true` (OpenTelemetry, needs the `otel` extra and
the usual `OTEL_EXPORTER_*` variables) and `OSR_OBSERVABILITY_SAMPLE_RATE` under heavy load. The hosted
platform image ([platform/README.md](https://github.com/isathish/OpenSmartRoute/blob/main/platform/README.md)) reads the same variables and adds the
workspace-scoped `GET /api/v1/trace/{request_id}`, `GET /api/v1/events` and the public `GET /api/v1/status`.

## Helm

```sh
helm install router deploy/helm/opensmartroute \
  --set-file config.targets=examples/targets.yaml \
  --set-file config.rules=examples/rules.yaml
```

Key values (see [values.yaml](https://github.com/isathish/OpenSmartRoute/blob/main/deploy/helm/opensmartroute/values.yaml) for all):

- `config.targets` / `config.rules` / `config.models` -> rendered into a ConfigMap at `/config`;
  the pod restarts on change (config checksum annotation).
- `autopilot.enabled` / `autopilot.intervalSeconds` / `autopilot.args` -> self-operation as above
  (needs `config.slm` and `persistence.enabled`). The autopilot promotes models per pod: with
  `replicaCount` above 1 each replica trains and serves its own champion, so run one replica (or
  one autopilot pod behind a shared `StateStore`) when the loop is on.
- `auth.tokens` / `auth.existingSecret` / `auth.required` -> access tokens for the HTTP API (a
  chart-managed Secret, or your own with a `tokens` key), and whether the pod may start without one.
- `persistence.enabled` -> keep bandit posteriors and the feedback log on a PVC (single replica or
  RWX class). For several replicas leave it off and use the Redis/SQL `StateStore`.
- `autoscaling`, `podDisruptionBudget`, `networkPolicy`, `ingress` -> standard knobs, all off or
  conservative by default.
- Security: `runAsNonRoot`, `readOnlyRootFilesystem`, all capabilities dropped, `automountServiceAccountToken: false`.

Secrets (MCP manifest signing key `OSR_MCP_KEY`, encrypted-state key) are injected with `env`/`envFrom`
`secretKeyRef`s - never in `values.yaml`.

Endpoints: `GET /healthz`, `GET /whoami`, `GET /targets`, `GET /stats`, `POST /route`, `POST /feedback`,
`GET /v1/models`, `POST /v1/chat/completions` (OpenAI-compatible proxy, `model: "auto"`).

<!-- Section: Concepts -->

---

<!-- Page: Architecture (https://opensmartroute.ai/docs/ARCHITECTURE) -->

<!-- OpenSmartRoute: Architecture. Source https://github.com/isathish/OpenSmartRoute/blob/main/docs/ARCHITECTURE.md; HTML https://opensmartroute.ai/docs/ARCHITECTURE -->
# Architecture

OpenSmartRoute is a **decision layer**, not a gateway. Given a request and a catalogue of targets it
returns *which target should handle this, how confident we are, and why* — and it learns from what
happened next. Executing the target is optional and pluggable.

```mermaid
flowchart LR
    subgraph In
        REQ[RouteRequest]
    end
    subgraph Middleware
        G[Guard] --> T[Tenant] --> C[Cache]
    end
    subgraph Core
        S[Signals] --> P[Policy] --> ST[Strategies] --> E[Ensemble] --> D[Decision + Trace]
    end
    subgraph Learn
        O[Outcome] --> AL[AutoLearner] --> ST
        O --> H[Health] --> P
    end
    REQ --> G
    C --> S
    D --> X[Executor / handler]
    X --> O
    CAT[(TargetRegistry)] --> P
```

## Layers

| Layer | Package | Responsibility | Depends on |
|---|---|---|---|
| **Core model** | `core` | `RouteTarget`, `RouteRequest`, `RouteDecision`, `RouteTrace`, `Outcome`, `TargetRegistry` | stdlib |
| **Signals** | `signals` | Deterministic features of a request: complexity, domain/action, language, modality, PII, jailbreak, token estimate; opt-in verbalised difficulty, draft-response and response-uncertainty features (`signals.uncertainty`) | core |
| **Policy** | `policy` | Hard constraints → admissible set + rejection reasons. Never traded off. | core, signals |
| **Strategies** | `strategies` | Soft scorers ∈ [0,1] with rationale: rules, capability fit, similarity, Thompson bandit, LLM judge, cascade executor; opt-in hidden-state probe, token budgets, edge/cloud tiers, auctions; post-answer controls (self-escalation, mixture-of-agents, protocol selection) | core, signals |
| **Learning** | `learning`, `math` | Strategies that update from outcomes (IRT, Bradley–Terry, LinUCB, Markov/MDP, multi-turn history embeddings, per-user shrinkage) and the pure-math they rest on (bandits, calibration, Dirichlet probe, energy model, mixture-cure hand-off); `AutoLearner` fan-out, drift, atomic persistence; the routing SLM (`RouterSLM`, `SLMStrategy`, `distill_router`) and its champion/challenger `SelfImprover` | strategies |
| **Discovery** | `discovery`, `retrieval` | Retrieve-then-rank narrowing, schema-aware tool matching, cache-preserving tool order, skill graphs and submodular skill sets for large catalogues | core, strategies |
| **Router** | `router` | Orchestrates signals → policy → strategies → confidence-weighted ensemble → decision; optional judge escalation; optional plan (persona → skill → model) | all above |
| **Real-time** | `realtime` | Circuit breaker, token bucket, rolling budget, `HealthRegistry`; `HealthPolicy` (hard) and `HealthStrategy` (soft) | policy, strategies, math |
| **Enterprise** | `enterprise` | `RouterBuilder`, `EnterpriseRouter`, middleware chain, telemetry/state/audit ports and in-process adapters | router, realtime, learning |
| **Security** | `security` | `InputGuard` (size, normalisation, confounder gadgets), `Redactor`, `GuardMiddleware`, prompt sanitiser, secret loading | enterprise |
| **Adapters** | `adapters` | OpenAI-compatible HTTP client + glue (judge, embedder, handler); agent harnesses; MCP tools and server recommendation, A2A cards, SKILL.md and persona loaders, semantic-router import; live model catalogue (OpenRouter prices, Hugging Face cards) and web-search knowledge (https-only, byte-capped, risk-scored); optional sentence-transformers and OpenTelemetry | strategies, enterprise |
| **Config** | `config` | JSON/YAML loaders for targets and rules with validation errors | core, strategies |
| **Surface** | `aio`, `server`, `cli`, `eval` | Async façade, FastAPI app, `osr` CLI, RouterBench-style evaluation, dataset collection from the Hugging Face Hub (`eval.collect`) | router, enterprise |

Dependency direction is strictly downward in this table. `math` depends on nothing but stdlib and
is usable on its own.

## The decision pipeline, precisely

1. **Signals** — `extract_signals(request)`; ~0.1 ms; no I/O.
2. **Candidate pool** — `registry.all(primary_only=True)`, optionally filtered by `kinds`.
3. **Policy** — `Policy.filter(pool)` → `(admissible, rejections)`. `HealthPolicy` adds breaker/rate/budget
   checks (non-consuming). Empty admissible set → `NoRouteError` with the reasons.
4. **Pinned rules** — a matching `Rule(pin=True)` narrows the admissible set (Arch-Router semantics).
5. **Scoring** — every strategy returns `{target_id: StrategyScore(score, rationale, confidence)}`.
6. **Ensemble** — quality estimate $\hat q = \sum_k w_k\kappa_k s_k / \sum_k w_k\kappa_k$ where $\kappa$ is
   the strategy's self-confidence (learners report $\kappa=\min(1,n/n_0)$, so they are silent until they
   have evidence). Utility $U=\,w_q\hat q - w_c\,\tilde c - w_\ell\,\tilde\ell$ with log-min-max normalised
   cost/latency and a hard quality floor.
7. **Confidence** — softmax margin over utilities ($\tau=0.1$). If below `escalate_llm_judge_below` and a
   judge is configured, re-score once with the judge included.
8. **Decision** — top target, ranked alternatives, full trace. Optionally a `RoutePlan` where each slot
   (persona, skill, llm) is its own sub-routing over that kind; a slot is left empty when its best
   candidate's quality estimate is below `slot_quality_floor`, so irrelevant skills are never attached.
9. **Execute** (optional, `execution.py`) — `Router.run()` walks the plan: the `skill` slot's handler runs as a
   pre-processor (may rewrite the request or attach `skill_output`), persona/skill/primary `instructions` are
   composed into `context["system"]`, then the primary handler is called (`chat_handler`, a skill function, or an
   agent harness via `CallableHarness`/`HTTPHarness`/`SubprocessHarness`). One `Outcome` per participant
   (`role=None|"persona"|"skill"`, shared `task_id`) is recorded — success, latency, cost from tokens × unit cost.
10. **Learn** — `Outcome` → every strategy's `update()`, drift detectors, health registry, feedback store.

## Extension points

| To add… | Implement | Register via |
|---|---|---|
| a target | `RouteTarget(...)` or a YAML entry | `registry.add()` / `load_targets()` |
| a signal | `SignalExtractor.extract()` | `Router(extractors=[...])` |
| a hard rule | `Policy.check()` → reason or `None` | `RouterBuilder.with_policy()` |
| a scorer | `Strategy.score()` (+ `update()` if it learns) | `with_strategy(s, weight)` |
| pre/post processing | `Middleware.__call__(request, next_)` | `with_middleware()` |
| metrics/tracing | `Telemetry.on_decision/on_outcome/on_error` | `with_telemetry()` |
| span/event capture | `EventSink.emit(event)` (+ `span_start/span_end` for live bridges) | `with_tracing(sink)` / `configure_tracing(sink)` |
| persistence | `StateStore.get/put/delete` | pass to learners / caches |
| a provider | `RouteTarget.handler = chat_handler(client, model)` | `adapters.OpenAICompatClient` |

## Concurrency model

- `Router.route()` is pure with respect to router state (no writes) and safe to call from many threads.
- Learners, health registry, caches and stores take their own locks on write.
- `AsyncRouter` runs routing in the default executor and awaits coroutine handlers.
- Multi-replica: learner updates are commutative sums (Beta counts, IRT gradients are small and
  order-insensitive in practice, BT strengths, LinUCB `A`/`b`, Markov counts), so replicas can be folded
  together with `merge()` / `learning.merge_learners()`, or readers can adopt a writer's snapshot through a
  `StateStore` with `AutoLearner.refresh()`. Corrupt snapshots are quarantined, never fatal
  (`AutoLearner.quarantined`).

## Performance envelope

Measured with `python scripts/bench.py --n 200 --scale` (CPython 3.11, x86-64 Windows laptop, pure
Python, no C extensions). Numbers are wall-clock per `route()` including signal extraction, policy,
every strategy and the ensemble; re-run the script on your own hardware before sizing.

**16-target example catalogue** (`examples/targets.yaml`, `examples/eval_dataset.jsonl`):

| Configuration | p50 | p95 | p99 |
|---|---|---|---|
| rules + capability + similarity + bandit | 5.8 ms | 6.2 ms | 6.5 ms |
| + IRT + preference + LinUCB + Markov + health | 8.6 ms | 12.9 ms | 17.8 ms |

**Scaling with catalogue size** (synthetic LLM catalogue, defaults + auto-learning). Cost is linear in
admissible targets x strategies until retrieval narrowing (BM25 + hashed dense, reciprocal-rank fusion)
caps the candidate pool; the default `Settings.routing` only narrows above 500 admissible targets (to 50),
`RouterBuilder.with_retrieval(narrow_above=32, narrow_to=24)` turns it on earlier:

| Targets | Narrowing | p50 | p95 | p99 |
|---|---|---|---|---|
| 16 | default (none) | 9.2 ms | 9.5 ms | 10.0 ms |
| 16 | top-24 above 32 (none triggered) | 9.2 ms | 9.6 ms | 10.5 ms |
| 64 | default (none) | 35.7 ms | 36.6 ms | 37.1 ms |
| 64 | top-24 above 32 | 14.5 ms | 15.1 ms | 15.6 ms |
| 256 | default (none) | 142 ms | 145 ms | 158 ms |
| 256 | top-24 above 32 | 16.7 ms | 17.6 ms | 18.5 ms |
| 1024 | default (auto top-50) | 38.7 ms | 43.5 ms | 49.1 ms |
| 1024 | top-24 above 32 | 24.1 ms | 28.2 ms | 32.9 ms |

Rule of thumb: keep the scored pool at or below ~50 targets and routing stays under 25 ms p99 in pure
Python regardless of catalogue size; the retrieval stage itself is ~10 ms at 1k targets and grows
roughly linearly in the catalogue. The LLM judge, when triggered, adds one provider round-trip; keep
`escalate_llm_judge_below` low (<= 0.5) so it fires only on genuinely ambiguous requests.

## Deployment shapes

1. **Library** — import in-process. Lowest latency. State on local disk or in-memory.
2. **Sidecar** — `osr serve` per application (container image from `deploy/Dockerfile`); shared
   `StateStore` for global learning.
3. **Control plane** — one HA deployment (Helm chart in `deploy/helm/opensmartroute`); a gateway
   (LiteLLM, Envoy, aisix) calls `/route`, executes the chosen target, and posts `/feedback`.

Multi-replica learning: run one *writer* replica that learns and saves through a `StateStore`, and have
readers call `AutoLearner.refresh()` on a timer; or let every replica learn independently and fold them
together periodically with `learning.merge_learners()`. See [deploy/README.md](https://opensmartroute.ai/docs/deploy.md).

## Non-goals

Model serving, protocol translation between providers, prompt versioning, chat UI. See
[ROADMAP.md](https://opensmartroute.ai/docs/ROADMAP.md#out-of-scope).

---

<!-- Page: Mathematics (https://opensmartroute.ai/docs/MATH) -->

<!-- OpenSmartRoute: Mathematics. Source https://github.com/isathish/OpenSmartRoute/blob/main/docs/MATH.md; HTML https://opensmartroute.ai/docs/MATH -->
# Mathematical Foundations

Every routing decision in OpenSmartRoute is the output of explicit, inspectable formulas.
This document lists them, the research they come from, and where they live in the code.

## 1. The routing problem

Given a request $x$ with signal vector $\phi(x)$ and candidate set $\mathcal{T}$ (after
policy filtering), choose

$$
t^\* = \arg\max_{t \in \mathcal{T}} \; U(x,t), \qquad
U(x,t) = w_q\,\hat q(x,t) - w_c\,\tilde c_t - w_l\,\tilde \ell_t
$$

subject to $\hat q(x,t) \ge q_{\min}$ (quality floor, Hybrid-LLM). Costs and latencies are
**log-min-max normalised** so one outlier (a \$0.50/1k human queue) doesn't flatten the rest:

$$
\tilde c_t = \frac{\log(1 + s\,c_t) - \min_k \log(1 + s\,c_k)}{\max_k \log(1 + s\,c_k) - \min_k \log(1 + s\,c_k)}
$$

Code: `router.py::_norm`, `Router._rank`.

## 2. Ensemble quality estimate

Each strategy $k$ returns a score $s_k(t) \in [0,1]$ and a self-confidence $\kappa_k(t)$.
The ensemble is a confidence-weighted mean:

$$
\hat q(x,t) = \frac{\sum_k w_k\,\kappa_k(t)\,s_k(t)}{\sum_k w_k\,\kappa_k(t)}
$$

Learners report $\kappa = \min(1, n/n_0)$ so they are silent until they have data —
this is what lets rules and declared capabilities dominate at cold start and
learned models take over as evidence accumulates.

## 3. Decision confidence

$$
\text{conf} = \frac{e^{U_1/\tau}}{\sum_j e^{U_j/\tau}}, \quad \tau = 0.1
$$

Low confidence (or high normalised entropy $H/\log|\mathcal T|$) triggers the
LLM-judge escalation (Router-R1 pattern). Calibration is measured with **ECE**
(`math.estimators.expected_calibration_error`).

## 4. Online learning (auto-learn)

### 4.1 Thompson sampling (Beta–Bernoulli) — `math.bandits.ThompsonBeta`
$\theta_t \sim \mathrm{Beta}(\alpha_t,\beta_t)$; on reward $r\in[0,1]$:
$\alpha_t \mathrel{+}= r$, $\beta_t \mathrel{+}= 1-r$. Optional forgetting
$\alpha \leftarrow \alpha_0 + \gamma(\alpha-\alpha_0)$ for non-stationarity.

### 4.2 UCB1 — `math.bandits.UCB1`
$\hat\mu_t + c\sqrt{2\ln n / n_t}$ (Auer, Cesa-Bianchi & Fischer 2002).

### 4.3 LinUCB (contextual) — `math.bandits.LinUCB`, `learning.LinUCBStrategy`
Per arm $A_t = I + \sum x x^\top$, $b_t = \sum r x$, $\hat\theta_t = A_t^{-1} b_t$,
score $x^\top \hat\theta_t + \alpha\sqrt{x^\top A_t^{-1} x}$ (Li et al., WWW 2010).
Context $x$ = `learning.signal_vector` (complexity, length, PII, jailbreak, tools,
language, multimodal, domain one-hots). This is the MixLLM formulation.

### 4.4 Cost-aware bandit (Lagrangian) — `math.bandits.CostAwareBandit`
Score $= \text{inner}(t) - \lambda c_t$ with dual ascent
$\lambda \leftarrow \max(0, \lambda + \eta(\bar c - B))$ so average spend tracks budget $B$
(C2MAB-V, Dai et al. 2024).

### 4.5 Item Response Theory (2PL) — `math.irt.IRTModel`, `learning.IRTStrategy`
$$P(\text{success}\mid t,i) = \sigma\big(a_i(\theta_t - b_i)\big)$$
Target ability $\theta_t$, item (domain × difficulty bucket) difficulty $b_i$,
discrimination $a_i$; fitted by online SGA on Bernoulli log-likelihood with L2.
Interpretable and cold-start friendly (IRT-Router, Song et al., ACL 2025).

### 4.6 Bradley–Terry / Elo — `math.preference`, `learning.PreferenceStrategy`
$$P(i \succ j) = \sigma(\xi_i - \xi_j)$$
Learned per domain → a *prompt-specific leaderboard* (Prompt-to-Leaderboard,
Frick et al. 2025; RouteLLM preference data). Selection probability via Luce:
$P(t) = e^{\xi_t}/\sum_k e^{\xi_k}$.

### 4.7 Markov chain & MDP — `math.markov`, `learning.MarkovStrategy`
Conversation state $s$ = dominant domain. Transition matrix with Dirichlet smoothing
$$P_{ij} = \frac{n_{ij} + \alpha}{\sum_k n_{ik} + \alpha|S|}$$
$k$-step prediction $\pi P^k$, first-passage probability to *escalate*, stationary
distribution by power iteration, next-state entropy.
Routing MDP (value iteration):
$$V(s) = \max_a\Big[R(s,a) + \gamma\sum_{s'}P(s'|s)V(s')\Big]$$
`MarkovStrategy` scores $(1-\beta)\,Q(s,a) + \beta\sum_{s'}P(s'|s)\,Q(s',a)$ —
prefer targets that are also good for where the conversation is *going*.

### 4.8 Drift detection — `math.estimators`
- **Page–Hinkley**: $m_T=\sum_{t\le T}(x_t-\bar x_t-\delta)$, alarm if $\max_t m_t - m_T > \lambda$.
- **ADWIN-lite**: split window, Hoeffding bound $\epsilon=\sqrt{\tfrac{1}{2m}\ln\tfrac{4}{\delta}}$.
`AutoLearner` flags drifted targets; `HealthStrategy`/`HealthPolicy` can demote them.

### 4.9 Uncertainty on success rates
**Wilson interval** lower bound is used as the reliability estimate so a target with 2/2
successes is *not* considered 100 % reliable.

## 5. Multi-objective decisions — `math.decision`
- **Pareto front** over (quality↑, cost↓, latency↓).
- **TOPSIS** closeness $C = d^-/(d^+ + d^-)$ to the ideal point.
- **Weighted sum** (the default scalarisation).

## 6. Real-time capacity — `math.decision`
- **Erlang-C**: $P(\text{wait}) = \dfrac{\frac{A^c}{c!}\frac{1}{1-\rho}}{\sum_{k<c}\frac{A^k}{k!} + \frac{A^c}{c!}\frac{1}{1-\rho}}$, $A=\lambda/\mu$, $\rho=A/c$.
- **Mean wait** $W_q = P(\text{wait})/(c\mu - \lambda)$; `servers_for_sla` sizes a human/agent pool.
- **Little's law** $L=\lambda W$.
- **Kingman** G/G/1: $W_q \approx \frac{\rho}{1-\rho}\cdot\frac{c_a^2+c_s^2}{2}\cdot\tau$ for bursty LLM endpoints.

## 7. Resilience state machines — `realtime`
Circuit breaker (closed → open after $k$ failures in window $W$ → half-open after
$T_r$ → closed on probe success), token-bucket rate limit, rolling-window budget.

## 8. Similarity — `strategies.similarity`
Hashing-trick embeddings (word + bigram, signed hashing, L2-normalised), cosine
similarity; score $= 0.6\,\tfrac{1+\max}{2} + 0.4\,\tfrac{1+\text{mean}_{k}}{2}$ (UniRoute-style
example-based target representation). Swap in sentence-transformers for semantic quality.

## 9. Adversarial robustness — `security.InputGuard`
Confounder-gadget score compares head vs tail of the prompt on: alphabetic-token ratio,
function-word share, Shannon entropy jump, all-unique tail. Based on the attack in
"Rerouting LLM Routers" (Shafran et al., 2025). Signals are computed on the *cleaned* head.

## 10. Evidential routing — `math.dirichlet.DirichletProbe`, `strategies.HiddenStateStrategy`
A linear head over a feature vector $x$ (host hidden state, or any embedding) outputs
concentrations $\alpha_k = 1 + \mathrm{softplus}(w_k^\top x + b_k)$ of a Dirichlet over the
$K$ targets, $S=\sum_k \alpha_k$. Prediction $P(\text{best}=k)=\alpha_k/S$; **epistemic
uncertainty** $u = K/S$ (all evidence near zero gives $u\to 1$). Training minimises the expected
cross-entropy under the Dirichlet plus a KL term that removes evidence from wrong classes
(Sensoy et al., 2018):
$$\mathcal L=\sum_k y_k\big(\psi(S)-\psi(\alpha_k)\big)+\lambda\,\mathrm{KL}\!\big(\mathrm{Dir}(\tilde\alpha)\,\|\,\mathrm{Dir}(\mathbf 1)\big),\qquad \tilde\alpha = y + (1-y)\odot\alpha,$$
by SGD with $\psi$ (digamma) and $\psi'$ (trigamma) in pure Python. `HiddenStateStrategy` uses
$1-u$ as its confidence so an unfamiliar state hands the decision to the other strategies.

## 11. Response-side uncertainty — `signals.uncertainty`
- **Semantic entropy** (Kuhn et al., 2023): cluster $n$ sampled answers by meaning (cosine over
  hashed or real embeddings, threshold $\tau$); with cluster mass $p_c$,
  $H=-\sum_c p_c\log p_c$ (nats; `response_uncertainty` normalises by $\log n$). Agreement $= \max_c p_c$.
- **P(True)** (Kadavath et al., 2022): an optional judge returns $P(\text{answer is correct})$.
- `response_uncertainty` $u = 0.5\,\tilde H + 0.3\,(1-\text{agreement}) + 0.2\,\min(1,\text{hedging})$,
  and with a judge $u \leftarrow 0.6\,u + 0.4\,(1-P(\text{True}))$; `UncertaintyGate` returns $1-u$ as
  a cascade quality gate. `EventTrigger` fires named actions on threshold crossings (2607.13048).

## 12. Optimal stopping during generation — `strategies.escalation.SelfEscalation`
Competence $\theta\sim\mathrm{Beta}(a,b)$, prior centred on the target's `quality_prior`. Each
streamed chunk adds pseudo-counts: hedges $+0.8$, refusals $+1.5$, self-corrections $+0.6$,
repetition $+1.0$, length overrun $+0.5$ to $b$; clean progress $+0.4$ to $a$. After each chunk
$$\text{escalate iff}\quad \theta<\theta_0\ \wedge\ c_{\text{esc}} + c_{\text{tok}}(t + T) \;<\; (1-\mathbb E[\theta])\,c_{\text{fail}} + c_{\text{tok}}\,(T-t),$$
where $t$ is tokens decoded, $T$ the expected answer length: restarting elsewhere pays the escalation
overhead and regenerates the answer, continuing risks a failed answer (2608.24087).

## 13. Mixture-cure hand-off — `learning.handoff`
A task fails once its cumulative teacher risk $R_t=\sum_{s\le t} r_s$ crosses a latent threshold;
a fraction $\pi$ of tasks is *cured* and never fails; tasks that end without failing are censored.
$$P(\text{fail by }R)=(1-\pi)\big(1-e^{-(R/\lambda)^k}\big),\qquad
P(\text{uncured}\mid\text{alive at }R)=\frac{(1-\pi)S_W(R)}{\pi+(1-\pi)S_W(R)}.$$
`MixtureCureModel.fit()` maximises the censored likelihood (density for failures, survival for
censored) over a grid in $(\pi,\lambda,k)$ with one refinement. `HandoffPolicy` hands over
permanently when $P(\text{uncured}\mid R)\ge\tau$ or the probability of failing within the next
$h$ steps at the trajectory's mean step risk exceeds $\tau_h$ (TACIT-Switch 2608.27911).

## 14. Error-aware reverse auction — `strategies.auction.AuctionStrategy`
Bidder $i$ claims success $\hat p_i$ at price $c_i$. Its claim is corrected by the observed bias
$\beta_i=\overline{\hat p_i - r_i}$ and realised rate $\bar r_i$ with shrinkage $w_i=n_i/(n_i+\kappa)$:
$\tilde p_i=(1-\tfrac{w_i}{2})(\hat p_i-w_i\beta_i)+\tfrac{w_i}{2}\bar r_i$. Surplus
$s_i=\tilde p_i V - c_i$; the highest surplus wins and pays the **second price**
$\max\!\big(c_w,\ \tilde p_w V - s_{(2)}\big)$ so over-claiming is not rewarded (EA-RAM 2608.12719).

## 15. Energy and carbon — `math.energy`
Per target, ridge regression on measured samples gives
$\mathrm{Wh}=e_0+e_{\text{in}}\,n_{\text{prompt}}+e_{\text{out}}\,n_{\text{output}}$ (normal equations,
$3\times3$ solve); $\mathrm{gCO_2}=\mathrm{Wh}\cdot g/1000$ with the grid factor $g$. Without a
meter, `HardwareProfile` estimates $\mathrm{Wh}=\mathrm{TDP}\cdot\mathrm{PUE}\cdot
(t_{\text{prefill}}+t_{\text{decode}}+t_{\text{fixed}})/3600$ from the accelerator's throughput
(2608.28044; HW-Router).

## 16. Personalisation by shrinkage — `learning.personal.UserAdaptiveStrategy`
$\hat p_{u,t}=w_u\,\bar\theta_{u,t}+(1-w_u)\big[w_n\,\bar\theta_{N(u),t}+(1-w_n)\,\bar\theta_{\cdot,t}\big]$,
$w=\tfrac{n}{n+\kappa}$: a user's own Beta posterior, the similarity-weighted posterior of the
$k$ nearest users (cosine over hashed profile vectors) and the global posterior. New users are routed
by their neighbours and the crowd and take over after $\kappa$ outcomes (GMTRouter 2511.08590).

## 17. Multi-turn joint embeddings — `learning.multiturn.HistoryTargetStrategy`
History vector $h=\mathrm{normalise}\big(\sum_k \lambda^{k}\,\phi(m_{n-k})\big)$ over the last $K$
role-tagged messages and the current text ($\phi$ = hashed word/bigram embedding, $\lambda<1$ recency
decay); target vector $e_t$ = catalogue embedding. Joint feature $z=\sqrt{d}\,(h\odot e_t)$ and

$$\hat p(\text{success}\mid h,t)=\sigma\big(w^\top z+b_t\big),$$

a bilinear form with a diagonal weight matrix shared by every target; only $b_t$ is target-specific,
so an unseen target is scored through $e_t$ alone (inductive). Online logistic SGD on the reward with
L2 shrinkage; $b_t$ is initialised at $\mathrm{logit}(\text{quality\_prior})$. Confidence
$0.15+0.85\,n_t/(n_t+\kappa)$. Incumbent continuity adds $\pm$`stay_bonus` (MTRouter 2604.23530).

## References
See [RESEARCH.md](https://opensmartroute.ai/docs/RESEARCH.md) for the full bibliography.

---

<!-- Page: Research (https://opensmartroute.ai/docs/RESEARCH) -->

<!-- OpenSmartRoute: Research. Source https://github.com/isathish/OpenSmartRoute/blob/main/docs/RESEARCH.md; HTML https://opensmartroute.ai/docs/RESEARCH -->
# Research Foundations

This document is the evidence base for OpenSmartRoute. It (1) states the routing problem formally,
(2) surveys the literature by family, (3) compares the systems that exist today, (4) lists the gaps
that motivate this project, and (5) maps every borrowed idea to the module that implements it.
It is maintained as the design changes; see [ROADMAP.md](https://opensmartroute.ai/docs/ROADMAP.md) for what is still open.

---

## 1. Problem statement

A request $x$ arrives with context $c$ (tenant, session, constraints). A catalogue
$\mathcal{T}$ of *targets* is available — models, agents, skills, personas, tools, workflows, humans.
Each target $t$ has a cost $c_t(x)$, latency $\ell_t(x)$ and an unknown quality $q_t(x)\in[0,1]$.
Routing chooses

$$
t^\* = \arg\max_{t\in\mathcal{A}(x)} \; \mathbb{E}\big[q_t(x)\big] - \lambda_c\, c_t(x) - \lambda_\ell\, \ell_t(x)
$$

where $\mathcal{A}(x)\subseteq\mathcal{T}$ is the *admissible* set after hard constraints (data
boundary, region, PII, budget, SLO). Three things make this hard:

1. **$q_t(x)$ is never observed before routing** — it must be predicted from $x$ and from $t$'s history.
2. **$\mathcal{T}$ changes** — targets are added, deprecated, and degrade; the router must be *inductive*.
3. **Feedback is delayed, partial and noisy** — a thumbs-up, a retry, a handoff, or nothing.

The survey by Varangot-Reille et al. (2025) formalises this as a performance–cost optimisation and
classifies solutions as **pre-generation** (predict, then call one target) or **post-generation**
(call, verify, escalate). OpenSmartRoute does both: strategies are pre-generation; `Cascade` and the
planned multi-round executor are post-generation.

---

## 2. Literature by family

### 2.1 Supervised routers (predict quality from the prompt)

| Work | Core idea | Result | Limitation |
|---|---|---|---|
| **FORC** — Šakota, Peyrard & West, WSDM 2024 | A *meta-model* predicts each LM's performance on an input; assign under a cost budget or minimise cost at a target quality | matches largest LM at −63 % cost on 14 datasets | fixed LM pool; retrain when the pool changes |
| **Hybrid LLM** — Ding et al., ICLR 2024 | Predict *query difficulty*; route to small vs large with a tunable quality threshold | −40 % large-model calls at equal quality | binary pool |
| **RouteLLM** — Ong et al., 2024 | Train routers on **human preference** (Chatbot Arena) with data augmentation; four router types (BT, matrix factorisation, BERT, causal LLM) | >2× cost reduction; routers **transfer** to new strong/weak pairs | pairwise (two targets) |
| **Zooter** — Lu et al., 2023 | Distil *reward-model* scores on training queries into a routing function; tag-based label smoothing | beats best single model on 26 subsets | needs an offline reward model |
| **RouterDC** — Chen et al., NeurIPS 2024 | Encoder + learnable LLM embeddings trained with **dual contrastive** losses (sample–LLM, sample–sample) | +2.8 % ID / +1.9 % OOD over best single LLM | handles "several are fine" better than pointwise |
| **Arch-Router** — Tran et al., 2025 | 1.5 B model maps query → user-defined **domain / action**; routing preferences encoded declaratively; new models added without retraining | SOTA on preference alignment for conversational data | routing policy still hand-authored |
| **Prompt-to-Leaderboard** — Frick et al., 2025 | LLM outputs **Bradley–Terry coefficients per prompt** → prompt-specific leaderboard; optimal router follows | #1 on Chatbot Arena (Jan 2025); power-law scaling | needs preference data at scale |

### 2.2 Inductive / representation-based routers (generalise to unseen targets)

| Work | Core idea | What we took |
|---|---|---|
| **GraphRouter** — Feng, Shen & You, ICLR 2025 | Heterogeneous graph (task, query, LLM); edge prediction of effect + cost; **new LLM = new node** | targets are described by declared capabilities, not learned IDs |
| **UniRoute** — Jitkrittum et al., 2025 | Represent an LLM by its prediction vector on a *representative prompt set*; cluster-based routing; excess-risk bound; 30+ unseen LLMs | `examples` on a target; nearest-example similarity |
| **EmbedLLM** — Zhuang et al., 2024 | Encoder–decoder learns compact LLM embeddings that forecast benchmark performance | roadmap v0.5: target embeddings from outcomes |
| **IRT-Router** — Song et al., ACL 2025 | Item Response Theory: target *ability* θ, query *difficulty* b, discrimination a; interpretable; semantic warm-up for cold start | `math.irt`, `learning.IRTStrategy` |

### 2.3 Online / bandit routers (learn from live feedback)

| Work | Core idea | What we took |
|---|---|---|
| **MixLLM** — Wang et al., NAACL 2025 | Contextual bandit; tag-enhanced query embedding; per-LLM quality & cost predictors; meta-decision for quality/cost/latency; continual training; handles pool changes | `LinUCBStrategy` on the signal vector; `Objective` weights |
| **C2MAB-V** — Dai et al., 2024 | Combinatorial MAB with versatile reward models and **cost constraints**; regret & violation bounds | `CostAwareBandit` (Lagrangian budget) |
| Thompson sampling / UCB1 (Thompson 1933; Auer et al. 2002) | Posterior sampling / optimism for exploration | `ThompsonBeta`, `UCB1` |
| LinUCB (Li et al., WWW 2010) | Linear payoff with confidence ellipsoid | `LinUCB` |

### 2.4 Cascades and multi-round routers (post-generation)

| Work | Core idea | What we took |
|---|---|---|
| **FrugalGPT** — Chen, Zaharia & Zou, 2023 | Call cheap first; a scorer decides whether to escalate | `Cascade` with pluggable `quality_gate` |
| **AutoMix** — Aggarwal et al., NeurIPS 2024 | Few-shot **self-verification** + **POMDP** router over verification noise; −50 % cost | roadmap v0.6: POMDP cascade |
| **Router-R1** — Zhang, Feng & You, NeurIPS 2025 | Router is itself an LLM; interleaves *think* / *route* actions across rounds; RL with format + outcome + **cost reward**; conditions on model descriptors (price, latency, examples) | `LLMJudgeStrategy` (conditioned on the same descriptors), confidence-gated escalation; roadmap v0.6 multi-round |
| **LLM-Blender** — Jiang, Ren & Lin, ACL 2023 | PairRanker over candidate outputs + GenFuser | aggregation step for the multi-round executor |

### 2.5 Multi-agent, tool and skill routing

| Work | Core idea | What we took |
|---|---|---|
| **MasRouter** — Yue et al., 2025 | *Multi-Agent System Routing*: choose collaboration mode → agent roles → LLM per role via cascaded controllers; −52 % overhead | `RoutePlan` slots (persona → skill → model) filled by sub-routing |
| **ToolLLM** — Qin et al., 2023 | 16 k APIs; **neural API retriever** recommends tools per instruction; DFS decision tree | tools are targets with `examples`; retrieval = `SimilarityStrategy` |
| **AnyTool** — Du et al., 2024 | **Hierarchical** API retriever (category → tool → endpoint) + self-reflection; revised evaluation protocol | roadmap v0.7: MCP catalogue import with hierarchical narrowing |
| **Agent Skills** (Anthropic, 2025) / skills as first-class units | Skills are declared with a description and triggers | `TargetKind.SKILL` with `capabilities.actions` |

### 2.6 Sequential decision making for conversations

Dialogue policy as an MDP dates to Levin, Pieraccini & Eckert (2000) and is the basis for
POMDP dialogue systems (Young et al., 2013). OpenSmartRoute applies the same lens to *who answers
next*: `MarkovChain` learns intent transitions, `RoutingMDP` runs value iteration with
reward = quality − λ·cost, and `MarkovStrategy` scores targets by their value on the predicted
next state, not only the current one.

### 2.7 Capacity and queueing

Skills-based routing in contact centres is a mature field (Koole & Mandelbaum 2002). Its tools
carry over directly to pools of humans or long-running agents: **Erlang-C** for P(wait) and
mean wait in M/M/c, **Little's law** for in-flight sizing, and **Kingman's** G/G/1 approximation
for bursty single endpoints. Implemented in `math.decision`.

### 2.8 Adversarial robustness of routers

**Rerouting LLM Routers** (Shafran, Schuster, Ristenpart & Shmatikov, 2025) defines *LLM control-plane
integrity* and shows query-independent **confounder gadgets** — short token sequences that push any
query to the strong model in white- and black-box settings, with low perplexity (so perplexity
filtering fails). Consequences for us: routing signals are computed on the *cleaned head* of a
prompt (`security.InputGuard`), the guard reports the suspicion in the trace, and the roadmap keeps
a learned detector open. See [SECURITY.md](https://opensmartroute.ai/docs/SECURITY.md).

The 2026 literature widens the control-plane threat model beyond gadgets:

| Work | Threat | Consequence for us |
|---|---|---|
| **ROPE** — Aug 2026 | Indirect prompt injection through tool results steering *state-changing* tool calls; deterministic **origin check** on sensitive parameters keeps attack success at 1.6–2.6 % with 82–100 % utility | origin policy for tool parameters inside executed `RoutePlan`s (v0.9) |
| Verifiable manifest signing for MCP pipelines — Jan 2026 | Tampered or drifting tool manifests | signed-manifest verification at import |
| "Recognition Without Enforcement" — Aug 2026 | Models recognise unsafe tool requests but still execute them | enforcement lives in policy, not in the model |
| "Beyond Max Tokens" — Jan 2026 | Resource amplification through tool chains | per-task depth / token caps |
| "When Safety Routing Breaks" — Sep 2026 | Routing itself as the safety bypass | adversarial routing regression suite |

### 2.9 Benchmarks and evaluation methodology

| Benchmark | Scale | Use here |
|---|---|---|
| **RouterBench** — Hu et al., 2024 | 405 k inference outcomes, 11 LLMs, 8 datasets; defines the quality–cost frontier and AIQ | `eval.cost_quality_frontier`, `area_under_frontier`; `eval.collect` lists the source (`routerbench`, gated: the Hub serves no data files - load a local copy with `DatasetCollector.store`) |
| **RouterEval** — Huang et al., 2025 | 200 M records, 8 500 LLMs, 12 evaluations; shows **model-level scaling** — a good router improves as the pool grows and can beat the best single model | motivates large catalogues; `eval.collect` source `routereval` (gated: needs a Hub token) |
| **LLMRouterBench** — Jan 2026 | 400 k instances, 21 datasets, 33 models, 10 baselines; **many routers, including commercial ones, do not reliably beat simple baselines**; large oracle gap from model-recall failures; larger ensembles show diminishing returns vs curation; latency-aware analysis | roadmap v0.4: mandatory baselines, curation report |
| **LLMRouter / xRouteBench** — Feng & You, Aug 2026 | Routing as a sequential decision with five components (context encoder, model encoder, scoring, decision rule, learning signal); 16+ routers; single/multi-turn, agentic and personalised tasks; learned routers +14.6 % over the best fixed model | component ablation report; personalised split |
| **RouterXBench / ProbeDirichlet** — Feb 2026 | Three axes: router ability, scenario alignment, cross-domain robustness; hidden-state Dirichlet router +16.7 / +18.9 % | cross-domain split; `math.DirichletProbe` + `strategies.HiddenStateStrategy` |
| **RouterArena** — 2026 | Public leaderboard with arena score, accuracy and cost per 1 k queries | v1.0 submission target |
| **"How Much of the Routing Gap Is Real?"** — Jul 2026 | 12–36 % of the router-to-oracle gap on open pools is single-draw **label noise**; multi-sample oracle ($k\ge 20$) and best-of-K recover the floor | `--oracle-samples`, reproducible-headroom figure |
| **"Most of the LLM Routing Gap Is Task Type"** — Aug 2026 | 21 methods within a fraction of a point; a **static per-task-type table** answered 262/294 at \$3.33 vs 245 at \$7.69 for the best single model; 5.4 % run-to-run flips | `TaskTableStrategy`; `--repeats` |
| **"When is Routing Meaningful?"** — Jul 2026 | Hierarchic social entropy for pool diversity; <10 behaviourally distinct agents suffice; kNN routers collapse under paraphrase, prompted routers stay stable | diversity / coreset report; paraphrase-robustness score |
| **MixInstruct** (LLM-Blender) | oracle pairwise comparisons | preference data format for `FeedbackStore.export_preferences` |
| **SkillFeed-Bench**, **Task2MCP**, **ComplexMCP** — 2026 | counterfactual profile-conditioned skill routing; MCP server recommendation; multi-server tool tasks | v0.7 discovery evaluation |

### 2.10 Risk control, calibration and abstention

| Work | Core idea | What we take |
|---|---|---|
| **RACER** — Feb 2026 | *α-VOR*: output a **set** of models with distribution-free mis-routing risk ≤ α, plus abstention; nested sets from concentration bounds | `RouteDecision.candidate_set`, abstain → `HUMAN` |
| **RouteNLP** — ACL 2026 (industry) | Conformal prediction to set cascade thresholds; distillation–routing co-optimisation by clustering escalation failures; −58 % cost, p99 1847 → 387 ms | conformal `escalate_below`; escalation-failure clustering feeds `osr train` |
| **CR²** — May 2026 | Cost-aware, risk-controlled routing | same family |
| Temperature scaling (Guo et al. 2017), isotonic regression, ECE | Post-hoc probability calibration | fitted confidence instead of fixed $\tau$ |
| Semantic entropy (Kuhn et al. 2023; Farquhar et al. 2024), P(True) (Kadavath et al. 2022) | Response-side uncertainty | `signals.semantic_entropy()`, `response_uncertainty()`, `UncertaintyGate`, `EventTrigger` |
| **Learning to defer** (Madras et al. 2018; Mozannar & Sontag 2020; Verma et al. 2023) | Classifier and expert trained jointly with a consistent surrogate; the expert has its own cost and accuracy | `TargetKind.HUMAN` as a deferral option |

### 2.11 Multi-turn, task-level and step-level routing for agents

| Work | Core idea | What we take |
|---|---|---|
| **TRACE-Router** — Jul 2026 | Agentic tasks have one **delayed, task-level** outcome; assign a backend once at admission with a contextual bandit, pin it, learn from the terminal reward (accuracy + latency); +7–8 pts on τ²-Bench, +7.1 pts / −36 % latency on Terminal-Bench | `Outcome.task_id`, admission-time routing, delayed-feedback bandits |
| **MTRouter** — ACL 2026 | History–model joint embeddings; turn-level utility from logged trajectories; beats GPT-5 on ScienceWorld at −58.7 % cost with fewer switches | `learning.HistoryTargetStrategy`: logistic model over `h * e_t` (recency-weighted hashed history, catalogue target embedding) with shared weights and per-target bias; incumbent bonus / failed-turn penalty from `context["last_target"]` |
| **ProgRouter** — EMNLP 2026 Findings | Multi-view task-progress scorer + dual-path progress predictor + meta-gating; step-wise routing under time & cost budgets | progress-guided step routing |
| **RLCascadeRouter** — Aug 2026 | Cascade routing as an MDP with *stop* / *select* actions trained on trajectory returns; **quality-estimator-free**; prediction loss ≠ decision loss; unseen models without retraining | cascade rewrite on `RoutingMDP` |
| **Bayesian self-escalation** — Aug 2026 | Intra-generation delegation as Bayesian optimal stopping over a learned competence posterior; time-varying threshold; $1/\sqrt{n}$ regret in calibration trajectories | `strategies.SelfEscalation` + `wrap_stream()`: Beta competence posterior per streamed chunk, stop when `loss(stop) < loss(continue)` |
| **TACIT-Switch** — Aug 2026 | Permanent handoff learned from interval-censored teacher intervention times; mixture-cure threshold model; +7.4–11.1 pp | `learning.MixtureCureModel` (Weibull mixture-cure, censored MLE) + `learning.HandoffPolicy` |
| **JiSi** — Jan 2026 | Query–response mixed routing, support-set aggregator selection, adaptive routing ↔ aggregation switch; 10 open LLMs beat Gemini-3-Pro at 47 % cost | `signals.DraftResponseSignal` (query–response features); `strategies.MixtureOfAgents` (routing / aggregation switch) |
| **GMTRouter** — EMNLP 2026 Findings | Heterogeneous graph (user, LLM, query, response, turn); few-shot personalisation | `learning.UserAdaptiveStrategy`: user → neighbour users → global shrinkage over hashed profile vectors |
| **TRouter** — ACL 2026 | Task-taxonomy-guided synthetic data for **cold start**; latent task-type variables with prior regularisation | `osr train --synth` |
| **SCX Router** — Sep 2026 | 0.6 B decoder-KV classifier predicts task type, difficulty, reasoning mode and output length before generation; task ontology (23 families / 115 types / 345 subtypes / 30 domains); separates learned prediction from per-task **policies** (eligibility, cost, cache, safety, sovereignty) | ontology-based task classifier; policy/prediction separation matches ours |
| **Think When Needed** — SIGIR 2026 | Pre-generation router head decides Think vs Non-Think; operating point chosen on the validation Pareto frontier; +6.3 % NDCG@10 at −49.5 % tokens | effort as a routable dimension |
| Agent-as-a-Router, SWE-Router, "Routing least learnable where most valuable" — 2026 | Routing gains shrink on the hardest agentic tasks | `eval.headroom.learnability_by_difficulty`: per-difficulty-bucket headroom and noise floor |
| BudgetMem, Gated-Memory Routing — 2026 | Routing applied to agent memory tiers | `strategies.MemoryRouter` (tiers with capacity / cost / latency, learned `ImportanceGate`, budgeted `recall`) |

### 2.12 Capability discovery at scale (models, agents, tools, skills)

| Work | Core idea | What we take |
|---|---|---|
| **Enrich-Retrieve-Rank** — Aug 2026 | MATS (models · agents · tools · skills) discovery; offline enrichment + retrieve-then-rank; in-context routing Match@1 falls 0.85 → 0.12 at N = 7 278 while retrieve-then-rank holds 0.81 → 0.39; crossover ≈ N = 500; 70× cheaper than full context | two-stage narrowing switched on by catalogue size |
| **SCOUT** — Aug 2026 (PayPal) | MCP gateway with `tool_search` / `execute_tool` meta-tools; BM25 + dense with RRF; 140 k → 1.3 k tool tokens (99 %) across 2 000+ tools / 200+ servers | meta-tools; hybrid retrieval |
| **Task2MCP / T2MRec** — 2026 | MCP *server* recommendation as retrieval + rank under engineering constraints | `adapters.recommend_servers()` over `ServerCard`s with constraint filters |
| SchemaRouter, CacheRouter, Nexus, ToolDNS, Semantic Tool Discovery — 2026 | Field-aware matching, cache-preserving tool subsets, retrieval-decoupled routing, DNS-based discovery | `discovery.schema_match()` / `SchemaAwareStrategy`; `discovery.CachePreservingSelector`; DNS discovery not adopted |
| **Best Prefix Selection** — Aug 2026 | Skill selection as **submodular** benefit minus context penalty under a token budget; $(1-1/e,\,1)$ bicriteria greedy; 0.73 vs 0.20–0.52 success, −28 % tokens | `RoutePlan.skills` set selection |
| **SkillFeed** — Aug 2026 | Profile-conditioned skill retrieval with a counterfactual benchmark; +35 pt when the profile matters | `learning.SkillAffinity` → `select_skill_set(relevance=)`; profile-swap evaluation |
| Compositional skill routing, CaSKG, systems foundation for agentic skills — 2026 | Skill graphs and composition | `discovery.SkillGraph` (`requires` / `conflicts` / `composes` + learned co-usage; `compose()`) |

### 2.13 Non-stationarity, budgets and latency

| Work | Core idea | What we take |
|---|---|---|
| **Drift-Aware Sparse Routing** — Sep 2026 | Non-stationary sparse contextual routing with **multiple knapsack** constraints; shadow-audit stream for pruned arms; pessimistic reward / optimistic cost; online shadow prices; regret $\tilde O(T\sqrt{s/\rho W} + W V_T + \sqrt T)$ | multi-knapsack `CostAwareBandit` |
| **OrcaRouter** — May 2026 | LinUCB over lexical + sentence embeddings; offline full-information reward matrix → ridge per arm → online bandit updates; #2 on RouterArena | offline warm start for `LinUCBStrategy` |
| **Latency-aware routing** — Jul 2026 | TTFT estimator that simulates batching; joint latency / accuracy / cost; +40 % utility at equal latency | queue-aware latency in `HealthStrategy` |
| **HW-Router** — DAC 2026; energy characterisation — IISWC 2026 | Hardware- and energy-aware routing | `math.EnergyModel` (per-target Wh regression, gCO2) and `hardware_profile()` priors; energy as a knapsack resource |
| **Federate the Router** — Jan 2026 | Federated MLP / K-means routers from decentralised sparse evaluations | mergeable learner state |
| D-UCB / SW-UCB (Garivier & Moulines 2011), delayed-feedback bandits (Joulani et al. 2013), bandits with knapsacks (Badanidiyuru et al. 2013), dueling bandits (Yue et al. 2012), multi-objective bandits (Drugan & Nowé 2013) | Classical machinery for drift, delay, budgets, preferences, Pareto | forgetting, delayed credit, knapsacks, `pareto_front` |
| Off-policy evaluation: IPS, SNIPS, doubly robust (Dudík et al. 2011) | Score a new policy on logged decisions | `eval` OPE; propensity logging |

### 2.14 Economics and mechanism design

| Work | Core idea | What we take |
|---|---|---|
| **EA-RAM** — Aug 2026 | Reverse auction: providers bid self-predicted success probability + cost; Bayesian incentive compatible under *dual error*; explicit welfare-loss bound | `strategies.AuctionStrategy`: bias-corrected claims, second-price payment |
| Strategy auctions for small agents — ICML 2026; coalition formation and communication pricing in skill-based agentic AI — 2026 | Market mechanisms among agents | research track |
| Dominant-resource fairness (Ghodsi et al. 2011) | Fair multi-resource allocation | tenant fairness under shared budgets |

---

## 3. Systems landscape (open source, September 2026)

| System | Targets | Learning | Explainable trace | Hard policy | Deps |
|---|---|---|---|---|---|
| RouteLLM (lmsys) | 2 LLMs | offline (preference) | no | no | torch |
| vLLM Semantic Router | LLMs (MoM) | classifier (offline) | partial | yes (PII, jailbreak, location) | Go/Rust/Python, models |
| Aurelio semantic-router | routes (intents) | embeddings | no | no | embeddings |
| NVIDIA llm-router | LLMs | classifier | no | no | Triton |
| LiteLLM / Portkey / aisix / nyro | LLMs (gateway) | none (rules, fallback) | no | rate/budget | service |
| NadirClaw / Misceo / ClawRouter / nexus-llm-router | LLMs (cheap↔premium) | heuristics | no | budget | Python/TS |
| LLMRouter (ulab-uiuc) | LLMs | 16+ trainable routers, xRouteBench | no | no | torch |
| SCX Router (checkpoint) | LLMs | 0.6 B zero-shot classifier + task ontology | task labels | per-task policies | torch |
| OrcaRouter | LLMs | LinUCB, offline warm start | no | no | Python, embeddings |
| SCOUT (MCP gateway) | tools | hybrid retrieval | no | gateway ACLs | service |
| **OpenSmartRoute** | **LLM · agent · skill · persona · tool · workflow · human** | **online (bandit, IRT, BT, LinUCB, MDP) + optional judge** | **yes, per strategy** | **yes, before scoring** | **0 in core** |

Gateways and OpenSmartRoute are complementary: the gateway translates protocols and executes; the
router decides.

---

## 4. Gaps this project addresses

1. **Heterogeneous targets.** All surveyed routers choose among LLMs. Real systems choose among a model,
   an agent with tools, a deterministic skill, a persona layered on a model, or a human. Treating these
   uniformly (one `RouteTarget` contract, one policy layer, one learning loop) is the central design
   decision.
2. **Composition.** MasRouter shows that *mode + role + model* beats picking a model. `RoutePlan` does
   this with declarative slots and no training.
3. **Inductive by default.** GraphRouter/UniRoute/IRT show it is possible; most shipped routers still
   retrain on pool change. Every strategy here works from declared capabilities on day one and learns
   from day two.
4. **Hard constraints before soft scores.** Cost/quality routers routinely trade away privacy or
   region. Policy runs first and cannot be outweighed.
5. **Control-plane integrity.** Almost no router considers adversarial inputs; Shafran et al. show they
   must.
6. **Explainability.** Regulated deployments need to say *why* a request went where it did. Every
   decision carries per-strategy scores and rationales.
7. **Honest evaluation.** Benchmarks in 2026 show most routers do not beat a static task table and
   that a third of the "oracle gap" is label noise. The evaluation harness must make these baselines
   and noise floors unavoidable, or learned strategies will be adopted on illusory gains.
8. **Task-level supervision.** Agentic workloads reward the *task*, not the call. The data model must
   carry `task_id` and trajectory position so learners can be credited correctly.
9. **Discovery at scale.** Flat similarity over thousands of tools and skills collapses; two-stage
   retrieve-then-rank and set-valued skill selection under a token budget are required.
10. **Risk, not just cost.** Set-valued decisions with abstention and conformal thresholds give
    operators a guarantee to sign off on; a tuned scalar threshold does not.

---

## 5. Idea → implementation map

| Idea | Source | Module |
|---|---|---|
| Difficulty-based routing with quality floor | Hybrid LLM | `signals.ComplexitySignal`, `Objective.quality_floor` |
| Domain/action preferences, pinned rules | Arch-Router | `signals.DomainActionSignal`, `strategies.RulesStrategy` |
| Example-based target representation | UniRoute, GraphRouter | `RouteTarget.examples`, `strategies.SimilarityStrategy` |
| Ability × difficulty model | IRT-Router | `math.irt`, `learning.IRTStrategy` |
| Per-domain pairwise preference | RouteLLM, P2L | `math.preference`, `learning.PreferenceStrategy` |
| Contextual bandit on signal vector | MixLLM, LinUCB | `math.bandits.LinUCB`, `learning.LinUCBStrategy` |
| Budget-constrained exploration | C2MAB-V | `math.bandits.CostAwareBandit` |
| Cheap-first cascade | FrugalGPT | `strategies.Cascade` |
| LLM-as-router conditioned on descriptors, cost-aware | Router-R1 | `strategies.LLMJudgeStrategy` + `escalate_llm_judge_below` |
| Mode/role/model composition | MasRouter | `Router._build_plan`, `RoutePlan` |
| Conversation lookahead | dialogue MDPs | `math.markov`, `learning.MarkovStrategy` |
| Quality–cost frontier evaluation | RouterBench | `eval` |
| Confounder-gadget defence | Rerouting LLM Routers | `security.InputGuard` |
| Capacity sizing for human pools | queueing theory | `math.decision.erlang_c`, `servers_for_sla` |
| Drift detection on target quality | Page–Hinkley, ADWIN | `math.estimators`, `learning.AutoLearner` |
| Static task-type table as default learner and baseline | "Routing gap is task type" | `strategies.TaskTableStrategy` (v0.4) |
| Multi-sample oracle, run-to-run flip rate | "How much of the routing gap is real?" | `eval` `--oracle-samples`, `--repeats` (v0.4) |
| Diversity / coreset and paraphrase robustness | "When is routing meaningful?" | `eval` reports (v0.4) |
| Ontology-based task / difficulty / reasoning-mode / length prediction | SCX Router, TRouter | `signals` learned classifiers, `osr train --synth` (v0.4) |
| Calibrated confidence | temperature / isotonic scaling | `router` confidence (v0.4) |
| Offline reward matrix → LinUCB warm start | OrcaRouter | `learning.LinUCBStrategy` (v0.5) |
| Effort as a target dimension | Think When Needed | `RouteTarget.effort` (v0.5) |
| Profile-conditioned routing | GMTRouter, SkillFeed | `RouteRequest.profile` (v0.5) |
| Task-level admission routing with delayed terminal reward | TRACE-Router | `Outcome.task_id`, delayed-feedback bandits (v0.6) |
| Cascade as MDP with a stop action | RLCascadeRouter, unified cascade routing | `strategies.Cascade` on `math.markov.RoutingMDP` (v0.6) |
| Progress-guided step routing | ProgRouter | v0.6 |
| Retrieve-then-rank narrowing, `tool_search` meta-tool | Enrich-Retrieve-Rank, SCOUT | `adapters.mcp`, two-stage selection (v0.7) |
| Submodular skill-set selection | Best Prefix Selection | `Router._build_plan` (v0.7) |
| Conformal thresholds, set-valued routing with abstention | RouteNLP, RACER | `RouteDecision.candidate_set` (v0.8) |
| Learning to defer | Madras; Mozannar & Sontag | `HUMAN` deferral objective (v0.8) |
| Multi-knapsack budgets with shadow audit | Drift-Aware Sparse Routing | `math.bandits.CostAwareBandit` (v0.8) |
| Forgetting in learners | D-UCB / SW-UCB | `math.irt`, `math.preference`, `math.bandits` (v0.8) |
| Queue-aware latency (TTFT) | Latency-aware routing | `realtime.HealthStrategy` + `math.decision` (v0.8) |
| Off-policy evaluation, shadow mode | IPS / DR | `eval`, `EnterpriseRouter(shadow=)` (v0.8) |
| Origin policy on tool parameters | ROPE | `security` (v0.9) |
| Contrastive (query, target) training over acceptable sets | RouterDC | `learning.ContrastiveRouter`, `ContrastiveStrategy`, `acceptable_set` |
| Reward-model-distilled soft labels | Zooter | `learning.soft_labels`, `ContrastiveRouter.fit(objective="distilled")` |
| A small routing model distilled from the full ensemble, calibrated, learning online | RouteLLM (matrix-factorisation router), Zooter, RouterDC | `learning.RouterSLM`, `distill_router`, `SLMStrategy` |
| Self-improvement from live outcomes and public routing datasets without regressions | RouterBench, RouterEval; champion/challenger evaluation | `learning.SelfImprover`, `eval.DatasetCollector`, `eval.rows_from_feedback` |
| Live cost/quality priors and discovery of new models | LLMRouterBench cost-quality frontier | `adapters.ModelCatalogue`, `adapters.WebKnowledge` |
| Decision loss vs prediction loss; end-to-end policy gradient | Router-R1, RLCascadeRouter | `learning.PolicyGradientStrategy`, `decision_reward`, `decision_regret` |
| Oracle gap decomposition, target diversity, minimal catalogue, scaling curve | "How much of the routing gap is real?", "When is routing meaningful?", RouterEval | `eval.headroom` (`routing_headroom`, `target_diversity`, `min_catalogue`, `scaling_curve`) |
| Learnability on the hardest tasks | SWE-Router, Agent-as-a-Router | `eval.headroom.learnability_by_difficulty` |
| Memory-tier routing under a token budget | BudgetMem, Gated-Memory Routing | `strategies.MemoryRouter`, `MemoryTier`, `ImportanceGate` |
| Modality-aware scoring and text-first escalation | LatentRouter, modality escalation, CUA routing | `strategies.ModalityStrategy`, `ModalityEscalation`, `request_modalities` |
| Semantic cache as the cheapest target | GPTCache, vLLM semantic router | `strategies.SemanticCache`, `SemanticCacheStrategy` |
| Annotator skill estimation and quorum selection | QUORUM; Dawid & Skene 1979 | `strategies.AnnotatorPool`, `quorum_accuracy`, `HumanRoutingStrategy` |
| Speculative draft/strong cascades with cancellation | speculative cascades; Differential Reasoning Router | `strategies.SpeculativeCascade`, `expected_mode_costs` |

---

## 6. Open research questions

Every item of the roadmap research track now has a module (see the research-track table in
[ROADMAP.md](https://opensmartroute.ai/docs/ROADMAP.md)); what remains open is measurement, not implementation:

- **How much routing headroom is real** on a given catalogue once label noise and task-type effects are
  removed — `eval.headroom.routing_headroom` reports it; the open question is what the numbers say on
  *your* pool, and therefore when a learned router is worth its complexity over a static table.
- **How few, how different** targets must be before routing beats the best single model
  (`target_diversity`, `min_catalogue`), and whether accuracy keeps scaling with catalogue size in
  heterogeneous pools (`scaling_curve`).
- **Prediction vs decision**: `decision_regret` puts end-to-end RL (`PolicyGradientStrategy`) next to
  predict-then-optimise ensembles on the same rows; when does decision loss matter in practice?
- **Where to escalate**: before generation (router), during (`SelfEscalation`), after
  (`Cascade`, `MixtureOfAgents`) or speculatively in parallel (`SpeculativeCascade`) — cost/quality
  frontiers of the four regimes on the same pool.
- **Contrastive vs pointwise** training when several targets are acceptable (`ContrastiveRouter` vs
  `TaskTableStrategy` / `LinUCBStrategy`), and how much reward-distilled soft labels help over one-hot.
- **Routing for the hardest agentic work** (web, SWE): `learnability_by_difficulty` on real agent traces.
- **Strategic providers**: how `AuctionStrategy`'s bias correction behaves against adaptive bidders;
  coalition pricing in multi-vendor catalogues.
- **Human pools**: Dawid-Skene assumes conditionally independent annotators; correlated errors inside
  a quorum (`AnnotatorPool.select_quorum`) are not yet modelled.
- Fairness under shared budgets; energy priors from `hardware_profile()` against metered fleets.

---

## 7. References

Entries without author lists are 2026 preprints cited by arXiv identifier; the italic text is a
descriptive title and the identifier is authoritative.

**Surveys**
- Varangot-Reille C., Bouvard C., Gourru A., Ciancone M., Schaeffer M., Jacquenet F. *Doing More with Less: A Survey on Routing Strategies for Resource Optimisation in LLM-Based Systems.* JAIR 2025. arXiv:2502.00409

**Supervised & preference routers**
- Šakota M., Peyrard M., West R. *Fly-Swat or Cannon? Cost-Effective Language Model Choice via Meta-Modeling (FORC).* WSDM 2024. arXiv:2308.06077
- Ding D. et al. *Hybrid LLM: Cost-Efficient and Quality-Aware Query Routing.* ICLR 2024. arXiv:2404.14618
- Ong I. et al. *RouteLLM: Learning to Route LLMs with Preference Data.* 2024. arXiv:2406.18665
- Lu K. et al. *Routing to the Expert: Efficient Reward-guided Ensemble of LLMs (Zooter).* 2023. arXiv:2311.08692
- Chen S., Jiang W., Lin B., Kwok J.T., Zhang Y. *RouterDC: Query-Based Router by Dual Contrastive Learning.* NeurIPS 2024. arXiv:2409.19886
- Tran C., Paracha S., Hafeez A., Chen S. *Arch-Router: Aligning LLM Routing with Human Preferences.* 2025. arXiv:2506.16655
- Frick E. et al. *Prompt-to-Leaderboard.* 2025. arXiv:2502.14855
- Stepanov I. et al. *SCX Router: Streaming Zero-Shot Model Selection with a Decoder-KV Classifier and a Real-World Task Ontology.* 2026. arXiv:2609.02292
- *TRouter: Task-Taxonomy-Guided Synthetic Data for Cold-Start LLM Routing.* ACL 2026. arXiv:2604.09377
- *VDAR-Router: Verbalized Difficulty Retrieval for LLM Routing.* 2026. arXiv:2607.18098
- *Scalable Prompt Routing via Latent Task Discovery.* 2026. arXiv:2603.19415
- Guo H. et al. *Think When Needed: Model-Aware Reasoning Routing for LLM-based Ranking.* SIGIR 2026. arXiv:2601.18146
- *When to Think Deeply.* 2026. arXiv:2606.06745

**Inductive / representation**
- Feng T., Shen Y., You J. *GraphRouter: A Graph-based Router for LLM Selections.* ICLR 2025. arXiv:2410.03834
- Jitkrittum W. et al. *Universal Model Routing for Efficient LLM Inference (UniRoute).* 2025. arXiv:2502.08773
- Zhuang R. et al. *EmbedLLM: Learning Compact Representations of Large Language Models.* 2024. arXiv:2410.02223
- Song W. et al. *IRT-Router: Effective and Interpretable Multi-LLM Routing via Item Response Theory.* ACL 2025. arXiv:2506.01048
- Sun Y., Xie E., Feng T., You J. *GMTRouter: Personalized LLM Router over Multi-turn User Interactions.* EMNLP 2026 Findings. arXiv:2511.08590
- *ProbeDirichlet / RouterXBench.* 2026. arXiv:2602.11877

**Online / bandits**
- Wang X. et al. *MixLLM: Dynamic Routing in Mixed Large Language Models.* NAACL 2025. arXiv:2502.18482
- Dai X., Li J., Liu X., Yu A., Lui J.C.S. *Cost-Effective Online Multi-LLM Selection with Versatile Reward Models (C2MAB-V).* 2024. arXiv:2405.16587
- Li L., Chu W., Langford J., Schapire R.E. *A Contextual-Bandit Approach to Personalized News Article Recommendation.* WWW 2010
- Auer P., Cesa-Bianchi N., Fischer P. *Finite-time Analysis of the Multiarmed Bandit Problem.* Machine Learning 47, 2002
- Thompson W.R. *On the Likelihood that One Unknown Probability Exceeds Another.* Biometrika 1933
- Bao Z. et al. *OrcaRouter: A Production-Oriented LLM Router with Hybrid Offline-Online Learning.* 2026. arXiv:2605.30736
- *Drift-Aware Sparse Routing: Nonstationary Contextual Routing under Multiple Knapsack Constraints.* 2026. arXiv:2609.00662
- Raj R. et al. *TRACE-Router: Task-Consistent and Adaptive Online Routing for Agentic AI.* 2026. arXiv:2607.22465
- *Federate the Router.* 2026. arXiv:2601.22318
- Patel S., Parthasarathy A.R., Mallick A., Joshi G. *Beyond Accuracy and Cost: Latency-Aware LLM Query Routing for Dynamic Workloads.* 2026. arXiv:2607.18253
- *HW-Router.* DAC 2026. arXiv:2608.14575
- Garivier A., Moulines E. *On Upper-Confidence Bound Policies for Switching Bandit Problems.* ALT 2011
- Joulani P., György A., Szepesvári C. *Online Learning under Delayed Feedback.* ICML 2013
- Badanidiyuru A., Kleinberg R., Slivkins A. *Bandits with Knapsacks.* FOCS 2013
- Yue Y., Broder J., Kleinberg R., Joachims T. *The K-armed Dueling Bandits Problem.* JCSS 2012
- Drugan M.M., Nowé A. *Designing Multi-Objective Multi-Armed Bandits Algorithms.* IJCNN 2013
- Dudík M., Langford J., Li L. *Doubly Robust Policy Evaluation and Learning.* ICML 2011

**Cascades & multi-round**
- Chen L., Zaharia M., Zou J. *FrugalGPT.* 2023. arXiv:2305.05176
- Aggarwal P. et al. *AutoMix: Automatically Mixing Language Models.* NeurIPS 2024. arXiv:2310.12963
- Zhang H., Feng T., You J. *Router-R1: Teaching LLMs Multi-Round Routing and Aggregation via RL.* NeurIPS 2025. arXiv:2506.09033
- Jiang D., Ren X., Lin B.Y. *LLM-Blender.* ACL 2023. arXiv:2306.02561
- Dekoninck J., Baader M., Vechev M. *A Unified Approach to Routing and Cascading for LLMs.* 2024. arXiv:2410.10347
- Huang S., Wang S., Ma H., Xu Z. *RLCascadeRouter: Quality-Estimator-Free Cascade Routing via Reinforcement Learning.* 2026. arXiv:2608.15817
- Shaikh N. *Knowing When to Ask for Help: Bayesian Self-Escalation in Hierarchical LLM Agents.* 2026. arXiv:2608.24087
- Lei J., Huang J. *TACIT-Switch: Cost-Aware Model Escalation for LLM Agents from Censored Supervision.* 2026. arXiv:2608.27911
- Li S., Abdelmoniem A.M., Wang S. *ProgRouter: Online Progress-Guided Orchestration for Multi-Agent LLM Workflows.* EMNLP 2026 Findings. arXiv:2608.25992
- *MTRouter: Multi-Turn LLM Routing.* ACL 2026. arXiv:2604.23530
- Tang S. et al. *Beyond Gemini-3-Pro: Revisiting LLM Routing and Aggregation at Scale (JiSi).* 2026. arXiv:2601.01330
- *Confidence-Guided Stepwise Model Routing.* 2025. arXiv:2511.06190
- *Differential Reasoning Router.* 2026. arXiv:2608.30224

**Risk control, calibration, deferral**
- *RACER: Risk-Controlled LLM Routing with Abstention.* 2026. arXiv:2603.06616
- *RouteNLP: Conformal Cascade Thresholds and Distillation–Routing Co-optimisation.* ACL 2026 Industry. arXiv:2604.23577
- *CR²: Cost-Aware Risk-Controlled Routing.* 2026. arXiv:2605.12001
- Guo C., Pleiss G., Sun Y., Weinberger K.Q. *On Calibration of Modern Neural Networks.* ICML 2017
- Kuhn L., Gal Y., Farquhar S. *Semantic Uncertainty.* ICLR 2023 · Kadavath S. et al. *Language Models (Mostly) Know What They Know.* 2022
- Madras D., Pitassi T., Zemel R. *Predict Responsibly: Improving Fairness and Accuracy by Learning to Defer.* NeurIPS 2018
- Mozannar H., Sontag D. *Consistent Estimators for Learning to Defer to an Expert.* ICML 2020
- Verma R., Barrejón D., Nalisnick E. *Learning to Defer to Multiple Experts.* AISTATS 2023

**Agents, tools, skills**
- Yue Y. et al. *MasRouter: Learning to Route LLMs for Multi-Agent Systems.* 2025. arXiv:2502.11133
- Qin Y. et al. *ToolLLM: Facilitating LLMs to Master 16000+ Real-world APIs.* 2023. arXiv:2307.16789
- Du Y., Wei F., Zhang H. *AnyTool: Self-Reflective, Hierarchical Agents for Large-Scale API Calls.* 2024. arXiv:2402.04253
- *Enrich-Retrieve-Rank: Discovery for Models, Agents, Tools and Skills at Scale.* 2026. arXiv:2608.22695
- *SCOUT: Scalable Tool Discovery for MCP Gateways.* PayPal, 2026. arXiv:2608.23992
- *Task2MCP / T2MRec: MCP Server Recommendation.* 2026. arXiv:2604.17234
- *Semantic Tool Discovery for MCP.* 2026. arXiv:2603.20313 · *SchemaRouter.* arXiv:2608.21375 · *CacheRouter.* arXiv:2608.22708 · *Nexus.* arXiv:2608.20397 · *ToolDNS.* arXiv:2607.18242 · *ComplexMCP.* arXiv:2605.10787
- *Best Prefix Selection: Optimal Skill Selection under Token Budgets.* 2026. arXiv:2608.19993
- *SkillFeed: Personalized Skill Routing.* 2026. arXiv:2608.28241
- *Compositional Skill Routing.* arXiv:2606.18051 · *CaSKG.* arXiv:2608.25500 · *A Systems Foundation for Agentic Skills.* arXiv:2608.29596 · *Microskill Architecture.* arXiv:2606.05720
- *Agent-as-a-Router.* arXiv:2606.22902 · *SWE-Router.* arXiv:2607.00053 · *Agentic Routing: a harness-native data flywheel.* arXiv:2607.11399 · *Routing Is Least Learnable Where It Is Most Valuable.* arXiv:2608.06171 · *LLMs Predict Failure Risk but Not Which Collaboration Protocol Pays Off.* arXiv:2608.14927
- *BudgetMem.* ICML 2026. arXiv:2602.06025 · *Gated-Memory Routing.* arXiv:2609.00237
- *LatentRouter.* arXiv:2605.11301 · *VLM Routing for Computer-Use Agents.* arXiv:2603.12823 · *Modality Escalation.* arXiv:2607.05438

**Economics & mechanism design**
- Chen H. et al. *Error-Aware Reverse Auction Mechanism for Large Language Model Routing (EA-RAM).* 2026. arXiv:2608.12719
- *Strategy Auctions for Small Agents.* ICML 2026. arXiv:2602.02751
- *Coalition Formation and Communication Pricing in Skill-Based Agentic AI.* 2026. arXiv:2608.07532
- Ghodsi A. et al. *Dominant Resource Fairness.* NSDI 2011

**Sequential decisions & queueing**
- Levin E., Pieraccini R., Eckert W. *A Stochastic Model of Human-Machine Interaction for Learning Dialog Strategies.* IEEE TSAP 2000
- Young S., Gašić M., Thomson B., Williams J.D. *POMDP-Based Statistical Spoken Dialog Systems: A Review.* Proc. IEEE 2013
- Koole G., Mandelbaum A. *Queueing Models of Call Centers: An Introduction.* Annals of OR 2002
- Kingman J.F.C. *The Single Server Queue in Heavy Traffic.* 1961 · Little J.D.C. *A Proof for the Queuing Formula L = λW.* 1961

**Security**
- Shafran A., Schuster R., Ristenpart T., Shmatikov V. *Rerouting LLM Routers.* 2025. arXiv:2501.01818
- *ROPE: Routed Origin Policy Enforcement against Indirect Prompt Injection.* 2026. arXiv:2608.27496
- *Verifiable Manifest Signing for MCP Pipelines.* 2026. arXiv:2601.23132
- *Recognition Without Enforcement.* 2026. arXiv:2608.28502
- *Beyond Max Tokens: Resource Amplification via Tool Chains.* 2026. arXiv:2601.10955
- *When Safety Routing Breaks.* 2026. arXiv:2609.01455
- *Malicious Prompt Detection via Bootstrap Aggregation.* 2026. arXiv:2602.08062

**Benchmarks**
- Hu Q.J. et al. *RouterBench: A Benchmark for Multi-LLM Routing System.* 2024. arXiv:2403.12031
- Huang Z. et al. *RouterEval: A Comprehensive Benchmark for Routing LLMs.* 2025. arXiv:2503.10657
- *LLMRouterBench: A Unified Benchmark for LLM Routing.* 2026. arXiv:2601.07206
- Feng T., You J. et al. *LLMRouter / xRouteBench: Routing as Sequential Decision Making.* 2026. arXiv:2608.06867
- *RouterXBench.* 2026. arXiv:2602.11877
- *How Much of the Routing Gap Is Real? Label Noise and the Oracle.* 2026. arXiv:2607.03436
- *Most of the LLM Routing Gap Is Task Type.* 2026. arXiv:2608.23023
- *When is Routing Meaningful? Diversity and Perturbation Robustness.* 2026. arXiv:2607.09197
- *Characterising Energy per Request and per Token for LLM Serving.* IISWC 2026. arXiv:2608.28044

**Drift**
- Page E.S. *Continuous Inspection Schemes.* Biometrika 1954 · Hinkley D.V. 1971
- Bifet A., Gavaldà R. *Learning from Time-Changing Data with Adaptive Windowing (ADWIN).* SDM 2007

**Systems**
- vLLM Project. *vLLM Semantic Router.* github.com/vllm-project/semantic-router · technical report arXiv:2603.04444
- LMSYS. *RouteLLM.* github.com/lm-sys/RouteLLM
- ulab-uiuc. *LLMRouter.* github.com/ulab-uiuc/LLMRouter · *GMTRouter.* github.com/ulab-uiuc/GMTRouter
- *Generating Workflow DAGs from Natural-Language Routing Rules.* 2026. arXiv:2608.30250

---

<!-- Page: Open Capability Manifest (https://opensmartroute.ai/docs/ocm) -->

<!-- OpenSmartRoute: Open Capability Manifest. Source https://github.com/isathish/OpenSmartRoute/blob/main/spec/ocm/README.md; HTML https://opensmartroute.ai/docs/ocm -->
# Open Capability Manifest (OCM) v1

A vendor-neutral, machine-readable description of **one routable capability** – an LLM, agent,
skill, persona, tool, workflow, human queue or destination – so that catalogues can be published,
validated and consumed by any router or gateway, not only OpenSmartRoute.

* Schema: [capability.schema.json](https://github.com/isathish/OpenSmartRoute/blob/main/spec/ocm/capability.schema.json) (JSON Schema 2020-12)
* Reference validator / converter: `opensmartroute.ocm` (`osr validate path/`)
* A `targets.yaml` entry with an `ocm: "1"` field *is* a manifest; `load_targets` accepts mixed files,
  a single manifest, or a directory of `capability.yaml` files.

```yaml
ocm: "1"
id: acme/support-agent
kind: agent
description: Handles orders, refunds and account issues via internal tools.
capabilities: { domains: [customer_support], actions: [action, qa], supports_tools: true }
constraints: { data_boundary: private, regions: [eu], pii_allowed: true }
cost: { usd_per_call: 0.02 }
latency_ms: { p50: 1500, p95: 4000 }
endpoints:
  - { protocol: a2a, url: https://agents.acme.com/support }
  - { protocol: http, url: https://eu.agents.acme.com/support, region: eu }
examples: ["I want a refund for order #12345, it arrived damaged."]
quality_prior: 0.8
```

## Field notes

| Field | Meaning |
|---|---|
| `id` | Stable, lowercase; namespaced with `/` (`publisher/name`). |
| `kind` | `llm` · `agent` · `skill` · `persona` · `tool` · `workflow` · `human` · `destination`. |
| `capabilities` | What it can do; `max_complexity`/`min_complexity` in `[0, 1]`. |
| `constraints` | Where it may run: `regions`, `data_boundary`, `pii_allowed`, `tenants`. |
| `cost` | `usd_per_call` for agents/workflows/tools/humans, `usd_per_1k_tokens` for models; energy and carbon optional. |
| `latency_ms` | A number or `{p50, p90, p95, p99}`; routers use `p50` for scoring. |
| `endpoints` | How to call it. `http` and `a2a` bind to executors directly (`bind_endpoint`); `mcp`/`queue` are wired by the operator. A `region` on an endpoint lets the router pick the in-region one. |
| `examples` | Representative prompts (fuel the similarity strategy and the eval dataset). |
| `quality_prior` | Prior quality belief before feedback (leaderboard score, past win rate). |
| `primary` | `false` for plan-slot-only capabilities (a persona layered on a model). |

## Versioning

`ocm` is a major version string. Additive fields ship under the same major; breaking changes bump it.
Unknown top-level fields are rejected so typos surface at `osr validate` time – extension data
belongs under `metadata`.

<!-- Section: Security -->

---

<!-- Page: Security model (https://opensmartroute.ai/docs/SECURITY) -->

<!-- OpenSmartRoute: Security model. Source https://github.com/isathish/OpenSmartRoute/blob/main/docs/SECURITY.md; HTML https://opensmartroute.ai/docs/SECURITY -->
# Security Model

OpenSmartRoute is an **LLM control plane**: it decides *who* answers. That makes its integrity a
security property in its own right (Shafran et al., "Rerouting LLM Routers", 2025). This document
lists assets, threats, and the controls implemented, mapped to OWASP Top 10 / OWASP LLM Top 10.

## Assets

1. Routing decisions (integrity, availability)
2. Request content (confidentiality – may contain PII / secrets)
3. Learner state (integrity – poisoning changes future decisions)
4. Configuration & credentials of downstream targets
5. Audit trail

## Threat model & controls

| # | Threat | OWASP ref | Control | Code |
|---|---|---|---|---|
| T1 | **Confounder gadgets** – attacker appends token soup to force routing to the expensive model (cost DoS) or to a weaker model (quality attack) | LLM01, LLM04 | Head/tail naturalness analysis; signals computed on cleaned head; optional reject; **learned** gadget detector (`InputGuard(learned=True)`, trained with `osr train --gadget`); router-directed sentences ("this is extremely complex", `[complexity=1]`) are scrubbed before signal extraction | `security.InputGuard`, `security.gadget.GadgetDetector`, `security.injection.strip_steering`, `GuardMiddleware` |
| T2 | **Prompt injection into LLM judge** – request text tells the judge which target to pick | LLM01 | Request wrapped in a fenced block; role markers and code fences neutralised; control/zero-width chars stripped; judge output parsed as strict JSON, item by item: non-object items, unparseable / NaN scores and ids not in the candidate list are dropped, scores clipped to [0,1]; judge only enabled below a confidence threshold; judge failures carry zero confidence so the ensemble ignores them | `security.sanitize_for_prompt`, `LLMJudgeStrategy` |
| T3 | **Policy bypass** – PII to a cloud model, EU data to US region, tenant reading another tenant's targets | A01 Broken access control | Policy evaluated *before* scoring and cannot be outweighed; tenant middleware; allow/deny lists | `policy.Policy`, `TenantMiddleware` |
| T4 | **Data leakage via logs / cache / feedback** | A09 Logging failures, LLM06 | Logs contain hash+length only; cache keys are SHA-256; `Redactor` masks PII before routing; feedback store records ids, not text | `LoggingTelemetry`, `CacheMiddleware`, `Redactor`, `FeedbackStore` |
| T5 | **Learner poisoning** – fake outcomes to steer routing | LLM03 Training-data poisoning | Outcomes only accepted via `learn()` from your code path (auth is your boundary); graded rewards clipped to [0,1]; Bayesian priors + confidence ramps limit the impact of few samples; Page–Hinkley drift alerts on sudden shifts; state files atomic and hash-addressable; corrupt / incompatible state is quarantined on load instead of being trusted or crashing (`AutoLearner.quarantined`); federated `merge()` adds only evidence beyond the prior | `AutoLearner`, `math.estimators`, `learning.merge_learners` |
| T6 | **Resource exhaustion** – huge prompts, unbounded state | A04 Insecure design / DoS | `max_chars`/`max_tokens` guard; every in-memory map is bounded; per-target rate limits and budgets; breaker isolates failing targets | `InputGuard`, `realtime` |
| T7 | **Path traversal in state store** | A01 | Keys hashed to filenames; writes are atomic to a fixed root | `FileStateStore` |
| T8 | **Secrets in config** | A02 Cryptographic failures / A05 Misconfig | Targets YAML never contains credentials; `load_secret()` reads env or `*_FILE` | `security.load_secret` |
| T9 | **Deserialisation** | A08 Software & data integrity | Only JSON / `yaml.safe_load`; no pickle anywhere | `TargetRegistry.from_file`, learners |
| T10 | **Audit tampering** | A09 | Hash-chained audit records; the chain resumes from the last record across restarts (no second genesis) and `FileAuditSink.verify(path)` re-walks it naming the first edited, removed or unparseable line | `FileAuditSink` |
| T11 | **Supply chain** | A06 Vulnerable components | Zero runtime dependencies in core; optional extras pinned by minimum version; CI runs `bandit`, `ruff -S`, `mypy` | `pyproject.toml`, CI |
| T12 | **Jailbreak content reaching a model** | LLM01 | `jailbreak_risk ≥ 0.5` → only `safety`-tagged targets or humans are admissible; `run_safety_suite()` / `osr safety` regression-tests that safety routing survives paraphrase, gadgets and steering | `Policy.check`, `security.safety` |
| T13 | **Poisoned catalogue entries** – MCP tool descriptions, A2A agent cards or skill files carry instructions to the router or model | LLM01, A08 | Instruction-injection lexicon + learned gadget score on every imported description; poisoned tools dropped at import time and reported; signed MCP manifests (HMAC-SHA256 or Ed25519, freshness window) so only vetted catalogues load | `security.injection`, `adapters.mcp.tools_from_mcp`, `sign_manifest` / `verify_manifest` |
| T14 | **Parameter provenance** – a tool result or retrieved document smuggles a value into a state-changing tool call ("send the refund to *this* account") | LLM01, A01 | `OriginPolicy`: sensitive parameters of state-changing tools must originate in the user turn; violations raise `OriginViolation` before execution | `security.provenance` |
| T15 | **Resource amplification in agentic plans** – runaway loops, recursive tool calls, token / cost blow-up | A04, LLM10 | Per-task caps on steps, tool calls, depth, tokens, cost and wall-clock enforced as middleware; multi-resource knapsack bandit checks hard meters **before** commitment and audits every pruned arm | `security.limits.ResourceLimiter`, `ResourceLimitMiddleware`, `math.bandits.MultiKnapsackBandit` |
| T16 | **Learner state at rest** – snapshots reveal usage patterns or are modified on disk | A02 | AES-256-GCM `EncryptedStateStore` with key rotation (`cryptography` extra); versioned store refuses to load unknown schema versions | `enterprise.stores.EncryptedStateStore`, `VersionedStateStore` |

## Secure defaults

- Core has **no network access**; it never calls a model unless you wire an `LLMJudgeStrategy`
  or a target `handler`.
- Logging is content-free, and so is tracing: span and event attributes carry ids, numbers, short labels
  and a text digest + length (`observability.text_digest`), never the request text. The hosted platform
  reads traces per workspace only (`GET /api/v1/trace/{request_id}` is 404 for another workspace's id).
- The LLM judge is off by default and, when on, only consulted below a confidence threshold.
- `Redactor` is opt-in (`GuardMiddleware(redact=True)`); when on, the original text is kept only in
  `request.context["_pii_map"]` for the executor and is never persisted by the SDK.

## What OpenSmartRoute does **not** do

- Authenticate callers. Put it behind your API gateway / service mesh.
- Encrypt state at rest by default. Use disk/volume encryption or wrap your store in
  `enterprise.stores.EncryptedStateStore`.
- Guarantee the downstream target's safety. Routing to a "safe" target is a policy you write.

## Threat-model delta: 0.3 -> 0.4

What changed for operators upgrading, and which residual risks remain:

| Area | 0.3 | 0.4 | Residual risk |
|---|---|---|---|
| Gadget / steering (T1) | heuristic head/tail analysis | + learned detector, + steering-sentence scrub before signal extraction | detector is trained on synthetic gadgets; re-train on your traffic (`osr train --gadget`) |
| Judge injection (T2) | fenced prompt, strict JSON | + per-item filtering, id allow-list, NaN / type guards, zero-confidence failures | a compromised judge can still bias *ranks* among legitimate ids; keep `escalate_llm_judge_below <= 0.5` |
| Catalogue poisoning (T13) | none | injection lexicon + gadget score on imported descriptions; signed manifests | lexicon is English-centric; sign manifests for anything you did not author |
| Parameter provenance (T14) | none | `OriginPolicy` for state-changing tools | policy is per tool / parameter; unlisted tools are not checked |
| Resource amplification (T15) | per-target rate limits and budgets | + per-task caps (steps, calls, depth, tokens, cost, wall-clock), knapsack meters before commitment | caps are enforced in the routing process; a handler that ignores the returned budget can still spend |
| Learner integrity (T5) | atomic writes | + quarantine of corrupt / incompatible state, versioned schema, federated merge adds evidence only | a *valid-looking* poisoned snapshot is indistinguishable from real learning; restrict write access to the store |
| Audit (T10) | hash chain per process | + chain continuity across restarts, `verify()` | the log file itself must be write-once at the OS level to make verification meaningful |
| State at rest (T16) | plain JSON | + AES-256-GCM store with rotation | key management is yours (`load_secret`) |
| Deployment | none | non-root container, hardened Helm chart (read-only rootfs, drop all capabilities, NetworkPolicy) | the chart exposes `/route` unauthenticated inside the cluster; front it with your gateway |

External review of this delta is outstanding; [SECURITY_REVIEW.md](https://opensmartroute.ai/docs/SECURITY_REVIEW.md) is the pack a reviewer
works from and its review log is where the report is linked (tracked in
[ROADMAP.md](https://opensmartroute.ai/docs/ROADMAP.md#v10-stable-api-complete)).

## Hardening checklist for operators

- [ ] Run `osr serve` behind TLS termination and authn/z (OIDC / mTLS).
- [ ] Enable `GuardMiddleware(redact=True)` for any tenant that may send PII.
- [ ] Set `TargetConstraints.pii_allowed=False` and `data_boundary` on every cloud target.
- [ ] Configure `HealthRegistry.configure(rate_per_s=…, budget_limit=…)` per expensive target.
- [ ] Ship `MetricsTelemetry` to your monitoring; alert on `AutoLearner.drifted` and breaker opens.
- [ ] Rotate and restrict access to the state directory; back it up (it *is* your learned policy).
- [ ] Alert on `AutoLearner.quarantined` (corrupt state was replaced) and run `FileAuditSink.verify()`
      on the audit log from a read-only replica.
- [ ] Sign MCP manifests you import (`osr mcp-manifest`) and fail closed on `verify_manifest` errors.
- [ ] Run `osr safety --learned-guard` in CI so safety routing regressions block a release.
- [ ] Pin the package version; watch the security advisories on this repo.

## Reporting

See [SECURITY.md](https://opensmartroute.ai/docs/security-policy.md).

---

<!-- Page: External review pack (https://opensmartroute.ai/docs/SECURITY_REVIEW) -->

<!-- OpenSmartRoute: External review pack. Source https://github.com/isathish/OpenSmartRoute/blob/main/docs/SECURITY_REVIEW.md; HTML https://opensmartroute.ai/docs/SECURITY_REVIEW -->
# External security review pack

Everything an independent reviewer needs to assess the OpenSmartRoute control plane against the threat
model in [SECURITY.md](https://opensmartroute.ai/docs/SECURITY.md): scope, trust boundaries, the evidence the project already produces,
how to reproduce it, the questions we want answered and how a report is published. It exists so that the
review named in the [roadmap](https://opensmartroute.ai/docs/ROADMAP.md#v09-security-hardening-of-the-control-plane-complete) can be run
by a third party without a briefing call.

## 1. Engagement

| | |
|---|---|
| Subject | `opensmartroute` 1.0.0 (`pip install opensmartroute`), the `osr serve` image, the hosted platform at `https://api.opensmartroute.ai` / `https://opensmartroute.ai` |
| Type | White-box: full source, tests, CI and deployment code are public in this repository |
| Rules of engagement | Test against your own `osr serve` / `python -m osr_platform.app` instance or a free-plan workspace you created; no load testing of the hosted platform; no access to other workspaces' data |
| Contact | `security@opensmartroute.org` or GitHub private vulnerability reporting ([../SECURITY.md](https://opensmartroute.ai/docs/security-policy.md)); acknowledgement within 72 h |
| Publication | Findings fixed or accepted first; then the report is linked in section 7 and in the roadmap |

## 2. System under review

```
caller ── HTTPS ──> platform/web (Next.js, proxies /api,/v1,/mcp) ──> platform/api (FastAPI, osr_platform)
                                                                        │  API keys, sessions, plans, metering,
                                                                        │  workspaces, audit, admin
                                                                        └──> opensmartroute (SDK: signals,
                                                                             policy, strategies, learners)
                                                                                └──> targets (Azure OpenAI,
                                                                                     MCP tools, agents, humans)
self-hosted: caller ──> osr serve (src/opensmartroute/server.py) ──> opensmartroute ──> targets
library:     your code ──> Router / RouterBuilder ──> targets
```

Trust boundaries, from the outside in:

1. **Internet -> web / API.** Anonymous: landing, docs, `/api/v1/models`, `/rankings`, `/stats/public`,
   `/status`, `/estimate` (rate limited), `/signup` (5 per IP per day). Everything else needs an API key
   (`osr_live_`, hashed at rest) or a browser session (`osr_sess_`, 30 days, SSO via PKCE with a signed
   state). Admin routes need `X-Admin-Token`. Cross-workspace reads are 404 by construction
   (`db.usage_by_request(account, request_id)` scopes every trace / event lookup).
2. **`osr serve`.** Optional bearer / `X-API-Key` middleware (`serve --token`, `--require-auth`,
   `OSR_SERVER_AUTH_TOKENS`); `OPEN_PATHS` are the probes, `/metrics`, `/whoami` and the OpenAPI UI. The
   Helm chart and the Docker image are documented as "front it with your gateway".
3. **Router -> request text.** `InputGuard` (size, gadget and steering scrub, optional PII redaction) runs
   before signals; `Policy` runs before scoring and cannot be outweighed; the LLM judge sees a fenced,
   sanitised prompt and its output is parsed item by item.
4. **Router -> catalogue.** Imported MCP tool descriptions, A2A cards and `SKILL.md` files pass the
   injection lexicon and the learned gadget score; MCP manifests can be signed (HMAC-SHA256 / Ed25519).
5. **Router -> execution.** `OriginPolicy` (parameter provenance), `ResourceLimiter` (steps, calls, depth,
   tokens, cost, wall-clock), breakers and budgets sit between a decision and a handler call.
6. **Router -> state.** JSON only, atomic writes, hashed keys, versioned schema, quarantine of corrupt
   state, optional AES-256-GCM at rest, hash-chained audit with `verify()`.

## 3. Threat model

[SECURITY.md, "Threat model & controls"](https://opensmartroute.ai/docs/SECURITY.md#threat-model--controls) lists T1-T16 with the OWASP
Top 10 / OWASP LLM Top 10 mapping and the implementing code; ["Threat-model delta: 0.3 -> 0.4"](https://opensmartroute.ai/docs/SECURITY.md#threat-model-delta-03---04)
lists the residual risk we already know about per threat. The reviewer is asked to confirm, refute or
extend that table rather than start from a blank page.

## 4. Evidence the project produces

| Evidence | Where | How to reproduce |
|---|---|---|
| Static analysis on every push: `ruff` (incl. the `S` bandit rules), `bandit`, `mypy` | [.github/workflows/ci.yml](https://github.com/isathish/OpenSmartRoute/blob/main/.github/workflows/ci.yml) | `python -m ruff check src tests scripts; python -m bandit -q -r src -c pyproject.toml; python -m mypy src` |
| CodeQL `security-and-quality` for Python and GitHub Actions, on push and weekly | [.github/workflows/codeql.yml](https://github.com/isathish/OpenSmartRoute/blob/main/.github/workflows/codeql.yml) | Security tab of the repository |
| Unit tests for the controls (guard, gadget detector, sanitiser, provenance, limits, signed manifests, audit chain, encrypted store) | [tests/test_security.py](https://github.com/isathish/OpenSmartRoute/blob/main/tests/test_security.py) (49 tests), [tests/test_enterprise_ops_stores.py](https://github.com/isathish/OpenSmartRoute/blob/main/tests/test_enterprise_ops_stores.py) | `python -m pytest tests/test_security.py -q` |
| Safety-routing red-team suite (paraphrase, gadget and steering attacks against safety routing) gates CI and every release | `security.safety.run_safety_suite`, [ci.yml](https://github.com/isathish/OpenSmartRoute/blob/main/.github/workflows/ci.yml) | `osr -t examples/targets.yaml safety --learned-guard` |
| Platform authorisation tests (API keys, sessions, workspaces, SSO state, device flow, admin token, trace scoping) | [platform/api/tests](https://github.com/isathish/OpenSmartRoute/blob/main/platform/api/tests) (108 tests) | `PYTHONPATH=src:platform/api python -m pytest platform/api/tests -q` |
| Zero runtime dependencies in the SDK core; optional extras imported lazily | [pyproject.toml](https://github.com/isathish/OpenSmartRoute/blob/main/pyproject.toml) | `pip install opensmartroute && pip show opensmartroute` (Requires: none) |
| Signed releases: SLSA build provenance and PEP 740 attestations, trusted publishing (OIDC, no long-lived PyPI token) | [.github/workflows/release.yml](https://github.com/isathish/OpenSmartRoute/blob/main/.github/workflows/release.yml) | `gh attestation verify dist/opensmartroute-*.whl --owner isathish` |
| Container hardening: non-root image, read-only root filesystem, dropped capabilities, NetworkPolicy, PDB in the Helm chart | [deploy/Dockerfile](https://github.com/isathish/OpenSmartRoute/blob/main/deploy/Dockerfile), [deploy/helm/opensmartroute](https://github.com/isathish/OpenSmartRoute/blob/main/deploy/helm/opensmartroute) | `helm template deploy/helm/opensmartroute` |
| Content-free logs and traces (ids, digests and lengths only) | `observability.text_digest`, `LoggingTelemetry` | grep the test suite for `text_digest`; run `osr serve` with a `FileSink` and inspect |
| Hash-chained audit log with tamper detection | `enterprise.FileAuditSink.verify` | edit one line of an audit file and run `verify()` |

## 5. Questions for the reviewer

Ordered by the impact a positive finding would have.

1. **Rerouting (T1, T2).** Can a request author steer the decision to a chosen target through the text
   alone - gadgets the learned detector misses, steering the scrub does not remove, or judge-prompt
   escapes past the sanitiser? Success = a reproducible prompt that moves a target's propensity by more
   than the ensemble's stated confidence margin on `examples/targets.yaml`.
2. **Policy bypass (T3).** Does any strategy, plan slot, fallback, cascade or `exclude`/`candidates`
   path reach a target that `Policy.check` rejected (PII to a `pii_allowed: false` target, region or
   tenant constraint)? The platform's `/api/v1/policy` overlay is in scope.
3. **Cross-workspace access (platform).** Any read or write that crosses `account_id`: traces, events,
   activity, savings, API keys, invites, SSO connections, marketplace listings, admin routes without
   the admin token, IDOR on `request_id`.
4. **Authentication (platform).** API-key hashing and comparison, session fixation / expiry, SSO PKCE
   and state signature, the RFC 8628 device flow (`slow_down`, code entropy, approval binding), signup
   rate limiting, the OpenAI-compatible `/v1` proxy's key handling.
5. **Catalogue poisoning (T13, T14).** An MCP tool description, A2A card or `SKILL.md` that passes the
   import filter and changes routing or smuggles a parameter past `OriginPolicy`.
6. **Learner poisoning (T5).** Feedback (`/feedback`, `Outcome`) sequences that move a target's learned
   quality faster than the documented prior / confidence ramp allows, or state files that load and
   are trusted when they should be quarantined.
7. **Resource amplification (T6, T15).** Inputs or plans that escape `InputGuard` limits, `ResourceLimiter`
   caps or the knapsack meters; unbounded growth in any in-memory map or state store.
8. **Data at rest and in logs (T4, T16).** Any code path that writes request text to logs, traces, cache
   keys, feedback, audit or metrics; weaknesses in `EncryptedStateStore` (nonce reuse, key rotation).
9. **Supply chain (T11).** Release workflow, attestation coverage, Docker base image, the `install.sh`
   / `install.ps1` bootstrap scripts served from the web app.

## 6. Known gaps (do not re-report)

- The SDK does not authenticate callers; `osr serve` is unauthenticated unless `--require-auth` or
  tokens are configured, and the Helm chart says so.
- The gadget detector ships trained on synthetic gadgets; the injection lexicon is English-centric.
- A compromised LLM judge can still bias ranks among legitimate candidate ids.
- A valid-looking poisoned learner snapshot is indistinguishable from real learning; write access to the
  state store is the boundary.
- Audit verification is only meaningful if the log file is write-once at the OS level.
- Encryption at rest is opt-in; key management is the operator's (`load_secret`).

## 7. Review log

| Date | Reviewer | Scope | Report | Findings (open / fixed) |
|---|---|---|---|---|
| - | none yet | - | - | - |

A completed review adds a row here, a "Threat-model delta" row in [SECURITY.md](https://opensmartroute.ai/docs/SECURITY.md) for anything
that changed, and a CHANGELOG entry for every fix. Until a row exists, the roadmap lists the independent
report as an open adoption item under v1.0, not as a shipped control: `python scripts/release.py readiness`
reads this table and reports the row as open while the only entry is the placeholder above.

---

<!-- Page: Reporting vulnerabilities (https://opensmartroute.ai/docs/security-policy) -->

<!-- OpenSmartRoute: Reporting vulnerabilities. Source https://github.com/isathish/OpenSmartRoute/blob/main/SECURITY.md; HTML https://opensmartroute.ai/docs/security-policy -->
# Security Policy

## Supported versions

| Version | Supported |
|---|---|
| 0.5.x | yes |
| 0.4.x | security fixes only |
| < 0.4 | no |

## Reporting a vulnerability

Please **do not** open a public issue. Email `security@opensmartroute.org` (or use GitHub's
private vulnerability reporting on this repository). Include a proof-of-concept and the
version affected. You will receive an acknowledgement within 72 hours and a fix or
mitigation plan within 14 days for high/critical issues.

## Scope

- Routing control-plane integrity (adversarial rerouting, confounder gadgets)
- Prompt-injection into the LLM-judge strategy
- PII / sensitive-data leakage through logs, caches, traces, or feedback stores
- Policy bypass (region, data boundary, tenant, budget)
- Denial of service via oversized inputs or state growth

See `docs/SECURITY.md` for the threat model and the controls implemented, and
`docs/SECURITY_REVIEW.md` if you are running an independent review.

<!-- Section: Agent skills -->

---

<!-- Page: osr-contributing (https://opensmartroute.ai/docs/skills/osr-contributing) -->

<!-- OpenSmartRoute: osr-contributing. Source https://github.com/isathish/OpenSmartRoute/blob/main/.claude/skills/osr-contributing/SKILL.md; HTML https://opensmartroute.ai/docs/skills/osr-contributing -->
---
name: osr-contributing
description: Work inside the OpenSmartRoute repository itself. Add a new strategy, signal, adapter or module to the codebase under src/opensmartroute; update the frozen public API snapshot in tests/public_api.json; keep the version literal and pyproject in sync; pass the ruff, mypy and pytest quality gates and the docs gate (docs/REFERENCE.md regenerated, every docs/*.md linked from README); write or update docs/*.md and .claude/skills SKILL.md packages so they render on the documentation site; follow the zero-dependency, settings and branding single-source-of-truth rules and the design patterns; write tests and CHANGELOG entries; prepare a release. Use when contributing code or documentation to the repo, fixing a failing gate, or reviewing a pull request against project conventions.
license: Apache-2.0
compatibility: OpenSmartRoute repository, Python >= 3.10
metadata:
  author: opensmartroute
  osr-domains: "coding"
  osr-tags: "opensmartroute contributing repo conventions"
  osr-quality-prior: "0.9"
  osr-primary: "false"
---

# Contributing to OpenSmartRoute

## Gates (run all four before finishing any change)

```
python -m ruff format src tests
python -m ruff check src tests          # rules E,F,I,UP,B,S; line length 120
python -m mypy src                      # mypy_path=src, packages=opensmartroute; must be clean
python -m pytest tests -q               # pythonpath=src
python scripts/api_reference.py         # regenerate docs/REFERENCE.md after any public-name change
```

`tests/test_docs.py` fails when `docs/REFERENCE.md` is stale or an exported name has no summary;
`python scripts/api_reference.py report` lists the undocumented names. It also requires every
`docs/*.md` to be linked from `README.md` and every `.claude/skills/<name>/` to contain a `SKILL.md`.
`tests/test_docs_claims.py` (SDK) and `platform/api/tests/test_platform_docs_claims.py` (platform) fail when a
doc names an `osr` command / flag, an `OSR_*` variable, an HTTP endpoint or a `/platform/dashboard/<page>` that
does not exist, when a user-facing doc is missing from the site catalogue, or when a CHANGELOG section
lacks a date, body or compare link - fix the doc or the code, never the test.
Ad-hoc scripts: `$env:PYTHONPATH="src"; python -X utf8 ...` (the package is not installed in the venv).

## Hard rules

- **Zero runtime dependencies in the core.** Anything optional (`yaml`, `fastapi`, `httpx`,
  `sentence_transformers`, `opentelemetry`, `cryptography`) is imported lazily inside the function
  that needs it and mapped to an extra in `pyproject.toml` (`yaml`, `server`, `embeddings`, `otel`,
  `crypto`, `all`).
- **Public API is frozen.** `tests/public_api.json` snapshots `opensmartroute.__all__`. Adding a
  name = add it to the JSON (sorted; uppercase first, then `__version__`, then lowercase). Removing
  or renaming = major version bump.
- **Version literal.** `__version__ = "x.y.z"` stays a literal in `src/opensmartroute/__init__.py`
  and must equal `version = "x.y.z"` in `pyproject.toml` (`scripts/release.py` rewrites both).
  Read it elsewhere through `branding.version()`.
- **No hardcoded tunables.** Thresholds, weights, scales go in `settings.py` dataclasses and are
  read through `resolve(settings).<group>.<field>`; document new fields (they become
  `OSR_<GROUP>_<FIELD>` automatically).
- **No brand literals.** Use `branding.PACKAGE / CLI / env_key / error_code / metadata_key /
  logger / user_agent`. `osr-*` metadata keys come from `metadata_key(field)`.
- **Typed errors only.** Raise subclasses of `OpenSmartRouteError` (`ConfigurationError`,
  `ValidationError`, `NoRouteError`, `TargetUnavailableError`, `ExecutionError`, `SecurityError`,
  `StateStoreError`) with `details={...}`; never bare `Exception`.
- **Every exported name is documented.** Classes and functions carry a one-line docstring; module-level
  constants and type aliases carry a same-line `# comment` (or a comment on the line above). The
  first line becomes its row in `docs/REFERENCE.md`.
- Strategies must degrade to neutral scores on internal failure, not raise during `route()`.

## Where things go

```
core/        types + TargetRegistry            strategies/  Strategy subclasses (name = snake_case noun)
signals/     SignalExtractor subclasses        policy/      PolicyRule callables + Policy chain
math/        pure numerics, no routing types   learning/    learners that persist state
realtime/    breakers, budgets, health         enterprise/  builder, middleware, telemetry, stores, ops
security/    guards, limits, provenance        adapters/    external systems; lazy optional imports
eval/        datasets, metrics, frontiers      sdk.py       ComponentRegistry + decorators
observability.py  Tracer, Span, Event, sinks   (emit with current_tracer().event(...); names in EVENT_NAMES)
settings.py  tunables                          branding.py  names (leaf module, imports nothing internal)
```

## Adding a strategy

1. Subclass `Strategy` in `strategies/<name>.py`; set `name`; implement
   `score(request, signals, candidates) -> dict[str, StrategyScore]`; optional `update(outcome)`,
   `state()/load_state()` for persistence.
2. Take `settings: Settings | None = None`; add a `WeightSettings.<name>` default.
3. Export from `strategies/__init__.py` (`__all__`) and, if public, `opensmartroute/__init__.py` +
   `tests/public_api.json`. Add to `default_strategies()` only if it needs no external service.
4. Tests: deterministic (`seed=0`), one behavioural test + one persistence round-trip if stateful.
5. Document in `docs/ARCHITECTURE.md` (strategies table) and `README.md` "Strategies"; CHANGELOG
   under `[Unreleased]`; run `python scripts/api_reference.py` so `docs/REFERENCE.md` picks it up.

## Adding an adapter

Return `RouteTarget`s or handlers; never import the optional client at module import time; validate
external input (`ConfigurationError` with `details["path"]`), pass descriptions through
`security.description_risk` when they come from a third party.

## Documentation

- Layout: `README.md` short overview; `docs/GUIDE.md` walkthroughs (numbered H2s); `docs/SDK.md` decorator
  SDK and settings; `docs/ARCHITECTURE.md` pipeline and strategies table; `docs/MATH.md` numbered sections
  with module references; `docs/RESEARCH.md` idea-to-module map; `docs/ROADMAP.md` rows name the module;
  `docs/REFERENCE.md` is generated - never edit by hand and never `> docs/REFERENCE.md` (the script writes it).
- The Next.js documentation site (`platform/web`, `/docs/<slug>`) is end-user documentation for the
  hosted platform, the Python SDK and the pip package. It publishes `docs/*.md` (`docs/PLATFORM.md` is
  the platform user guide), the root `README`/`CHANGELOG`/`CONTRIBUTING`/`SECURITY`, `deploy/README.md`,
  `spec/ocm/README.md`, every `.claude/skills/*/SKILL.md` and a REST API reference generated from
  `platform/api/openapi.json`. Internal documents (`docs/PLATFORM_PLAN.md`, `docs/GO_TO_MARKET.md`,
  `docs/BRAND.md`, `docs/sales/*.md`, `platform/README.md`) stay in the repository and are linked from
  `README.md` only. A new user-facing `docs/*.md` needs an entry in
  `platform/web/src/lib/docs/catalogue.ts` and a link in `README.md`; skills are discovered
  automatically. Details: the `osr-platform` skill and `platform/README.md`.
- Markdown that renders well on the site: one H1, H2/H3 headings (they become the table of contents and
  the search index), GitHub-flavoured tables, fenced code with a language, ```` ```mermaid ```` for
  diagrams, relative links to other `.md` files (rewritten to site routes) and to source files
  (rewritten to GitHub). Do not rename headings that other files link to without updating the anchors.
- Skill packages: `.claude/skills/<name>/SKILL.md` with frontmatter `name` (= directory), `description`
  (<= 1024 chars, several request-like sentences - they are the similarity examples), `license`,
  `compatibility`, `metadata.osr-domains` (ontology names only, e.g. `coding general`),
  `osr-tags`, `osr-quality-prior` (0.8-0.95), `osr-primary: "false"`. Omit `osr-actions`. Add a
  representative query to `tests/test_skills.py` that must select the new skill for the plan slot;
  `osr skills .claude/skills` validates the folder.

## Tests

- `tests/test_router.py`, `test_math.py`, `test_security.py`, `test_enterprise.py`,
  `test_adapters.py`, `test_execution.py`, `test_sdk.py`, `test_integrations.py`, `test_public_api.py`.
- Use `tmp_path` for state, `monkeypatch` for env, `ComponentRegistry()` instances rather than the
  global `components`; restore `configure(before)` in a `finally` when touching settings.

## Release

Edit `CHANGELOG.md` `[Unreleased]` as you go (Added / Changed / Fixed). `scripts/release.py` (run by the
`Prepare release` workflow) bumps the version literal in both files and dates the section; the
`release.yml` workflow publishes. Do not hand-edit released sections. `scripts/release.py readiness`
computes the v1.0 readiness table (ROADMAP) from the repo - release rows block `check` for 1.x, adoption
rows (`docs/SECURITY_REVIEW.md` review log, `examples/leaderboard/results/README.md` *Listings* rows with
`accepted`, `ADOPTERS.md` *Production users* rows) are tracked only. Never fake a row - the table is the
honest state of 1.0. Version mirrors outside `release.py prepare`: platform/api/{pyproject.toml (version and
`opensmartroute[...]>=`), osr_platform/__init__.py}, platform/web/package{,-lock}.json (top two entries),
deploy/helm/opensmartroute/Chart.yaml (version + appVersion), azure.yaml `template@`, deploy/Dockerfile and
deploy/README.md image tags, README `OSR_VERSION=`, docs/sales/DEMO_SCRIPT.md, docs/SECURITY_REVIEW.md
subject; then regenerate `platform/api/openapi.json` and `docs/REFERENCE.md`.

---

<!-- Page: osr-decorator-sdk (https://opensmartroute.ai/docs/skills/osr-decorator-sdk) -->

<!-- OpenSmartRoute: osr-decorator-sdk. Source https://github.com/isathish/OpenSmartRoute/blob/main/.claude/skills/osr-decorator-sdk/SKILL.md; HTML https://opensmartroute.ai/docs/skills/osr-decorator-sdk -->
---
name: osr-decorator-sdk
description: Extend OpenSmartRoute in Python with the decorator SDK. Write a custom strategy scorer, a custom policy rule, a signal extractor, middleware or telemetry sink and register them with @strategy, @policy_rule, @signal, @middleware, @telemetry; declare handlers as targets with @tool, @skill, @agent or @target; assemble everything with ComponentRegistry and components.router() or components.builder(); load plugins; change default weights and thresholds through Settings, configure() and OSR_* environment variables. Use when the user wants custom routing logic, a new strategy or rule, or to register components with decorators.
license: Apache-2.0
compatibility: OpenSmartRoute >= 0.4, Python >= 3.10
metadata:
  author: opensmartroute
  osr-domains: "coding general"
  osr-tags: "opensmartroute sdk decorators settings"
  osr-quality-prior: "0.85"
  osr-primary: "false"
---

# OpenSmartRoute decorator SDK

`opensmartroute.sdk` stores *blueprints* (factories) in a `ComponentRegistry`; decorators are
transparent (they return the decorated object) and `osr.components` is the process-wide registry the
top-level decorators bind to. Use a private `ComponentRegistry()` in tests and multi-tenant apps.

## Declaring components

```python
import opensmartroute as osr

@osr.tool("weather", domains=["travel"], actions=["lookup"], cost=0.0, examples=["weather in paris"])
def weather(request, **kw):
    """Current weather for a city."""            # first docstring paragraph -> target.description
    return lookup(request.text)                  # handler(request, **kw) -> response

@osr.agent("planner", domains=["travel"], min_complexity=0.3, quality_prior=0.9, latency_ms=4000)
def planner(request, **kw): ...

@osr.skill("cite-sources", domains=["research"], primary=False)   # skill slot pre-processor
def cite(request):                                               # may return RouteRequest | str | None
    return "Always cite primary sources."

@osr.strategy(weight=0.8)                        # fn(request, signals, candidates) -> {id: 0..1 | StrategyScore}
def geo_affinity(request, signals, candidates):
    region = request.constraints.region
    return {t.id: 1.0 if region in t.constraints.regions else 0.4 for t in candidates}

@osr.strategy                                    # Strategy subclass; name inferred -> "recency"
class RecencyStrategy(osr.Strategy):
    def score(self, request, signals, candidates): ...
    def update(self, outcome): ...               # optional: learn from Outcome

@osr.signal(order=-1)                            # order < 0 runs before built-in extractors
def urgency(request, signals):
    return {"urgent": "asap" in request.text.lower()}   # Signals fields are set; other keys -> signals.extra

@osr.policy_rule                                 # fn(target, request, signals) -> rejection reason | None
def business_hours_only(target, request, signals):
    if target.kind is osr.TargetKind.HUMAN and request.context.get("after_hours"):
        return "human queues are closed"
    return None

@osr.middleware                                  # fn(request, next_) -> RouteDecision
def tag_tenant(request, next_):
    request.context.setdefault("tenant", "public")
    return next_(request)

@osr.telemetry                                   # Telemetry subclass only
class Collect(osr.enterprise.Telemetry):
    def on_decision(self, request, decision): ...
```

`@target(id, kind, *, name, description, domains, actions, languages, modalities, tags,
min_complexity, max_complexity, supports_tools, cost, latency_ms, quality_prior, examples,
instructions, primary, constraints, metadata, family, effort)` is the general form.

## Assembling

```python
router = osr.components.router(registry)                    # Router; registered strategies replace defaults
app = osr.components.builder(registry).with_auto_learning().build()      # EnterpriseRouter
RouterBuilder(registry).with_defaults().with_components(reg, middleware=False)  # selective wiring
```

- `registry.strategies() / weights() / extractors() / policy() / middlewares() / telemetry_sinks() /
  targets() / target_registry(base)` materialise fresh instances.
- `add(*instances, weight=)` registers ready-made objects; duplicates raise `ConfigurationError`
  (`register(..., replace=True)` to override); `unregister`, `clear`, `merge` manage the catalogue.
- Plugins: `registry.include("pkg.module")` (decorators run on import; `"pkg.module:setup"` calls
  `setup(registry)`), `registry.discover()` loads `opensmartroute.plugins` entry points.

## Settings (all tunables, no hardcoded numbers)

```python
from opensmartroute.settings import RoutingSettings, WeightSettings
osr.configure(routing=RoutingSettings(narrow_above=200), weights=WeightSettings(rules=3.0))
osr.Router(reg)                               # uses configured settings
osr.Router(reg, settings=osr.Settings())      # isolated library defaults
```

Groups: `routing` (softmax_temperature, slot_quality_floor, skill_set_margin, narrow_above,
narrow_to, cost/latency_norm_scale), `policy` (jailbreak_threshold, safety_tag, data_boundaries,
wildcard_language), `rules`, `capability`, `bandit`, `weights` (one per strategy name).
Environment overlay `OSR_<GROUP>_<FIELD>` (e.g. `OSR_WEIGHTS_CAPABILITY=1.5`); bad values raise
`ConfigurationError(details={"env": key})`. `osr settings [--json]` lists every key.

## Conventions when writing components

- Strategy names are snake_case nouns; decorated classes drop the `Strategy`/`Signal`/`Middleware`/
  `Telemetry` suffix automatically. Scores are clipped to `[0, 1]`; omit a candidate to abstain.
- Never read thresholds from literals - take a `settings: Settings | None = None` argument and call
  `opensmartroute.settings.resolve(settings)`.
- Brand-derived names come from `opensmartroute.branding` (`env_key`, `error_code`, `metadata_key`,
  `logger`, `user_agent`).

---

<!-- Page: osr-deploy-serve (https://opensmartroute.ai/docs/skills/osr-deploy-serve) -->

<!-- OpenSmartRoute: osr-deploy-serve. Source https://github.com/isathish/OpenSmartRoute/blob/main/.claude/skills/osr-deploy-serve/SKILL.md; HTML https://opensmartroute.ai/docs/skills/osr-deploy-serve -->
---
name: osr-deploy-serve
description: Run OpenSmartRoute as a service - the `osr serve` FastAPI app and its /route, /feedback, /targets, /stats, /healthz, /whoami and OpenAI-compatible /v1 endpoints, access tokens for the self-hosted server (`serve --token` / `--generate-token` / `--require-auth`, OSR_SERVER_AUTH_TOKENS, `osr token generate`, `osr login --url ... --token`), the self-operating autopilot (`serve --autopilot`, /autopilot/cycle), the Docker image and entrypoint environment (OSR_TARGETS, OSR_RULES, OSR_STATE, OSR_MODELS, OSR_SLM, OSR_AUTOPILOT, OSR_LLM_BASE_URL, OSR_HOST, OSR_PORT), the Helm chart (auth.tokens), OSR_* settings overrides, state volumes and health checks. Use when deploying, containerising, exposing the router over HTTP, securing it with tokens, or debugging a running instance.
license: Apache-2.0
compatibility: OpenSmartRoute >= 0.4, Python >= 3.10, Docker, Kubernetes/Helm 3
metadata:
  author: opensmartroute
  osr-domains: "coding general"
  osr-tags: "opensmartroute deploy docker helm fastapi"
  osr-quality-prior: "0.85"
  osr-primary: "false"
---

# OpenSmartRoute serve and deploy

## HTTP API (`curl -LsSf https://opensmartroute.ai/install.sh | sh`, or `pip install opensmartroute[server,yaml]`)

```
osr -t targets.yaml [-r rules.yaml] [-s /state] [-m models.json] serve --host 0.0.0.0 --port 8000
osr -t targets.yaml serve --generate-token            # prints an osr_local_ token once; all routes need it
osr -t targets.yaml serve --require-auth --token $T   # production: refuse to start without a token
```

| Method | Path | Body -> response |
|---|---|---|
| GET | `/healthz` | `{"status": "ok", targets, uptime_s}` (liveness) |
| GET | `/readyz` | readiness; 503 until the catalogue has targets and a probe request routes end to end |
| GET | `/metrics` | Prometheus text: `MetricsTelemetry.prometheus()` (`osr_route_decisions_total{target}`, `osr_route_latency_ms_bucket`, `osr_outcome_*`, `osr_decision_cache_*`) + tracer counters + `osr_targets` |
| GET | `/whoami` | `{service, edition: "self-hosted", version, auth_required, authenticated, targets}` - what `osr whoami` prints |
| GET | `/targets` | catalogue (`registry.to_dicts()`) |
| GET | `/stats` | feedback aggregates (+ `health` from an EnterpriseRouter, + `autopilot` status when enabled) |
| POST | `/route` | `{text, context, history, objective, constraints, kinds, top_k=3, plan=false}` -> `RouteDecision.to_dict()`; 422 when no route |
| POST | `/feedback` | `{request_id, target_id, success, quality, cost_usd, latency_ms, domains, complexity, preferred_over}` -> `{"status":"recorded"}` |
| POST | `/autopilot/cycle` | (with `--autopilot`) schedule an improvement cycle now -> `{scheduled, status}` |
| GET | `/v1/models` | OpenAI list: `auto` + every primary target id |
| POST | `/v1/chat/completions` | OpenAI body; `model: "auto"` or `"opensmartroute"` routes (plan on), a target id pins; 400 stream, 404 unknown model, 422 no route, 502 execution error |

Python: `from opensmartroute.server import create_app; app = create_app(router, proxy_aliases=..., autopilot=..., auth_tokens=[...])`
- mount it inside your own FastAPI app or pass an `EnterpriseRouter`-built `router`.

## Access tokens

- Open by default (sidecar on a private network). With one or more tokens (`--token`, `--generate-token`,
  `OSR_SERVER_AUTH_TOKENS=a,b`, `OSR_SERVER_AUTH_TOKENS_FILE=/run/secrets/tokens`, `ServerSettings.auth_tokens`)
  every path except `/healthz`, `/readyz`, `/metrics`, `/whoami`, `/docs`, `/openapi.json`, `/redoc` needs
  `Authorization: Bearer <t>` or `X-API-Key: <t>` (`server.OPEN_PATHS`, constant-time compare); otherwise
  401 + `WWW-Authenticate: Bearer`. `--require-auth` / `OSR_SERVER_REQUIRE_AUTH=1` fails start-up without a token.
- Tokens are opaque strings; `osr token generate [--count N]` mints `osr_local_...` ones
  (`credentials.generate_token`). Sign a CLI in with `osr login --url http://host:8000 --token <t>`
  (stored per profile under `~/.config/opensmartroute/credentials.json`, `OSR_CONFIG_DIR` overrides);
  `osr whoami` calls `/whoami`. There is no account or key registry on a self-hosted server - rotation is
  "add the new token, restart, remove the old one".
- Helm: `auth.tokens: [..]` (chart-managed Secret), `auth.existingSecret` (key `tokens`, comma separated),
  `auth.required`. Docker: pass the same `OSR_SERVER_*` variables; the entrypoint needs no change.

## Self-operation (`serve --autopilot`)

```
osr -t targets.yaml --slm slm.json -s /state serve --autopilot [--autopilot-interval 3600] \
    [--cache-dir DIR] [--catalogue cat.json] [--source NAME ...] [--tier TIER=TARGET ...] [--search Q ...] [--offline]
```

`learning.Autopilot` runs `SelfImprover` cycles in a daemon thread: on the schedule
(`OSR_SLM_AUTOPILOT_INTERVAL_S`), early when `DriftMonitor` (Page-Hinkley on outcome success /
quality, fed through `Router.observers`) alarms, or on `POST /autopilot/cycle`; never closer than
`OSR_SLM_AUTOPILOT_MIN_GAP_S`. Feedback becomes training rows through the router's request memory
(`OSR_SLM_AUTOPILOT_REMEMBER`); an accepted challenger is hot-swapped into the served `SLMStrategy`
and saved (to the served file when writable, else `<cache-dir>/slm.json`). A failing cycle is counted
in `/stats` -> `autopilot.errors`, never fatal. Container: `OSR_AUTOPILOT=1` (+ `OSR_CACHE_DIR`,
`OSR_CATALOGUE`, `OSR_AUTOPILOT_ARGS="--offline ..."`); the entrypoint prefers a previously promoted
`$OSR_STATE/autopilot/slm.json` over `OSR_SLM` on restart. Helm: `autopilot.enabled`,
`autopilot.intervalSeconds`, `autopilot.args` (needs `config.slm` + `persistence.enabled`).

## Container

```dockerfile
# deploy/Dockerfile installs .[yaml,server,fast] (numpy: autopilot retrains 10x faster); defaults OSR_TARGETS=/config/targets.yaml OSR_STATE=/state; EXPOSE 8000
docker build -t opensmartroute -f deploy/Dockerfile .
docker run -p 8000:8000 -v $PWD/examples:/config:ro -v osr-state:/state \
  -e OSR_TARGETS=/config/targets.yaml -e OSR_RULES=/config/rules.yaml opensmartroute
```

`deploy/entrypoint.sh` reads `OSR_TARGETS` (required), `OSR_RULES`, `OSR_STATE`, `OSR_MODELS`,
`OSR_SLM` (routing SLM from `osr slm train`, joins the ensemble as the `slm` strategy),
`OSR_HOST` (default `0.0.0.0`), `OSR_PORT` (default `8000`) and execs `osr ... serve`. Health check
hits `/healthz`. Mount `/state` on a persistent volume or learner state is lost on restart.

The `/v1/chat/completions` proxy only *executes* when LLM targets have handlers. Set
`OSR_LLM_BASE_URL` to any OpenAI-compatible endpoint (OpenAI, Azure OpenAI, vLLM, Ollama at
`http://host.docker.internal:11434/v1`, OpenRouter, LiteLLM) plus `OSR_LLM_API_KEY` (or
`OSR_LLM_API_KEY_ENV=<VAR>`); each LLM target's `metadata.model` is the upstream model name and
`OSR_LLM_MODEL` the default for targets without one. Without it the proxy still routes but LLM
targets answer 502 "has no handler". Verified end to end: `X-OSR-Target` / `X-OSR-Request-Id` headers,
`model: "auto"`, `"auto:cheap"`, a pinned target id, 404 unknown model, 400 stream.

## Helm

```
helm install osr deploy/helm/opensmartroute -f values.yaml
```
`values.yaml` covers image, replicas, HPA, ingress, service account, a ConfigMap for
`targets.yaml`/`rules.yaml`, extra env/`envFrom` for secrets (`OSR_MCP_KEY`, encrypted-state key)
and a PVC for `OSR_STATE`. With more than one replica use a shared `StateStore` (Redis/SQL) via a
small Python entrypoint, or run one writer and read-only replicas.

## Tuning without code

Every tunable is an `OSR_<GROUP>_<FIELD>` variable - run `osr settings` to list them with current
values. Common ones: `OSR_ROUTING_SOFTMAX_TEMPERATURE`, `OSR_ROUTING_NARROW_ABOVE`,
`OSR_POLICY_JAILBREAK_THRESHOLD`, `OSR_WEIGHTS_RULES`, `OSR_WEIGHTS_CAPABILITY`,
`OSR_BANDIT_CONFIDENCE_WINDOW`. Bad values fail fast at startup with `ConfigurationError`.

## Operating

- Route latency is sub-millisecond for local strategies; only an LLM judge adds network time - keep
  `escalate_llm_judge_below` low and use `TimeoutMiddleware`.
- Post `Outcome`s to `/feedback` from the caller that observed the result; `/stats` shows them.
- Watch `route_latency_ms.p95`, `policy_rejections_total`, `errors_by_type` from
  `MetricsTelemetry.snapshot()`; `GET /metrics` renders the same store as Prometheus text
  (`MetricsTelemetry.prometheus(namespace="osr", extra=...)`). Scrape it with a `ServiceMonitor` or
  `prometheus.io/scrape` annotations. `RouterBuilder.with_cache(ttl_s, max_size)` adds a decision cache whose
  hits and misses appear as `osr_decision_cache_*`; `with_audit(sink, outcomes=True)` chains outcomes too.
- Tracing: `GET /metrics` also carries the tracer's `osr_spans_total` / `osr_span_duration_ms` per stage,
  `GET /trace/{request_id}` returns every span and event of one request (`X-OSR-Trace-Id` on each
  response; send `traceparent` to join your own trace), `GET /events?name=route.*` filters the buffer.
  `OSR_OBSERVABILITY_LOG_EVENTS=true` for JSON lines, `OSR_OBSERVABILITY_EVENTS_FILE=/state/events.jsonl`
  for a file, `OSR_OBSERVABILITY_OTEL=true` (+ `otel` extra) for OpenTelemetry, `OSR_OBSERVABILITY_SAMPLE_RATE`
  under load. See `docs/OBSERVABILITY.md`. The hosted platform image reads the same variables; there the
  buffer is workspace-scoped (`GET /api/v1/trace/{request_id}` with outcomes and audit records,
  `GET /api/v1/events`, public `GET /api/v1/status` for status pages) - `osr-platform` skill.
- Use `osr -t targets.yaml eval data.jsonl --min-accuracy 0.9` in the image build pipeline so a
  catalogue change cannot ship a regression.
- Secrets: env or `<NAME>_FILE` mounts via `load_secret`; never bake them into the image or YAML.

---

<!-- Page: osr-enterprise-builder (https://opensmartroute.ai/docs/skills/osr-enterprise-builder) -->

<!-- OpenSmartRoute: osr-enterprise-builder. Source https://github.com/isathish/OpenSmartRoute/blob/main/.claude/skills/osr-enterprise-builder/SKILL.md; HTML https://opensmartroute.ai/docs/skills/osr-enterprise-builder -->
---
name: osr-enterprise-builder
description: Assemble a production OpenSmartRoute router with RouterBuilder - auto-learning (IRT, Bradley-Terry, LinUCB, Markov), health circuit breakers and budgets, tenant/cache/timeout/guard middleware, logging and metrics telemetry, hash-chained audit, Redis/SQL/encrypted state stores, calibration, retrieval narrowing, shadow/A-B rollout and fair share. Use when hardening a router for multi-tenant or high-volume use, persisting learner state, or adding observability.
license: Apache-2.0
compatibility: OpenSmartRoute >= 0.4, Python >= 3.10
metadata:
  author: opensmartroute
  osr-domains: "coding general"
  osr-tags: "opensmartroute enterprise builder middleware telemetry"
  osr-quality-prior: "0.85"
  osr-primary: "false"
---

# OpenSmartRoute enterprise builder

`RouterBuilder(registry, settings=None)` is a fluent builder; every `with_*` returns the builder and
`build()` returns an `EnterpriseRouter` (thread-safe facade: `route`, `learn`, `execute`, `run`,
`health_snapshot`). It raises `ConfigurationError` for an empty registry, no strategies, or duplicate
strategy names.

## Reference chain

```python
from opensmartroute.enterprise import (RouterBuilder, CacheMiddleware, TenantMiddleware, TimeoutMiddleware,
                                       LoggingTelemetry, MetricsTelemetry, FileAuditSink)
from opensmartroute.observability import MemorySink, MetricsSink
from opensmartroute.security import GuardMiddleware

metrics = MetricsTelemetry()
app = (
    RouterBuilder(registry)
    .with_rules(load_rules("rules.yaml"))            # declarative preferences (RulesStrategy)
    .with_defaults()                                 # capability + similarity + Thompson bandit
    .with_auto_learning(state_dir=".osr-state")      # IRT + Bradley-Terry + LinUCB + Markov, persisted
    .with_health(latency_slo_ms=3000)                # breakers, rate limits, budgets; HealthPolicy + HealthStrategy
    .with_middleware(GuardMiddleware(redact=True),   # order = execution order (outermost first)
                     TenantMiddleware({"acme": {"max_cost_per_1k": 0.005, "data_boundary": "private",
                                                "deny_targets": ["llm-frontier"]}}),
                     CacheMiddleware(max_size=4096, ttl_s=30),
                     TimeoutMiddleware(budget_ms=250))
    .with_telemetry(LoggingTelemetry(), metrics)     # on_decision / on_outcome / on_error
    .with_tracing(MemorySink(), MetricsSink())       # spans + events of every stage (docs/OBSERVABILITY.md)
    .with_audit(FileAuditSink("audit.jsonl"))        # hash-chained, tamper-evident
    .with_calibration(conformal_alpha=0.1)           # temperature scaling + conformal candidate sets
    .with_retrieval(narrow_above=500, narrow_to=50)  # BM25 + dense RRF narrowing for big catalogues
    .with_llm_judge(LLMJudgeStrategy(judge), escalate_below=0.6)
    .with_objective(Objective(cost=0.3))
    .build()
)
```

Other options: `with_strategy(s, weight)`, `with_policy(policy)` (wrapped by `HealthPolicy` when
health is on), `with_feedback(FeedbackStore)`, `with_state_store(store, save_every)`,
`with_queue_awareness(slo_ms)` (in-flight tracking + Erlang-C latency), `with_fair_share(weights)`,
`with_shadow(candidate_router, mode="shadow"|"ab")` (SPRT-judged), `with_router_options(**kw)`,
`with_components(registry)` (decorator SDK).

Optional learners and controls added with `with_strategy(...)` (each is a `Strategy`; those with a
`model` are persisted by `with_auto_learning` / `with_state_store` like the built-in ones):
`learning.HistoryTargetStrategy()` (multi-turn: history-target joint embeddings),
`learning.UserAdaptiveStrategy()` (per-user shrinkage on `context["user_id"]`),
`strategies.EdgeCloudStrategy()` (`metadata.tier` edge/cloud vs latency SLO and data boundary),
`strategies.TokenBudgetStrategy()` (elastic `<id>@<budget>` siblings), `strategies.AuctionStrategy()`
(bias-corrected second-price bids), `strategies.HiddenStateStrategy(state_fn, targets, dim)`,
`discovery.SchemaAwareStrategy()` (tool `input_schema` coverage). After the answer:
`signals.UncertaintyGate` as the `Cascade` quality gate, `strategies.ProtocolPolicy` to pick
single / cascade / aggregate / debate / handoff per request, `strategies.MixtureOfAgents` for top-k
aggregation, `strategies.SelfEscalation` + `wrap_stream` for mid-stream escalation and
`learning.HandoffPolicy` for permanent hand-off of failing agent tasks. Energy and carbon:
`math.EnergyModel` / `hardware_profile()` fill `cost.wh_per_1k_tokens` / `gco2_per_1k_tokens` that
`Objective(energy=, carbon=)` weighs.

## Middleware contract

```python
class Middleware(ABC):
    def __call__(self, request: RouteRequest, next_: RouteFn) -> RouteDecision: ...
```
Raise `SecurityError`/`ValidationError` to reject; mutate `request.constraints` to enforce tenant
defaults (that is what `TenantMiddleware` does); wrap `next_(request)` for caching/timing.

## Telemetry contract

`Telemetry.on_decision(request, decision)`, `on_outcome(outcome)`, `on_error(request, error)`. Never
log raw text - `LoggingTelemetry` logs a SHA-256 prefix + length. `MetricsTelemetry.snapshot()` gives
`decisions_by_target`, `route_latency_ms{p50,p95,p99}`, `policy_rejections_total`, `mean_confidence`.
`opensmartroute.adapters.OpenTelemetryTelemetry` needs the `otel` extra.

## Tracing contract

Telemetry sees decisions; the tracer sees *how* they were made. `with_tracing(*sinks_or_tracer,
sample_rate=)` (no arguments: `Tracer.from_settings()` / `OSR_OBSERVABILITY_*`) records a `request` span per
call with nested `route` / `plan` / `execute` spans and events `route.signals`, `route.policy`, `route.rank`
(per-strategy ms), `cache.hit|miss`, `tenant.rejected`, `route.slow`, `guard.*`, `health.breaker`,
`shadow.*`, `fairshare.*`, `learn.outcome`. Sinks: `MemorySink` (`trace(request_id)`), `MetricsSink`
(`prometheus()`), `LoggingSink`, `FileSink`, `adapters.optional.OpenTelemetrySink`. Inside your own
middleware emit with `current_tracer().event("name", **attrs)` - never pass text, only ids and numbers.
See `docs/OBSERVABILITY.md`.

## Persisting learner state

`with_auto_learning(state_dir)` writes JSON files; for shared infrastructure use
`enterprise.stores`: `RedisStateStore`, `SQLStateStore`, plus wrappers `EncryptedStateStore`
(needs `crypto` extra), `VersionedStateStore(Migration...)`, `BatchedStateStore`,
`NamespacedStateStore`. Pass with `with_state_store(store, save_every=10)`. Multi-replica: one
writer, readers reload; state is quarantined on load failure rather than trusted blindly.

## Closing the loop

```python
d = app.route(RouteRequest("...", constraints=RequestConstraints(tenant="acme")))
app.learn(Outcome(d.request_id, d.target.id, success=True, quality=0.9, cost_usd=0.002, latency_ms=1200,
                  domains=d.trace.signals.domains, complexity=d.trace.signals.complexity))
res = app.run("...")          # route -> execute plan -> learn, with the full middleware chain
```

`app.health_snapshot()` exposes breaker state per `target.health_key` (family-aware);
`app.shadow` returns the `ShadowMiddleware` for A/B statistics.

## Checklist before shipping

- Middleware order: guard -> tenant -> cache -> timeout.
- `TenantMiddleware(require=True)` unless anonymous traffic is expected.
- Set `latency_slo_ms`, `Objective.quality_floor`, and per-tenant `max_cost_per_1k`.
- Persist state (`state_dir` or a `StateStore`) and mount it in the container (`OSR_STATE`).
- Export `metrics.snapshot()` and keep the audit sink on durable storage.

---

<!-- Page: osr-evaluation (https://opensmartroute.ai/docs/skills/osr-evaluation) -->

<!-- OpenSmartRoute: osr-evaluation. Source https://github.com/isathish/OpenSmartRoute/blob/main/.claude/skills/osr-evaluation/SKILL.md; HTML https://opensmartroute.ai/docs/skills/osr-evaluation -->
---
name: osr-evaluation
description: Evaluate and benchmark an OpenSmartRoute router - author JSONL evaluation datasets, run `osr eval` with cost/quality frontiers, baselines, robustness, calibration (ECE, Brier, conformal) and ablation reports, load RouterBench-style presets, run off-policy evaluation of logged decisions, and gate CI on minimum accuracy. Use when measuring routing accuracy, comparing strategies, or building a regression suite for routing quality.
license: Apache-2.0
compatibility: OpenSmartRoute >= 0.4, Python >= 3.10
metadata:
  author: opensmartroute
  osr-domains: "data_analysis coding general"
  osr-tags: "opensmartroute evaluation benchmark calibration"
  osr-quality-prior: "0.85"
  osr-primary: "false"
---

# OpenSmartRoute evaluation

## Dataset format (JSONL, one row per request)

```json
{"text": "Hi there! How's it going?", "acceptable": ["llm-small"]}
{"text": "Prove sqrt(2) is irrational", "expected": "llm-frontier", "context": {"tier": "pro"}}
{"prompt": "Summarise this PDF", "scores": {"llm-small": 0.4, "llm-mid": 0.8}, "task_type": "summarize"}
```

Fields: `text` (or `prompt`), `expected` (single id), `acceptable` (list of ids), `context`,
`profile`, `task_type`, `scores` `{id: quality}` (RouterBench style), `samples` `{id: [floats]}`,
`tokens` `{id: tokens spent}` (effort / elastic siblings).
Unknown keys are ignored. A row is correct when the decision is `expected` or in `acceptable`; with
`scores`, `mean_quality` reports the quality of the chosen target.

Agentic tasks (`--agentic`) use a different JSONL: one multi-step task per line with a per-agent,
per-step success / latency table (`eval.agentic.AgentTask`):

```json
{"task_id": "t1", "steps": ["look up the order", "reason through the refund conflict step by step"],
 "success": {"fast-agent": [0.97, 0.45], "strong-agent": [0.94, 0.90]},
 "latency_ms": {"fast-agent": [400, 400], "strong-agent": [2000, 2000]}, "domain": "customer_support"}
```

Presets: `--preset routerbench|llmrouterbench|routereval|xroutebench|routerxbench` maps public
benchmark files onto your catalogue (`eval.datasets.load_benchmark(path, preset, model_map, limit)`).

## CLI

```
osr -t targets.yaml -r rules.yaml eval data.jsonl [--limit N] [--json]
    --frontier       2-D cost/quality frontier + area under it
    --frontier3      quality/cost/latency Pareto frontier + hypervolume
    --baselines      random / cheapest / best-prior / task-table / oracle (+ noise floor); --quality-floor 0.7
    --robustness     repeat flip-rate, paraphrase stability, profile-swap fairness (first 200 rows)
    --calibration    ECE, Brier, reliability diagram, conformal set coverage / size; plus held-out ECE after
                     fitting a temperature / isotonic calibrator on the even rows (>= 20 rows)
    --ablation       leave-one-strategy-out and leave-one-extractor-out deltas
    --multi-round    route-execute-judge-refine loop vs best single target (rows with scores); --threshold --max-rounds
    --effort         effort routing vs always-highest-effort (rows with scores + tokens; targets declare effort)
    --agentic        DATASET is agentic tasks; per-step routing vs best single agent on accuracy-latency; --retries N
    --min-accuracy X exit status 2 when accuracy < X (CI gate)
osr -t targets.yaml ope logs.jsonl [--max-weight 20]     # IPS / SNIPS / DR off-policy evaluation
osr -t targets.yaml safety [--learned-guard] [--json]    # red-team safety-routing suite
osr collect --cache-dir data --tier frontier=llm-frontier --tier mid=llm-mid --tier small=llm-small
    # ten public sources (routellm-battles, arena-55k/100k/140k, ppe-human, webdev-arena, mt-bench-human,
    # reward-bench, ultrafeedback, routellm-gpt4), resumable after a 429; --source NAME picks, --refresh refetches
osr -t targets.yaml slm train --out slm.json --cache-dir data --report   # every *-tiers cache; numpy when installed
    # judge --report against the constant baseline: "always mid" is acceptable ~64 % of the time on the arena corpora
OSR_SLM_ENCODER=attention osr ... slm train ... --report  # compare the transformer-block encoder on the holdout
osr slm eval slm.json data.jsonl --min-accuracy 0.8       # CI gate for the routing SLM itself
```

## Python

```python
from opensmartroute.eval import evaluate, load_dataset, cost_quality_frontier, area_under_frontier, calibration_report
rows = load_dataset("data.jsonl")
r = evaluate(router, rows)             # EvalResult
print(r.summary())                     # n, accuracy, mean_cost_per_1k, mean_latency_ms, mean_confidence,
                                       # mean_quality, ece, brier, abstain_rate, set_coverage, mean_set_size
r.confusion                            # {expected: {chosen: count}}
r.errors                               # rows that missed, with the decision trace
```

Also: `eval.baselines.baseline_suite`, `eval.robustness.{repeat_flip_rate, paraphrase_robustness,
profile_swap_fairness}`, `eval.frontier.hypervolume`, `eval.ope.off_policy_evaluate`.
Roadmap exit criteria as runnable measurements: `eval.criteria` (`cold_start_ratio`,
`effort_token_savings`, `multi_round_vs_best_single`, `match_at_1_at_scale`, `conformal_coverage`,
`knapsack_never_exceeds_cap`, `ope_within_live_ci`, `run_all`) and `eval.agentic.task_routing_frontier`;
`python scripts/exit_criteria.py [--full]` prints the table quoted in `docs/ROADMAP.md`.

## Interpreting results

- Accuracy below baselines' `best-prior` means the catalogue (examples / complexity bands) is wrong,
  not the weights. Fix data first.
- `ece > 0.1` -> confidence is miscalibrated: run `router.calibrate()` on logged outcomes or lower
  `OSR_ROUTING_SOFTMAX_TEMPERATURE`. `calibration_report()["held_out"]` shows what a fitted
  `TemperatureScaler` / `IsotonicCalibrator` reaches out of sample (on the public human-preference suites raw
  ECE 0.25-0.32 drops to 0.04-0.06 isotonic); ship the fitted calibrator rather than tuning weights.
- `--ablation` deltas near zero identify strategies you can drop to cut latency.
- `--robustness` flip-rate > 0 with identical inputs means a stochastic strategy (bandit) is
  dominating; seed it or lower its weight.
- Use `--frontier` when the goal is "same quality, less cost": pick the objective weights on the knee.

## CI gate pattern

```yaml
- run: osr -t examples/targets.yaml -r examples/rules.yaml eval examples/eval_dataset.jsonl --min-accuracy 0.9 --json
```

Keep the dataset in the repo, deterministic (`seed=0` strategies), and add a row for every routing
bug you fix.

---

<!-- Page: osr-integrations (https://opensmartroute.ai/docs/skills/osr-integrations) -->

<!-- OpenSmartRoute: osr-integrations. Source https://github.com/isathish/OpenSmartRoute/blob/main/.claude/skills/osr-integrations/SKILL.md; HTML https://opensmartroute.ai/docs/skills/osr-integrations -->
---
name: osr-integrations
description: Connect OpenSmartRoute to the outside world - OpenAI-compatible LLM clients and handlers, MCP tool catalogues (stdio client, signed manifests), A2A agent cards, agent harnesses (callable, HTTP, subprocess), LangGraph nodes and conditions, Microsoft Agent Framework executors, OpenAI function-calling tool specs, persona loaders and Agent-Skills SKILL.md packages. Use when wiring a real provider or framework into the router, importing tools or agents, or authoring a SKILL.md that the router can select.
license: Apache-2.0
compatibility: OpenSmartRoute >= 0.4, Python >= 3.10
metadata:
  author: opensmartroute
  osr-domains: "coding general"
  osr-tags: "opensmartroute integrations mcp a2a langgraph openai skills"
  osr-quality-prior: "0.85"
  osr-primary: "false"
---

# OpenSmartRoute integrations

Everything below produces `RouteTarget`s or handlers; the router itself has no I/O.

## LLM providers (`adapters.openai_compat`)

```python
from opensmartroute.adapters import OpenAICompatClient, chat_handler, embedder, judge_fn
openai = OpenAICompatClient()                                   # OPENAI_API_KEY (or *_FILE) from env
ollama = OpenAICompatClient("http://localhost:11434/v1", api_key="ollama")
registry.get("llm-frontier").handler = chat_handler(openai, model="gpt-4o", system_prompt=None)
registry.get("llm-small").handler   = chat_handler(ollama, model="llama3.1")
router = Router(registry,
    strategies=[CapabilityStrategy(), SimilarityStrategy(embedder=embedder(openai, "text-embedding-3-small"))],
    llm_judge=LLMJudgeStrategy(judge_fn(openai, "gpt-4o-mini")), escalate_llm_judge_below=0.5)
```
`chat_handler` returns `ChatResult(text, model, prompt_tokens, completion_tokens, latency_ms, raw)`;
execution derives cost from tokens x `usd_per_1k_tokens`. Local embeddings:
`sentence_transformers_embedder(model)` (`embeddings` extra).

## MCP tools (`adapters.mcp`)

```python
from opensmartroute.adapters import tools_from_mcp, connect_mcp, tools_from_manifest
targets = tools_from_mcp(tools_list_json, server="github", call=lambda name, args: client.call(name, args),
                         cost_per_call_usd=0.0, latency_ms=300, quality_prior=0.6, max_description_risk=0.6)
client, targets = connect_mcp(["npx", "-y", "@modelcontextprotocol/server-filesystem", "."], server="fs")
```
Ids are `server:tool`; `inputSchema` is kept in `metadata`; risky descriptions are dropped. Prefer
signed manifests in production (`sign_manifest` / `verify_manifest` / `tools_from_manifest`).

Which *server* to connect in the first place (`adapters.mcp_servers`):

```python
from opensmartroute.adapters import ServerCard, recommend_servers
cards = [ServerCard.from_dict(d) for d in registry_json]            # name, description, tools, tags, auth,
for rec in recommend_servers("open a pull request", cards, k=2,    #   latency_ms, region, data_boundary
                             constraints=req.constraints, allowed_auth=["none", "token"]):
    rec.server.name, rec.score, rec.rationale, rec.matched_tools
```
BM25 + hashed-dense fusion over the card text; hard filters on latency, region, data boundary and
auth kind run before ranking, so a private request never gets a public-only server recommended.

## vLLM semantic-router configs (`adapters.semantic_router`)

```python
from opensmartroute.adapters import load_semantic_router_config
imported = load_semantic_router_config("config.yaml")            # or a dict
router = Router(imported.registry, strategies=[RulesStrategy(imported.rules), *default_strategies()])
imported.categories, imported.default_model, imported.warnings
```
Categories with `model_scores` become `RouteTarget`s (`quality_prior` = mean score, cost from
`model_config.pricing`, `effort: high` when every category uses reasoning) plus one `Rule` per
category; `system_prompt` lands in the top target's `instructions`.

## A2A agents (`adapters.a2a`)

```python
from opensmartroute.adapters import fetch_agent_card, agent_from_card, skills_from_card, a2a_handler
card = fetch_agent_card("https://agents.example.com/travel")          # https only, /.well-known/agent.json
agent = agent_from_card(card, call=a2a_handler(card["url"], token=load_secret("A2A_TOKEN")))
for s in skills_from_card(card): registry.add(s)                       # primary=False, id "<agent>/<skill>"
```

## Agent harnesses (`adapters.harness`)

`CallableHarness(fn)` (`fn(task, context, history) -> str | dict | HarnessResult`),
`HTTPHarness(url, api_key_env=..., timeout_s=300)` (POST `{"task","context","history"}`),
`SubprocessHarness(argv, json_output=True, system_flag="--system")`. Wrap with
`harness_handler(harness)` and assign to a `kind: agent` target. Return `HarnessResult(text, success,
cost_usd, prompt_tokens, completion_tokens, quality)` so outcomes carry real cost/quality.

## Frameworks (`adapters.frameworks`)

```python
node = langgraph_node(router, execute=False, plan=False)      # state["messages"] -> route_target, route
graph.add_conditional_edges("route", langgraph_condition(by="target", default="__end__"), {...})
executor, handoffs = maf_router_executor(router)               # Microsoft Agent Framework
spec, handle = openai_tool_spec("route_request"), openai_tool_handler(router)   # function calling
decision, result = route_and_execute(router, text, history=..., context=..., execute=True)
```

## Personas and skills

- Personas: `load_personas(path)` accepts a directory of `*.md`, `.json`, `.jsonl` or `.csv`;
  `persona_target(name, prompt, description, domains=..., primary=False)`. Their `instructions` are
  composed into `context["system"]` as `# Persona: <name>` when they fill a plan slot.
- Skills: `load_skills(root)` globs `*/SKILL.md`; `load_skill(dir)` loads one.

### Authoring a SKILL.md the router can select

```markdown
---
name: sql-reporting                      # must equal the directory name; ^[a-z0-9]+(-[a-z0-9]+)*$, <= 64
description: Writes warehouse SQL for recurring reports. Use for SQL or analytics questions.   # <= 1024
license: Apache-2.0
metadata:
  osr-domains: "data_analysis coding"    # routing hints (space/comma separated)
  osr-languages: "en"
  osr-tags: "sql"
  osr-quality-prior: "0.82"
  osr-primary: "false"                   # slot-only: layered on a model in the plan
  osr-latency-ms: "0"
  osr-requires: "warehouse-schema"       # optional skill-graph edges, space/comma separated ids
  osr-conflicts: ""
  osr-composes: "chart-builder"
---
# Body = instructions, disclosed only when the skill is selected (progressive disclosure)
```
Only `name`/`description` are used for routing; write the description for the *request* the skill
should attract (it seeds `examples`, one per sentence). `osr-domains`/`osr-actions` must use the
signal ontology names (`signals.DOMAIN_LEXICON` / `ACTION_LEXICON`, e.g. `coding`, `data_analysis`,
`code_generation`, `reasoning`); an unmatched declared scope is penalised, so omit `osr-actions`
when the skill spans many request types. Validate with `osr skills <root>`; register with
`registry.add(skill)` or `--skills <root>` on the CLI; the router fills the plan's skill slot
(`router.route(req, plan=True)`) or scores skills directly via `candidates=[...]`.
`osr-requires` / `osr-conflicts` / `osr-composes` feed `discovery.SkillGraph(load_skills(root))`;
`graph.compose(["sql-reporting"])` returns the dependency-ordered skill set and the ids dropped for
conflicts, and `graph.observe(skill_ids, success)` learns co-usage companions.

## OpenAI-compatible proxy

`osr -t targets.yaml serve` exposes `POST /v1/chat/completions`; `model: "auto"` (or
`"opensmartroute"`) lets the router pick, a real target id pins it. See the deploy skill.

---

<!-- Page: osr-platform (https://opensmartroute.ai/docs/skills/osr-platform) -->

<!-- OpenSmartRoute: osr-platform. Source https://github.com/isathish/OpenSmartRoute/blob/main/.claude/skills/osr-platform/SKILL.md; HTML https://opensmartroute.ai/docs/skills/osr-platform -->
---
name: osr-platform
description: Work on the hosted OpenSmartRoute platform under platform/ - the FastAPI platform API (osr_platform, accounts, API keys, plans, metered /api/v1/route, OpenAI-compatible /v1 proxy, admin, editions, the committed openapi.json snapshot) and the Next.js 16 web app (landing, models catalogue, rankings, playground, pricing, dashboard, route-handler proxies). Add or change a page of the end-user documentation site (/docs) rendered from the repository Markdown via sync-content.mjs, lib/docs/catalogue.ts and lib/docs/render.ts, the generated REST API reference (/docs/api) or the version menu; publish a new docs file or skill; fix the docs build, search index or sitemap. Build and deploy both containers to Azure Container Apps with azd (azure.yaml, infra/, the platform GitHub workflow). Use when editing platform/api, platform/web, the documentation site, azure.yaml or infra/.
license: Apache-2.0
compatibility: OpenSmartRoute repository, Python >= 3.11, Node >= 22, Azure Developer CLI
metadata:
  author: opensmartroute
  osr-domains: "coding general"
  osr-tags: "opensmartroute platform nextjs fastapi docs azure azd"
  osr-quality-prior: "0.85"
  osr-primary: "false"
---

# OpenSmartRoute hosted platform

Two deployables under `platform/`, one repository - deployed as three services: the API (`osr-api`), the
public website (`osr-web`, `opensmartroute.ai/`, `/docs`) and the platform (`osr-platform`, everything under
`opensmartroute.ai/platform/...`; the web image with the website forwarding `/platform` to it). Read
`platform/README.md` first (developer
documentation, repository only); the end-user guide is `docs/PLATFORM.md`, published at `/docs/PLATFORM`.

| Directory | Service | Stack | Checks |
|---|---|---|---|
| `platform/api` | `osr-platform-api` | FastAPI, SQLite, `osr_platform` package | `ruff format --check platform; ruff check platform; mypy platform/api/osr_platform; pytest platform/api/tests -q` |
| `platform/web` | `osr-platform-web` | Next.js 16 App Router, React 19, Tailwind 4 | `npm run check && npm run build` (in `platform/web`; `check` = eslint + tsc + `vitest run` over `tests/*.test.ts`), then `npm run e2e` (Playwright: customer journey + operator console against the standalone build and a throw-away API bootstrapped with the `OPERATOR` from `e2e/global-setup.ts`; `OSR_E2E_WEB` / `OSR_E2E_API` / `OSR_E2E_OPERATOR_USERNAME` / `_PASSWORD` point it at servers you already run, e.g. the dev stack with `admin` / `admin-dev-only`) |

Run the whole platform in Docker with live updates: `docker compose up --watch` (root `compose.yaml`, project
`opensmartroute-dev`; dev images `platform/api/Dockerfile.dev` and `platform/web/Dockerfile.dev`, context = repository
root with their own `Dockerfile.dev.dockerignore`; `--profile llm` adds Ollama wired through
`deploy/compose/providers.ollama.yaml`; knobs `OSR_API_PORT`, `OSR_WEB_PORT`, `OSR_OLLAMA_PORT`, `OSR_OLLAMA_MODEL`).
Compose Watch syncs saves into the containers - new source folders or content sources need a `develop.watch`
entry there. The self-hosted production stack (PostgreSQL, Redis, Kafka) is `platform/docker-compose.yml`.
Or without Docker: `$env:PYTHONPATH="src;platform/api"; python -m osr_platform.app` (env `OSR_PLATFORM_*`,
port 8080) and `OSR_API_URL=http://localhost:8080 npm run dev` in `platform/web` (port 3000). The browser
only talks to the web origin: `src/app/api/[[...path]]`, `v1/[[...path]]` and `openapi.json` route
handlers stream to `OSR_API_URL`. A new API response header must be added to
`FORWARD_RESPONSE_HEADERS` in `src/lib/api/proxy.ts` or the browser never sees it.

## Platform API (`platform/api/osr_platform`)

- `app.py` builds the FastAPI app (`create_app(settings)`), `settings.py` maps `OSR_PLATFORM_<FIELD>`;
  `api.py` public/account routes, `proxy.py` OpenAI-compatible `/v1`, `admin.py`, `billing.py`, `docs.py`
  (JSON mirror of the Markdown at `/api/v1/docs`; the web site does not use it).
- Two editions: `OSR_PLATFORM_EDITION=community` (core `Router`) or `enterprise` (`RouterBuilder` with
  auto-learning, health, guard, metrics, hash-chained audit; enables tenants, stats, audit). Both are built
  in `editions.py` behind `Engine` (`controls()`, `health_snapshot()`, `observability_snapshot()`); the
  guard with PII redaction and `MetricsTelemetry` run in both.
- Observability (`observability.py`): `RequestContextMiddleware` (echo/generate `X-Request-Id`,
  `Server-Timing`, JSON access log `osr.platform.access`, `HttpMetrics` per route template), `/healthz`,
  `/readyz` (`readiness()` checks database, catalogue, router, audit file), `/metrics` (`prometheus_text`;
  `OSR_PLATFORM_METRICS_PUBLIC=false` gates it behind `X-Admin-Token`). Public catalogue GETs return
  through `api.cacheable(request, payload, max_age_s)` (ETag, `Cache-Control`, 304 on `If-None-Match`;
  `OSR_PLATFORM_HTTP_CACHE_MAX_AGE_S`). `OSR_PLATFORM_ROUTE_CACHE_TTL_S` adds `CacheMiddleware`;
  `OSR_PLATFORM_RETENTION_DAYS` drives `db.purge_usage` from a background sweep in `app.py`.
- Governance (`api.py`): `POLICY_KEYS` = tenant keys + `daily_budget_usd` / `monthly_budget_usd`;
  `validate_policy(config, registry)` (400 on unknown keys, unknown targets, allow/deny overlap) is used by
  both `PUT /tenants` and `PUT /policy`; `apply_policy(constraints, cfg)` merges (fill boundaries, stricter
  caps, union deny, intersect allow). `metered()` calls `check_budget` only when `meta["spends"]` (execute /
  chat) and records `tenant` on usage rows (`db.spend`, `db.spend_by_tenant`); exhaustion = 429 +
  `Retry-After` + `X-Budget-Limit|Used|Period`. `GET /governance` -> `PlatformAPI.governance(p)`. Web pages
  `app/platform/dashboard/governance/page.tsx` (all editions) and `app/platform/dashboard/health/page.tsx` (enterprise, `stats`).
  New response headers must be added to `EXPOSED_HEADERS` in `app.py` and `FORWARD_RESPONSE_HEADERS`.
- Traces and events (`api.py`): `event_buffer()` finds the tracer's `MemorySink`; `trace_payload(p, rid)`
  (`GET /trace/{request_id}`, 404 unless `db.usage_by_request(account, rid)` finds the row - always
  account-scope trace reads), `events_payload(...)` (`GET /events`, joined to the account through
  `db.request_ids(account, since)`; request-less events only with the `stats` feature), `status_payload()`
  (public `GET /status`, 503 when `readiness()` fails), `mark_traced(items)` on `/activity`,
  `trace_id_of(rid)` -> `trace_id` + `X-OSR-Trace-Id` on `/route`. Feedback is persisted per request
  (`db.record_feedback` / `db.feedback_for`, table `feedback`); `audit_records(p, rid)` scans the audit file
  for the request (plans with `audit`). Web: `components/observability/
  trace-view.tsx` (`TraceView`, `TraceLoader`, `stageTone`, `summarise`) used by `app/platform/dashboard/events/
  page.tsx`, the activity drawer and the playground `DecisionPanel`; `system-status.tsx` (`useStatus`,
  `StatusPill`, `SystemStatusCard`). Event attributes never carry prompt text - keep it that way.
- Learning (`editions.py`): `PlatformSettings.slm` -> `RouterSLM.from_file` + `builder.with_strategy(SLMStrategy)`;
  `PlatformSettings.autopilot` -> `_build_autopilot` (mirror of `cli._build_autopilot`: `SelfImprover` on
  `engine.core.feedback` / `.memory`, `Autopilot(tracer=engine.tracer)` so cycles land in the platform's tracer,
  `engine.core.observers.append(pilot.observe)`; `with_router_options(remember_requests=...)` keeps prompts in
  memory for feedback rows). A promoted model is saved to `settings.autopilot_dir / "slm.json"` and preferred
  over the mounted file on restart. `Engine.learning_snapshot()` (`GET /api/v1/learning`, any plan),
  `Engine.feedback_snapshot()` (NaN -> None via `_finite`), `slm_info(model)`, `Engine.start()/stop()` wired
  as FastAPI startup/shutdown handlers in `app.py`. Admin `POST /admin/autopilot/cycle` -> `pilot.trigger("api")`.
  Web: `app/platform/dashboard/learning/page.tsx` (nav icon `brain`), types `LearningSnapshot`, `SlmInfo`,
  `AutopilotStatus`, `ImprovementReport`. SDK side: `EnterpriseRouter._bookkeep` runs `Router.observers`;
  `SelfImprover` emits `learn.improve` / `learn.promote`. Tests `platform/api/tests/test_learning.py`.
- Marketplace import (`harvest.py`, `docs/MARKETPLACE.md` "Imported catalogue"): `SkillsShSource` (skills.sh
  sitemaps -> every `SKILL.md` per repo through `GitHub.tree/raw`), `GitHubSearchSource` (code search, token
  required, size buckets <= 1000 hits), `MCPRegistrySource` (stream `?version=latest`, active only -> `tool`
  rows). `listing_for(h)` builds the OCM manifest (`metadata.source = {provider, repository, path, ref, url}`,
  slug `owner-repo-skill` / `mcp-<name>`, <= 63 chars + hash), `import_rows(store, rows, refresh=)` writes in
  one transaction per batch, `Harvester(work_dir)` keeps a resumable JSONL cache per source. Admin
  `GET|POST /api/v1/admin/registry/import` (max 500 items) and `scripts/harvest_skills.py collect|stats|publish`
  drive it. `RegistryStore.facets()` is cached (`FACETS_TTL_S`, `invalidate()` on write); `sources()` counts
  listings per provider. Seeded manifests have `metadata.source` as a *string* - always `isinstance` guard.
  Tests swap the transport: `monkeypatch.setattr(harvest, "fetch", fake)`. Never `git add` the work dirs
  (`.osr-harvest*/`). Whole-catalogue publishes go OFFLINE (`publish --data-dir`, stop the API, replace
  `platform.sqlite3` on the Azure Files share, start it): sustained bulk POSTs over SMB corrupted the listings
  btree once ("database disk image is malformed"). The endpoint is for incremental batches. Keeping current:
  `MarketplaceRefresher` (harvest.py) runs in the process owning `marketplace` when
  `OSR_PLATFORM_MARKETPLACE_REFRESH_S > 0` (Bicep param `marketplaceRefreshS`, weekly) - cache under
  `<data>/harvest/`, `existing_slugs()` filter, batches of 100; `POST /admin/registry/import/refresh` = run now.
  Big catalogues need the covering indexes (`ix_listings_browse`, `ix_listings_popular`) BUILT OFFLINE and the
  file swapped: a cold `SELECT tags FROM listings` over 100k wide rows on Azure Files took 14 minutes.
- Signup `POST /api/v1/signup {email,name[,password]}` (5/IP/day). Route body: `text`, `objective` (Objective field
  names `quality/cost/latency`), `constraints`, `kinds`, `plan` (Pro-gated), `execute`.
- Operator console (`/platform/admin`, `api.admin_router()`): operators (`db.operators`, scrypt hashes via `auth.hash_password`
  / `verify_password`, `osr_op_` sessions 12 h, roles `operator` / `superadmin`) sign in at `POST /admin/auth/login`;
  the `self.admin` dependency yields an `OperatorPrincipal` from a Bearer operator token *or* `X-Admin-Token`
  (counts as superadmin) - keep every `/api/v1/admin/*` route on it, call `require_superadmin` for operator
  management and `audit(op, "<noun>.<verb>", ...)` for mutations (an `admin` event). `bootstrap_operator()` runs
  at start-up from `OSR_PLATFORM_ADMIN_USERNAME/_PASSWORD`. Users: `users.password_hash`, `POST /auth/password/login`,
  `POST|DELETE /auth/password`, `settings.password_login`. Web: `lib/auth/operator.tsx` (`OperatorProvider`,
  `useOperator`, `useAdminApi`; token in localStorage `osr.platform.operatorToken`), `components/admin/admin-shell.tsx`,
  pages under `app/platform/admin/(console)/`, `app/platform/admin/login`; `robots.ts` disallows `/platform/admin`.
- Shared state (`cluster.py`, `db.py` backends): `Database(path_or_url)` picks `SqliteBackend` or `PostgresBackend`
  (`to_postgres()` rewrites `?` -> `%s`, `INSERT OR IGNORE`, `REAL` -> `DOUBLE PRECISION`, `AUTOINCREMENT` -> identity;
  qualify columns in `ON CONFLICT ... DO UPDATE SET x=table.x+1`). Run the suite on PostgreSQL with
  `OSR_TEST_DATABASE_URL` (one schema per test). `make_limiter` -> `RedisRateLimiter` (Lua sliding window) when
  `OSR_PLATFORM_REDIS_URL`; `make_event_bus` -> `KafkaEventBus` when `OSR_PLATFORM_KAFKA_BOOTSTRAP`
  (`api.emit(topic, payload, key)`; topics `usage`, `feedback`, `admin`; `MemoryEventBus` in tests via
  `c.app.state.api.bus = ...`). `/readyz` reports `redis`, `events`, `database_dialect` without failing readiness.
  `platform/docker-compose.yml` = production stack (PostgreSQL, Redis, KRaft Kafka, API, web); `infra/`
  `clusterMode` adds `osr-postgres` / `osr-redis` / `osr-kafka` container apps (internal TCP ingress, one share
  each, `nobrl` mounts) and multi-replica `osr-api`. Kafka image: `CONTROLLER://localhost:9093` (0.0.0.0 fails
  validation), `CLUSTER_ID` must be 22 base64url chars; kafka-python `bootstrap_connected()` goes false after
  metadata - use `_metadata.brokers()` for health.
- SLM service (`slm_service.py`, entry `osr-platform-slm`): `SLMWorker` (cold-starts an untrained bundle to
  `autopilot_dir/slm.json`, `ingest(topic, payload)` for `training` {request_id,text} into `RequestMemory` and
  `feedback` into a `FeedbackStore`, `cycle()` = `SelfImprover.cycle`, Kafka consumer group `osr-slm`), FastAPI
  `/healthz /model /predict /reports /cycle` on `settings.slm_port` (8090). API side: `settings.slm_url`,
  `training_events` (`api.note_training` next to `note_decision` in /route and /v1 chat), `slm_reload_s`
  (`api.maybe_reload_slm()` swaps core/calibrator/targets/meta in place from the promoted file's mtime);
  `build_engine` serves the promoted file when `slm_url` is set even without `OSR_PLATFORM_SLM`. Admin proxies
  `/api/v1/admin/slm*` use urllib (404 without `slm_url`, 503 unreachable). Web `app/platform/admin/(console)/slm/page.tsx`.
  Tests `test_slm_service.py` (in-process TestClient + a stdlib HTTP relay for the proxy).
- Domain services (`services.py`): `SERVICES` specs (name, gateway prefixes, entry `osr-platform-<name>`), ports
  in `settings.SERVICE_PORTS` (docs 8091 ... admin 8098, mcp 8099, openai 8100, routing 8101, slm 8090),
  `create_service_app(name)` = `PlatformContext.routers(name)` + `/healthz /readyz /metrics`; the API includes
  `gateway_router` for a domain when `OSR_PLATFORM_<NAME>_URL` is set (`X-OSR-Service` on the answer,
  `text/event-stream` relayed chunk by chunk through `forward_async`). Include order in `create_platform_app`:
  rankings (`/stats/public`) before routing (`/stats`), routing (`/admin/autopilot`) + marketplace + providers
  before the `admin` catch-all, onboarding before accounts (`/auth`). `ctx.owns(name)` decides background work
  (owner of `routing` runs the autopilot and attaches the telemetry store). Adding a service = SERVICE_PORTS +
  SERVICES + `main_<name>` + pyproject script + `routers()` + loop tuple + both compose files + bicep
  `domainServices`/`domainServiceEnv` + platform/README table + `test_services.py`.
- Notifications (`notifications.py`): `NotificationStore` (tables `notification_channels`, `notifications` =
  one episode per alert id with `state` firing/resolved and `read_at`, `notification_deliveries`),
  `Notifier(api, interval_s)` = daemon loop started by the `api` role (`PlatformContext.start`), leader through
  `SET osr:alerts:leader NX EX` in Redis; `tick()` retries last round's failures, re-evaluates
  `api.alerts_payload(p)` for every active workspace and `alerts.deployment_alerts(...)` for the deployment,
  `_reconcile` opens/refreshes/resolves episodes, emits Kafka topic `alerts`, delivers `firing` / `escalated`
  / `resolved` through `send(ch, n, event, context)` (email via `api.mail`, webhook JSON + `X-OSR-Event`,
  `X-OSR-Delivery`, `X-OSR-Signature` = `sign(secret, body, ts)`, Slack Block Kit, Teams MessageCard; `https`
  only, `_post_json` never raises). `notifications_router` (workspace, `accounts` domain, tag `account`) and
  `admin_notifications_router` (deployment scope, `admin` domain) - add new alert rule names to the `rules`
  list in each router. Settings `alerts_interval_s` (`OSR_PLATFORM_ALERTS_INTERVAL_S`; tests pass `0.0` and call
  `notifier.tick(force=True)`). Web: `components/notifications/notifications-center.tsx` (shared by
  `app/platform/dashboard/notifications/page.tsx` with `useApi` and `app/platform/admin/(console)/notifications/page.tsx` with
  `useAdminApi`), `notifications-bell.tsx` in the dashboard header. Tests `test_notifications.py` (a stdlib
  `Sink` HTTP server receives the webhooks; header names arrive title-cased - compare lower-cased).
- Console extras (`admin_console.py`): `admin_console_router` (`admin` domain) serves `/analytics` (SQL grouped by
  the portable `usage.day` column; signups bucketed in Python), `/activity` (cross-workspace request log, cursor
  `before`), `/lookup?q=`, `/settings` (`settings_payload`: dataclass fields -> `OSR_PLATFORM_<NAME>`, `_mask` /
  `_scrub` hide secret-looking names and nested `client_secret` / `api_key` keys - add new secret markers to
  `_SECRET_MARKERS`, new groups to `_GROUPS`), `/retention` + `/retention/purge`; `admin_console_local_router`
  serves `/health` from the routing process (register it with the other local admin routers before the gateway).
  Web: `app/platform/admin/(console)/{analytics,health,activity,leads,plans,marketplace,settings,search}/page.tsx`;
  `ADMIN_NAV` in `components/admin/admin-shell.tsx` (also the header search box -> `/platform/admin/search?q=`).
  Tests `platform/api/tests/test_admin_extras.py`.
- CLI sign-in (`osr login`) is the RFC 8628 device grant in `auth_router()`: `POST /auth/device/code`
  (anonymous, `device:{ip}` limiter 10/min + 30/h, returns `user_code` `XXXX-XXXX` from
  `auth.generate_user_code`, `verification_uri = <web_url>/platform/cli/authorize`), `GET /auth/device/{user_code}`
  and `POST /auth/device/approve|deny` (signed in; approve needs `admin` and a free key slot; `db.decide_device_code`),
  `POST /auth/device/token` (`authorization_pending` / `slow_down` / `access_denied` / `expired_token` as
  HTTP 400 JSON; success mints a key via `create_key` and `consume_device_code`). Table `device_codes`
  stores only the SHA-256 of the device code (`DEVICE_CODE_TTL_S` 15 min). Web page:
  `app/platform/(auth)/cli/authorize/page.tsx` + `components/auth/device-authorize.tsx`. SDK side lives in
  `src/opensmartroute/credentials.py` (`device_login`, `CredentialStore`). Tests: `tests/test_device_flow.py`.
- The installer scripts `install.sh` / `install.ps1` at the repository root are copied by `sync-content.mjs`
  and served verbatim by `app/install.sh/route.ts` and `app/install.ps1/route.ts` (`lib/installers.ts`);
  install one-liners for the site come from `install` in `lib/config/site.ts`.
- Chat proxy metadata key is `opensmartroute`; `models[]` = candidate list with fallbacks; `stream` = SSE.
- Tests use `platform/api/tests/conftest.py` fixtures (temp data dir, `examples/targets.yaml`).
  Smoke a deployment with `python platform/api/scripts/smoke.py <base-url>`.
- `platform/api/openapi.json` is a committed snapshot of the enterprise-edition OpenAPI document
  (`OPENAPI_TAGS`, `SECURITY_SCHEMES` and `openapi_snapshot()` in `app.py`). After changing any route,
  model or docstring run `python -X utf8 platform/api/scripts/export_openapi.py`;
  `test_openapi_snapshot_is_current` and the CI `--check` step fail on drift. Tag descriptions become
  the intro of each `/docs/api/<tag>` page; the `admin` and `docs` tags and every `/api/v1/admin/*`
  path (`HIDDEN_TAGS`, `HIDDEN_PATH` in `lib/docs/openapi.ts`) are hidden from the site; give a new tag
  a title in `TAG_TITLES`.

## Web app (`platform/web`)

- Route groups: `(marketing)` (landing, marketplace, models, rankings, estimate, pricing, roi,
  compare, plus the prose pages `support`, `terms`, `privacy` built on
  `components/marketing/prose-page.tsx`), `(docs)`, and everything signed-in under `platform/` - the
  **platform** half of the site, served at `/platform/...`: `platform/(auth)` (login, signup, password
  recovery, `auth/callback`, `invite/[token]`, `cli/authorize`), `platform/dashboard/`, `platform/admin/`
  (operator console) and `platform/(marketing)` (playground, `marketplace/publish` - public-site chrome,
  the layout re-exports `(marketing)/layout.tsx`). `/platform` itself redirects to the dashboard. Write
  page paths with the prefix (`/platform/dashboard/keys`, `/platform/login`); `src/lib/routes.ts` has
  `PLATFORM_PREFIX`, `platformPath()` and `LEGACY_PLATFORM_PATH` (the pre-split paths that `src/proxy.ts`
  308-redirects). In production the website and the platform are two container apps running the same
  image: `osr-web` has `OSR_WEB_PLATFORM_URL` and forwards `/platform` requests to `osr-platform`
  (`NextResponse.rewrite`); a single process (`next dev`, Compose) serves both. Site-wide names and nav live
  in `src/lib/config/site.ts`: `headerNav` (`groups` render as navigation-menu panels on desktop and
  sections in the mobile sheet, `links` are top-level; icon names resolve in
  `components/layout/nav-icons.tsx`), `footerNav` (Product / Developers / Resources / Company groups)
  and `legalNav` (footer bottom bar). Every layout
  renders `<main id="main">` - the header's skip link targets it. New public pages also go into
  `SearchCommand`'s `PAGES`, `app/sitemap.ts` and the container smoke step of `.github/workflows/platform.yml`.
  Server-side API reads in `src/lib/api/server.ts` (`serverApi`, memoised); client calls in `src/lib/api/client.ts`.
- UI toolkit: `src/components/ui` is the shadcn/ui `new-york` kit (`components.json`: rsc, slate,
  CSS variables, lucide icons; aliases `utils -> @/lib/utils/format`, `ui -> @/components/ui`,
  `hooks -> @/hooks`). Add a component with `npx shadcn@latest add <name>`, then fix the generator
  output: it writes `import { cn } from "cn"` (must be `@/lib/utils/format`), it may scaffold
  `next-themes`/dark-mode code (remove it - the dark surface is the `theme-dark` class, not a
  theme provider) and its `Math.random`/`setState`-in-effect patterns fail the React Compiler lint.
  Radix comes from the `radix-ui` umbrella package only - never add `@radix-ui/react-*`.
  Brand tokens map onto the kit variables in `globals.css` (`--background`, `--muted-foreground`,
  ...): use `text-muted-foreground`, `bg-muted`, `border-input`, `ring-ring` rather than legacy
  `text-muted`. Legacy-named wrappers kept for call sites: `Badge`/`Card`/`Alert`/`EmptyState`/
  `StatCard` in `ui/card.tsx`, `Button`/`ButtonLink` (`external`) in `ui/button.tsx`, `Input`/
  `Textarea`/`SearchInput`/`Select`/`Switch`/`Field` in `ui/field.tsx`, `Avatar`/`Progress`/
  `Separator` in `ui/primitives.tsx`, `DataTable` + `Table*` (`TableRow interactive`, `TableCell
  numeric`) in `ui/table.tsx`, `FilterChip` in `ui/toggle.tsx`, `SegmentedControl` in `ui/tabs.tsx`.
  `Select` renders a sizing wrapper around the native `<select>`: width/height/text-size classes go
  on `className` (e.g. `className="h-8 w-52 text-xs"`), never padding. No hand-rolled `<table>`,
  `<select>`, `<details>`, chip buttons or dropdowns in pages - reach for the kit component.
- Design system (light, editorial; see the website sections of `docs/BRAND.md`): fonts come from
  `next/font` in `app/layout.tsx` - Inter (`--font-sans`, also `font-display`), Instrument Serif
  (`--font-serif`, weight 400 only, never `font-bold`), Geist Mono (`--font-mono`), Quicksand only
  for the wordmark (`font-wordmark`). Utilities in `globals.css`: `heading-serif` for display titles
  with `text-display-sm/md/lg/xl`, `eyebrow` (mono uppercase muted label), `container-x` (76rem).
  Marketing sections are `<section className="border-b border-line py-20 sm:py-28">` with
  `SectionHeading` (`eyebrow`, serif `title` with an `<em className="text-slate-ink">` second clause,
  `description`; left-aligned by default, `as="h1"` for the page title, `size` sm/md/lg) and
  `divide-y divide-line` ledgers with `font-mono text-sm text-muted-foreground tabular-nums`
  numerals - not icon cards. Alternate bands use `bg-snow`. Buttons: `primary`/`default` = ink fill,
  `accent` = brand-blue (rare), `secondary` = mist, `outline`, `inverse` on dark panes. Inline links
  are `text-ink underline decoration-ink/30 underline-offset-4 hover:decoration-ink`, highlighted
  cards use `border-ink` (no coloured rings), hover borders `hover:border-ink/30`, shadows hairline
  (`shadow-card`) or none. The header is always `h-16`; the docs topbar sticks at `top-16` and the
  docs sidebar at `top-[6.75rem]`. Dashboard page titles are `text-xl font-semibold` (`PageHeader`);
  the dashboard sidebar is `bg-sidebar` (snow) and the content area white. Docs prose stays plain.
- Next 16 lint rules (react-hooks v7): no `setState` directly in a `useEffect` body, no ref writes
  during render, use `<Link>` for internal paths (even `/api/...`).
- SEO: every public page exports `metadata = pageMetadata({ title, description, path, keywords?, type? })`
  from `src/lib/seo.ts` (canonical, Open Graph, Twitter card, merged `seo.keywords` from
  `lib/config/site.ts`; `noIndex` for one-time links) - dynamic routes call it from `generateMetadata`
  and return `robots: { index: false }` for unknown ids. Structured data goes through
  `<JsonLd data={...}>` (`components/seo/json-ld.tsx`, escapes `<`) with the builders in `lib/seo.ts`
  (`organizationJsonLd`/`websiteJsonLd` in the root layout, `softwareJsonLd` on the landing page,
  `breadcrumbJsonLd` + `articleJsonLd` on docs/compare pages, `offersJsonLd` + `faqJsonLd` on pricing,
  a `SoftwareApplication` per marketplace listing, `itemListJsonLd` on `/vendors`, `/vendors/[vendor]`
  and `/compare`). The default `og:image` is the generated card `app/og/route.tsx`
  (`/og?title&subtitle&kicker`, `next/og` + Poppins read from `public/brand/fonts` with `fs` - the
  standalone server has no asset `fetch`; clamped text; `ogImageUrl()` builds the
  URL, `absoluteTitle` pages keep the static social card). `robots.ts` disallows `/platform/dashboard`, `/api/`, `/v1/`,
  `/mcp`, `/auth/`, `/cli/`, `/platform/invite/` and `?view=`/`?plan=`/`?next=` variants; `sitemap.ts` lists
  canonical paths only (no query strings) including `/vendors/<vendor>` and `/compare`. Programmatic hubs:
  `(marketing)/vendors` (from `serverApi.llms().vendors`) and `(marketing)/vendors/[vendor]`
  (models table + FAQ, `vendorHref()` in `components/models/reference-model.tsx`); `(marketing)/compare`
  indexes `COMPARISONS`. Machine feeds: `app/llms.txt/route.ts` (llmstxt.org map built from
  `docSections()`), `app/feed.xml/route.ts` (Atom from `releases()`, advertised through
  `ALTERNATE_TYPES` in `pageMetadata` and the root layout - a page-level `alternates` replaces the
  layout's), `app/indexnow.txt/route.ts` (serves runtime `INDEXNOW_KEY`, 404 when unset) with
  `scripts/indexnow.mjs` submitting the sitemap after deploy (workflow step, `OSR_INDEXNOW_KEY`).
- Analytics: `components/analytics/google-analytics.tsx` (rendered once in the root layout) loads the
  Google tag with Consent Mode v2 defaults denied, replays the stored choice, sends `page_view` per
  client navigation and mounts `ConsentBanner`; it renders nothing without
  `NEXT_PUBLIC_GA_MEASUREMENT_ID` or under `/platform/dashboard`. `lib/analytics.ts` exposes `track(event,
  params)` (used for the GA4 `sign_up` / `login` conversions in `components/auth/*`; when
  `analytics.googleAdsConversions[event]` is set it also fires the Ads `conversion` hit), `trackCta`,
  `trackOutbound`, `trackWebVital` (`useReportWebVitals` -> LCP/INP/CLS events), `writeConsent`
  and `subscribeConsent` (`useSyncExternalStore`, never setState-in-effect). Server components mark
  CTAs with `data-track="<name>" data-track-location="<where>"`; one delegated listener in
  `GoogleAnalytics` turns those and every external `<a>` into events, and `CopyButton` sends
  `copy_code`. Ids and verification
  tokens are build args (`azure.yaml` `OSR_GA_MEASUREMENT_ID`, `OSR_GOOGLE_ADS_ID`,
  `OSR_GOOGLE_ADS_CONVERSIONS`, `OSR_GOOGLE_SITE_VERIFICATION`, `OSR_BING_SITE_VERIFICATION`); the
  privacy page documents the tag and renders `ConsentControls`.
- `output: "standalone"`; the image is `platform/web/Dockerfile` (context = `platform/web`). A running
  standalone server (`node .next/standalone/server.js`) holds `.next/standalone` locked on Windows and
  makes `next build` fail with EBUSY - stop it first.

## Documentation site (`/docs`)

The site is **end-user documentation** for the hosted platform, the Python SDK and the pip package.
It renders the repository Markdown itself at build time - no API call, fully static. Internal material
(`docs/PLATFORM_PLAN.md`, `docs/GO_TO_MARKET.md`, `docs/BRAND.md`, `docs/sales/`, `platform/README.md`)
stays in the repository and is deliberately not in the catalogue; Markdown links to it fall back to
GitHub.

1. **Content snapshot** - `node scripts/sync-content.mjs` copies `README.md`, `CHANGELOG.md`,
   `CONTRIBUTING.md`, `SECURITY.md`, `docs/**` (minus `docs/sales`), `deploy/README.md`,
   `spec/ocm/README.md`, `examples/leaderboard/README.md`, `platform/api/openapi.json` and
   `.claude/skills/*/SKILL.md` into
   `platform/web/content/` (git-ignored, same relative layout as the repo). npm `predev`/`prebuild`,
   the azd `prepackage` hook and the CI container job all run it; the Dockerfile copies `content/`
   into the runtime image.
2. **Catalogue** - `src/lib/docs/catalogue.ts` is the single source of truth for sections, order,
   slugs, titles, one-line summaries and icons. Sections: Start, Hosted platform (`PLATFORM`,
   `MARKETPLACE`, the `api` overview and its discovered `api/<tag>` children), Python SDK (`SDK`, `ENTERPRISE`, `REFERENCE`),
   Self-hosting (`deploy`), Concepts, Security, Agent skills (discovered) and Releases (`changelog`,
   `ROADMAP`, `contributing`). `SECTION_TABS` drives the top bar. Slugs: file stem for `docs/*.md`
   (`/docs/GUIDE`), plus `readme`, `changelog`, `contributing`, `security-policy`, `deploy`, `ocm`.
3. **Renderer** - `src/lib/docs/render.ts`: remark-parse + GFM -> rehype-raw -> strip the first H1
   (page header renders it) -> `rehype-slug` ids (GitHub-compatible; `README` anchors such as
   `#10-routing-latency` keep working) -> TOC (h2/h3) -> autolinked headings -> link rewriting
   (relative `.md` to `/docs/<slug>`, other repo paths to GitHub blob/tree, images to raw GitHub) ->
   Shiki (`github-dark-default`, languages listed in `CODE_LANGS`) -> HTML. ```` ```mermaid ```` fences
   become `<pre class="mermaid">` and render client-side in `components/docs/docs-article.tsx`.
4. **REST API reference** - `src/lib/docs/openapi.ts` reads the snapshot and groups operations by
   tag; `components/docs/api-reference.tsx` renders the overview (`/docs/api`: auth schemes, base URL,
   conventions, groups) and one page per tag (parameters, request body, responses, curl and JSON
   examples). Example bodies come from `exampleBodyFor()`; add a hint there for a new request model.
5. **Versions** - `src/lib/docs/versions.ts` parses `## [x.y.z] - date` headings from `CHANGELOG.md`;
   `components/docs/docs-topbar.tsx` shows the current release and a menu that links each release (and
   the unreleased section) to `/docs/changelog#<anchor>`. A release needs no code change here.
6. **Pages** - `src/app/(docs)/docs/page.tsx` (index: intro, install, entry points per audience, one
   list per section), `[...slug]/page.tsx`
   (`generateStaticParams` from the catalogue, `dynamicParams = false`, breadcrumb, reading time, edit
   link, prev/next; API pages branch on `kind === "api"`), `search.json/route.ts` (static index; API
   pages index their operations) consumed by `components/layout/search-command.tsx`. `sitemap.ts`
   lists every catalogue entry.

To publish a new user-facing document: add the Markdown under `docs/`, add a `STATIC_SECTIONS` entry
in `catalogue.ts` (slug, file, title, description, icon), link it from `README.md`
(`tests/test_docs.py` requires every `docs/*.md` to be linked), run `npm run build` and open
`/docs/<slug>`. Internal documents are linked from `README.md` only. A new
`.claude/skills/<name>/SKILL.md` needs no code change. When a heading is renamed, search for the old
anchor across `docs/`, `README.md` and `platform/web/src`.

Style: the docs pages are plain reference pages - headings, paragraphs, tables and code blocks. No
icon cards, badges, pills, uppercase eyebrow labels, hover animations or drop shadows; no reading
time or page counts; copy states what an endpoint or option does, without marketing adjectives.

## Deploy

```
azd provision            # infra/main.bicep: Container Apps osr-api + osr-web, Azure OpenAI, ACR, Files share, DNS zone
azd deploy api           # platform/api/Dockerfile, context = repo root
azd deploy web           # platform/web/Dockerfile, context = platform/web (prepackage hook syncs content/)
azd deploy platform      # same Dockerfile -> osr-platform (serves /platform; osr-web forwards to it)
.\scripts\domains.ps1    # custom domain: dns (zone + name servers) -> bind (certs) -> verify; -Phase clean retires osr-platform
.\scripts\sso.ps1        # sign-in providers: microsoft (az app registration + secret) | google | github | gitlab | show | push | apply
```

JSON-valued deployment inputs (`OSR_PLATFORM_SSO_PROVIDERS_B64`, `OSR_PLATFORM_STRIPE_PRICES_B64`) travel
base64-encoded: azd substitutes `${VAR}` textually into `infra/main.parameters.json`, so raw JSON breaks every
provision ("invalid character after object key:value pair"); `infra/main.bicep` decodes with `base64ToString`.
`sso.ps1` is the only writer of the SSO map (local azd env -> `gh secret set` -> `azd provision`); every provider
uses the single redirect `<WEB_URL>/auth/callback` (kept without the `/platform` prefix on purpose - the web app
308s it to `/platform/auth/callback`, so the URI registered at every provider never changes). Google and GitHub have no API for OAuth clients, so those
commands print the console steps and prompt for the id and hidden secret.

`.github/workflows/platform.yml` runs API tests and the OpenAPI snapshot check, `npm run check && npm
run build`, a two-container smoke (`/`, `/docs`, `/docs/GUIDE`, `/docs/PLATFORM`, `/docs/api`,
`/docs/api/routing`, `/docs/skills/osr-contributing`, `/docs/search.json`, `/pricing`, `/api/v1/info`;
`/docs/sales/*` must 404) and then `azd up` on `main`. After deploying, verify `<WEB>/docs/api` and
`<WEB>/api/v1/info` (`edition`, `sdk_version`). `NEXT_PUBLIC_SITE_URL` is baked at build time via
`buildArgs` in `azure.yaml`; SQLite on Azure Files needs `nobrl` mount options and a single API replica.
The public domain is an Azure DNS zone (`infra/dns.bicep`, azd `OSR_DNS_ZONE=opensmartroute.ai`): apex A to the
environment static IP, `www`/`api` CNAMEs, `asuid.*` TXT; the registrar delegates to the zone's name servers.
`www` redirects come from `platform/web/src/proxy.ts` (`OSR_WEB_REDIRECT_HOSTS`), not from an edge proxy.
Apex managed certificates use `HTTP` domain-control validation, subdomains `CNAME` (`TXT` never completes for an
apex). The `azd up` job signs in with an OIDC federated credential whose subject is
`repo:isathish/OpenSmartRoute:environment:production` (repository variables `AZURE_CLIENT_ID`, `AZURE_TENANT_ID`,
`AZURE_SUBSCRIPTION_ID`, custom-domain `OSR_*` vars; secret `OSR_PLATFORM_ADMIN_TOKEN`) and seeds
`SERVICE_*_IMAGE_NAME` from the running apps before `azd provision`; setup in `platform/README.md`
("Continuous deployment").

---

<!-- Page: osr-routing-catalogue (https://opensmartroute.ai/docs/skills/osr-routing-catalogue) -->

<!-- OpenSmartRoute: osr-routing-catalogue. Source https://github.com/isathish/OpenSmartRoute/blob/main/.claude/skills/osr-routing-catalogue/SKILL.md; HTML https://opensmartroute.ai/docs/skills/osr-routing-catalogue -->
---
name: osr-routing-catalogue
description: Define and tune an OpenSmartRoute routing catalogue. Add a new target to targets.yaml (LLM, agent, skill, persona, tool, human) with capabilities, domains, a complexity band, cost, latency and a quality prior. Write a rules.yaml rule that prefers, avoids or pins targets, for example pin PII requests to an on-prem model. Set hard constraints (region, data boundary, tenant, max cost) and objective weights per request. Build plans and close the learn loop with Outcomes. Debug why a request routed to a given target with the trace. Use when editing targets or rules YAML or explaining a routing decision.
license: Apache-2.0
compatibility: OpenSmartRoute >= 0.4, Python >= 3.10
metadata:
  author: opensmartroute
  osr-domains: "coding general"
  osr-tags: "opensmartroute routing catalogue"
  osr-quality-prior: "0.85"
  osr-primary: "false"
---

# OpenSmartRoute routing catalogue

Routing is `signals -> policy -> strategies -> ensemble -> decision (+plan)`. The catalogue is
data: add a target and it is routable immediately, no retraining.

## Target schema (`targets.yaml` / `.json`, or `TargetRegistry.from_dicts`)

Top level is `targets: [...]` (a bare list also works). Keys are exactly the `RouteTarget` fields -
unknown keys raise `ConfigurationError`.

```yaml
targets:
  - id: llm-small                      # required, unique; used in rules, outcomes, plans
    kind: llm                          # llm | agent | skill | persona | tool | workflow | human | destination
    name: Small fast model
    description: Cheap, fast generalist for simple chat, classification and extraction.
    capabilities:
      domains: [general, customer_support]      # match signals.domains
      actions: [qa, classify, extract, summarize]
      languages: [en, es]                       # default [en]; "*" = any
      modalities: [text]                        # default [text]
      min_complexity: 0.0                       # request complexity band this target fits
      max_complexity: 0.45
      context_window: 128000                    # optional; policy rejects oversize inputs
      supports_tools: true
      tags: []
    constraints:
      regions: [us, eu]                         # empty = anywhere
      data_boundary: public                     # public | private | on_prem
      pii_allowed: true
      max_tokens_in: null
      tenants: []                               # empty = every tenant
    cost: { usd_per_1k_tokens: 0.0002 }         # also wh_per_1k_tokens, gco2_per_1k_tokens
    latency_ms: 300
    quality_prior: 0.55                         # 0..1 belief before any outcomes
    examples: ["Hi, how are you today?"]        # anchor the similarity strategy
    instructions: ""                            # system prompt fragment, disclosed only when selected
    primary: true                               # false = slot-only (persona/skill layered on a model)
    family: ""                                  # siblings share breaker/budget state
    effort: ""                                  # reasoning-effort variant label
    enabled: true
    metadata: {}                                # free-form; keys read by optional strategies below
```

`metadata` keys that optional strategies read (all opt-in, none required):
- `tier: edge | cloud` - `strategies.EdgeCloudStrategy` (edge decode time vs SLO, cloud upload
  penalty, cloud excluded for private / on_prem requests).
- `input_schema: {properties: {...}, required: [...]}` on `kind: tool` - `discovery.SchemaAwareStrategy`
  scores how many required parameters the request text can fill (MCP imports set it automatically).
- `requires`, `conflicts`, `composes` (lists of skill ids) on `kind: skill` - `discovery.SkillGraph`;
  SKILL.md packages declare them as `osr-requires` / `osr-conflicts` / `osr-composes` frontmatter.
- Elastic models: do not hand-write `<id>@<budget>` siblings; call
  `strategies.expand_elastic(parent, [BudgetVariant(tokens, quality=, cost_scale=)])` and add the result;
  it sets `metadata.token_budget`, which `strategies.TokenBudgetStrategy` scores against the tokens
  the answer needs.

Guidance:
- Give every target 2-5 `examples` phrased like real requests; similarity is the cheapest strong signal.
- Set `min_complexity`/`max_complexity` bands so the capability strategy can penalise overkill and
  too-hard routes; overlap bands slightly.
- `quality_prior` is a prior, not a score - bandits/IRT move it with `Outcome`s.
- Personas and skills are usually `primary: false`; they fill plan slots.
- Python handlers: `registry.get("id").handler = fn` where `fn(request, **kw) -> response`.

## Rules (`rules.yaml`, Arch-Router style)

```yaml
rules:
  - name: pii-stays-onprem
    when: { contains_pii: true }                  # domains, actions, language, min/max_complexity,
    prefer: [llm-onprem, human-escalation]        #   contains_pii, needs_tools, context: {key: value}
    avoid: [llm-frontier]
    weight: 1.0
    pin: false                                    # true = restrict candidates to `prefer`
```

Rules push scores; they never override policy. There is no `boost` key - use `weight`.

## Hard constraints and objectives (per request)

```python
from opensmartroute import RouteRequest, RequestConstraints, Objective
req = RouteRequest("Summarise this contract",
    constraints=RequestConstraints(region="eu", data_boundary="private", max_cost_per_1k=0.01,
                                   max_latency_ms=3000, allowed_kinds=["llm"], deny_targets=[],
                                   tenant="acme", require_tools=False),
    objective=Objective(quality=1.0, cost=0.3, latency=0.05, quality_floor=0.6, energy=0.0, carbon=0.0))
```

Constraints are filtered by `Policy` (rejections appear in `decision.trace.policy_rejections`);
objective weights shape the utility `quality - cost*norm(cost) - latency*norm(latency)`.

Request `context` keys that strategies read: `user_id` (`learning.UserAdaptiveStrategy`),
`last_target` and `last_failed` (`learning.HistoryTargetStrategy` incumbent bonus / penalty),
`difficulty` (`signals.VerbalisedDifficultySignal`), `draft` (`signals.DraftResponseSignal`),
`session_id` (`MarkovStrategy` transitions). Pass prior
turns in `RouteRequest.history=[{"role": ..., "content": ...}]` so multi-turn strategies see them.

## Route, plan, learn

```python
from opensmartroute import Router, Outcome, load_targets, load_rules
router = Router(load_targets("targets.yaml"), strategies=[load_rules("rules.yaml"), *default_strategies()])
d = router.route(req, plan=True)          # d.target, d.confidence, d.alternatives, d.plan.slots, d.trace
print(d.trace.explain())                  # per-strategy scores for every candidate
router.learn(Outcome(d.request_id, d.target.id, success=True, quality=0.9, cost_usd=0.002,
                     latency_ms=1200, domains=d.trace.signals.domains, complexity=d.trace.signals.complexity))
res = router.run(req)                     # route + execute plan; res.text, res.outcomes, res.system_prompt
```

CLI equivalents: `osr -t targets.yaml -r rules.yaml route "text" [--plan] [--json]`,
`osr -t targets.yaml targets`, `osr -t targets.yaml stats`.

## Debugging a surprising route

1. `d.trace.explain()` - check `policy: N -> M candidates` and the rejection reasons first.
2. Compare `capability` (domain/action/complexity fit) vs `similarity` (examples) columns; fix the
   catalogue (examples, complexity band) before touching weights.
3. Weights and thresholds live in `Settings` (`osr settings`); override with `OSR_WEIGHTS_<NAME>`
   or `Router(weights={...})` only after the catalogue is right.

---

<!-- Page: osr-security-hardening (https://opensmartroute.ai/docs/skills/osr-security-hardening) -->

<!-- OpenSmartRoute: osr-security-hardening. Source https://github.com/isathish/OpenSmartRoute/blob/main/.claude/skills/osr-security-hardening/SKILL.md; HTML https://opensmartroute.ai/docs/skills/osr-security-hardening -->
---
name: osr-security-hardening
description: Harden OpenSmartRoute deployments against prompt injection, gadget/steering attacks, PII leakage, runaway agents and untrusted tool arguments - InputGuard and GuardMiddleware, learned gadget detector, PII redaction, ResourceLimits for tools and agents, OriginPolicy for tool-parameter provenance, signed MCP manifests, secret loading and the safety-routing red-team suite. Use when reviewing security of a routing setup, handling PII or private data boundaries, or wiring MCP tools safely.
license: Apache-2.0
compatibility: OpenSmartRoute >= 0.4, Python >= 3.10
metadata:
  author: opensmartroute
  osr-domains: "coding legal general"
  osr-tags: "opensmartroute security injection pii mcp"
  osr-quality-prior: "0.85"
  osr-primary: "false"
---

# OpenSmartRoute security hardening

Threat model: the router sits between untrusted text (users, retrieved documents, tool output) and
expensive or state-changing targets. Defend the input, the catalogue, and the execution budget.

## 1. Input guard (every deployment)

```python
from opensmartroute.security import InputGuard, GuardMiddleware
guard = InputGuard(max_chars=32_000, max_tokens_est=8_000, gadget_threshold=0.6,
                   reject_on_gadget=False, learned=False, strip_steering=True)
app = RouterBuilder(reg).with_defaults().with_middleware(GuardMiddleware(guard, redact=True)).build()
```

- `guard.inspect(text) -> GuardReport(ok, reasons, gadget_suspected, clean_text, score, steering_removed)`.
- `GuardMiddleware` raises `SecurityError` when `not ok`; with `redact=True` PII is replaced before
  routing (original in `request.context["_original_text"]`, map in `context["_pii_map"]`).
- `learned=True` uses `GadgetDetector.default()`; train your own with `osr train --gadget gadget.json`
  and `GadgetDetector.load`.
- Pure functions for ad-hoc checks: `injection_risk(text)`, `inspect_injection(text)`,
  `strip_steering(text)`, `sanitize_for_prompt(text)`, `shannon_entropy(text)`.

## 2. Data boundaries and PII in the catalogue

- Targets declare `constraints.data_boundary: public|private|on_prem` and `pii_allowed`. Requests set
  `RequestConstraints(data_boundary=..., contains_pii=...)`; the `PIISignal` also detects PII.
- Policy rejects mismatches (`DataBoundaryRule`, `pii` rule) - verify in `trace.policy_rejections`.
- Pair with a rule: `when: {contains_pii: true} prefer: [llm-onprem] pin: true`.
- Tenant isolation: `constraints.tenants` on targets + `TenantMiddleware(require=True)`.

## 3. Budgets for tools, agents and workflows

```python
from opensmartroute.security import ResourceLimits, ResourceLimiter, ResourceLimitMiddleware, apply_limits
limiter = ResourceLimiter(ResourceLimits(max_steps=50, max_tool_calls=100, max_depth=4,
                                         max_total_tokens=500_000, max_cost_usd=5.0, max_wall_s=900.0))
apply_limits(registry, limiter, kinds=("tool", "agent", "workflow"))   # wraps handlers per task_id
builder.with_middleware(ResourceLimitMiddleware(limiter))
```
Exceeding a limit raises `ResourceLimitExceeded` (a `SecurityError`). Budgets key on
`request.context["task_id"]`.

## 4. Tool-parameter provenance (indirect injection)

```python
from opensmartroute.security import OriginPolicy, OriginRule, mark_untrusted, apply_origin_policy
mark_untrusted(request, retrieved_text, source="retrieved")         # tag text you did not author
policy = OriginPolicy(rules=[OriginRule(state_changing=["delete_*", "send_*"], allowed_origins=["user"])])
apply_origin_policy(registry, policy)                                # wraps tool handlers
```
A state-changing tool whose arguments were copied from untrusted text raises `OriginViolation`.

## 5. MCP catalogues

- `tools_from_mcp(payload, max_description_risk=0.6, guard=InputGuard())` drops tools whose
  descriptions look like steering ("ignore previous instructions ...").
- Sign catalogues offline and verify at load: `osr mcp-manifest tools.json --key-env OSR_MCP_KEY
  --server name --out manifest.json`; then `tools_from_manifest(manifest, key, require_signature=True,
  max_age_s=86400)`. `--algorithm ed25519` needs the `crypto` extra.

## 6. Secrets

`load_secret("OPENAI_API_KEY")` reads the env var or the file named by `OPENAI_API_KEY_FILE`
(Kubernetes secret mounts). Never put secrets in `targets.yaml`; handlers read them at call time.
Telemetry never logs raw text.

## 7. Red-team suite

`osr -t targets.yaml safety [--learned-guard] [--json]` or `run_safety_suite(app.route)` runs
`SafetyCase`s (jailbreaks, PII exfiltration, boundary hops, cost bombs) and reports `pass_rate` and
`by_category`. Add it next to `osr eval --min-accuracy` in CI.

## Review checklist

- [ ] `GuardMiddleware` first in the chain, `redact=True` when any target is `pii_allowed: false`.
- [ ] Every private/on-prem target has `data_boundary` and `regions` set; requests carry tenant.
- [ ] `apply_limits` on all `tool`/`agent`/`workflow` targets; `max_cost_usd` matches the tenant budget.
- [ ] MCP tools loaded from a signed manifest; `max_description_risk` left at or below 0.6.
- [ ] Secrets via `load_secret`; audit sink enabled; `osr safety` passes in CI.

<!-- Section: Releases -->

---

<!-- Page: Changelog (https://opensmartroute.ai/docs/changelog) -->

<!-- OpenSmartRoute: Changelog. Source https://github.com/isathish/OpenSmartRoute/blob/main/CHANGELOG.md; HTML https://opensmartroute.ai/docs/changelog -->
# Changelog

All notable changes are documented here. Format follows [Keep a Changelog](https://keepachangelog.com);
versions follow [SemVer](https://semver.org). Releases are cut by the `Prepare release` workflow
(`scripts/release.py`), which turns the Unreleased section into a dated section.

## [Unreleased]

### Added

Workspace self-service: outcome reporting in the dashboard, key rotation, data export
- Outcomes without curl: `components/dashboard/outcome-form.tsx` - good / poor, a quality slider and "this
  target would have been better" (recorded as a pairwise preference) - on every expanded activity row and in
  the request trace; the onboarding step "report an outcome" now links to the activity page (or billing on
  plans without the feature).
- API keys: `PATCH /api/v1/keys/{id}` renames, `POST /api/v1/keys/{id}/rotate` issues a new secret for the
  same key in one transaction (shown once; the old secret dies immediately; id, name and creation date stay,
  so audit rows and dashboards remain attached). The keys page renames inline and rotates with a warning.
- Data portability: `GET /api/v1/me/export?days=` returns one JSON document with the person (profile,
  identities, sessions, memberships, history) and the workspace (plan, members, key prefixes, tenants,
  policy, SSO, usage, request log, reported outcomes); *Download my data* on the account page. Secrets and
  password hashes never appear; tests assert the export never contains the API key or password.

Machine-readable site: real llms.txt, Markdown pages, complete OpenAPI
- Every documentation page is served as Markdown at `/docs/<page>.md` (relative links resolved to the site or
  the repository, SKILL.md frontmatter stripped) and advertises it with a `text/markdown` alternate link;
  `/llms.txt` now links to those Markdown sources as llmstxt.org intends and `/llms-full.txt` is the whole
  documentation in one file. `/openapi.json` on the website serves the complete committed OpenAPI document
  (146 paths, `servers` set to the site) instead of proxying the gateway process, whose own document only
  lists the routes it serves itself once domains run as separate services (4 paths locally, 19 in
  production). Footer: `llms-full.txt` added, *Discussions* (not enabled on the repository, a 404) replaced
  by *Issues*; the support page sends questions to a labelled issue. Tests: `platform/web/tests/llms.test.ts`.

Operator console: analytics, health, request log, marketplace moderation, leads, plans, configuration, search
- `platform/api/osr_platform/admin_console.py`: `GET /api/v1/admin/analytics?days=` (deployment-wide daily
  requests / failures / cost / tokens / active workspaces / mean latency, per endpoint, per target, busiest
  workspaces, signups per day), `GET /api/v1/admin/activity` (every metered request across workspaces with
  cursor paging and endpoint / target / failed / workspace / request-id filters), `GET /api/v1/admin/lookup?q=`
  (users, workspaces, API keys by prefix, operators, tenants, request ids - each with its console page),
  `GET /api/v1/admin/settings` (effective configuration grouped with the `OSR_PLATFORM_*` variable of every
  value; secrets and nested `client_secret` / `api_key` values are masked), `GET /api/v1/admin/retention` and
  `POST /api/v1/admin/retention/purge` (run the sweep now), and `GET /api/v1/admin/health?window=` served by
  the routing process itself (readiness, HTTP series, last five minutes, circuit breakers, observability,
  deployment alerts, the alert evaluator, replica host).
- Console pages `/platform/admin/analytics`, `/health`, `/activity`, `/leads`, `/plans`, `/marketplace`
  (review queue and every status, approve / reject / unpublish with a note, featured and verified flags,
  imported catalogue per source, scheduled refresh with *run now*), `/settings` (configuration + retention)
  and `/search`; a search box in the console header opens the search page. Navigation regrouped into
  Deployment, Directory, Learning, Platform and Access.

Operators see both portals
- Superadmins open any workspace's dashboard from the console: `POST /api/v1/admin/accounts/{id}/session`
  mints a one-hour browser session on the workspace as its owner (provider `operator`, visible in the owner's
  session list, audited as `workspace.open`); *Open dashboard* on `/platform/admin/workspaces` and on each
  workspace page switches to the workspace portal, which shows an *Operator view* banner (who, which
  workspace, when the session ends) with *Back to console* and *End operator view* (revokes the session).
  The view sees every plan feature (statistics, audit, tenants ...) whatever the workspace's plan - quotas
  stay the workspace's - so no page is hidden behind an upgrade prompt for the operator.
  The console's account menu leads to the workspace dashboard and the dashboard's account menu back to the
  console while an operator session exists. `GET /api/v1/me` now includes `session` (null for API keys).

True streaming on the OpenAI-compatible endpoint
- `POST /v1/chat/completions` with `stream: true` now relays each provider token as it arrives instead of
  completing the request first and replaying the finished text as chunks (time to first token was the whole
  completion time - measured 2.55 s to the first of 39 chunks that then all landed within 130 ms). The final
  chunk still carries `usage` and the `opensmartroute` metadata, the call is metered and learned from exactly
  like the plain response, quota / routing / provider errors before the first token are ordinary HTTP errors,
  `X-OSR-Target` / `X-OSR-Confidence` name the routed target, and the automatic fallback to the next target
  applies only before the first token (`committed` in `_execute_with_fallback`).
- SDK: `OpenAICompatClient.chat(..., on_delta=)` / `stream_chat()` consume the upstream `stream: true`
  server-sent events (with `stream_options.include_usage`; token counts estimated when the provider sends
  none) and fall back to a single delta for providers that ignore `stream`; `chat_handler` forwards the
  callback, so `execute(decision, request, on_delta=...)` streams through the whole SDK pipeline.
  `platform/api/tests/test_streaming.py` drives a paced SSE upstream through the SDK client, the TestClient
  and a real uvicorn server (tokens spread over the provider's pacing, not one burst).

Operator console: directory, audit and mail administration
- `GET /api/v1/admin/users` filters by `disabled`, `verified` (email confirmed) and `method` (`password` /
  `sso`), sorts by `created_at`, `last_login_at`, `email` or `name` (`order=asc|desc`), returns the filtered
  `total` and downloads the matching rows with `format=csv`; `POST /api/v1/admin/users/bulk` applies
  `disable`, `enable`, `verify`, `revoke_sessions`, `reset_link` or `delete` to up to 500 users and reports each
  as done or failed. `GET /api/v1/admin/accounts` gains `plan`, `disabled`, `sort`, `order` and `format=csv`.
- `GET /api/v1/admin/audit-log` filters by `actor_type`, `actor_id`, `after` and free text `q` (actor, IP,
  details), returns the distinct `actions` with counts for the filter menu and exports with `format=csv`.
  Every operator action (`operator.*`, `user.*`, `account.*`, `mail.*`, `users.bulk`, `*.export`) is now
  recorded in `audit_log` with the operator as actor, not only mirrored on the event bus.
- Outbox: `GET /api/v1/admin/mail` filters by `kind` and `failed`, pages with `offset`, returns `total` and a
  `summary` (composed, unsent, failed, kinds); `POST /api/v1/admin/mail/{id}/resend` delivers a message again
  (409 without SMTP), `DELETE /api/v1/admin/mail/{id}` removes it, `POST /api/v1/admin/mail/test` sends a
  delivery check to an address. `Mailer.resend()` and `Mailer.test_message()`.
- `GET|DELETE /api/v1/admin/operators/{id}/sessions`: superadmins inspect any operator's console sessions and
  revoke one (`session_id`) or all of them; operators see their own.
- Console pages: `/platform/admin/users` (status / email / sign-in filters, sortable columns, last sign-in and
  verified columns, row selection with a bulk-action bar, *Export CSV*), `/platform/admin/workspaces` (kind,
  plan and status filters, sortable columns, *Export CSV*), `/platform/admin/audit` (action menu with counts,
  actor, time window, user / workspace and text filters, *Load older*, *Export CSV*, click an action badge to
  filter by it), `/platform/admin/mail` (counters, kind and delivery filters, paging, *Resend*, *Delete*,
  *Send a test message*), `/platform/admin/operators` (*Sessions* of another operator with per-session and
  sign-out-everywhere revocation).

SLO-aware routing (the platform plan's open "rolling p50/p90/p99 per endpoint" row)
- `realtime.LatencyWindow`: each target in `HealthRegistry` keeps its last 256 observed latencies with
  nearest-rank p50 / p90 / p99 (`health_snapshot()` gains `latency_p50_ms`, `latency_p90_ms`, `latency_p99_ms`;
  the platform's `/api/v1/stats` and the Health page show them next to the breaker state).
- `RequestConstraints.preferred_max_latency_ms`: a *soft* latency target. `HealthStrategy` now scores the
  observed p90 (once five calls are in the window; EWMA, then the declared latency before that) against the hard
  `max_latency_ms`, else this soft target, else `with_health(latency_slo_ms=...)`, so a target with a good mean
  and a bad tail is penalised without being excluded. Accepted by `osr route --constraint
  preferred_max_latency_ms=...`, the MCP `route` tool, `POST /api/v1/route` constraints and workspace / tenant
  policy (`preferred_max_latency_ms` in `POLICY_KEYS`; the stricter of request and policy wins).

Dashboard chart kit
- `components/charts`: the platform's own chart primitives on top of the time-series ones - `RankedBars`
  (horizontal ranking with values), `Donut` (share with a centre label), `Bars`, `ScatterPlot`, `Sparkline`,
  `Meter` (bounded value against a threshold), `ShareBar` (stacked composition, HTML only) and `Heatmap`
  (day x hour density, HTML only); `ChartTooltip`, `ChartLegend`, `ChartEmpty`, a `SERIES` palette with
  `seriesColor()` and the theme tokens `--chart-axis`, `--chart-grid`, `--chart-cursor`, `--chart-other`
  (light and dark). `components/dashboard/analytics.tsx` renders the API's breakdowns as charts instead of
  tables: `TargetBreakdown`, `EndpointShare`, `TrafficHeatmap`, `SavingsChart` (routed against the baseline
  per day), `SavingsByTarget`, `PlanMix`, `DecisionsBreakdown`, `LatencyHistogram`, `RouteTraffic`,
  `BreakerGrid`, `StrategyWeights`, `OutcomesByTarget`. `StatCard` takes a `trend` (delta pill with
  direction) and an `info` tooltip. Tests: `platform/web/tests/charts.test.tsx`.

Notifications and alert delivery
- Alerts are now delivered, not only shown. `platform/api/osr_platform/notifications.py`: the API
  re-evaluates the workspace and deployment alert rules every `OSR_PLATFORM_ALERTS_INTERVAL_S` (60 s; one
  replica through a Redis lease) and turns each condition into a notification episode (opened, escalated,
  resolved) kept in an inbox with unread counts, published to the `alerts` Kafka topic in cluster mode and
  delivered to channels: e-mail (platform mailer), `https` webhooks (JSON with `X-OSR-Event`,
  `X-OSR-Delivery` and an HMAC `X-OSR-Signature`), Slack and Teams incoming webhooks. Channels filter by
  minimum severity and rule names, keep last delivery / last error, have *Send test*; failed webhooks are
  retried on the next rounds (three attempts). Endpoints `GET /api/v1/notifications[/unread]`,
  `POST /api/v1/notifications/read`, `GET|POST /api/v1/notifications/channels`,
  `PATCH|DELETE /api/v1/notifications/channels/{id}`, `POST .../channels/{id}/test`,
  `GET /api/v1/notifications/deliveries`; operator equivalents under `/api/v1/admin/notifications` plus
  `POST /api/v1/admin/notifications/evaluate`. Dashboard page `/dashboard/notifications` with a bell and
  unread badge in the header; console page `/admin/notifications`.

The platform's own observability
- The platform observes itself; no external metrics, tracing or alerting system is involved (the OTLP bootstrap
  and the Prometheus / Grafana / Jaeger compose profile added earlier were removed). `osr_platform.telemetry`:
  a `DbEventSink` on the tracer persists every span and event to the platform database from a bounded queue
  and a daemon writer (never on the request path; `OSR_PLATFORM_TELEMETRY_STORE`, on by default;
  `OSR_PLATFORM_TELEMETRY_RETENTION_DAYS`, 14, purged by the retention sweep), so `GET /api/v1/events`
  (`since` / `until`, `source`, `persisted`, `retention_start`) and `GET /api/v1/trace/{id}` (`source`) survive
  restarts and `GET /api/v1/activity` marks persisted requests `traced`. `HttpMetrics.series` keeps per-minute
  HTTP counters for 24 hours. `GET /api/v1/telemetry/series?window=1h|6h|24h|7d|30d` buckets the workspace's
  metered traffic (requests, failures, p50 / p95, cost, tokens; per target and endpoint) and, with `stats`, the
  deployment's HTTP requests, 4xx / 5xx and p50 / p95 / p99. `osr_platform.alerts` + `GET /api/v1/alerts`: rules
  evaluated in-process - budgets at 80 % / exhausted (workspace and tenants), failing requests, and on `stats`
  plans readiness, 5xx rate, latency SLO, open breakers, drift alarms, autopilot errors, a stale SLM, tracing
  off - each naming the dashboard page that fixes it. Web: `AlertsPanel` and 24-hour traffic / latency series
  on the overview, deployment HTTP series with a window selector and the alerts on Health, a 1 h - 7 d window on
  Events served from the store, trace view labels the source. Tests: `platform/api/tests/test_telemetry.py`.

Platform services and model providers
- The platform API runs as one process or as several: `docs`, `rankings`, `marketplace`, `providers`,
  `onboarding`, `accounts` (sign-in, sessions, workspaces, keys, tenants, policy, governance, usage), `billing`,
  `admin`, `mcp` (the `/mcp` Model Context Protocol endpoint), `openai` (the `/v1` proxy; streamed completions
  are relayed through the gateway chunk by chunk) and `routing` (`/api/v1/route`, feedback, targets, stats,
  audit, the trace / event / learning reads of its decisions and the autopilot) each have their own entry point
  (`osr-platform-<name>`, `python -m osr_platform.services <name>`,
  ports 8091-8101) on the API image (`platform/api/osr_platform/services.py`). The API is the gateway: with
  `OSR_PLATFORM_<NAME>_URL` set it forwards that domain's paths (headers, body, `X-Forwarded-For`) and marks
  the answer with `X-OSR-Service`; unset, it serves the domain in-process. `/api/v1/info`, `/status`,
  `/estimate` and the health / metrics endpoints always stay with the API; every service also serves
  `/healthz`, `/readyz` and `/metrics`. Each process owns its background
  work (`PlatformContext`: the API bootstraps operators and evaluates alerts, the owner of `routing` runs the
  autopilot and the telemetry store, `rankings` refreshes the
  reference catalogue, `marketplace` seeds the registry). `GET /api/v1/info` -> `services`,
  `GET /api/v1/admin/services` (topology + live health) and the `/admin/services` page; the dev
  `compose.yaml` and `platform/docker-compose.yml` start every service next to the gateway.
- Model providers managed at run time: `ProviderStore` (tables `providers`, `provider_models`) with
  `GET|POST /api/v1/admin/providers`, `GET|PATCH|DELETE /api/v1/admin/providers/{id}`,
  `POST /api/v1/admin/providers/{id}/check` (probes `/models`, records latency and the upstream model ids)
  and `PUT|DELETE /api/v1/admin/providers/models/{target_id}`; presets for OpenAI, Azure, OpenRouter,
  Anthropic, Mistral, Groq, Together, Fireworks, DeepSeek, Ollama, vLLM, LiteLLM. The mounted
  `OSR_PLATFORM_PROVIDERS` file is imported once and stays merged underneath; every write re-binds the
  handlers at once and other API replicas re-read the store every `OSR_PLATFORM_PROVIDERS_RELOAD_S` (30 s).
  Public `GET /api/v1/providers`: the OpenRouter-style catalogue (provider, upstream model, reference prices,
  context, health, 7-day traffic per executable target; never credentials) and its page on the site, `/providers`.
  Console page `/admin/providers`.
- `GET /api/v1/onboarding`: the getting-started checklist of a workspace (e-mail confirmed, key, first route,
  first outcome, provider connected, teammate invited) with links and the completing API call; the dashboard
  overview shows it until every step is done. Signup,
  verification, password reset and invitation links moved into the `onboarding` router (same paths).

### Changed

Website and platform as two services, one hostname
- The signed-in product now lives under `/platform`: `/platform/login`, `/platform/signup`, password recovery,
  invitations and `/platform/cli/authorize`; the dashboard at `/platform/dashboard/...`; the operator console at
  `/platform/admin/...`; the playground at `/platform/playground` and the publish flow at
  `/platform/marketplace/publish`. The website (landing, `/docs`, `/models`, `/vendors`, `/rankings`,
  `/pricing`, `/marketplace`, `/compare`, `/roi`, `/estimate`) keeps the root. The old paths redirect (308,
  query string preserved), the OAuth redirect URI registered at providers stays `<web>/auth/callback`, and
  links the API generates (emails, invitations, device sign-in, Stripe return, alert `href`s) use the new paths.
- Deployment: a third container app, `osr-platform` (azd service `platform`, the web image, internal ingress),
  serves `/platform`; `osr-web` forwards those requests to it (`OSR_WEB_PLATFORM_URL`, `platform/web/src/proxy.ts`).
  A single web process without that variable serves both halves (`next dev`, Compose, the CI smoke container).

### Fixed

- Azure cluster mode connected to PostgreSQL, Redis and Kafka through `<app>.internal.<domain>` names, which
  resolve to the HTTP ingress and never answer on a TCP port; the API hung inside `psycopg.connect` until the
  probes killed it and the platform silently kept serving from the previous SQLite revision. `infra/` now uses
  the bare container-app names (`osr-postgres:5432`, `osr-redis:6379`, `osr-kafka:9092`), the database URL
  carries `connect_timeout=10` (also the default in `Database` for PostgreSQL URLs without one) and the API and
  domain services have a Startup probe so a slow first boot is not restarted mid-migration.

## [1.0.0] - 2026-09-07

1.0 freezes the public API. Every `opensmartroute.*` name in `tests/public_api.json` is stable from here;
removals go through `errors.deprecated()` for at least one minor first. The release rows of the v1.0
readiness table are met from the repository (`python scripts/release.py readiness`); the adoption rows -
independent security report, accepted leaderboard listings, two production users - stay open and are
tracked in `docs/ROADMAP.md`, `docs/SECURITY_REVIEW.md`, `examples/leaderboard/results/README.md` and
`ADOPTERS.md`.

### Added

Routing SLM and datasets
- `RouterSLM.info()`: metadata, shape (encoder, dims, embedder), calibration, size and the fit history of a
  model without its weights; `osr slm info` prints it and the platform's `GET /api/v1/learning` uses it.
- `osr collect --file NAME=PATH`: load a benchmark you downloaded by hand (`.jsonl` / `.json` / `.csv` in the
  `--preset` shape; `--tier` folds model columns onto your targets) into the dataset cache - the path for
  gated RouterBench / RouterEval copies.

Release readiness
- `scripts/release.py readiness` distinguishes **release** rows (repository evidence; block a `1.x`
  version in `check`) from **adoption** rows (third-party evidence; tracked and printed, never blocking),
  and counts the freeze step `0.Y.0 -> 1.0.0` as a non-breaking release step when the changelog has no
  removals and the tagged `tests/public_api.json` only grew.

Platform administration and the operator console
- Operators sign in to `/admin` with a **username and password** (`POST /api/v1/admin/auth/login` mints an
  `osr_op_` session valid for twelve hours; scrypt password hashes; sign-in attempts rate limited per address).
  `OSR_PLATFORM_ADMIN_USERNAME` / `OSR_PLATFORM_ADMIN_PASSWORD` create - or reset - the first `superadmin` at
  API start-up; the static `OSR_PLATFORM_ADMIN_TOKEN` keeps working and counts as a superadmin. Superadmins
  manage operators (`/api/v1/admin/operators`); every operator changes their own password and sessions.
- Full administration API and web console: deployment overview (`/api/v1/admin/overview`), users (search,
  create with personal workspace / plan / password / first key, detail with workspaces, identities and sessions,
  rename, disable, reset password, sign out everywhere, delete), workspaces (search, create organizations for an
  owner email, plan / name / slug / disable / delete, members with roles, API keys, tenants with validated
  configuration, policy, SSO and usage), every tenant across workspaces, plans. Pages `/admin`,
  `/admin/users[/{id}]`, `/admin/workspaces[/{id}]`, `/admin/tenants`, `/admin/operators`, `/admin/login`.
  Every mutation is written to the access log as `operator:<username>` and published as an `admin` event.
- Email + password for workspace users: `POST /api/v1/signup` accepts a `password` (and then also returns a
  browser session), `POST /api/v1/auth/password/login` signs in by email, `POST|DELETE /api/v1/auth/password`
  set, change or remove it (`has_password` on the user); `/login`, `/signup` and `/dashboard/account` have the
  forms. `OSR_PLATFORM_PASSWORD_LOGIN=false` turns it off.

Self-hosted cluster: PostgreSQL, Redis, Kafka as containers
- `OSR_PLATFORM_DATABASE_URL=postgresql://...` runs the platform store on PostgreSQL through `psycopg` (the same
  SQL is written once; the backend rewrites placeholders, `INSERT OR IGNORE`, `REAL`/identity columns); the
  whole test suite passes against PostgreSQL (`OSR_TEST_DATABASE_URL`). SQLite stays the single-replica default.
- `OSR_PLATFORM_REDIS_URL` shares rate-limit windows between replicas (atomic sliding window in Redis, falls
  back to per-replica limiting when Redis is unreachable) and persists the enterprise learners' state in Redis
  instead of the state directory.
- `OSR_PLATFORM_KAFKA_BOOTSTRAP` publishes `usage`, `feedback` and `admin` events as JSON to Kafka topics
  `osr.<name>` (any Kafka-protocol broker; never prompt text; broker outages drop events instead of failing
  requests). `GET /readyz` reports `database_dialect`, `redis` and `events`; `GET /api/v1/info` and `/admin`
  report `storage` (`{database, clustered, redis, events}`).
- `platform/docker-compose.yml`: the complete platform on any Docker host with PostgreSQL, Redis, a single-node
  KRaft Kafka broker, the API and the web app - no cloud service involved. `infra/`: `OSR_PLATFORM_CLUSTER_MODE`
  (default true) adds `osr-postgres`, `osr-redis` and `osr-kafka` container apps with internal TCP ingress and
  their own Azure Files shares, and scales `osr-api` to `OSR_PLATFORM_API_MAX_REPLICAS` replicas; the API image
  ships the `psycopg`, `redis` and `kafka-python` clients (`osr-platform-api[cluster]`).
- The routing SLM as its own service: `osr-platform-slm` (`osr_platform.slm_service`, the API image with another
  entry point; `slm` in the Compose stack, `osr-slm` on Container Apps) consumes the `training` topic (request id +
  prompt, published by the API when `OSR_PLATFORM_TRAINING_EVENTS=true`) and `feedback`, runs the `SelfImprover`
  champion/challenger cycle on a schedule or on demand and writes promotions to `/data/autopilot/slm.json`; API
  replicas reload that file every `OSR_PLATFORM_SLM_RELOAD_S` seconds (and serve it even without a mounted bundle).
  Operator endpoints `GET /api/v1/admin/slm`, `/slm/reports`, `POST /slm/cycle`, `/slm/predict` proxy to the
  service (`OSR_PLATFORM_SLM_URL`); the console page `/admin/slm` shows model, evidence, cycles and a prompt probe.
- Microservice deployment of the platform: `platform/docker-compose.yml` runs `docs`, `rankings`, `marketplace`,
  `providers`, `onboarding` and `slm` as their own containers (`python -m osr_platform.services <name>`) behind the
  API gateway (`OSR_PLATFORM_<NAME>_URL`, answers carry `X-OSR-Service`); Container Apps cluster mode adds the
  internal apps `osr-docs`, `osr-rankings`, `osr-marketplace`, `osr-providers`, `osr-onboarding` (HTTP ingress, 1-3
  replicas, shared configuration and `/data` share) and points `osr-api` at them. The operator console page
  `/admin/services` shows the topology with a live health probe per remote service; CI's cluster smoke asserts each
  domain is served by its own container.

- `compose.yaml`: the whole hosted platform on a workstation with `docker compose up --watch` - development
  images `platform/api/Dockerfile.dev` (editable installs, uvicorn `--reload`) and `platform/web/Dockerfile.dev`
  (`next dev`), Compose Watch syncing `src/`, `platform/`, `docs/`, `examples/` and the skills into the running
  containers so edits show immediately, a persistent data volume, and an optional `llm` profile that starts
  Ollama with `deploy/compose/providers.ollama.yaml` so chat completions execute end to end without cloud keys.
- `python scripts/release.py readiness`: the v1.0 readiness table of `docs/ROADMAP.md` computed from the
  repository - frozen API snapshot and test, performance envelope, reference deployments, review pack and CI
  safety suite, published leaderboard run, two consecutive non-breaking minor steps (changelog sections
  without removals, verified against the tagged `tests/public_api.json` when both tags exist), independent
  report rows in `docs/SECURITY_REVIEW.md`, `accepted` rows under *Listings* in
  `examples/leaderboard/results/README.md` and *Production users* rows in the new `ADOPTERS.md`.
  `--require` exits 1 while a row is open and `release.py check` applies the same rule to any `1.x`
  version, so the *Release* workflow cannot tag 1.0.0 before the evidence exists; the workflow prints the
  table in its job summary.
- Tests for the OpenTelemetry bridge (`OpenTelemetrySink` driven through a fake `opentelemetry` API:
  spans, nesting through the context, `traceparent` parent, events, counters, duration histogram, error
  status, missing-extra error).
- Documentation trust gates: `tests/test_docs_claims.py` fails when any Markdown source names an `osr`
  command or flag the CLI parser does not have, or an `OSR_*` variable nothing in the code or deployment
  configuration reads; `platform/api/tests/test_platform_docs_claims.py` fails when a documented HTTP endpoint is not
  a route of the platform app (method included), a documented `/dashboard/<page>` has no page, a dashboard
  page or nav entry is undocumented, a user-facing `docs/*.md` is missing from the site catalogue, or a
  CHANGELOG section lacks a date, body or compare link. Historical tags `v0.2.0`, `v0.3.0`, `v0.4.0` mark
  the commits that carried those versions so the compare links and the readiness snapshot check resolve.

### Changed

- Marketing and docs honesty: the editions matrix credits the community edition with the guard, PII
  redaction, metrics and tracing it actually runs; the SLM showcase caveats the 0.94 figure as a
  single-source number; the pricing page labels the `sso` feature; the hero shows the version only when the
  API reports it; the Learning page points operators at `POST /api/v1/admin/autopilot/cycle` instead of a
  button that could only fail; `platform/README.md` documents `MODELS_REFRESH`, `REGISTRY_AUTO_PUBLISH` and
  `REGISTRY_SEED`; the Helm chart and deploy guide state that autopilot promotion is per pod; RESEARCH.md
  reflects the gated RouterBench / RouterEval sources.

## [0.5.0] - 2026-09-06

### Added

Roadmap: everything 1.0 needs from the repository
- `examples/leaderboard/results/`: the published 0.5.0 benchmark run made with the leaderboard recipe (SLM trained on
  Arena-55k + RouteLLM battles, seed 0; MT-Bench human, PPE human and WebDev Arena held out at the suite level,
  `--limit 1000`): one JSON per suite for the learned router (`--baselines --calibration --robustness`) and for the
  declarative router, plus a README with the data table, the SLM SHA-256 and the result table (learned 0.52-0.57 vs
  declarative 0.19-0.42 and static task table 0.54-0.67; paraphrase 0.88-1.00; held-out isotonic ECE 0.03-0.05).
  Published on the documentation site as `/docs/leaderboard-results`.
- `docs/SECURITY_REVIEW.md`: the external security review pack - engagement terms, system and trust boundaries,
  the threat-model reference, an evidence table with a reproduction command per row, nine ordered reviewer
  questions, known gaps that need not be re-reported and the review log where a third-party report is linked.
  `SECURITY.md` supported versions updated to 0.5.x / 0.4.x.
- `docs/ROADMAP.md`: v0.9 is Complete (its exit criterion now names the published review pack; the independent
  report moved to the v1.0 readiness table as the adoption item it is) and v1.0 has an explicit readiness table:
  five repository items met, one of two stable minors done, three adoption rows open (independent report, accepted
  leaderboard listings, two production users).

### Added

Hosted platform: billing and SSO as deployment inputs
- `infra/main.bicep` accepts `stripeSecret`, `stripeWebhookSecret`, `stripePrices`, `ssoProviders` and
  `metricsPublic` (azd `OSR_PLATFORM_STRIPE_SECRET`, `_STRIPE_WEBHOOK_SECRET`, `_STRIPE_PRICES`,
  `_SSO_PROVIDERS`, `_METRICS_PUBLIC`; GitHub secrets / variables of the same names in the deploy job). Container
  secrets are added only when set, so a provision without them changes nothing; with the Stripe values the hosted
  pricing page goes to Checkout instead of the operator instructions. `GET /metrics` on the Azure deployment now
  requires `X-Admin-Token` by default.
- `platform/api/tests/test_docs_links.py` checks that every `/docs/<slug>#anchor` link in the web app and every
  `file.md#anchor` link between the Markdown sources points at an existing heading (the site's `rehype-slug` ids).
  The container smoke in `.github/workflows/platform.yml` now also requests `/models`, `/rankings`, `/marketplace`,
  `/playground`, `/estimate`, `/roi`, `/mcp`, `/login`, `/signup`, `/cli/authorize`, `/docs/REFERENCE`,
  `/sitemap.xml`, `/openapi.json` and `/api/v1/status`.
- The web app has a unit test suite (`platform/web/tests/*.test.ts`, vitest, `npm test`; part of `npm run check`
  and therefore of the CI web job): the documentation catalogue against the content snapshot, the Markdown
  renderer on the real documents (heading ids, link rewriting, Shiki, skill frontmatter), changelog version
  parsing, the `/api` / `/v1` proxy (forwarded headers, 304/204, 502), `pageMetadata` and JSON-LD builders and
  the formatting helpers.
- Browser end-to-end journey for the hosted web app (`platform/web/e2e/customer-journey.spec.ts`, Playwright,
  `npm run e2e`; the CI web job runs it against the standalone build and a throw-away platform API): sign up in
  the UI and read the one-time key, dashboard overview, every sidebar page, a playground decision and the free-plan
  upgrade hint, key create and revoke, the activity row, sign-out and the login redirect, a rejected key, and the
  public pages with live data. The dashboard sidebar is a `navigation` landmark ("Dashboard").

### Fixed

- `osr login --url <server> --token osr_local_...` printed "Signed in to URL as URL": a self-hosted `osr serve`
  token has no account behind it, so the message now names the URL once ("Signed in to URL (self-hosted edition)").
- Web: the dashboard sidebar never listed Governance, Events, Learning and Health (the pages existed and were in
  `dashboardNav`, but not in the sidebar's section map); they are now an "Operate" section, and any future nav
  entry without a section renders under "More" instead of disappearing. The section label reads "Organization"
  like the rest of the site.
- Web: on 1024-1279 px viewports the header's "Compare" link rendered over the search box; the wide search box now
  appears only where it fits (md-lg without the desktop nav, xl and up with it) and the icon button elsewhere.
- Web: documentation sidebar entries were hard-clipped mid-word ("Cost estimates and the MCP se") because Radix
  ScrollArea lays its content out as a table; the viewport content is now block-level and long titles wrap.
- Web: the support and privacy pages linked `/docs/PLATFORM#13-errors` and `#12-privacy-and-data-handling`, two
  sections behind the current numbering (`#15-errors`, `#14-privacy-and-data-handling`). `/dashboard/billing`
  points at support when self-serve billing is disabled instead of only showing the operator command.
- `platform/README.md`: `OSR_PLATFORM_SSO_PROVIDERS` is a JSON map keyed by provider id, not a list.

### Added

Marketplace: the public skills ecosystem
- `osr_platform.harvest` imports the Agent Skills ecosystem and the official MCP registry into the marketplace:
  every `SKILL.md` in each repository on the skills.sh leaderboard, GitHub code search for the long tail (token
  required), and the latest active version of every MCP server as a `tool` listing with its remote endpoints.
  Rows become OCM manifests with `metadata.source` (provider, repository, path, ref, url), the upstream owner as
  publisher, the author's licence and an install command in the readme; the listing page shows a *Source* card
  that links back. `platform/api/scripts/harvest_skills.py collect|stats|publish` runs it (resumable JSONL cache,
  batches through the new admin endpoints `GET|POST /api/v1/admin/registry/import`, or straight into SQLite).
  Marketplace facets are cached for 60 s and the index paginates with a window (the hosted catalogue now has
  140 000+ listings). Every listing payload carries a compact `source` (provider, url, repository) and cards
  show a provenance badge. The catalogue keeps itself current from inside the platform process:
  `OSR_PLATFORM_MARKETPLACE_REFRESH_S` (weekly on Azure) harvests what is new upstream and publishes the
  slugs the store lacks in small batches (`harvest.MarketplaceRefresher`; status under
  `GET /api/v1/admin/registry/import`, `POST /api/v1/admin/registry/import/refresh` runs it now). From a
  machine, `publish --only-new` asks `POST /api/v1/admin/registry/import/check` which slugs exist and sends
  only the rest. Browsing 100k+ listings stays fast: covering indexes for facets, search counts and the
  popular sort, cached stats and a start-up prewarm. `docs/MARKETPLACE.md` "Imported catalogue".

Roadmap exit criteria: v0.5 and v0.6 closed
- `eval.criteria.effort_token_savings()` and `synthetic_effort_rows()` measure the effort-routing half of the
  v0.5 exit criterion: a reasoning model exposed through `expand_elastic()` as `effort="low"` / `effort="high"`
  siblings, routed by the real `Router` with `EffortStrategy` + `TokenBudgetStrategy`, spends 61.6 % of the
  always-think tokens at equal quality (2 000 rows). `EvalRow.tokens` (per-target tokens spent) is the new
  dataset field and `osr eval DATASET --effort` runs the same comparison on rows collected from live
  thinking / non-thinking pairs.
- `eval.agentic` (`AgentTask`, `load_agentic_tasks()`, `synthetic_agentic_tasks()`, `task_routing_frontier()`)
  measures the tau-bench half of the v0.6 exit criterion: multi-step tasks with a per-agent per-step success /
  latency table are replayed through `ProgressRouter` + `TaskCredit` with `CallableHarness` agents and compared
  with every always-use-agent-X policy on common random numbers; routed accuracy 0.987 vs 0.977 for the best
  single agent at 45.4 % of its latency (1 000 tasks). `osr eval TASKS.jsonl --agentic [--retries N]` runs it on
  a real table.
- `scripts/exit_criteria.py` and `criteria.run_all()` include both; 8/8 criteria met. `docs/ROADMAP.md` marks
  v0.5-v0.8 **Complete**; v0.9 closes with the review pack and v1.0 gets its readiness table (see above).
- v0.4 public-suite measurement published in `docs/ROADMAP.md`: on the held-out MT-Bench / PPE / WebDev Arena
  human-preference suites relabelled to three tiers, the learned router beats the declarative router by 10-14 pp
  and every naive baseline, but not the always-best-single-tier task table (weak pairwise labels, noise floor not
  measurable on single-sample rows); ECE <= 0.10 holds only after fitting a calibrator.
- `calibration_report()` returns `held_out`: a `TemperatureScaler` and an `IsotonicCalibrator` fitted on the
  even rows and scored on the odd rows (raw vs temperature vs isotonic ECE / Brier), so `osr eval --calibration`
  shows what a fitted calibrator would achieve out of sample instead of only the raw ECE.
- `examples/leaderboard/`: the three-tier catalogue (`targets.yaml`) and the collect / train / evaluate recipe
  behind the v0.4 table, i.e. the reproducible config the v1.0 leaderboard submissions link to; published on the
  documentation site as "Benchmarks and leaderboard recipe" (`/docs/leaderboard`, Python SDK section).

### Changed

Web site redesign
- The hosted web app has a light, editorial design: Inter for text, Instrument Serif for display titles (marketing,
  docs and auth pages), Geist Mono for code and numerals, Quicksand kept for the wordmark; a warm neutral palette
  (white pages, hairline rules, ink buttons and links) with the brand hues reserved for the mark, charts and target
  kinds. The landing page has a new section structure and copy (hero with the live ledger of most-routed targets,
  vendor strip, six routing principles, how it works, featured models, model landscape, code, live catalogue,
  editions, research, get started) while keeping every live data feed. Marketing pages, the light footer, the auth
  pages, the dashboard shell and the documentation shell (sidebar, topbar, titles) follow the same system; the
  documentation prose itself is unchanged. `docs/BRAND.md` records the website palette and typography.
- The site header dropdown menus render in the shared centered viewport (they previously overflowed the right
  viewport edge and flickered on hover). The landing page gains a live usage band (tokens routed all time and in
  the last 24 hours, requests, success rate, models, accounts), a value section with concrete savings / audit /
  cost numbers, per-industry routing examples and a real healthcare `rules.yaml`, and restrained motion (scroll
  reveals, animated counters, a vendor-logo marquee, all disabled under `prefers-reduced-motion`). A later
  typography pass removed the italic mid-headline pivot from every display heading, the decorative hero glow
  and the card tilt.
- Vendor icon coverage matches the live catalogue (Kwaipilot, Meituan LongCat, IBM Granite added; the Moonshot AI
  mark switched to the visible monochrome variant) and curated vendor display names win over harvested ones.
  The hero prompt examples are chip pills, a floating back-to-top button appears on every page, the rankings
  page shows the daily per-target traffic and vendor/domain share charts, and alert notices flow inline text
  correctly.
- A dark landing-page band presents the routing SLM: the collect / train / eval / serve pipeline, the measured
  holdout numbers, and the autopilot, champion-challenger and distillation loops.

### Added

Account lifecycle end to end
- Anyone can sign up with any email address and a password, and recover the account: `POST /api/v1/auth/password/forgot`
  emails a single-use, one-hour reset link (always `202`, never reveals whether an address exists) and
  `POST /api/v1/auth/password/reset` sets the new password, revokes every session, confirms the address and signs the
  person in. Web pages `/forgot-password`, `/reset-password` and a *forgot password?* link on `/login`.
- Email confirmation: signup sends a link to `/verify-email`; SSO profiles with a verified address, accepted invitations
  and password resets confirm it too. `email_verified` and `last_login_at` on the user object; the account page shows
  a banner with *Send again* (`POST /api/v1/auth/verify`, `POST /api/v1/auth/verify/send`).
- Self-service deletion (`POST /api/v1/me/delete` with the email typed and the password): sessions, identities and
  memberships go, solo workspaces are disabled with their keys, an organization with other members needs another
  owner first; a notice is emailed. *Delete my account* on `/dashboard/account`.
- Invitations are emailed to the invitee (the link is still returned once to the inviter); a removed member's
  sessions on that workspace are revoked.
- Account audit log (`audit_log` table): signups, sign-ins and failed attempts, password and email changes,
  invitations, joins, role changes, removals, deletions and operator actions, mirrored on the `admin` event topic.
  `GET /api/v1/me/audit` (own history, shown on the account page), `GET /api/v1/workspace/audit` (admins),
  `GET /api/v1/admin/audit-log` and the `/admin/audit` console page.
- Outbound email (`osr_platform/mail.py`): standard-library SMTP from `OSR_PLATFORM_SMTP_URL` (`smtp://` STARTTLS or
  `smtps://`) with `OSR_PLATFORM_MAIL_FROM`; every message is also kept in an outbox, so without a mail server operators
  hand the links over from `/admin/mail` (`GET /api/v1/admin/mail[/{id}]`). `mail` in `GET /api/v1/info`; Bicep/azd
  parameters and the deploy workflow carry the two variables.
- Operator console per user: *Send reset link* (`POST /api/v1/admin/users/{id}/reset-link`, link returned for
  support), *Resend confirmation* / *Mark verified* (`POST .../verify-link`, `PATCH` with `email_verified`), lifecycle
  history (`GET .../audit`), last sign-in; users created without a password get a reset link to choose one.

Single sign-on configuration from the CLI
- `scripts/sso.ps1` configures the hosted platform's sign-in providers end to end: `microsoft` creates the Microsoft
  Entra app registration with `az` (work/school and personal accounts, `<WEB_URL>/auth/callback` redirect, Graph
  `openid email profile User.Read`, service principal, two-year client secret), `google` / `github` / `gitlab` walk
  through the provider consoles and prompt for the client pair (no API exists for those OAuth clients), `show`,
  `remove`, `push` (GitHub Actions secret) and `apply` (`azd provision` keeping the running images, then verifies
  `GET /api/v1/auth/providers`). Secrets are never printed.
- JSON-valued deployment inputs are now base64-encoded parameters (`OSR_PLATFORM_SSO_PROVIDERS_B64`,
  `OSR_PLATFORM_STRIPE_PRICES_B64`, decoded in `infra/main.bicep`): `azd` substitutes them textually into
  `main.parameters.json`, so a raw JSON value made every provision fail.

Web site analytics and search optimisation
- Google Analytics 4 on the public pages of the hosted web app with Consent Mode v2: a consent banner (Accept /
  Decline, stored in `localStorage` as `osr-consent`, changeable on the privacy page), denied defaults until the
  visitor accepts, truncated IPs, one `page_view` per client-side navigation and `sign_up` / `login` events as the
  conversions for Google Ads. The tag never loads under `/dashboard` and is absent when the build has no
  `NEXT_PUBLIC_GA_MEASUREMENT_ID` (azd: `OSR_GA_MEASUREMENT_ID`; optional `OSR_GOOGLE_ADS_ID`,
  `OSR_GOOGLE_SITE_VERIFICATION`, `OSR_BING_SITE_VERIFICATION`).
- Every public page now ships a canonical URL, keywords, Open Graph and Twitter card (`pageMetadata` in
  `platform/web/src/lib/seo.ts`); the landing page has a descriptive title, and the model, marketplace, comparison
  and documentation pages have per-entity titles and descriptions. schema.org JSON-LD: `Organization` and `WebSite`
  site-wide, `SoftwareApplication` for the product and for each marketplace listing (with ratings), `Product` offers
  and `FAQPage` on pricing, `BreadcrumbList` and `TechArticle` on documentation, model and comparison pages.
  `robots.txt` blocks the dashboard, proxies, one-time links and query-string variants; the sitemap lists canonical
  paths only. The privacy page and the platform guide describe the analytics tag.
- Search and social distribution end to end: generated Open Graph cards for every page (`/og?title=...`, brand
  typeface, used automatically by `pageMetadata`), programmatic vendor hubs (`/vendors` and `/vendors/<vendor>`
  with per-vendor prices, context windows, benchmarks and FAQ), a `/compare` index for the alternatives pages,
  `/llms.txt` for AI assistants, an Atom release feed at `/feed.xml` (advertised on every page), a `SearchAction`
  on the `WebSite` schema and `ItemList` schema on hub pages. IndexNow: the web app serves `INDEXNOW_KEY` at
  `/indexnow.txt` (azd: `OSR_INDEXNOW_KEY`) and the platform workflow submits the sitemap to Bing/Yandex after
  each deploy (`platform/web/scripts/indexnow.mjs`). Analytics: Core Web Vitals reported to GA4, outbound-link
  and `data-track` call-to-action events, `copy_code` on install snippets and Google Ads conversion labels per
  event (`OSR_GOOGLE_ADS_CONVERSIONS`, e.g. `sign_up=AW-123/label`). Vendors and Compare join the header,
  footer and command palette.

### Added

CLI installers and sign-in
- Native installers for the `osr` command: `curl -LsSf https://opensmartroute.ai/install.sh | sh` (Linux, macOS)
  and `irm https://opensmartroute.ai/install.ps1 | iex` (Windows). Both pick `uv tool install`, `pipx` or a private
  venv (never the system Python), bootstrap uv when nothing else is available, honour `OSR_VERSION`, `OSR_EXTRAS`,
  `OSR_INSTALLER` and `OSR_NO_MODIFY_PATH`, put `osr` on PATH and verify with `osr --version`. The scripts live at
  the repository root, ship in the sdist and with every GitHub release, and the web app serves them at
  `/install.sh` and `/install.ps1`; `osr --version` prints the SDK version.
- `osr login` signs in to the hosted platform (community or enterprise edition) with the OAuth 2.0 device
  authorization grant (RFC 8628): the CLI prints a short code, opens `/cli/authorize`, the user approves in the
  browser and the platform mints a workspace API key for that machine. `osr login --token` / `--with-token` store a
  pasted key or a self-hosted server token instead; `--url` and `--profile` keep several deployments side by side.
  `osr whoami`, `osr logout [--all]`, `osr token generate|create|list|revoke` and `osr mcp --remote` (bridge to the
  signed-in workspace without copying the key) complete the set. Credentials are stored per profile in
  `~/.config/opensmartroute/credentials.json` (`%APPDATA%\opensmartroute` on Windows, `OSR_CONFIG_DIR`); flags,
  then `OSR_API_URL` / `OSR_API_KEY`, then the saved profile decide which one applies.
- `opensmartroute.credentials`: `config_dir`, `CredentialStore`, `Credential`, `PlatformClient`, `device_login`,
  `whoami`, `generate_token`, `token_kind`, `redact` - stdlib only, injectable transport. `branding` gains
  `WEBSITE`, `REPOSITORY`, `INSTALL_SCRIPT_SH|PS1`, `API_URL_ENV`, `API_KEY_ENV`, `CONFIG_DIR_ENV`, `TOKEN_PREFIX`,
  `LOCAL_TOKEN_PREFIX` and `platform_url()`; `errors.AuthenticationError` (`OSR_AUTH`).
- Self-hosted `osr serve` can now require access tokens: `serve --token T` (repeatable), `--generate-token`
  (prints an `osr_local_...` token once with the matching `osr login` command), `--require-auth`, or
  `ServerSettings` (`OSR_SERVER_AUTH_TOKENS`, `OSR_SERVER_AUTH_TOKENS_FILE`, `OSR_SERVER_REQUIRE_AUTH`);
  `create_app(router, auth_tokens=[...])`. Every path except `/healthz`, `/readyz`, `/metrics`, `/whoami` and the
  OpenAPI documents then needs `Authorization: Bearer` or `X-API-Key` (constant-time compare, 401 with
  `WWW-Authenticate`). New `GET /whoami` describes the server and whether the caller is authenticated. Helm chart:
  `auth.tokens`, `auth.existingSecret`, `auth.required`.
- Platform API: `POST /api/v1/auth/device/code` (anonymous, rate limited), `GET /api/v1/auth/device/{user_code}`,
  `POST /api/v1/auth/device/approve|deny` (admin role, plan key quota) and `POST /api/v1/auth/device/token`
  (`authorization_pending`, `slow_down`, `access_denied`, `expired_token`); device codes are stored hashed and
  expire after 15 minutes. Web: `/cli/authorize` approval page (sign-in redirect, code entry, key name, approve/deny),
  install one-liners on the docs landing page, footer and integrations page.

Tracing and observability
- `opensmartroute.observability`: a `Tracer` that every stage of a request reports to - spans `request`,
  `http.request`, `route`, `plan`, `execute`, `autopilot.cycle` and events for signals, policy, pinning,
  narrowing, shortlist, ranking (per-strategy timings), escalation, exploration, abstention, fallback, plan
  slots, execution steps, outcomes, task credit, calibration, breaker transitions, cache hits and misses,
  tenant rejections, slow decisions, guard verdicts, shadow comparisons and A/B verdicts, fair-share
  throttling and autopilot drift (`SPAN_NAMES`, `EVENT_NAMES`). Sinks: `MemorySink` (ring buffer with
  `events()` / `trace(request_id)`), `MetricsSink` (counters, p50/p95/p99, `prometheus()`), `LoggingSink`,
  `FileSink` (JSONL) and `adapters.optional.OpenTelemetrySink` (`otel` extra: OTel spans nested through
  the OTel context and joined to an inbound `traceparent`, span events, counters and a duration histogram).
  `configure_tracing(*sinks)` / `get_tracer()` for the process-wide tracer, `Router(tracer=)`,
  `RouterBuilder.with_tracing(...)`, `current_tracer()` / `use_tracer()` for components; trace ids come
  from `traceparent`, a UUID request id or a hash of the request id. Request text never enters a span
  (`text_digest()`); attributes are bounded and JSON-clean; a failing sink is counted, never raised.
  `ObservabilitySettings` (`OSR_OBSERVABILITY_METRICS|MEMORY_EVENTS|LOG_EVENTS|EVENTS_FILE|OTEL|SAMPLE_RATE|ATTRIBUTE_MAX_LEN`).
- `osr serve`: `http.request` span per request (probes and observability endpoints excluded),
  `X-OSR-Trace-Id` response header, `GET /events` (filter by request id, trace id, name glob, kind, level),
  `GET /trace/{request_id}`, tracer counters appended to `GET /metrics`, `observability` block in `/stats`.
  `osr route --events` prints the trace of the decision; every CLI router traces into the sinks named by
  `OSR_OBSERVABILITY_*`. The hosted platform wires the same tracer into both editions (`/api/v1/stats`).
  New `docs/OBSERVABILITY.md`.

Metrics, caching and AI governance end to end
- `enterprise.MetricsTelemetry` is a real metrics store: `snapshot()` returns decision counts per target,
  policy rejections, errors by type, a routing-latency histogram with p50/p95/p99, mean confidence,
  outcomes per target (success rate, cost, quality, latency) and uptime; `prometheus(namespace=, extra=)`
  renders the Prometheus exposition format (`osr_route_decisions_total`, `osr_route_latency_ms_bucket`,
  `osr_outcome_cost_usd_total`, `osr_decision_cache_hits_total`, ...). `CacheMiddleware.stats()`,
  `invalidate()` and `len()`; `RouterBuilder.with_cache(ttl_s, max_size)` and `with_audit(sink,
  outcomes=True)` so the hash chain also records how each decision turned out.
- `osr serve`: `/healthz` and `/readyz` probes, `GET /metrics` in Prometheus format (router metrics plus
  tracer counters and a `targets` gauge), request-latency and error observation on every routed call. The
  Helm chart and the Azure Container Apps template use `/readyz` for readiness; the chart's default
  `podAnnotations` carry `prometheus.io/scrape|path|port`.
- Platform API observability (`osr_platform.observability`): every response carries `X-Request-Id`
  (echoed when the client sends one) and `Server-Timing`; JSON access log on `osr.platform.access` with
  route template, status, latency and account; per-route HTTP counters, latency histogram and in-flight
  gauge. `GET /healthz` (targets, uptime), `GET /readyz` (503 with the failing check: database,
  catalogue, router, audit file) and `GET /metrics` (`osr_platform_http_*` plus the router's metrics;
  public by default, `OSR_PLATFORM_METRICS_PUBLIC=false` restricts it to `X-Admin-Token`).
  `GET /api/v1/stats` adds `http`, `controls`, `cache` and `observability` blocks; breaker snapshots
  serialise cleanly (`budget_remaining: null` instead of infinity).
- Platform caching: public catalogue reads (`/api/v1/info`, `/models*`, `/rankings`, `/llms*`,
  `/catalogue`, `/stats/public`) return `ETag` and `Cache-Control: public, max-age, stale-while-revalidate`
  and answer `304` to `If-None-Match` (`OSR_PLATFORM_HTTP_CACHE_MAX_AGE_S`, `0` disables). An optional
  decision cache in front of either edition (`OSR_PLATFORM_ROUTE_CACHE_TTL_S`) reuses recent decisions for
  identical requests; hit rates appear in `/api/v1/stats` and `/metrics`.
- Platform governance: a **workspace policy** (`GET|PUT|DELETE /api/v1/policy`, admin role) applies the
  tenant constraint keys plus `daily_budget_usd` / `monthly_budget_usd` to every request of the
  workspace; tenants accept the same budget keys. Policy and tenant constraints merge deterministically
  (boundaries fill in, caps take the stricter value, deny lists union, allow lists intersect); unknown
  keys, unknown targets and allow/deny overlaps are rejected with `400`. Executed spend is attributed per
  tenant (`X-OSR-Tenant`); once a budget is exhausted routed calls return `429` with `Retry-After` and
  `X-Budget-Limit|Used|Period`. `GET /api/v1/governance` returns the whole posture in one payload:
  controls in force (input guard, PII redaction, steering strip, metrics, tracing, audit chain, circuit
  breakers, decision cache, learning persistence), audit-chain verification, retention, policy, budgets
  and spend, quota, tenants with monthly spend, and the catalogue's data boundaries and PII-capable
  targets. Usage rows older than `OSR_PLATFORM_RETENTION_DAYS` (default 365, `0` keeps everything) are
  purged by a background sweep.
- Web: `/dashboard/governance` (policy editor with budgets, controls, budget bars, audit and retention,
  catalogue boundaries, tenant spend) and `/dashboard/health` (enterprise: HTTP traffic, error rate,
  routing decisions and latency histogram, circuit breakers, decision cache hit rate, per-route table,
  auto-refresh). The tenant editor gains daily and monthly budgets. The proxy forwards `If-None-Match`,
  `ETag`, `Server-Timing` and the `X-Budget-*` headers; `ApiError` exposes `requestId` and `budget`.

Observability portal: traces, events and readiness in the dashboard
- Platform API: `GET /api/v1/trace/{request_id}` returns the activity row of one of the workspace's
  requests plus every span and event the tracer buffered for it (`trace_id`, `spans`, root `duration_ms`;
  `404` for another workspace's id, `events: []` once evicted). `GET /api/v1/events` lists recent tracer
  events scoped to the workspace through its request ids (`name` with `*` wildcard, `kind`, `level`,
  `request_id`, `limit` up to 2000; deployment-wide events without a request on plans with `stats`;
  `404` when `OSR_OBSERVABILITY_MEMORY_EVENTS=0`). `GET /api/v1/status` is a public readiness endpoint
  for status pages (checks, edition, versions, uptime, controls, tracing buffer; `503` while a check
  fails, `no-store`). `POST /api/v1/route` responses carry `trace_id` and `X-OSR-Trace-Id`;
  `GET /api/v1/activity` marks rows still in the buffer with `traced` and returns the buffer state.
  `usage(request_id)` is indexed.
- Closed loop: `POST /api/v1/feedback` is now stored per request (`feedback` table, purged with usage
  retention); `GET /api/v1/trace/{request_id}` returns the request's `outcomes` and its hash-chained
  `audit` records (decision and outcome, plans with `audit`), and activity rows carry an `outcome`
  summary (`reports`, `success`, mean `quality`).
- Web: `/dashboard/events` (live event stream with auto-refresh, stage/kind/level filters, most-frequent
  names, click-through to the trace); a **trace drawer** on every traced activity row and a *Trace* tab
  in the playground rendering a span waterfall (offsets and durations relative to the root span, nested
  events, level badges, expandable attributes, problems filter) with a *How it was routed* story
  (signals, policy rejections and guard verdicts, ranking bars, decision with fallbacks and execution
  steps), the reported outcome (or the feedback call to make) and the audit records; an *Outcome* column
  in the activity log; a *Systems* card on `/dashboard/health`
  (also shown in the community edition) and a status pill on the overview polling `/api/v1/status`.
  New `components/observability/trace-view.tsx` and `system-status.tsx`; the proxy forwards
  `X-OSR-Trace-Id`.

Routing SLM and autopilot on the hosted platform
- SDK: `EnterpriseRouter` now runs `Router.observers` on the auto-learning path too, so an autopilot's
  drift hook sees every outcome in the enterprise edition (it was silently dead there). `SelfImprover`
  reports each cycle as a `learn.improve` event (rows, champion vs challenger accuracy, verdict, reason)
  and each promotion as `learn.promote`; `Autopilot(tracer=)` names the tracer cycles report to when the
  host's tracer is not the process-wide one.
- Platform: `OSR_PLATFORM_SLM` serves a routing SLM as the `slm` strategy; `OSR_PLATFORM_AUTOPILOT`
  (with `_OFFLINE`, `_SOURCES`, `_INTERVAL_S`, `_MIN_ROWS`) runs the self-improvement loop in-process on
  the platform's own feedback, promoting a challenger only when it beats the champion and saving it to
  `DATA_DIR/autopilot/slm.json` (preferred over the mounted file after a restart). `GET /api/v1/learning`
  (any plan) returns the strategy ensemble and weights, learner state (persistence, drift resets,
  quarantine), per-target outcome statistics (JSON-safe), the SLM in service (rows, sources, encoder,
  calibration, fit history) and the autopilot status with recent improvement reports;
  `POST /api/v1/admin/autopilot/cycle` schedules a cycle; `controls` / `info` gain `slm` and `autopilot`.
- Web: `/dashboard/learning` (strategy weights, outcomes per target, SLM card with accuracy sparkline and
  fit history, autopilot card with drift, cycles, promotions, champion-vs-challenger and a *Run cycle*
  action, improvement-cycle table); Governance lists the SLM and autopilot controls.

Azure DNS zone and custom domain
- `infra/dns.bicep` (parameter `dnsZoneName`, azd `OSR_DNS_ZONE`): public Azure DNS zone for the platform with
  apex A to the Container Apps environment static IP, `www`/`api` CNAMEs to the app FQDNs, `asuid.*` TXT records
  for hostname validation and SPF/DMARC reject records; outputs `OSR_DNS_NAME_SERVERS`. `scripts/domains.ps1` is
  now Azure-only (phases `dns`, `bind`, `verify`, `clean`; no Cloudflare token): it creates the zone, waits for the
  registrar delegation, binds the hostnames with managed certificates, rebuilds web with the domain baked in and
  checks site, API, MCP and redirects end to end; `clean` retires the superseded `osr-platform` container app.
- Web: `platform/web/src/proxy.ts` 301s alias hostnames listed in `OSR_WEB_REDIRECT_HOSTS` (set by Bicep to the
  web alias domains, e.g. `www.`) to the canonical `NEXT_PUBLIC_SITE_URL`.
- Apex managed certificates validate over HTTP (`TXT` validation never completed for the apex); subdomains keep
  CNAME validation.
- Continuous deployment: the `azd up` job in `.github/workflows/platform.yml` signs in with an OIDC federated
  credential bound to the `production` GitHub environment, reads the custom-domain parameters from repository
  variables and seeds `SERVICE_*_IMAGE_NAME` from the running apps so a provision never swaps them to the placeholder
  image. Setup commands in `platform/README.md` ("Continuous deployment").

Quotes, recommended models and the MCP server
- `opensmartroute.estimate`: quote a request before sending it. `estimate(router, request,
  output_tokens=, kinds=, prices=, quality_tolerance=, decision=)` routes as usual and prices every
  ranked candidate (`TargetEstimate`: input / output / per-call cost, latency, quality estimate,
  context fit, energy and CO2, rationale) into a `RequestEstimate` with the `recommended`,
  `cheapest` (within a quality tolerance of the best), `best_quality` and `fastest` picks and
  `savings_usd`; `estimate_tokens` / `estimate_messages_tokens` heuristics, `target_prices` (split
  `usd_per_1k_input` / `usd_per_1k_output` when declared), `PriceHook` for live price feeds. CLI
  `osr estimate TEXT [--output-tokens N] [--cost-weight W] [--monthly R] [--json]`.
- `opensmartroute.mcp_server`: a standard-library Model Context Protocol server (`2025-06-18`, JSON-RPC
  2.0) over any router. Tools `route`, `estimate`, `recommend` (priority balanced / cost / quality /
  speed), `explain`, `list_targets`, `feedback`, plus `ask` (route and execute) and
  `marketplace_search` / `marketplace_get` when the `execute` / `marketplace` hooks are given;
  resources `osr://targets` and `osr://stats`; `serve_stdio` transport, `RemoteMCP` / `bridge_stdio`
  HTTP forwarder. CLI `osr mcp [--list-tools]` serves a catalogue over stdio, `osr mcp --url <platform>/mcp
  --api-key` bridges to a hosted platform. `osr serve` mounts `POST /mcp` and `GET /mcp`.
- Platform: `POST /api/v1/estimate` (anonymous, rate limited per client, or keyed and metered as
  `estimate`; vendor / model names, live catalogue prices, optional `monthly_requests` projection),
  `GET /api/v1/models/recommended` (recommended, cheapest, best-quality and fastest pick per use case -
  chat, code, reasoning, summarise, extraction, pii - from live quotes, plus a leaderboard, cached one
  minute) and the MCP endpoint `POST /mcp` / `GET /mcp` (tag `mcp`; scoped to the workspace, metered
  as `mcp`, `ask` on the plan tier, marketplace tools backed by the registry). Web: `/estimate` page
  (quote form, four picks, signals, candidate table, monthly projection), "Recommended per use case"
  on `/models`, *Dashboard -> Integrations* with copy-paste MCP configuration for VS Code, Cursor,
  Windsurf, Claude Code and Claude Desktop, and `/mcp` proxied on the web origin. `docs/MCP.md`.

Web app chrome and platform pages
- Header: the main navigation collapses into the menu below the `lg` breakpoint (it overflowed on
  tablets), the menu closes on navigation and lists the signed-in shortcuts, active items carry
  `aria-current`, and a "Skip to content" link targets `<main id="main">`.
- Footer: plain headings, four groups (Product, Developers, Resources, Company) with the REST and
  Python references labelled apart, licence / SDK version / edition as a definition list, and a legal
  bar (Terms, Privacy, Support, Security).
- New pages `/support` (documentation entry points, issues and discussions, account and billing,
  security reports, enterprise, service status), `/privacy` (what the platform stores per table, request
  content handling, browser storage, processors, retention, access and deletion) and `/terms` (accounts
  and keys, plans and payment, content, acceptable use, marketplace, availability, suspension, open
  source, liability). Linked from the footer, the signup form, the dashboard footer, the 404 and error
  pages, the command palette and the sitemap; the CI container smoke test requests them.
- Command palette lists every public page (marketplace, estimate, pricing, REST API, support).

Marketplace and declarative stacks
- `opensmartroute.stack`: declarative *stacks* - one YAML/JSON document (`osr: "1"`, `kind: stack`)
  holding targets (plain or OCM), rules, settings, objective and `imports` of other stacks, catalogues
  or marketplace templates (`registry://slug@version`). `load_stack`, `validate_stack`, `plan_stack`
  (diff against a deployed stack), `dump_stack`, `starter_stack` and `Stack.router()` mirror the
  infrastructure-as-code check / diff / apply workflow; the CLI gains `osr stack init | validate |
  plan | apply` (`--registry`, `--against`, `--out`, `--route`). `examples/stack.yaml` is a complete
  support-desk example.
- Hosted marketplace (`platform/api/osr_platform/registry.py`, tag `marketplace`): listings of kind
  template, agent, skill, persona, tool, llm and prompt with slug, semver versions and changelog,
  publisher, licence, tags, Markdown readme, price, installs and star ratings. Public browse and
  detail (`GET /api/v1/registry`, `/registry/kinds`, `/registry/{slug}`, `/manifest?version=`,
  `/reviews`), publisher lifecycle draft -> review -> published -> archived (`POST /registry`,
  `PATCH`, `/submit`, `/versions`, `/archive`, `/unarchive`, `GET /me/listings`), installs and
  purchases (`POST /registry/{slug}/install`, `GET /me/installs`, one-time Stripe checkout confirmed by
  webhook), reviews (`PUT`/`DELETE /registry/{slug}/reviews`) and moderation under `/api/v1/admin/registry`
  (`approve`, `reject`, `flags`, `unpublish`). Manifests are validated against the OCM / stack schemas;
  template manifests may be sent as `{"yaml": "..."}`. Settings `OSR_PLATFORM_REGISTRY_AUTO_PUBLISH`
  (free listings go live on submit) and `OSR_PLATFORM_REGISTRY_SEED` (seed the repository's skills,
  the starter and example templates, a persona and prompts, all verified).
- Web: `/marketplace` (kind rail, search, filters, featured, pagination), `/marketplace/{slug}`
  (about, use-it snippets, manifest, reviews with histogram, versions, install / buy / rate),
  `/marketplace/publish` (four-step wizard, no code needed for prompts and personas) and
  `Dashboard -> Marketplace` (my listings with status and reviewer notes, edit page with new-version
  form, installed items). Marketplace links in the main navigation, footer, dashboard and sitemap.
  User docs: `docs/MARKETPLACE.md` (site page `/docs/MARKETPLACE`).
- OCM cost block accepts `context_tokens`; `capability_from_target` emits it.
- `PlatformAPI.optional_principal` dependency (anonymous or authenticated) and
  `Billing.listing_checkout_url` / `Billing.on_purchase` for one-time purchases.

Documentation site
- The hosted web app (`platform/web`) now renders the documentation itself as a static Next.js site at
  `/docs`: `scripts/sync-content.mjs` snapshots the repository Markdown (`README`, `CHANGELOG`,
  `CONTRIBUTING`, `SECURITY`, `docs/**`, `deploy/README.md`, `spec/ocm/README.md`,
  `.claude/skills/*/SKILL.md`) into `content/`; `src/lib/docs/catalogue.ts` is
  the single source of truth for sections, slugs and summaries; `src/lib/docs/render.ts` renders with
  remark/rehype (GFM, GitHub-compatible heading ids, Shiki highlighting, Mermaid, relative-link
  rewriting). Pages get a collapsible sidebar, scroll-spy table of contents, breadcrumbs, reading time,
  code copy buttons, previous/next links and an "Edit on GitHub" link. New pages: every Agent-Skills
  package (`/docs/skills/<name>`, with its frontmatter), `/docs/deploy`, `/docs/ocm`,
  `/docs/contributing`, `/docs/security-policy`.
  `/docs/search.json` feeds documentation pages and headings into the Ctrl/Cmd+K palette; the sitemap
  lists every page. The docs no longer depend on the platform API being up.
- The site is organised for end users of the hosted platform, the Python SDK and the pip package:
  sections Start, Hosted platform, Python SDK, Self-hosting, Concepts, Security, Agent skills and
  Releases, with a section tab bar and audience paths on the index. Internal material
  (`docs/PLATFORM_PLAN.md`, `docs/GO_TO_MARKET.md`, `docs/BRAND.md`, `docs/sales/`,
  `platform/README.md`) is no longer published on the site and stays in the repository.
- `docs/PLATFORM.md`: platform guide for end users (authentication, `/api/v1/route` fields and
  constraints, the `/api/v1/estimate` quote, execution, the OpenAI-compatible endpoint, feedback,
  plans and quotas, organizations and roles, organization SSO, tenants, the marketplace, the MCP
  server, dashboard pages, privacy, error handling). `docs/MARKETPLACE.md` is published under Hosted
  platform. `docs/GUIDE.md` lists every `osr` subcommand grouped by task.
- REST API reference generated from the platform's OpenAPI document: `/docs/api` (authentication
  schemes, base URL, conventions, endpoint groups) and one page per tag (`/docs/api/routing`,
  `/docs/api/openai`, `/docs/api/public`, `/docs/api/account`, `/docs/api/workspaces`,
  `/docs/api/auth`, `/docs/api/billing`, `/docs/api/mcp`, `/docs/api/marketplace`) with parameters,
  request and response schemas, curl and JSON examples; operations are searchable from the palette.
  Operator-only `/api/v1/admin/*` paths are left out whatever tag they carry.
- Version menu in the docs top bar built from `CHANGELOG.md` (current release, unreleased changes,
  links to each release section and PyPI).
- `.claude/skills/osr-platform`: Agent-Skills package for the hosted platform (API, web app, the
  documentation site and `azd` deployment); root `AGENTS.md` and `.github/copilot-instructions.md`
  point coding agents at the gates, the easy-to-break rules and the skill for each task.

Platform API
- The OpenAPI document declares `bearer` and `apiKey` security schemes (replacing the per-operation
  `authorization` / `x-api-key` header parameters), tag descriptions for every route group and a
  richer info block. `platform/api/openapi.json` is a committed snapshot regenerated with
  `python platform/api/scripts/export_openapi.py`; `--check` (run in CI and by
  `test_openapi_snapshot_is_current`) fails on drift.

Routing SLM and self-improvement
- `learning.RouterSLM`: a self-contained small routing model (hashed dual encoder, temperature
  calibration, target snapshots) that trains on `EvalRow`s, predicts a calibrated distribution and a
  quality/cost/latency utility per candidate, learns online from outcomes, and saves to a compact JSON
  file (`save` / `from_file`, `SLM_FORMAT`). `learning.distill_router()` compresses the whole ensemble
  (rules, capability fit, learners, judge) into one SLM from its traces or propensities.
  `learning.SLMStrategy` puts it into the ensemble (weight `OSR_WEIGHTS_SLM`) with online updates;
  `Router` accepts it like any other strategy and `osr --slm model.json route` loads one from the CLI.
- `adapters.ModelCatalogue` / `ModelCard`: live model catalogue from OpenRouter (prices, context,
  modalities, tool support) and Hugging Face model cards (downloads, likes, `model-index` benchmarks ->
  `quality_from_benchmarks`, `quality_from_popularity`), merged per model id and persisted; `targets()`
  filters by quality/price into `RouteTarget`s (`card_to_target` replaces injection-risky third-party
  descriptions), `cost_benchmark()`, `frontier()` (Pareto), `to_markdown()`; fail-soft `refresh()`.
- `adapters.WebKnowledge`: web-search discovery with keyless providers (`huggingface_search`,
  `duckduckgo_search`) and `brave_search` when `BRAVE_API_KEY` is set; `fetch_bytes` / `fetch_json` /
  `fetch_page_text` / `html_to_text` (https-only, byte-capped, risk-scored pages); a cached hit list
  that `discover_models()` feeds into the catalogue.
- `eval.DatasetCollector` / `DatasetSource` / `KNOWN_SOURCES`: collect routing datasets from the
  Hugging Face datasets-server (`fetch_hf_rows`, `iter_hf_rows`, `collect_dataset`) into a JSONL
  cache, plus `rows_from_feedback()` (outcomes -> labelled rows) and `synthetic_rows()` (ontology
  seed prompts scored by capability fit); `corpus()` dedupes and caps, `split()` is a deterministic
  holdout. `DEFAULT_SOURCES` are the publicly served pairwise battle sets `routellm-battles`
  (RouteLLM GPT-4-judged battles) and `arena-55k` (LMArena human preference); the RouterBench and
  RouterEval presets stay in `KNOWN_SOURCES` flagged `gated` (the Hub does not serve their rows
  without authentication). `from_pairwise_row()` turns a battle into an `EvalRow` (ties make both
  sides acceptable); `model_quality()` fits a Bradley-Terry model over the battles and returns each
  model's win probability against the field, which the catalogue applies as an `arena` quality
  prior. `ARENA_TIERS` / `tier_of()` / `tier_model_map()` fold the retired arena model names onto
  the operator's tiers (`osr collect --tier frontier=llm-frontier --tier mid=llm-mid --tier
  small=llm-small`, also on `osr improve`); `DatasetCollector.derive()` remaps an existing cache
  offline so tiered corpora never need a second download. `collect()` keeps partial pages when the
  Hub rate-limits mid-way and never replaces a fuller cache with a shorter one; `fetch_bytes` retries
  429/5xx with exponential backoff honouring `Retry-After` (`OSR_SLM_FETCH_RETRIES`,
  `OSR_SLM_FETCH_BACKOFF_S`, `OSR_SLM_FETCH_BACKOFF_MAX_S`) and pages pause `OSR_SLM_PAGE_PAUSE_S`.
- `adapters.fetch_leaderboard_quality()`: Open LLM Leaderboard results (IFEval, BBH, MATH, GPQA, MuSR,
  MMLU-Pro) per model via the datasets-server; `ModelCatalogue.apply_quality()` maps them (and the
  arena win rates) onto cards by exact id or `model_key()` (case/variant-insensitive tail), ranking
  sources popularity < catalogue < arena < leaderboard so a measured prior is never downgraded;
  `refresh(leaderboard=True)`, `targets(measured=True)` and `osr catalogue --leaderboard
  --quality-from DATA_DIR --measured`; `osr slm train --catalogue catalogue.json --source NAME` trains
  on measured, priced catalogue models and on selected caches only.
- `adapters.attach_chat_handlers()`: give every LLM target a `chat_handler` on one OpenAI-compatible
  client (`metadata.model` names the upstream model, or a default). The CLI does this at startup when
  `OSR_LLM_BASE_URL` is set (`OSR_LLM_API_KEY`, `OSR_LLM_API_KEY_ENV`, `OSR_LLM_MODEL`), so the
  container's `/v1/chat/completions` executes against OpenAI, Azure OpenAI, vLLM, Ollama, OpenRouter
  or LiteLLM instead of answering 502. `huggingface_search()` limits model hits to `text-generation`
  and falls back to per-keyword queries (the Hub matches repo names, not natural language);
  `fetch_json()` returns `None` for an empty 200 body (DuckDuckGo does that under load).
- Deployment: `OSR_SLM` (Docker entrypoint, Helm `config.slm` -> `/config/slm.json`) loads a routing
  SLM into the served ensemble; `OSR_LLM_BASE_URL` / `OSR_LLM_MODEL` are declared in the image.
- `learning.SelfImprover` / `ImprovementReport`: the closed loop - refresh catalogue, discover models
  and datasets on the web, gather evidence (feedback, collected datasets, synthetic), train a
  challenger on the train split, calibrate it on the holdout, and promote it over the champion only
  when holdout accuracy improves by `OSR_SLM_MIN_GAIN`; JSONL history, `run(interval_s)` loop.
- CLI: `osr catalogue`, `osr collect`, `osr slm train|eval|predict|info`, `osr improve` and the global
  `--slm` option; `settings.SLMSettings` (`OSR_SLM_*`) and `WeightSettings.slm`.
- Transformer encoders for the SLM. `learning.AttentionEncoder` is a pure-Python transformer block
  (hashed token embeddings, sinusoidal positions, multi-head self-attention, residual, attention
  pooling, hand-derived gradients) that `ContrastiveRouter(attention=...)` adds to the hashed query
  encoder so tokens can condition on each other; opt in with `OSR_SLM_ENCODER=attention`
  (`OSR_SLM_ATTENTION_HEADS`, `_HEAD_DIM`, `_MAX_TOKENS`, `_VOCAB`); the block is persisted in the
  model file (`core.encoder`, `core.attention`) and restored by `from_file`. `learning.EmbeddingFeaturizer`
  / `load_embedder()` append a frozen pretrained sentence-transformer embedding (extra `embeddings`, or
  any `Callable[[list[str]], list[list[float]]]`) as dense features, so `W` learns a linear head on top
  of it; `RouterSLM(embedder=...)`, `OSR_SLM_EMBEDDER=<model>` / `OSR_SLM_EMBEDDER_SCALE`, the file
  records the embedder name and `from_file(embedder=...)` takes a custom one back.
- Self-operation: `learning.Autopilot` runs `SelfImprover` cycles inside a live process on a schedule
  and on drift (`learning.DriftMonitor`, Page-Hinkley over the success indicator and quality of every
  outcome the router learns from), rate-limited by a minimum gap, hot-swapping an accepted challenger
  into the served `SLMStrategy` and rewriting the model file; failures are recorded, never fatal.
  `Router.observers` is the outcome hook it (and any drift monitor or metric) plugs into.
  `osr serve --autopilot [--autopilot-interval S --cache-dir DIR --catalogue FILE --source NAME
  --search QUERY --tier TIER=TARGET --offline --min-rows N]`, `create_app(router, autopilot=...)`
  exposes `autopilot` in `GET /stats` and `POST /autopilot/cycle`; settings
  `OSR_SLM_AUTOPILOT_INTERVAL_S`, `_MIN_GAP_S`, `_DRIFT_THRESHOLD`, `_DRIFT_MIN_N`, `_REMEMBER`;
  container / Helm: `OSR_AUTOPILOT=1`, `OSR_AUTOPILOT_ARGS`, `OSR_CACHE_DIR`, `OSR_CATALOGUE`.
  `status()["busy"]` says a (possibly multi-minute) cycle is in flight. `DatasetCollector.corpus()` /
  `cached()` skip the improver's own `improve-history.jsonl` and any non-row line in the cache dir -
  the second cycle used to fail with `EvalRow.__init__() missing 'text'` when the log shared the dir.
- Routing datasets: `osr collect` knows ten public sources verified against the datasets-server -
  the 2024-25 LMArena releases (`arena-100k`, `arena-140k`, `ppe-human`, `webdev-arena`), `mt-bench-human`,
  `reward-bench`, `ultrafeedback` (per-model judge scores) and `routellm-gpt4` (RouteLLM's own "is the
  cheap model good enough" labels), next to `routellm-battles` and `arena-55k`; gated ones stay listed but
  off by default. Parsers understand the newer layouts (typed conversation turns, `winner` labels, per-model
  score lists) and carry the code / language / math / hard-prompt tags into `context`; `tier_of()` has
  name rules and a parameter-count fallback for the 2024-26 model generations (`gpt-4o-mini` -> mid,
  `gemini-2.5-pro` -> frontier, `llama-3.1-8b` -> small ...) and `DatasetSource.tiers` folds every side
  onto your catalogue at parse time (`--tier` no longer needs an exhaustive model table). Collection is
  resumable: a `<name>.meta.json` records the raw offset, a run cut short by a 429 keeps its rows and the
  next `osr collect` continues from there (`DatasetCollector.progress()`); retries back off up to five
  times / 60 s. `corpus()` fills `OSR_SLM_MAX_ROWS` round-robin across sources so one large dataset cannot
  crowd the others out.
- SLM training is faster and generalises better. With numpy installed (extra `fast`, now in the serve
  image) `ContrastiveRouter.fit()` runs the same SGD on dense arrays - about 10x faster (18 000 rows in
  under a minute) - and inference projects through a dense mirror of `W`; `OSR_SLM_BACKEND=auto|numpy|python`
  picks the arithmetic, both paths seed identically and produce the same model file (tested to 1e-9), so a
  pure-Python deployment reads a numpy-trained model unchanged. The hashed featurizer caches token hashes.
  The model gains a learned per-target logit bias (`B`, the base rate a unit-norm dot product cannot
  express; older files load with none), inverse-time learning-rate decay (`OSR_SLM_LR_DECAY`, default 2) and
  regularised defaults (`epochs` 4, `l2` 1e-3): on the 18 000-row mixed corpus the previous defaults
  overfit below the constant "always the mid tier" baseline (0.62 vs 0.64 on a random holdout); the new
  ones sit at 0.64-0.66 with holdout loss 0.98-1.00 vs 1.11 and are stable across seeds.
- The SLM no longer harms mixed catalogues. Scored rows train a softmax over the *scored population* (the
  targets any row of the corpus scores, plus `OSR_SLM_NULL_TARGETS` learned "none of these" logits) instead
  of the whole catalogue, so a corpus of LLM battles leaves the embeddings of tools, skills and agents it
  never compared untouched (they used to be driven to p=0.0005 and any SLM took `examples/eval_dataset.jsonl`
  from 0.83 to 0.43; it is 0.80 now). `SLMStrategy` / `ContrastiveStrategy` abstain on targets the model
  has never compared (`ContrastiveRouter.B` records them; `RouterSLM.predict_proba()` with no explicit
  candidates ranks only those), score the rest centred (best 1.0, mean 0.5, so a flat softmax is "no
  preference" rather than a boost) and scale confidence by the share of the request's candidates they know.
  Chosen on three leave-one-source-out suites (PPE 0.561, MT-Bench 0.508, WebDev Arena 0.506 - see
  `docs/ROADMAP.md` v0.4).
- `eval.load_dataset()` raises `ConfigurationError` naming the file and line for a line that is not JSON, not
  an object or has no text (it used to surface as a `TypeError` traceback), and `osr catalogue | collect | slm
  | improve` print every SDK error as one `error:` line with its path / line / preset / backend detail and exit
  1. `ContrastiveRouter(backend=...)` with an unknown backend is a `ConfigurationError` too.

Agentic core (PLATFORM_PLAN Phase 0)
- `signals.EventSignal`, `WorkflowSignal`, `parse_event()`, `EventInfo` and the `EVENT_*` lexicons:
  textless requests (`context["event"]`, `context["workflow"]`) get domain / action signals so events
  route to workflows and agents; rules match `event:` globs and `workflow: true`.
- `strategies.SessionAffinityStrategy` / `SessionState`: keep multi-turn conversations on the target
  that owns the session unless quality drops; `Router.route(..., exclude=[...])` and fallback
  candidates; `PLAN_ROLES` is kind-aware (agent / workflow plans need no persona); per-call pricing
  on targets (`cost.usd_per_call`, `RouteTarget.cost_per_call`, `estimated_cost()`); `observe` hook.
- Proxy: `osr/auto:<variant>` presets (`cheap`, `fast`, `quality`, `private`, `agentic`, `llm`),
  `osr` request block (exclude, objective, plan, fallbacks), `X-OSR-App` attribution and
  `X-OSR-Target` / `X-OSR-Request-Id` response headers; metadata carries alternatives and cost.
- Executors: `adapters.http_handler()`, `mcp_tool_handler()` and queue-backed targets
  (`Queue` protocol, `InMemoryQueue`, `queue_handler()`, `PendingResult`) so agents, tools and human
  queues execute through the same plan runner.
- Open Capability Manifest: `spec/ocm/` (JSON Schema + README), `opensmartroute.ocm`
  (`validate_capability`, `target_from_capability`, `capability_from_target`, `load_capabilities`,
  `bind_endpoint`), `load_targets()` accepts manifests and mixed catalogues, `osr validate` and
  `osr export-ocm`.
- Routing Audit and savings: `eval.routing_audit()` / `AuditReport` / `load_audit_log()` replay
  logged traffic against a catalogue (savings vs baseline, policy violations, route mix);
  `osr audit logs.jsonl --baseline ... --markdown|--json [--min-savings]`;
  `enterprise.SavingsLedger` telemetry sink (`RouterBuilder.with_savings()`, `EnterpriseRouter.savings`)
  with per-request baseline-vs-routed entries and `SavingsReport.to_markdown()`.
- CLI: `osr route --constraint KEY=VALUE` (hard constraints) and `--kinds a,b`; `--event`, `--session`,
  `--exclude`.
- Platform: `GET /api/v1/savings`, `GET /api/v1/signals`, `GET /api/v1/admin/pql` (product-qualified
  leads); web `/roi` calculator, `/compare/{openrouter,litellm,portkey,diy}` pages and the dashboard
  savings view.
- Docs: sales enablement kit under `docs/sales/` (discovery guide, demo script, Routing Audit,
  pilot charter, ROI calculator, security pack, battlecards, pricing and proposal, case study and QBR).

Research track (every previously `Research` roadmap row now has code, tests and a module reference)
- `learning.ContrastiveRouter` and `ContrastiveStrategy`: RouterDC-style contrastive training on
  (query, target) pairs. Softmax cross-entropy over `q . e_t / tau` against a row's *acceptable set*
  (`acceptable_set(row, slack)`, every target within `slack` of the best score), so several right
  answers are not penalised; `fit(objective="distilled")` trains against Zooter-style reward-distilled
  `soft_labels(scores, temperature)` instead of one-hot labels; hashed encoders, pure-Python SGD,
  online updates from outcomes, `state()/load()`.
- `learning.PolicyGradientStrategy`, `decision_reward()`, `decision_regret()` and `RegretReport`:
  end-to-end REINFORCE routing (Router-R1 / RLCascadeRouter) on hashed request features with an EWMA
  baseline and entropy bonus, trained on the *decision* utility rather than a quality prediction;
  `decision_regret(rows, targets, choose, predict)` reports decision regret next to prediction MAE so
  the two losses can be compared on the same dataset.
- `eval.headroom`: `routing_headroom()` splits oracle minus best-single into the label-noise floor and
  measurable headroom; `target_diversity()` (pairwise disagreement, winner entropy / share);
  `min_catalogue(rows, targets, fraction)` (smallest subset keeping a fraction of the oracle);
  `scaling_curve()` (oracle / best-single / headroom against catalogue size);
  `learnability_by_difficulty()` (per-difficulty-bucket headroom and noise floor, the "least learnable
  where most valuable" check for agentic pools).
- `strategies.MemoryRouter`, `MemoryTier`, `MemoryItem`, `ImportanceGate`, `RecallResult`: BudgetMem-style
  memory-tier routing. Tiers carry capacity, write / read cost per token and latency; a hashed logistic
  gate decides what is worth writing and learns from `feedback(used_ids, recalled)`; value decays per
  `tick()`, lowest-value residents are demoted when a tier is full; `recall(query, budget_tokens)`
  fills a token budget by relevance x value.
- `strategies.ModalityStrategy`, `ModalityEscalation`, `EscalationResult`, `request_modalities()`:
  penalise text-only targets when a request carries images / screens / audio / video / files that a
  text surrogate (OCR, caption, transcript) only describes; text-first escalation keeps a Beta
  posterior on "the surrogate sufficed" per modality set and calls the multimodal target first when
  that probability is low.
- `strategies.SemanticCache`, `SemanticCacheStrategy`, `CacheEntry`, `CacheHit`: a semantic cache as a
  `DESTINATION` target. `SemanticCache.target()` serves near-duplicate answers (cosine over the router's
  embedder, TTL, LRU bound, source-based invalidation); the strategy scores only cache targets from the
  hit similarity and learns per-similarity-band hit quality from outcomes.
- `strategies.AnnotatorPool`, `AnnotatorSkill`, `QuorumPlan`, `quorum_accuracy()`,
  `HumanRoutingStrategy`: QUORUM-style routing among `HUMAN` targets. Per-domain annotator accuracy is
  estimated by Dawid-Skene EM over labels (with gold labels and direct quality reports as evidence,
  prior seeded from `quality_prior`); `select_quorum(domain, target_accuracy, budget)` picks the
  cheapest majority-vote quorum that reaches the target accuracy.
- `strategies.SpeculativeCascade`, `SpeculativeResult`, `expected_mode_costs()`: speculative
  draft / strong cascades. A Beta acceptance posterior per complexity bucket decides between
  `draft_only`, `speculate` (both in parallel, strong cancelled when the draft is accepted) and
  `strong_only` by expected cost + latency under the request's `Objective`.
- `learning.HistoryTargetStrategy`, `HistoryTargetModel` and `history_vector()`: multi-turn routing
  on history-target joint embeddings (MTRouter). A logistic model over `h * e_t` (recency-weighted
  hashed history and current text, catalogue target embedding) with shared weights and a per-target
  bias learns that the same follow-up routes differently depending on the conversation so far;
  inductive for unseen targets; `context["last_target"]` earns an incumbent bonus that becomes a
  penalty when `context["last_failed"]` is set; `state()/load()/merge()`, persisted by `AutoLearner`.
- `errors.deprecated(name, since=, removal=, replacement=)` and `errors.OpenSmartRouteDeprecationWarning`
  implement the public API deprecation policy documented in CONTRIBUTING.md.
- `signals.VerbalisedDifficultySignal` and `parse_difficulty()`: a small model's verbalised
  difficulty (`0.7`, `7/10`, "hard") from `context["difficulty"]` or a callable blends into
  `complexity` and raises `reasoning_need`. `signals.DraftResponseSignal` and `draft_features()`
  derive hedging, self-correction, query overlap and length ratio from a cheap draft answer.
- `signals.semantic_entropy()`, `semantic_clusters()`, `response_uncertainty()` (meaning clusters over
  sampled answers, hedging, optional P(True) judge), `signals.UncertaintyGate` (a `Cascade` quality
  gate / `MixtureOfAgents` trigger built on them) and `signals.EventTrigger` / `TriggerRule`
  (threshold rules that fire named actions).
- `math.DirichletProbe` (evidential Dirichlet head with digamma loss and KL regulariser, pure Python
  SGD, `predict()` returns probabilities and epistemic uncertainty `K/S`) and
  `strategies.HiddenStateStrategy(state_fn, targets, dim)` that routes on host hidden states and
  lowers its confidence on unfamiliar states.
- `math.EnergyModel` (per-target ridge fit of `Wh = e0 + e_in * prompt + e_out * output`, gCO2 via a
  grid factor) and `math.HardwareProfile` / `hardware_profile()` priors for A100, H100, L4, RTX 4090,
  16-core CPU and an edge NPU.
- `strategies.expand_elastic(parent, [BudgetVariant(...)])` creates `<id>@<budget>` siblings of an
  elastic model; `strategies.TokenBudgetStrategy` scores budget fit (truncation risk vs unused
  capacity).
- `strategies.EdgeCloudStrategy`: Thompson posterior of edge success per complexity bucket, decode
  time against the latency SLO, upload penalty for the cloud tier and a hard exclusion when the
  request's data boundary forbids a public target.
- `strategies.AuctionStrategy`, `AuctionResult`, `default_bid()`: error-aware reverse auction where
  each bidder's claimed success is corrected by its observed bias; second-price payment.
- `strategies.ProtocolPolicy`, `ProtocolRule`, `DEFAULT_RULES`, `failure_risk()`: risk / budget /
  task-type rules choose `single`, `cascade`, `aggregate`, `debate` or `handoff`; a per-protocol
  ledger reports which protocol paid.
- `strategies.SelfEscalation` and `wrap_stream()`: Bayesian competence posterior updated per streamed
  chunk with an optimal-stopping rule; `SelfEscalation.for_target()` seeds the prior from
  `quality_prior`.
- `strategies.MixtureOfAgents`, `majority_vote()`, `AggregateResult`: call the top-k alternatives when
  the decision is unsure and the budget allows, aggregate by semantic majority (or a supplied
  synthesiser) and emit one `Outcome` per participant.
- `learning.UserAdaptiveStrategy` (per-user Beta posteriors shrunk toward neighbour users and the
  global posterior; `observe_user()` warm start), `learning.SkillAffinity` (profile-bucket skill
  relevance for `select_skill_set(relevance=)`), `profile_vector()`, `profile_bucket()`.
- `learning.MixtureCureModel` (Weibull mixture-cure on cumulative trajectory risk with censoring, grid
  MLE) and `learning.HandoffPolicy` (permanent hand-off on eventual-failure probability or horizon
  hazard; releases the task's `TaskPins` pin).
- `opensmartroute.discovery`: `schema_match()` / `extract_entities()` / `SchemaAwareStrategy`
  (coverage of required `input_schema` properties by typed entities in the request),
  `CachePreservingSelector` (per-session prefix-stable tool ordering with lazy eviction and
  `prefix_hit_ratio()`), `SkillGraph` (`requires` / `conflicts` / `composes` edges plus learned
  co-usage; `compose()` returns a dependency order).
- `adapters.recommend_servers()`, `ServerCard`, `ServerRecommendation`: rank MCP servers for a task
  with BM25 + hashing-cosine fusion over server cards and hard constraint filters.
- `adapters.load_semantic_router_config()` / `SemanticRouterImport`: import a vLLM semantic-router
  `config.yaml` (categories, model scores, reasoning flags, pricing, system prompts) as targets and
  rules.
- `load_skills` exposes `osr-requires`, `osr-conflicts` and `osr-composes` frontmatter as
  `metadata["requires" | "conflicts" | "composes"]` for `SkillGraph`.
- `tests/test_roadmap_research.py` covers every item above; `tests/public_api.json` lists the new
  names.

Decorator SDK
- `opensmartroute.sdk`: `ComponentRegistry` plus the decorators `@strategy`, `@signal`,
  `@policy_rule`, `@middleware`, `@telemetry` and `@target` / `@tool` / `@skill` / `@agent`. Plain
  functions are adapted through `FunctionStrategy`, `FunctionSignal` and `FunctionMiddleware`;
  classes are registered as factories. `registry.router()` / `registry.builder()` assemble a
  `Router` / `RouterBuilder` from everything declared; `include("pkg.mod[:hook]")` and
  `discover()` (entry-point group `opensmartroute.plugins`) load plugins. The process-wide registry
  is exported as `opensmartroute.components` and the top-level decorators bind to it.
- `RouterBuilder.with_components(registry, targets=, strategies=, signals=, policy=, middleware=,
  telemetry=)` wires a registry into an existing builder.
- `strategies.default_strategies(seed=, settings=, state_dir=)` builds the default ensemble.

Settings
- `opensmartroute.settings`: immutable `Settings` grouped into `RoutingSettings`,
  `PolicySettings`, `RulesSettings`, `CapabilitySettings`, `BanditSettings` and `WeightSettings`;
  `configure()`, `get_settings()`, `Settings.from_env()` (`OSR_<GROUP>_<FIELD>` overlay with typed
  parsing and `ConfigurationError` on bad values) and `Settings.env_keys()`. `Router`,
  `RouterBuilder`, `Policy`, `RulesStrategy`, `CapabilityStrategy` and `BanditStrategy` accept
  `settings=`; every previously hardcoded threshold, weight and scale now resolves from it.
- `Router.weight_of(strategy)` reports the effective ensemble weight.
- `osr settings [--json]` prints the effective tunables, their `OSR_*` keys and which are overridden.

Agent skills
- `.claude/skills/`: eight Agent-Skills packages (`osr-routing-catalogue`, `osr-decorator-sdk`,
  `osr-enterprise-builder`, `osr-evaluation`, `osr-security-hardening`, `osr-integrations`,
  `osr-deploy-serve`, `osr-contributing`) that teach coding agents such as Claude Code how to use and
  extend the library. They are valid `TargetKind.SKILL` targets: `tests/test_skills.py` loads them
  with `load_skills`, checks the routing hints against the signal ontology and asserts the router
  fills the plan's skill slot with the right package for a representative request.
- `osr skills [ROOT] [--json]` validates and lists SKILL.md packages (default root
  `branding.SKILLS_DIR` = `.claude/skills`); `osr --skills ROOT ...` adds them to any routing command.

Documentation coverage
- `docs/REFERENCE.md`: generated API reference listing every module under `src/opensmartroute` and
  every exported name with its kind, signature and one-line summary; re-exports link to their
  definition. Produced by `scripts/api_reference.py` (`write` | `check` | `report`), stdlib only.
- Every exported class, function, constant and type alias now carries a one-line summary (docstring,
  or a same-line comment for constants). `tests/test_docs.py` fails on an undocumented name, a stale
  reference, a module missing from the reference, or a `docs/*.md` file not linked from the README.

Policy composition
- `Policy` is an ordered chain of `PolicyRule` callables (`enabled`, `allow_list`, `deny_list`,
  `allowed_kinds`, `region`, `DataBoundaryRule`, `pii`, `tenant`, `cost_budget`, `latency_slo`,
  `input_tokens`, `context_window`, `tools`, `modalities`, `LanguageRule`, `JailbreakRule`).
  `Policy.with_rules(*rules)` appends rules (subclasses that override `check` are folded in as a
  single rule); `Policy.rule_names` and `default_rules()` expose the chain.

Branding
- `opensmartroute.branding`: single source of truth for brand-derived names (`BRAND`, `PACKAGE`,
  `CLI`, `ENV_PREFIX`, `ERROR_CODE_PREFIX`, `METADATA_PREFIX`, `STATE_DIR`, `ENTRY_POINT_GROUP`)
  with `env_key()`, `error_code()`, `metadata_key()`, `logger()`, `user_agent()` and `version()`.
  Error codes, `osr-*` frontmatter keys, the HTTP `User-Agent`, MCP `clientInfo`, the OpenAI proxy
  alias and the FastAPI title/version derive from it.

Learning
- Judge-score calibration: `LLMJudgeStrategy(calibrate=True, min_fit=20, calibrator=)` keeps an
  `IsotonicCalibrator` per judge, pairs each raw judge score with the served target's observed
  outcome and re-fits on every `update()`; `Router.learn()` forwards outcomes to the judge even
  when it is only used for escalation. Raw scores stay visible in the rationale (`raw=`); state
  round-trips through `state()` / `load()`.
- `learning.warm_start_from_matrix(rows, strategies, weight=)`: offline full-information reward
  matrix `(prompt, {target: reward})` replayed into every learner (LinUCB ridge per arm, IRT,
  Thompson posteriors, task table, Markov rewards) before going online.
- `MarkovStrategy(state_from="auto" | "task_type" | "domain", decay=)`: conversation state is the
  learned task type joined with the domain (not only the domain); per-request state memory so an
  outcome credits the state that was scored. `MarkovChain(decay=)` forgets transition counts,
  `RoutingMDP(decay=)` keeps exponentially weighted rewards.
- Federated merge on every learner: `MarkovChain.merge`, `RoutingMDP.merge`, `MarkovStrategy.merge`,
  `BanditStrategy.merge`, `TaskTableStrategy.merge`; `learning.merge_learners(local, remote)` pairs
  same-named strategies across replicas. `AutoLearner.refresh()` implements the writer / reader
  pattern for replicas sharing one `StateStore`.
- Cascade steps feed every learner: `CascadeStep.cost_usd`, `CascadeResult.total_cost_usd`,
  `CascadeResult.outcomes(request)` (one per-step `Outcome`, rejected steps are failures) and
  `Router.learn_cascade(request, result)`. Learners keep their per-request memory while
  `Outcome.step` is set so multi-step trajectories credit every step.
- Strategies condition on the plan role: `Router._build_plan` tags sub-route signals with
  `signals.extra["role"]`; `BanditStrategy` contexts, `TaskTableStrategy` rows (`skill:<task>` plus
  `family:skill:<fam>`), `MarkovStrategy` states and every learner's request memory are keyed by
  role (`strategies.base.role_of` / `memory_key`), so a target's record as a skill or persona never
  leaks into its record as the primary answerer. `TaskTableStrategy` now learns from role outcomes.

Economics
- Energy and carbon as cost dimensions: `RouteTarget.unit_energy` (`cost["wh_per_1k_tokens"]`),
  `RouteTarget.unit_carbon` (`cost["gco2_per_1k_tokens"]`, or energy x `cost["gco2_per_wh"]`) and
  `Objective(energy=, carbon=)` weights applied to the log-normalised values in the utility.

Resilience
- `AutoLearner.load()` quarantines corrupt or incompatible learner state instead of failing: the file
  is renamed `*.corrupt-<ts>` (or the store key copied under `quarantine/`), the learner starts blank
  and `AutoLearner.quarantined` lists `(strategy, reason)`. A failing `StateStore.get` is recorded,
  not raised.
- `FileAuditSink` resumes its hash chain from the last line on disk across restarts and gains
  `FileAuditSink.verify(path) -> (ok, n, first_problem)` detecting edited, removed or unparseable lines.
- `LLMJudgeStrategy` filters judge output item by item: non-dict items, unparseable / NaN scores and
  hallucinated or injected target ids are skipped without discarding the rest of the ranking;
  non-string judge returns are a failure mode with zero confidence.
- `scripts/bench.py --scale` sweeps 16 / 64 / 256 / 1024 targets with and without retrieval narrowing
  and reports p50 / p95 / p99; `docs/ARCHITECTURE.md` documents the measured envelope.
- `tests/test_resilience.py`: audit-chain verification, judge JSON failure modes, state-corruption
  recovery (files and stores), multi-replica hand-off.

Deployment
- `deploy/Dockerfile` (non-root, multi-stage, `OSR_*` environment to `osr serve`), `deploy/entrypoint.sh`
  and the Helm chart `deploy/helm/opensmartroute` (ConfigMap-driven catalogue, hardened pod security
  context, HPA, PDB, NetworkPolicy, optional PVC with a multi-replica warning).

Roadmap exit criteria
- `eval.criteria`: every offline-measurable exit criterion in `docs/ROADMAP.md` as a simulation that
  drives the real `Router` / learners and returns a `CriterionResult` - `cold_start_ratio()` (v0.5),
  `multi_round_vs_best_single()` (v0.6), `match_at_1_at_scale()` with `synthetic_tool_catalogue()`
  (v0.7), `conformal_coverage()`, `knapsack_never_exceeds_cap()` and `ope_within_live_ci()` (v0.8),
  plus `bootstrap_ci()` and `run_all()`. `scripts/exit_criteria.py [--full] [--json]` runs them and
  exits non-zero when one fails.
- `osr eval --multi-round [--threshold] [--max-rounds]` compares `MultiRoundExecutor` against the
  best single target on a dataset whose rows carry per-target `scores`.
- Cold-start exploration: `Router(explore_rate=, explore_min_samples=, explore_seed=)` and
  `RoutingSettings.explore_rate` / `explore_min_samples` occasionally serve the least-seen viable
  candidate until it has enough outcomes; propensities include the exploration mass and
  `signals.extra["explored"]` marks the request.
- `MultiKnapsackBandit(on_capped="release" | "abstain")` and `capped(scores)`: when every arm would
  breach a hard cap the bandit can abstain instead of releasing the cheapest arm.
- `MultiRoundExecutor(failures_before_switch=)` forwards the switch policy to `ProgressRouter`.
- Thread safety: `Router.lock` (re-entrant) guards strategy scoring, every learner update, task
  pins, request memory, retrieval narrowing and the exploration RNG; the LLM judge is scored outside
  it. `AutoLearner(lock=)` shares that lock (`EnterpriseRouter` / `RouterBuilder` pass it), snapshots
  are deep-copied and sequence-numbered so a slow `save()` never overwrites a newer one, and a
  separate I/O lock orders writers against `load()` / `refresh()`. `SemanticCache`, `MemoryRouter`,
  `AnnotatorPool` and `ModalityEscalation` lock their own state.
- `SpeculativeCascade(concurrency=, draft_timeout_s=, history_size=)`: bounded worker pool sized for
  concurrent speculative runs, draft timeout, draft exceptions escalate to the strong target
  (`details["draft_error"]` / `["draft_timeout"]`), late strong failures are swallowed after an accepted
  draft, `draft_failures` / `strong_failures` counters and `stats()`.
- `ModalityEscalation.run` escalates when the text target raises (`details["reason"]`) instead of failing.
- `MultiRoundExecutor`: rounds whose handler raised become `Round(error=...)` and the loop continues
  on another target; one judged learn per round (no provisional + correction double count);
  `deadline_ms`, `return_best`, `credit_task`, `RoundResult.stop_reason` / `best` / `errors` /
  `total_cost_usd`; `arun()` for coroutine handlers and an `async` judge. `ProgressRouter.run_step`
  (`learn=`) routes via `router.route`, records a failed step before re-raising, estimates step cost
  from the target's unit cost when the handler reports none; `arun_step` twin.
- `tests/test_e2e_scenarios.py`: real-scenario suite with no stubs - the shipped `examples/`
  catalogue and rules driven end to end through an in-process OpenAI-compatible provider fleet over
  HTTP (PII boundary, tenant deny lists and data boundaries, persona plans, metrics, hash-chained
  audit, learner state surviving a restart), a provider outage opening and recovering a circuit
  breaker, the FastAPI service over an `EnterpriseRouter` (OpenAI proxy, feedback, stats, error
  mapping), the `osr` CLI (`route`, `eval --frontier`, `--min-accuracy` gate, `targets`, `stats`),
  `ProgressRouter` / `MultiRoundExecutor` agentic loops, a real MCP server subprocess over stdio, and
  24 concurrent requests through the full middleware stack.
- `EnterpriseRouter.credit_task()`: delivers the delayed task reward through the router and persists
  the re-credited learner state; `MultiRoundExecutor(EnterpriseRouter)` now closes the task-credit loop.
- Execution: a primary target without a handler (an instructions-only skill or persona) runs on the
  plan's `llm` slot - its instructions are disclosed in the system prompt, the model answers, and the
  model is credited with a `role="llm"` outcome - instead of failing with "has no handler".
- Execution enforces the PII boundary set up by `GuardMiddleware(redact=True)`: targets whose
  constraints allow PII receive the original text (`context["pii_restored"]`), every other target
  keeps the placeholders (`context["pii_redacted"]`), and the mapping never reaches a handler.
- `create_app()` accepts an `EnterpriseRouter` (middleware, health, telemetry and audit apply to every
  HTTP request; `/stats` includes the health snapshot) and maps `SecurityError` / `ValidationError`
  to 400, `NoRouteError` to 422, `TargetUnavailableError` to 503 and `ExecutionError` to 502.
- `DomainActionSignal` detects arithmetic and unit conversions (`17% of 2,450`, `12 * 4`,
  `5 miles to km`, percent / calculate / square root ...) as the `math` domain, so calculator-style
  tools and `max_complexity` math rules fire; lexicon keywords made of symbols (`%`) now match.
- `examples/targets.yaml`: the public cloud models and the research agent carry
  `constraints.pii_allowed: false`, making PII a hard policy stop instead of only a rule preference.

Hosted platform (`platform/`, separate from the zero-dependency SDK): `platform/api` (package
`osr-platform-api`, FastAPI) and `platform/web` (`osr-platform-web`, Next.js 16)
- `osr_platform.app.create_platform_app()` / `osr-platform-api`: the routing API as a service:
  `/api/v1/route` (decision, signals, ranked breakdown, policy rejections; `plan` / `execute`),
  `/feedback`, `/targets`, `/catalogue`, `/usage`, `/keys`, `/tenants`, `/stats`, `/audit`, the
  OpenAI-compatible `/v1/models` + `/v1/chat/completions` (`model: "auto"` or a pinned target), and
  `/api/v1/docs` serving the repository Markdown (README, CHANGELOG, `docs/`) rendered to HTML with a
  table of contents and Mermaid blocks as JSON. `OSR_PLATFORM_WEB_URL` / `OSR_PLATFORM_CORS_ORIGINS`
  point `/` and Stripe redirects at the web app and allow it as a browser origin.
- Web app: landing page with a live catalogue, documentation (sidebar, table of contents, Mermaid),
  pricing, a playground (decision view with ranked utility bars, signals, policy rejections, plans,
  execution; OpenAI-compatible chat with `auto` or pinned model), signup / sign-in (key shown once,
  stored in the browser only) and a dashboard (overview with usage chart, API keys, usage by target
  and endpoint, tenants editor, audit trail with chain verification, plan and billing). Next.js route
  handlers proxy `/api/*`, `/v1/*` and `/openapi.json` to `OSR_API_URL`, so the browser only talks to
  the web origin; server components render from the API at request time.
- API keys (`osr_live_...`, SHA-256 at rest, `Authorization: Bearer` or `X-API-Key`), plans
  (free / pro / enterprise with per-minute and per-day quotas, key and tenant caps, feature gates;
  `OSR_PLATFORM_PLAN_OVERRIDES`), usage metering per key / endpoint / target with tokens and cost,
  admin API (`X-Admin-Token`) and an optional Stripe checkout + signed-webhook billing hook that is
  inert until `OSR_PLATFORM_STRIPE_*` is configured. State is SQLite (WAL, with a rollback-journal
  fallback for SMB mounts).
- Editions: `OSR_PLATFORM_EDITION=community` runs the core `Router`; `enterprise` runs `RouterBuilder`
  with auto-learning persistence, health circuit breakers, `GuardMiddleware`, metrics and a
  hash-chained `FileAuditSink`, and unlocks tenants (per-tenant data boundary / region / deny lists /
  cost caps applied as hard constraints), `/stats` and `/audit` (filtered per account).
- `OSR_PLATFORM_PROVIDERS` maps targets to OpenAI-compatible providers (Azure OpenAI, OpenAI, vLLM,
  Ollama ...) through `adapters.openai_compat`; targets without a provider stay routable as decisions.
- `platform/api/Dockerfile` and `platform/web/Dockerfile` (non-root, health checks; the API keeps a
  `/data` volume), `infra/` Bicep (Container Apps `osr-api` + `osr-web`, Azure Files, ACR, Log
  Analytics, Azure OpenAI with gpt-4.1 family deployments), `azure.yaml` for `azd up`, and the
  `Platform` workflow (API tests, web lint/typecheck/build, two-container smoke, OIDC `azd` deploy).
  Reference deployment: `rg-osr-prod` in Sweden Central.
- Public catalogue and rankings, measured on the deployment rather than curated: `GET /api/v1/models`
  (model cards with capabilities, constraints, pricing per 1k / 1M tokens, executable flag, requests,
  tokens, cost, success rate, latency, domains and week-over-week trend), `GET /api/v1/models/{id}`
  (daily series, health, effort siblings), `GET /api/v1/rankings?days=&domain=` (share of tokens and
  requests, trend, per-domain leaders) and `GET /api/v1/stats/public` (headline figures). Usage rows
  now store `request_id`, `model`, `domain` and `complexity` (additive SQLite migration) so every
  figure is reproducible from the same table that meters billing.
- `GET /api/v1/activity`: newest-first per-account request log with `endpoint` / `target` / `key`
  filters and `before=` cursor paging.
- `POST /v1/chat/completions` extensions: `models` (candidate list; the router scores only those and
  falls back down the ranking when an upstream call fails, up to three times), `stream: true`
  (`text/event-stream` chunks, `[DONE]` terminator) and an `osr` object (`objective`, `constraints`,
  `tenant`, `plan`, `fallbacks`). The `opensmartroute` metadata gained `plan`, `fallback_from`,
  `cost_usd` and `latency_ms`; responses carry `X-Request-Id`, `X-OSR-Target`, `X-OSR-Confidence`.
- Web: `/models` explorer (search, domain, executable, kind and sort), `/models/{id}` pages with traffic
  chart, routed-domain bars, capabilities, constraints, examples and copyable call snippets for the
  live base URL, `/rankings` leaderboard (window, metric and domain filters), dashboard `/activity`
  with expandable request rows and feedback snippet, playground compare mode (up to four targets in
  parallel with cheapest / fastest badges and fallback markers; `?model=` deep link from model pages),
  and a landing page fed by the public stats (hero figures, most-routed targets, domain leaders,
  candidates + streaming sample, three-step getting started). Sitemap lists models and rankings.
- Platform single sign-on: `GET /api/v1/auth/providers`, `GET /auth/{provider}/start?next=` and
  `POST /auth/{provider}/callback` (PKCE, HMAC-signed state, GitHub / Google / Microsoft / GitLab presets
  and custom OIDC issuers via `OSR_PLATFORM_SSO_*`), `osr_sess_` browser sessions with 30-day TTL
  (`GET /auth/sessions`, `POST /auth/logout?everywhere=`, `DELETE /auth/sessions/{id}`), linked
  identities (`GET /auth/identities`, `DELETE /auth/identities/{provider}`; refuses to remove the only way
  in), `/api/v1/me` reports `via` and `identities`, `/api/v1/info` lists `sso` providers. New SQLite
  tables `identities` and `sessions` (additive migration).
- Web: "Continue with GitHub / Google / ..." on login and signup, `/auth/callback` exchange page (shows the
  first API key once when the account is created), `/dashboard/account` (profile, linked identities,
  browser sessions with revoke and sign out everywhere). Shared UI toolkit under `components/ui`
  (Radix-based button variants, dialog, dropdown menu, tooltip, select, switch, progress, avatar, table,
  sonner toasts) and a Recharts chart kit under `components/charts` (`TimeBars`, `TimeArea`,
  `RankedBars`, `Donut`, `Sparkline`) with one palette and shared tooltip/legend/empty states.
  Redesigned header (compact nav, Ctrl/Cmd+K catalogue search, account menu), light hero with a prompt
  box that opens the playground pre-filled (`/playground?q=`), rankings charts (share of requests,
  traffic by domain), dashboard usage stacked bars and quota progress, and a table view on `/models`.
- Platform users, workspaces and roles (B2C and B2B): new SQLite tables `users`, `memberships`, `invites`
  and `sso_connections`, `accounts.kind` (`personal` / `organization`) and `accounts.slug`; existing
  databases are migrated on start (every account becomes a personal workspace owned by its user).
  `POST /signup` and first SSO sign-in create the user plus a personal workspace; `GET/POST /workspaces`,
  `POST /workspaces/{id}/switch`, `PATCH/DELETE /workspace`, `GET /workspace/members`,
  `PATCH/DELETE /workspace/members/{user_id}` (owner > admin > member, last owner protected),
  `POST/DELETE /workspace/invites` (link shown once, 14-day TTL, seats = `plan.max_members`: free 3, pro 10,
  enterprise 500), `GET /auth/invites/{token}` and `POST /auth/invites/{token}/accept`. Key and tenant
  mutations need admin; API keys act as owner of their workspace. `/me` adds `user`, `role`, `workspaces`;
  usage and activity rows carry `user_id`. Organization SSO (enterprise plan feature `sso`):
  `GET/PUT/DELETE /workspace/sso` stores the company IdP (Google, Microsoft, GitLab, GitHub or any OIDC
  issuer, allowed email domains, default role), served as provider `org-<slug>` with just-in-time
  membership; `GET /auth/discover?email=` returns the sign-in options for an email domain.
- Web: workspace switcher in the dashboard sidebar, `/dashboard/members` (invite, roles, remove, pending
  invitations), `/dashboard/workspace` (create organization with its first key, rename/slug, organization
  SSO connection, leave), `/invite/[token]` acceptance page, and the sign-in page discovers company SSO
  from the typed email ("Continue with <Company SSO>").
- Custom domains: Bicep parameters `webAliasDomains`, `apiCustomDomain`, `customDomainCertificates`
  (managed certificates, TXT validation for the apex), outputs for the static IP and domain verification
  id; `scripts/domains.ps1` configures Cloudflare DNS, redirects (`.com` to `.ai`, `www` to apex) and
  SSL settings and binds the domains in two provision phases (see `platform/README.md`).

### Changed
- Web: the UI toolkit under `platform/web/src/components/ui` is now the shadcn/ui `new-york` kit
  (`components.json`, `radix-ui` umbrella package, `class-variance-authority`, `cmdk`, `vaul`, `tw-animate-css`)
  with the brand tokens mapped onto the kit's CSS variables in `globals.css`. Header (navigation menu with
  grouped product/resources panels, mobile sheet, command search), footer, dashboard shell (sidebar), docs
  sidebar (collapsible sections, mobile sheet) and version menu (dropdown) are rebuilt on it. Hand-rolled
  tables, filter chips, selects, switches, search inputs, stat cards, accordions and sliders across the
  marketing pages, models explorer/catalogue, marketplace, rankings, playground, estimator, ROI
  calculator and dashboard pages use the kit primitives (`DataTable`, `StatCard`, `FilterChip`,
  `SearchInput`, `Select` wrapper, `Switch`, `Progress`, `Accordion`, `Slider`). The per-scope
  `@radix-ui/react-*` packages, `cn` and `next-themes` dependencies are removed.
- `Router(slot_quality_floor=, narrow_above=, narrow_to=)` and
  `RouterBuilder.with_retrieval(narrow_above=, narrow_to=)` default to `None` (= settings) instead
  of literals; `Router.calibrator` defaults to `TemperatureScaler(settings.routing.softmax_temperature)`
  and plan-slot margins use it instead of a private softmax.
- `RouterBuilder.with_auto_learning()`, `with_health()` and `with_queue_awareness()` no longer
  inject hardcoded weights; the defaults come from `WeightSettings`.
- `MultiKnapsackBandit`: the sliding-window spend meter is O(1) per step (running sums over a
  bounded deque); shadow prices are updated in budget units (`avg / B - 1`) so a budget expressed in
  tokens no longer blows the penalty up; the "release the cheapest arm" fallback only triggers when
  *every* arm is blocked, not when utilities happen to be negative.
- Documentation: the README is a short overview (pitch, install, quick start, what it routes, how it
  decides, production, security, documentation index, roadmap); the detailed walkthroughs (targets,
  execution, real providers, learning, decorator SDK, enterprise builder, research-track modules,
  CLI, routing latency, repository layout) moved to `docs/GUIDE.md`. `docs/ROADMAP.md` marks every
  research-track row as implemented with its module; `docs/RESEARCH.md` and `docs/MATH.md` describe
  the corresponding techniques and formulas (sections 10 to 16).

### Fixed
- `PageHinkley` accumulated `x - mean - delta`, so a perfectly stationary quality stream alarmed
  after `threshold / delta` samples and `AutoLearner` reset healthy targets; the tolerance now has the
  textbook sign (`+ delta`) and a constant stream never drifts.
- `Bandit.mean` is abstract and `LinUCB.mean()` returns the intercept estimate instead of a stub.
- `OpenAICompatClient` derives its `User-Agent` from the package version instead of a literal.
- `enterprise.ops.ABTest.observe` no longer divides by zero when the control arm has no samples yet.
- `server.py`: request bodies were parsed as query parameters (HTTP 422) because
  `from __future__ import annotations` stringified the locally defined pydantic models; removed.
- `execute()` re-raised an `ExecutionError` from a provider handler (upstream 4xx, malformed body)
  without recording a failed outcome, so learners and health never saw those failures.
- `EnterpriseRouter.learn()` with auto-learning bypassed the router's feedback store, task-credit
  buffer and pins (only the strategies were updated); `RouterBuilder.with_feedback()` no longer
  double-records through a learner sink.
- `CacheMiddleware` keyed only on text / constraints / objective, so a pinned (`candidates=`) or
  `plan=True` call could be served another call's cached decision; per-call routing options are now
  part of the key.

## [0.4.0] - 2026-09-04

### Added

Execution and plans
- `Router.run()`, `AsyncRouter.run()` and `EnterpriseRouter.run()` route and execute the resulting
  plan through the new `opensmartroute.execution` module: the `skill` slot's handler runs as a
  pre-processor (may return a rewritten `RouteRequest`, a `str` attached as `context["skill_output"]`,
  or `None`), persona / skill / primary `instructions` are composed into `context["system"]`, the
  primary handler (LLM, skill or agent harness) is called, and one `Outcome` per participant is
  recorded and learned from. Returns `ExecutionResult(response, text, target_id, steps,
  system_prompt, effective_request, outcomes)`.
- `RouteTarget.instructions` (persona prompt, skill body or agent standing instructions, disclosed
  only when the target is part of the plan), `RouteTarget.effort` (reasoning-effort variant) and
  `RouteTarget.family` (siblings share breaker and budget state).
- `Outcome.task_id`, `Outcome.role` and `Outcome.step` so delayed, task-level rewards can be joined
  to every call in a trajectory and plan slots learn separately from the primary.
- `RouteRequest.profile` (user / tenant attributes) and `RouteRequest.history` (session turns) feed
  `ProfileSignal` and `HistorySignal`.
- `Router(slot_quality_floor=0.5)`: a persona / skill slot is only filled when its best candidate's
  quality estimate clears the floor.

Strategies and learning
- `TaskTableStrategy` (static task-type table with Wilson intervals; `fit_task_table()` from outcomes),
  `DeferStrategy` (learning-to-defer for `TargetKind.HUMAN`), `EffortStrategy` and `decide_effort()`
  (think vs. no-think), `ProgressRouter` and `MultiRoundExecutor` (progress-guided step routing and a
  Router-R1 style route / execute / judge loop), `CascadePlanner` with `BeliefTracker`
  (cascade as a finite-horizon MDP or POMDP with a stop action).
- `learning.TaskCredit` (uniform / discounted / last / blended credit assignment from a terminal
  task reward), `learning.TaskPins` (admission-time pinning), `ExampleMiner`, `RequestMemory`,
  `SimilarityFallback`, `target_embedding()`, `nearest_targets()` and `warm_start()` for cold-start
  targets.
- `math.calibration`: `TemperatureScaler`, `IsotonicCalibrator`, `ConformalCalibrator` (split
  conformal candidate sets with a coverage guarantee). `Router(calibrator=, conformal=)` exposes
  calibrated confidence and `RouteDecision.candidate_set`.
- Forgetting (`decay < 1`) in `IRTModel` and `BradleyTerry`; `BradleyTerry.merge()` for federated
  sufficient statistics; Sherman-Morrison rank-1 updates and surfaced posterior variance in `LinUCB`.

Signals
- `signals.ontology.TaskOntology` (families, types, subtypes with seed templates) and
  `signals.models` (`HashedFeaturizer`, `HashedClassifier`, `HashedRegressor`, `synthesize_dataset()`)
  behind `TaskTypeSignal`, `LearnedDifficultySignal`, `ReasoningNeedSignal`, `OutputLengthSignal`,
  `SensitivitySignal`; weights ship as JSON (`SignalModelBundle`) and load with
  `Router(extractors=learned_extractors(SignalModelBundle.from_file(path)))` or `osr --models path`.
- `osr train --out models.json [--from rows.jsonl] [--synth] [--gadget detector.json]`.

Catalogue interop and scale
- `opensmartroute.retrieval`: `BM25Index`, `DenseIndex`, `Retriever`, `rrf()` (reciprocal-rank fusion),
  `select_skill_set()` (submodular skill-set selection under a token budget), `tool_search_target()` and
  `execute_tool_target()` meta-tools. `Router(retriever=, narrow_above=500, narrow_to=50)` switches to
  retrieve-then-rank for large catalogues.
- `adapters.mcp`: `tools_from_mcp()`, `connect_mcp()` (stdio JSON-RPC), `enrich_description()`,
  `sign_manifest()` / `verify_manifest()` (HMAC-SHA256 or Ed25519), `description_risk()`;
  `osr mcp-manifest sign|verify`.
- `adapters.a2a`: `fetch_agent_card()`, `agent_from_card()`, `skills_from_card()`, `a2a_handler()`.
- `adapters.frameworks`: `langgraph_node()`, `langgraph_condition()`, `maf_router_executor()`,
  `openai_tool_spec()`, `route_and_execute()`.
- `adapters.personas`: `load_personas()` / `persona_target()` from Markdown, JSON, CSV and Copilot
  `*.agent.md` / `*.chatmode.md` files.
- Agent-harness adapters: `AgentHarness`, `HarnessResult`, `CallableHarness`, `HTTPHarness`
  (408 / 429 / 5xx map to `TargetUnavailableError`), `SubprocessHarness`, `harness_handler()`.
- SKILL.md loader (`load_skill`, `load_skills`, `skill_from_markdown`) for
  [agentskills.io](https://agentskills.io/specification) folders; works with or without PyYAML.
- OpenAI-compatible proxy in the HTTP server: `POST /v1/chat/completions` (`model: "auto"` routes,
  executes and learns) and `GET /v1/models`; `GET /healthz`.

Evaluation
- `eval.baselines`: random, cheapest, best-prior, most-expensive, static task table, oracle,
  `multi_sample_oracle()` and `noise_floor()`; `osr eval --baselines`.
- `eval.robustness`: `repeat_flip_rate()`, `paraphrase_robustness()`, `profile_swap_fairness()`,
  `coreset()`, `diversity()`; `osr eval --robustness`.
- `eval.frontier`: `frontier3()`, `hypervolume()`, `ablation_report()`; `osr eval --frontier3 --ablation`.
- `eval.ope`: IPS, SNIPS and doubly-robust off-policy estimates with weight clipping and effective
  sample size; `osr ope logs.jsonl`.
- `eval.datasets`: presets for RouterBench, LLMRouterBench, RouterEval, xRouteBench and RouterXBench;
  `osr eval --preset`.
- `calibration_report()` (ECE, Brier, reliability bins); `osr eval --calibration`.
- `osr eval --min-accuracy X` exits with status 2 below the threshold; CI uses it as the routing
  accuracy gate.

Enterprise
- `enterprise.stores`: `RedisStateStore`, `SQLStateStore` (DB-API 2.0; PostgreSQL, SQLite, MySQL),
  `VersionedStateStore` (schema envelope with forward migrations), `EncryptedStateStore`
  (AES-256-GCM, key rotation; `crypto` extra), `BatchedStateStore` (write-behind), `NamespacedStateStore`.
  `RouterBuilder.with_state_store()` persists learner state through any of them.
- `enterprise.ops`: `ShadowMiddleware` and `ABTest` (shadow or A/B routing judged by Wald's SPRT),
  `FairShareMiddleware` (dominant-resource fairness across tenants), `InflightTracker` and
  `QueueAwareStrategy` (Erlang-C / Kingman wait estimates from live in-flight counts).
  `RouterBuilder.with_shadow()`, `.with_fair_share()`, `.with_queue_awareness()`,
  `.with_calibration()`, `.with_retrieval()`, `.with_router_options()`.
- `opensmartroute.settings`: frozen, environment-overridable `Settings` (`OSR_<GROUP>_<FIELD>`) for
  every tunable constant; `opensmartroute.branding` centralises package identifiers.
- `deploy/`: non-root reference `Dockerfile`, `entrypoint.sh` and a Helm chart (Deployment, Service,
  Ingress, HPA, ConfigMap). Releases publish `ghcr.io/isathish/opensmartroute:<version>`.

Security
- `security.gadget.GadgetDetector` (learned confounder-gadget classifier; `InputGuard(learned=True)`),
  `security.injection.injection_risk()` and `inspect_injection()`, `security.limits.ResourceLimiter`
  and `ResourceLimitMiddleware` (per-task caps on steps, tool calls, depth, tokens, cost, wall-clock),
  `security.provenance.OriginPolicy` (sensitive parameters of state-changing tools must originate
  from the user turn), `security.safety.run_safety_suite()` (red-team routing cases);
  `osr safety [--learned-guard]` exits with status 2 on any failure.

Project
- Public API snapshot test (`tests/public_api.json`, `tests/test_public_api.py`): adding or removing an
  exported name fails CI until the snapshot is updated deliberately.
- Release automation: `scripts/release.py` (version consistency, changelog sections, release
  preparation), `Prepare release` and `Release` workflows (release PR, tag, GitHub release with
  changelog notes, PyPI trusted publishing with PEP 740 attestations, SLSA build provenance, GHCR
  image), CodeQL for Python and workflows, Dependabot for actions and Python dependencies.
- CI matrix covers CPython 3.10 to 3.13 on Linux plus 3.12 on Windows, checks formatting, builds and
  smoke-tests the container image.
- `examples/end_to_end.py`, `examples/skills/sql-reporting/SKILL.md`; `instructions` on the example
  personas and skills.

### Changed
- `Router.execute()` / `AsyncRouter.execute()` keep returning the raw handler response but now go
  through the plan-aware executor. `EnterpriseRouter` gains `execute()` (returns `ExecutionResult`).
- `load_targets` and `load_rules` are exported from the package root.
- Project metadata: repository URLs point at `isathish/OpenSmartRoute`; classifiers declare
  CPython 3.13 and `Typing :: Typed`; the sdist no longer ships brand assets and fonts.

### Fixed
- `config.load_document` and the eval dataset loader read files as UTF-8 explicitly; on Windows the
  locale default (cp1252) failed on the shipped `examples/targets.yaml`.

## [0.3.0] - 2026-09-04

### Added
- `opensmartroute.adapters`: stdlib `OpenAICompatClient` (timeouts, exponential backoff with
  `Retry-After`, typed error mapping, key never logged) with `judge_fn`, `embedder`, `chat_handler`
  glue; lazy `sentence_transformers_embedder` and `OpenTelemetryTelemetry`.
- `opensmartroute.config`: `load_targets` / `load_rules` / `load_document` with `ConfigurationError`
  reporting file, entry and reason. `TargetRegistry.from_file` now delegates to it.
- `scripts/bench.py` — reproducible routing-latency benchmark; `scripts/brand_build.py` — logo system
  generator with WCAG 2.2 audit and platform-standard exports.
- Brand identity: rounded-geometric wordmark (Quicksand) and Poppins text, vendored under OFL; light
  and dark palettes verified ≥ 4.5:1 text / ≥ 3:1 graphics.
- `docs/ROADMAP.md` with per-release exit criteria; `docs/RESEARCH.md` rewritten as a survey with a
  formal problem statement, taxonomy, systems comparison and idea→module map.

### Fixed
- `HealthPolicy` consumed a rate-limit token for **every** admissible candidate on every route; it now
  only checks availability, and `EnterpriseRouter` reserves one token for the chosen target.
- `/feedback` endpoint dropped `complexity`, so IRT never learned from HTTP feedback.

### Changed
- `cli._load_rules` (private, but used by examples) replaced by public `config.load_rules`.
- Removed the earlier gradient ring / wordmark SVGs in favour of the generated system.

## [0.2.0] - 2026-09-04

### Added
- `opensmartroute.math`: Thompson/UCB1/LinUCB/ε-greedy bandits, cost-aware Lagrangian wrapper,
  IRT (2PL) model, Bradley–Terry + Elo, Markov chain + routing MDP (value iteration),
  EWMA/Welford, Page–Hinkley and ADWIN-lite drift, Wilson interval, Pareto/TOPSIS,
  Erlang-C / Little's law / Kingman queueing.
- `opensmartroute.learning`: `IRTStrategy`, `PreferenceStrategy`, `LinUCBStrategy`,
  `MarkovStrategy`, `AutoLearner` (fan-out + drift detection + atomic persistence).
- `opensmartroute.realtime`: circuit breaker, token-bucket rate limit, rolling budget,
  `HealthRegistry`, `HealthPolicy`, `HealthStrategy`.
- `opensmartroute.enterprise`: `RouterBuilder`, `EnterpriseRouter`, middleware chain
  (`CacheMiddleware`, `TenantMiddleware`, `TimeoutMiddleware`), telemetry ports
  (`LoggingTelemetry`, `MetricsTelemetry`), `StateStore` ports, hash-chained `FileAuditSink`.
- `opensmartroute.security`: `InputGuard` (confounder-gadget detection), `Redactor`,
  `GuardMiddleware`, `sanitize_for_prompt`, `load_secret`.
- `opensmartroute.aio.AsyncRouter`.
- Typed exception hierarchy in `opensmartroute.errors`; `py.typed` marker.
- Pinned rules (`pin: true`) and `primary: false` plan-only targets.
- CI (ruff, mypy, bandit, pytest, eval gate), SECURITY.md, CONTRIBUTING.md.

### Changed
- Lexicon matching is now whole-word (regex `\b`) — fixes false positives like `sql` in `australia`.
- Cost/latency normalisation is log-scaled; default `Objective` weights lowered.
- `NoRouteError` now derives from `OpenSmartRouteError` and carries `details["rejections"]`.

## [0.1.0] - 2026-09-04

- Initial release: core types, registry, signals, policy, rules/capability/similarity/bandit/
  llm-judge/cascade strategies, router with plans, feedback store, eval harness, CLI, FastAPI server.
  Predates the repository history: the code is the initial commit tagged `v0.2.0`.

[Unreleased]: https://github.com/isathish/OpenSmartRoute/compare/v1.0.0...HEAD
[1.0.0]: https://github.com/isathish/OpenSmartRoute/compare/v0.5.0...v1.0.0
[0.5.0]: https://github.com/isathish/OpenSmartRoute/compare/v0.4.0...v0.5.0
[0.4.0]: https://github.com/isathish/OpenSmartRoute/compare/v0.3.0...v0.4.0
[0.3.0]: https://github.com/isathish/OpenSmartRoute/compare/v0.2.0...v0.3.0
[0.2.0]: https://github.com/isathish/OpenSmartRoute/releases/tag/v0.2.0
[0.1.0]: https://github.com/isathish/OpenSmartRoute/tree/v0.2.0

---

<!-- Page: Roadmap (https://opensmartroute.ai/docs/ROADMAP) -->

<!-- OpenSmartRoute: Roadmap. Source https://github.com/isathish/OpenSmartRoute/blob/main/docs/ROADMAP.md; HTML https://opensmartroute.ai/docs/ROADMAP -->
# Roadmap

Status values: **Shipped** (in a tagged release), **Complete** (every item on `main` and the exit
criterion measured and met; ships with the next tag), **In progress** (code on `main`, part of the
exit criterion still open), **Planned**, **Research** (tracked, not scheduled). Where a criterion is
open, the section names exactly what is missing and what would close it.

Versioning follows SemVer. Each release has an **exit criterion** that must be met before tagging.
Items are ordered by dependency, not by preference. Dates are deliberately absent; the project ships
when the criteria are met.

This document is rebuilt from two inputs: (a) an engineering audit of the `v0.3.0` code base
(what is implemented, what is heuristic, what is missing; section 0), and (b) a literature scan of LLM /
agent / tool routing up to **September 2026** (summarised as design principles in section 0.3, cited per
item below, catalogued in [RESEARCH.md](https://opensmartroute.ai/docs/RESEARCH.md)). Every planned item names the paper or
concept it operationalises so the *why* survives the *what*.

`v0.4.0` shipped first implementations across the v0.4 to v0.9 themes and `v0.5.0` the research track,
the routing SLM and autopilot, estimates and the MCP server, the CLI sign-in flow and end-to-end tracing
(see [CHANGELOG.md](https://opensmartroute.ai/docs/changelog.md)).
The research track that followed (verbalised difficulty, draft-response and hidden-state
representations, elastic budgets, personalisation, self-escalation, hand-off, aggregation, protocol
selection, schema-aware and cache-preserving discovery, skill graphs, MCP server recommendation,
semantic-router import, energy models, edge/cloud and auctions) is implemented on `main` with tests;
each row below names the module. The per-version sections keep their exit criteria; an item is only
**Shipped** when its code is released, and a version is only closed when its exit criterion has been
measured and published.

---

## 0. Where we were: engineering audit of v0.3.0

### 0.1 What exists (and how it works)

| Layer | Implemented | Mechanism |
|---|---|---|
| Signals | `LengthSignal`, `LanguageSignal`, `ModalitySignal`, `ComplexitySignal`, `DomainActionSignal`, PII / jailbreak regexes | **Lexicon + regex heuristics**, <1 ms, zero deps. Nothing is learned. |
| Policy | `Policy.check/filter` | Hard constraints before scoring: enable, allow/deny, kind, region, boundary, PII, tenant, cost/latency/token SLO, modality, language, jailbreak ≥ 0.5 → safety/human only |
| Strategies | `Rules` (pin/prefer/avoid), `Capability`, `Similarity` (hashing embedder, optional sentence-transformers), `Bandit` (Beta–Bernoulli TS per domain), `LLMJudge`, `Cascade` (executor, not a scorer) | Ensemble $\hat q=\sum_k w_k\kappa_k s_k / \sum_k w_k\kappa_k$; utility $U=w_q\hat q-w_c\tilde c-w_\ell\tilde\ell$ with log-min-max normalisation and a hard quality floor; confidence = softmax margin at fixed $\tau=0.1$ |
| Learning | `IRTStrategy` (2PL, online SGA), `PreferenceStrategy` (Bradley–Terry / Elo per domain), `LinUCBStrategy` (disjoint LinUCB on a 8+|domains| signal vector), `MarkovStrategy` (Dirichlet-smoothed chain + value-iteration MDP), `AutoLearner` (fan-out, Page–Hinkley drift, atomic JSON persistence) | Learners are silent until $n \ge n_0$ ($\kappa=\min(1,n/n_0)$) so declarative strategies dominate at cold start |
| Maths | `ThompsonBeta` (with decay), `UCB1`, `LinUCB`, `EpsilonGreedy`, `CostAwareBandit` (Lagrangian dual ascent), `IRTModel`, `BradleyTerry`, `Elo`, `MarkovChain`, `RoutingMDP`, `EWMA`, `Welford`, `PageHinkley`, `WindowDrift`, Wilson, ECE, Pareto, TOPSIS, Erlang‑C, Little, Kingman | Pure Python, deterministic, serialisable |
| Real-time | `CircuitBreaker`, `TokenBucket`, `Budget`, `HealthRegistry/Policy/Strategy` | Non-consuming checks in policy; one token reserved for the chosen target only |
| Enterprise | `RouterBuilder`, `EnterpriseRouter`, middleware (`Guard`, `Tenant`, `Cache`, `Timeout`), `StateStore` (memory, file), `Telemetry` (logging, metrics), `AuditSink` (hash-chained JSONL) | Ports are ABCs; only in-process / file adapters ship |
| Security | `InputGuard` (size, token estimate, head/tail naturalness gadget score), `sanitize_for_prompt`, `Redactor`, `GuardMiddleware`, `load_secret` | Threat model T1–T10 in [SECURITY.md](https://opensmartroute.ai/docs/SECURITY.md) |
| Adapters | `OpenAICompatClient` (stdlib, retries, typed errors), `judge_fn`, `embedder`, `chat_handler`, agent harnesses (`CallableHarness`, `HTTPHarness`, `SubprocessHarness` → `HarnessResult`), SKILL.md loader (`load_skills`, agentskills.io frontmatter → `RouteTarget`), sentence-transformers, OpenTelemetry | No connection pooling |
| Execution | `execution.execute/aexecute`, `Router.run`, `AsyncRouter.run`, `EnterpriseRouter.run` | Plan-aware: skill pre-processor → persona/skill/primary `instructions` composed into `context["system"]` → primary handler; one `Outcome` per participant with `role` and shared `task_id`; slots gated by `slot_quality_floor` |
| Eval | `evaluate`, `cost_quality_frontier`, `area_under_frontier`, `expected_calibration_error` | Offline, single-draw labels, no counterfactual estimation |
| Surface | `Router`, `AsyncRouter`, FastAPI `create_app`, `osr` CLI, `config.load_targets/load_rules`, `FeedbackStore` (+ RouteLLM preference export) | |

### 0.2 What was missing (from the audit) and where it landed

| Gap | Where it bit | Roadmap home | Status |
|---|---|---|---|
| All signals are lexicon heuristics; no learned difficulty / task-type / reasoning-need classifier | `signals/__init__.py` | v0.4 | Shipped in 0.4.0 (`signals.models`, `signals.ontology`) |
| No **static task-type table** baseline or "always-strongest" baseline in `osr eval` | `eval/__init__.py` | v0.4 | Shipped in 0.4.0 (`eval.baselines`, `TaskTableStrategy`) |
| Single-draw oracle in eval; no paraphrase-robustness or pool-diversity metrics | `eval/__init__.py` | v0.4 | Shipped in 0.4.0 (`multi_sample_oracle`, `eval.robustness`) |
| Confidence uses fixed $\tau$; ECE computed but never used to **recalibrate**; no abstention / set-valued output | `router.py::_softmax_margin` | v0.4, v0.8 | Shipped in 0.4.0 (`math.calibration`, `RouteDecision.candidate_set`) |
| Target = one model; **reasoning effort / think-mode** is not a routable dimension | `core/types.py` | v0.5 | Shipped in 0.4.0 (`RouteTarget.effort`, `EffortStrategy`) |
| No personalisation: user / tenant profile does not condition scores | `core/types.py`, strategies | v0.5 | Shipped in 0.4.0 (`RouteRequest.profile`, `ProfileSignal`) |
| `Cascade` runs outside the ensemble and its steps are not fed back to learners; no "stop" action; fixed order | `strategies/cascade.py` | v0.6 | Shipped in 0.4.0 (`CascadePlanner`, `BeliefTracker`) |
| No trajectory step index; learners treat each outcome as immediate, so **delayed, task-level reward** is not joined | `core/types.py::Outcome`, `learning/` | v0.6 | Shipped in 0.4.0 (`Outcome.step`, `TaskCredit`, `TaskPins`) |
| Plan slots emit outcomes but strategies do not condition on role | `strategies/`, `learning/` | v0.6 | Shipped in 0.4.0 (`signals.extra["role"]`, role-keyed learner memory: `strategies.base.role_of` / `memory_key`) |
| No MCP / A2A import; no retrieve-then-rank narrowing for large catalogues; `SimilarityStrategy` is flat top-k | `adapters/` | v0.7 | Shipped in 0.4.0 (`adapters.mcp`, `adapters.a2a`, `retrieval`) |
| Only in-memory / file `StateStore`; per-outcome writes; no schema version in saved state | `enterprise/__init__.py`, `learning/__init__.py` | v0.7, v0.8 | Shipped in 0.4.0 (`enterprise.stores`) |
| IRT / BT / LinUCB / Markov have **no forgetting**; only `ThompsonBeta` decays | `math/irt.py`, `math/preference.py`, `math/bandits.py`, `math/markov.py` | v0.8 | Shipped in 0.4.0 (`decay=` / `discount=` on every learner) |
| `CostAwareBandit` handles one budget; no multi-resource knapsack, no shadow-audit stream | `math/bandits.py` | v0.8 | Shipped in 0.4.0 (`MultiKnapsackBandit`) |
| Latency is a static `latency_ms`; queue state / TTFT is not modelled although Erlang-C exists | `realtime/`, `math/decision.py` | v0.8 | Shipped in 0.4.0 (`enterprise.ops.QueueAwareStrategy`) |
| No off-policy evaluation (IPS / DR), no shadow mode, no A/B harness | `eval/` | v0.8 | Shipped in 0.4.0 (`eval.ope`, `enterprise.ops.ShadowMiddleware`) |
| No learning-to-defer objective for `TargetKind.HUMAN` | strategies | v0.8 | Shipped in 0.4.0 (`DeferStrategy`) |
| `bandits._inverse` is a naive Gauss-Jordan inverse (can go singular); posterior variance not surfaced in `StrategyScore` | `math/bandits.py` | v0.8 | Shipped in 0.4.0 (Sherman-Morrison updates, variance) |
| Energy / carbon not a cost dimension | `core/types.py::cost` | v0.8 | Shipped in 0.4.0 (`unit_energy`, `unit_carbon`, `Objective(energy=, carbon=)`) |
| Gadget detector is heuristic; no origin / provenance check on tool parameters in plans; no federated learning of router weights | `security/` | v0.9 | Shipped in 0.4.0 (`GadgetDetector`, `OriginPolicy`, `merge()` on every learner + `learning.merge_learners`) |
| Untested: cascade path, audit-chain verification, judge JSON failure modes, state-corruption recovery, multi-replica consistency | `tests/` | every release | Shipped in 0.4.0 (`tests/test_resilience.py`, `test_cascade_*`; `FileAuditSink.verify`, `AutoLearner.quarantined` / `refresh`) |

### 0.3 Evidence-driven principles (literature, 2025–2026)

These findings change defaults, not just features. Each is cited in [RESEARCH.md](https://opensmartroute.ai/docs/RESEARCH.md).

1. **Beat the boring baselines first.** Under unified evaluation many routers, including commercial ones,
   fail to reliably outperform simple baselines, and a large router-to-oracle gap remains (LLMRouterBench,
   Jan 2026). A static *task-type → model* table captured 21 of 29 routable questions in one study
   ("Most of the LLM Routing Gap Is Task Type", Aug 2026). ⇒ `osr eval` must always report
   *always-strongest*, *always-cheapest* and *static task table* alongside any learned strategy, and a
   `TaskTableStrategy` becomes a first-class, cheapest-possible learner.
2. **The oracle is noisy.** 12–36 % of the reported oracle gap on open-model pools is single-draw label
   noise that no single-commit router can recover ("How Much of the Routing Gap Is Real?", Jul 2026).
   ⇒ evaluation adopts a multi-sample oracle protocol and reports a *reproducible headroom* figure.
3. **Curate, don't pile.** Larger ensembles show diminishing returns versus careful curation
   (LLMRouterBench); hierarchic social entropy saturates below ~10 behaviourally distinct actors
   ("When is Routing Meaningful?", Jul 2026). ⇒ ship a diversity / coreset report for catalogues.
4. **Robustness is a metric.** Surface-form paraphrases should route identically; kNN-style routers
   collapse under perturbation while prompted routers stay stable (same paper). ⇒ a paraphrase
   robustness score in `osr eval`.
5. **Route the task, not the call.** Agentic workflows have one delayed, task-level outcome; per-call
   routers mis-attribute feedback (TRACE-Router, Jul 2026; MTRouter, ACL 2026; ProgRouter, EMNLP 2026).
   ⇒ `Outcome` gains `task_id`; admission-time routing with pinned backend; terminal reward.
6. **Escalation is an optimal-stopping problem**, not a threshold (Bayesian self-escalation, Aug 2026;
   TACIT-Switch, Aug 2026; RLCascadeRouter, Aug 2026). ⇒ cascade becomes an MDP with a *stop* action.
7. **Control risk, not just cost.** Conformal / distribution-free calibration gives set-valued routing
   with abstention and provable mis-routing bounds (RACER, Feb 2026; RouteNLP, ACL 2026 industry;
   CR², May 2026). ⇒ conformal thresholds replace hand-tuned `escalate_below`.
8. **Budgets are knapsacks and the world drifts.** Multi-resource constraints, shadow-audit streams,
   pessimistic reward / optimistic cost, hard meters before commitment (Drift-Aware Sparse Routing,
   Sep 2026). ⇒ generalise `CostAwareBandit`; add forgetting to every learner.
9. **Latency is queue state, not a constant** (Latency-aware routing with a TTFT estimator, Jul 2026;
   HW-Router, DAC 2026). ⇒ wire `math.decision` queueing into `HealthStrategy`.
10. **Discovery at scale is retrieve-then-rank.** In-context selection collapses from Match@1 0.85 → 0.12
    as registries grow to thousands of models / agents / tools / skills; retrieve-then-rank crosses over
    at N≈500 (Enrich-Retrieve-Rank, Aug 2026); hybrid BM25 + dense with RRF cut MCP tool tokens by 99 %
    in production (SCOUT, Aug 2026). ⇒ two-stage catalogue narrowing before scoring.
11. **Skill sets are submodular knapsacks** (Best Prefix Selection, Aug 2026). ⇒ `RoutePlan` skill slots
    are selected as a set under a token budget, not top‑k.
12. **Reasoning effort is a routable dimension** (Think When Needed, SIGIR 2026; SCX Router predicts
    task type, difficulty, reasoning mode and output length before generation, Sep 2026). ⇒ targets
    carry `effort` variants; signals predict *whether to think*.
13. **Personalisation matters exactly when it changes the answer** (GMTRouter, EMNLP 2026; SkillFeed
    profile-conditioned skill routing, Aug 2026; xRouteBench personalised tasks). ⇒ profile-conditioned
    scoring with counterfactual tests.
14. **Routing data is fragmented and private.** Federated router training improves the accuracy–cost
    frontier over client-local routers ("Federate the Router", Jan 2026). ⇒ mergeable learner state.
15. **The control plane is an attack surface.** Beyond confounder gadgets: indirect prompt injection via
    tool content (ROPE, Aug 2026), unsigned MCP manifests, and recognition-without-enforcement gaps.
    ⇒ origin checks on state-changing tool parameters inside routed plans.

---

## v0.3: Real integrations (Shipped)

**Goal:** route to real providers with the same code that runs in tests.

| Item | Status | Notes |
|---|---|---|
| OpenAI-compatible HTTP client (stdlib, retries, typed errors) | Shipped | `adapters.OpenAICompatClient`; works with OpenAI, Azure, vLLM, Ollama, LiteLLM, OpenRouter, Groq |
| Judge / embedder / handler glue for the client | Shipped | `judge_fn`, `embedder`, `chat_handler` |
| sentence-transformers embedder | Shipped | `adapters.sentence_transformers_embedder` (extra: `embeddings`) |
| OpenTelemetry telemetry adapter | Shipped | `adapters.OpenTelemetryTelemetry` |
| Public config loaders with validation errors | Shipped | `config.load_targets / load_rules` |
| Rate-limit tokens consumed only for the chosen target | Shipped | bug fix in `HealthPolicy` / `EnterpriseRouter` |

**Exit criterion:** an end-to-end test routes, escalates to an LLM judge, and executes against an
in-process OpenAI-compatible server with zero mocks of the transport. Met.

---

## v0.4: Learned signals and honest evaluation (Shipped)

**Goal:** replace lexicon heuristics with trained, still-tiny models, without adding torch to the core,
and make `osr eval` hard to fool.

Ordering rationale: principle 1 (baselines) and 2 (noisy oracle) come **first** because they decide
whether anything in v0.5+ is worth building.

### 0.4a Evaluation harness

| Item | Status | Notes |
|---|---|---|
| Mandatory baselines in `osr eval`: always-strongest, always-cheapest, random, **static task-type table**, oracle | Shipped | `osr eval --baselines`; `eval.baselines` reports gap-to-oracle and the label-noise floor (LLMRouterBench 2601.07206; "Most of the routing gap is task type" 2608.23023) |
| Multi-sample oracle protocol | Shipped | `multi_sample_oracle()` and `noise_floor()` decompose the gap into *label-noise floor* vs *recoverable* (2607.03436) |
| Run-to-run variance | Shipped | `repeat_flip_rate()` in `osr eval --robustness` (5 % run-to-run flips observed in 2608.23023) |
| Paraphrase-robustness score | Shipped | `paraphrase_robustness()`: deterministic surface-form rewrites, % identical decisions (2607.09197) |
| Pool-diversity report | Shipped | `eval.robustness.diversity()` and greedy k-center `coreset()` |
| Latency-aware frontier | Shipped | `eval.frontier.frontier3()` and `hypervolume()`; `osr eval --frontier3` |
| Dataset adapters | Shipped | `eval.datasets` presets: `routerbench`, `llmrouterbench`, `routereval`, `xroutebench`, `routerxbench`; `osr eval --preset` |
| Component-level ablation report | Shipped | `ablation_report()` drops one strategy or extractor at a time; `osr eval --ablation` |

### 0.4b Learned signals (zero core deps)

| Item | Status | Notes |
|---|---|---|
| `TaskTableStrategy` | Shipped | per-(task type x target) success table with Wilson intervals; `fit_task_table()` from outcomes |
| Task-type classifier | Shipped | `TaskTypeSignal`: multinomial logistic regression on hashed n-grams over `signals.ontology.TaskOntology` (SCX Router 2609.02292); weights ship as JSON; falls back to `DOMAIN_LEXICON` |
| Difficulty classifier | Shipped | `LearnedDifficultySignal`; training on RouterBench / LLMRouterBench outcomes via `osr train --from` |
| Reasoning-need signal | Shipped | `ReasoningNeedSignal` (Think When Needed 2601.18146); consumed by `EffortStrategy` |
| Expected-output-length signal | Shipped | `OutputLengthSignal` regression head |
| Verbalised-difficulty retrieval | Shipped | `signals.VerbalisedDifficultySignal`: reads a small model's verbalised difficulty (number, `7/10`, or words) from `context["difficulty"]` or a callable, blends it into `complexity` and raises `reasoning_need`; `parse_difficulty()` (VDAR-Router 2607.18098) |
| Training CLI `osr train --from feedback.jsonl` | Shipped | produces the `SignalModelBundle` JSON; `--gadget` also trains the gadget detector |
| Cold-start synthetic supervision | Shipped | `synthesize_dataset()` generates labelled prompts per ontology node (TRouter 2604.09377; SCX); `osr train --synth` |

### 0.4c Calibration

| Item | Status | Notes |
|---|---|---|
| Temperature / isotonic fitting of `confidence` | Shipped | `TemperatureScaler`, `IsotonicCalibrator`; `osr eval --calibration` reports ECE, Brier and reliability bins |
| Judge-score calibration | Shipped | `LLMJudgeStrategy(calibrate=True, min_fit=20)` keeps an `IsotonicCalibrator` per judge instance, pairs each raw judge score with the served target's observed `quality`/`success` and re-fits on every outcome; raw scores are kept in the rationale (`raw=`) once the mapping is active; calibrator state is persisted with `state()/load()` |
| Confidence exposed per strategy | Shipped | posterior variance surfaced for LinUCB and IRT |

**Exit criterion:** on `examples/eval_dataset.jsonl` and one public suite, learned signals beat the
static task table by a margin larger than the measured label-noise floor; `osr eval` reports ECE <= 0.10;
paraphrase robustness >= 0.9; core dependency count remains **zero**.

**Measurement (published with 0.5.0, partially met).** Command:
`osr -t <targets> [--slm slm.json] eval <suite> --baselines --calibration --robustness --limit 1000`.
The public suites are the human-preference battles relabelled onto three tiers (`llm-small`,
`llm-mid`, `llm-frontier`) by `osr collect`; the SLM bundle was trained on Arena-55k + RouteLLM
battles and the suites below are held out.

| Suite | Router (learned) | Declarative | Static task table | Random | Cheapest | Paraphrase | ECE raw | ECE held-out isotonic |
|---|---|---|---|---|---|---|---|---|
| `examples/eval_dataset.jsonl` (n=30, 17 targets) | 0.833 | 0.833 | 0.233 | 0.133 | 0.067 | 0.867 | 0.245 | 0.261 (n=15) |
| MT-Bench human (n=1000) | 0.519 | 0.421 | 0.541 | 0.441 | 0.372 | 0.973 | 0.316 | 0.050 |
| PPE human (n=1000) | 0.561 | 0.418 | 0.651 | 0.445 | 0.174 | 0.898 | 0.279 | 0.036 |
| WebDev Arena (n=1000) | 0.533 | - | 0.670 | 0.391 | 0.000 | 0.872 | 0.253 | 0.060 |

- Met: learned signals beat the declarative router by +10 to +14 pp on every held-out suite and beat
  random / cheapest / best-prior everywhere; zero core dependencies; paraphrase robustness >= 0.9 on
  two of three public suites. With a calibrator fitted on half the suite, held-out ECE is 0.04-0.06
  (isotonic) and 0.05-0.13 (temperature), i.e. the <= 0.10 target is met **after** fitting.
- Not met: on the public suites the static task table (which, with no task-type labels, degenerates to
  "always the best single tier") still wins by 2-14 pp, because a pairwise preference between two
  models is a weak label for a three-tier decision; the label-noise floor is not measurable on these
  rows (one sample per prompt, `noise_floor()` needs `samples`); raw, unfitted ECE is 0.25-0.32.
- What this means: ship the router with a fitted `TemperatureScaler`/`IsotonicCalibrator` (see
  `calibration_report()["held_out"]`), and use multi-sample suites (RouterBench presets) to beat the
  task table by more than the noise floor - that measurement is tracked under v1.0 leaderboard runs.

**Re-measurement (0.5.x SLM recipe, leave-one-source-out).** The routing SLM now trains on ten public
sources, so each suite below was held out in turn and the SLM trained on the other nine (same command,
same targets, `--limit 1000`). The recipe change that matters for mixed catalogues - a corpus of model
battles no longer pushes down tools, skills and agents it never compared, and the strategy abstains on
them - takes `examples/eval_dataset.jsonl` *with* the SLM from 0.43 to 0.80 (0.83 without it; the old
recipe made any SLM harmful there). On the tier suites the numbers are unchanged within noise: PPE 0.561,
MT-Bench 0.508, WebDev Arena 0.506 (declarative 0.42 / 0.42 / -, static table 0.651 / 0.541 / 0.670);
held-out isotonic ECE 0.03 / 0.07 / 0.07. The conclusion stands: learned signals beat the declarative
router by +9 to +14 pp, and a pairwise battle remains too weak a label to beat "always the best tier".

---

## v0.5: Target representations, effort and personalisation (Complete)

**Goal:** learn what a target is good at from its own outcomes, so `examples` and `quality_prior` are
derived, not hand-written, and widen "target" to cover *how hard the model should think* and *for whom*.

### 0.5a Representations

| Item | Status | Notes |
|---|---|---|
| Target embeddings from outcome logs | Shipped | `learning.target_embedding()`: centroid of description, capability lexicon and examples; `nearest_targets()` routes to **unseen** targets via neighbours (UniRoute / EmbedLLM) |
| Offline reward-matrix warm start | Shipped | `learning.warm_start_from_matrix(rows, strategies, weight=)`: replays a full-information `(prompt, {target: reward})` matrix into every learner before going online (LinUCB ridge per arm, IRT ability, Thompson posteriors, task table, Markov rewards); `weight < 1` shrinks the offline evidence toward the prior (OrcaRouter 2605.30736) |
| Automatic `examples` mining | Shipped | `learning.ExampleMiner` promotes high-quality outcomes into `target.examples`, de-duplicated by cosine |
| Cold-start via IRT warm-up | Shipped | `learning.warm_start()` copies shrunk IRT ability, Thompson posteriors and Bradley-Terry strengths from neighbours (IRT-Router) |
| Target similarity to fallback ordering | Shipped | `learning.SimilarityFallback` |
| Query-response mixed representation | Shipped | `signals.DraftResponseSignal`: hedges, self-corrections, query overlap and length ratio of a cheap draft (`context["draft"]` or a drafter callable) become `extra["draft_uncertainty"]` and lift `complexity` (JiSi 2601.01330) |
| Hidden-state Dirichlet router | Shipped | `math.DirichletProbe` (evidential Dirichlet head, digamma loss, KL regulariser, pure Python SGD) and `strategies.HiddenStateStrategy(state_fn, targets, dim)`; epistemic uncertainty `K/S` becomes the strategy confidence so an unfamiliar state defers to the other strategies (ProbeDirichlet, RouterXBench 2602.11877) |

### 0.5b Effort as a routable dimension

| Item | Status | Notes |
|---|---|---|
| `Target.effort` variants | Shipped | `RouteTarget.effort` in `{none, low, medium, high}` with `RouteTarget.family` grouping siblings for breaker / budget purposes |
| Think-vs-non-think decision | Shipped | `EffortStrategy` / `decide_effort()` match `ReasoningNeedSignal` to effort level (Think When Needed 2601.18146; When to Think Deeply 2606.06745) |
| Token-budget-aware routing to elastic models | Shipped | `strategies.expand_elastic(parent, [BudgetVariant(...)])` turns one elastic model into `<id>@<budget>` siblings sharing `family` and handler; `TokenBudgetStrategy` scores the fit between `token_budget` and the tokens the answer needs (truncation risk vs unused capacity) (Nemotron Elastic 2511.16664, Star 2605.07182) |

### 0.5c Personalisation

| Item | Status | Notes |
|---|---|---|
| `RouteRequest.profile` | Shipped | opaque user / tenant profile features; `ProfileSignal` hashes them into the LinUCB context |
| Few-shot user adaptation | Shipped | `learning.UserAdaptiveStrategy`: per-user x target Beta posteriors shrunk toward neighbour users (cosine over hashed profile vectors) and the global posterior, `w = n / (n + kappa)`; `observe_user()` warm-starts from logged interactions; `state()/load()` (GMTRouter 2511.08590) |
| Counterfactual personalisation test | Shipped | `eval.robustness.profile_swap_fairness()` measures how often the decision changes with the profile (SkillFeed methodology 2608.28241) |

**Exit criterion:** adding a new target with *only* `id`, `kind` and `cost` reaches >= 90 % of its
hand-configured routing accuracy after 200 outcomes; effort routing reduces tokens >= 30 % at equal
quality on a reasoning benchmark. **First half met:** `eval.criteria.cold_start_ratio()` strips the
`legal` expert down to `id`/`kind`/`cost` next to seven configured experts and a generalist, streams
mixed traffic through the real `Router` with `explore_rate=0.1`, and measures 1.000 x the
hand-configured accuracy after 200 outcomes for the newcomer (0.000 before learning; 1 643 requests,
180 of them exploratory). Making this pass required cold-start exploration in the router
(`Router(explore_rate=, explore_min_samples=)`), because a target with no description or examples is
never ranked first and so never earns an outcome. Reproduce with `python scripts/exit_criteria.py`.

**Effort-token half met** in `eval.criteria.effort_token_savings()`: one reasoning model exposed through
`expand_elastic()` as `reasoner@fast` (`effort="low"`) and `reasoner@think` (`effort="high"`), 2 000
rows carrying per-sibling `scores` *and* `tokens` (60 % plain lookups / rewrites where both siblings
score 0.90, 40 % proof / analysis prompts where the fast sibling drops to 0.50 - the pattern Think When
Needed and When to Think Deeply report for live thinking / non-thinking pairs). The real `Router` with
`default_strategies() + EffortStrategy() + TokenBudgetStrategy()` sends 59 % of the rows to the fast
sibling and spends **61.6 %** of the always-think tokens (1 234 vs 2 003 per row; oracle 1 231) at
quality 0.900 vs 0.900. `EvalRow.tokens` is the new dataset field and `osr eval DATASET --effort` runs the
same comparison on rows collected from live thinking / non-thinking pairs; the synthetic figure is the
roadmap claim until a live pair is published from the hosted platform.

---

## v0.6: Multi-step, multi-turn and agentic routing (Complete)

**Goal:** treat routing as a sequence, not a single pick; align the unit of learning with the unit of
supervision (the task).

### 0.6a Data model

| Item | Status | Notes |
|---|---|---|
| `Outcome.task_id`, `Outcome.step`, `Outcome.role` | Shipped | populated by `Router.run()`; `learning.TaskCredit` joins a terminal task-level reward to every call in the trajectory (TRACE-Router 2607.22465) |
| Delayed-feedback bandit updates | Shipped | `TaskCredit` buffers outcomes per task and credits on completion with uniform / discounted / last / blended schemes (Joulani et al. 2013) |
| Session history in `RouteRequest.history` | Shipped | `HistorySignal` turn-level features; `learning.HistoryTargetStrategy` scores each target by a logistic model over the history-target joint embedding `h * e_t` (recency-weighted hashed history, catalogue target embedding, shared weights + per-target bias, online SGD), with an incumbent bonus that flips to a penalty when `context["last_failed"]` is set; `state()/load()/merge()` (MTRouter 2604.23530) |

### 0.6b Strategies

| Item | Status | Notes |
|---|---|---|
| Admission-time task routing | Shipped | `learning.TaskPins` pins a task to its first target and releases on failure or after `max_steps` (TRACE-Router; MTRouter) |
| Cascade as MDP with a *stop* action | Shipped | `CascadePlanner(mode="mdp")`: finite-horizon expected value over candidate orderings (RLCascadeRouter 2608.15817; Dekoninck 2024). `CascadeResult.outcomes()` / `Router.learn_cascade()` turn every executed step into a per-step `Outcome` (rejected = failure with the gate score, accepted = success) that every learner consumes |
| POMDP self-verification cascade (AutoMix) | Shipped | `CascadePlanner(mode="pomdp")` with `BeliefTracker` |
| Bayesian self-escalation | Shipped | `strategies.SelfEscalation` keeps a Beta competence posterior updated per streamed chunk (hedges, refusals, self-corrections, repetition, overrun) and stops when `loss(stop) < loss(continue)`; `wrap_stream(chunks, monitor)` cuts any text iterator at the escalation point; `SelfEscalation.for_target()` seeds the prior from `quality_prior` (2608.24087) |
| Permanent-handoff policy from censored teacher signals | Shipped | `learning.MixtureCureModel` (Weibull mixture-cure on cumulative risk, grid MLE with censoring) and `learning.HandoffPolicy(threshold, hazard_threshold, horizon_steps, fallback_target, pins)`: hands a task over once and releases its `TaskPins` pin (TACIT-Switch 2608.27911) |
| Progress-guided step routing | Shipped | `ProgressRouter`: step index, failures, budget spent and last target condition each step's route (ProgRouter 2608.25992; 2511.06190) |
| Routing / aggregation switch | Shipped | `strategies.MixtureOfAgents(k, confidence_below, budget_usd, aggregator)`: calls the top-k alternatives when the decision is unsure and the budget allows, aggregates by semantic majority vote (`majority_vote`) or a supplied synthesiser, and emits one `Outcome` per participant (JiSi 2601.01330; Mixture-of-Agents) |
| Multi-round executor (Router-R1) | Shipped | `MultiRoundExecutor`: route, execute, judge, refine loop with bounded rounds |
| Plan execution | Shipped | `Router.run()` / `execution.py`: skill pre-processes, persona and skill `instructions` set the system prompt, primary answers; per-participant outcomes; `slot_quality_floor` |
| Conversation-state MDP from real sessions | Shipped | `MarkovStrategy(state_from="auto")` keys transitions and rewards on the learned task type (ontology leaf from the task-type classifier) joined with the domain, `"task_type"` / `"domain"` force one axis; per-request state memory so the outcome credits the state that was scored; `decay=` forgets stale sessions |
| Collaboration-protocol selection | Shipped | `strategies.ProtocolPolicy`: `failure_risk()` from decision confidence, signals and response uncertainty; an ordered `ProtocolRule` table (risk, budget, task type) picks `single / cascade / aggregate / debate / handoff`; per-protocol ledger reports which protocol actually paid (2608.14927) |

**Exit criterion:** on tau-bench style agentic tasks, task-level routing beats the best single target on
the accuracy-latency frontier; on multi-hop QA the executor beats the best single target at <= 60 % of
its cost, reproducibly via `osr eval --multi-round`; cascade outcomes are consumed by every learner.
**Multi-round half met** on the synthetic scored dataset in `eval.criteria.multi_round_vs_best_single()`
(1 000 rows, small / medium / large targets with per-row scores): `MultiRoundExecutor` with
`failures_before_switch=1` reaches quality 0.911 (best single target `large`: 0.900, oracle 0.912) at
**49.8 %** of the best single target's cost, 1.8 rounds on average. `osr eval DATASET --multi-round`
runs the same comparison on any dataset whose rows carry `scores`. Cascade outcomes are consumed by
every learner (`Router.learn_cascade()`, tested).

**tau-bench half met** in `eval.agentic.task_routing_frontier()`: 1 000 synthetic tasks of 3-5 steps
(customer support, coding, travel, finance; 30 % reasoning-heavy steps) with a per-step success /
latency table for three agents - `fast-agent` (400 ms, 0.97 on plain steps, 0.45 on hard ones),
`balanced-agent` (900 ms, 0.96 / 0.72) and `strong-agent` (2 000 ms, 0.94 / 0.90) - so no single agent
dominates per step, the shape tau-bench reports across model tiers. Each agent is a `CallableHarness`
behind `harness_handler()`; the real `ProgressRouter` (escalate on failure, switch after two) routes every
step with `EffortStrategy` and the auto-learning quartet, `TaskCredit` credits the task result back, and
both policies get one retry per step on common random numbers. Routed accuracy **0.987** vs 0.977 for the
best single agent (`strong-agent`) at **45.4 %** of its latency (3.9 s vs 8.5 s per task; 67 % of steps
go to `fast-agent`, 33 % to `strong-agent`). `AgentTask` / `load_agentic_tasks()` define the JSONL
format and `osr eval TASKS.jsonl --agentic` runs the same comparison on a table collected from live
harnesses; the synthetic frontier is the roadmap claim until a live one is published.

---

## v0.7: Catalogue interop and discovery at scale (Complete)

**Goal:** stop hand-writing `targets.yaml` for things that already describe themselves, and keep routing
accurate when the catalogue has thousands of entries.

### 0.7a Import

| Item | Status | Notes |
|---|---|---|
| MCP tool catalogue import to `TargetKind.TOOL` | Shipped | `adapters.mcp.tools_from_mcp()` (offline `tools/list`), `connect_mcp()` (stdio JSON-RPC), `enrich_description()` at import time (Enrich-Retrieve-Rank 2608.22695) |
| MCP *server* recommendation | Shipped | `adapters.recommend_servers(task, servers, k, constraints, allowed_auth)`: BM25 + hashing-cosine fusion over `ServerCard` text (name, description, tool names, tags) with hard filters on latency, region, data boundary and auth kind; rationale names the matched tools (Task2MCP / T2MRec 2604.17234; ComplexMCP 2605.10787) |
| A2A agent cards to `TargetKind.AGENT` | Shipped | `adapters.a2a.fetch_agent_card()`, `agent_from_card()`, `skills_from_card()`, `a2a_handler()` |
| Agent-harness adapters to `TargetKind.AGENT` | Shipped | `CallableHarness`, `HTTPHarness`, `SubprocessHarness` with a common `HarnessResult` and `task_id`-carrying outcomes (2607.11399; SWE-Router 2607.00053) |
| Skill packages to `TargetKind.SKILL` | Shipped | `load_skills()` reads agentskills.io `SKILL.md` folders |
| Persona catalogue import | Shipped | `adapters.personas.load_personas()` from Markdown, JSON, CSV and Copilot agent / chatmode files; persona choice learned from plan-level outcomes (MasRouter) |
| OpenAI-compatible **proxy mode** | Shipped | `POST /v1/chat/completions` with `model: "auto"` routes, executes and learns; `GET /v1/models` |
| LangGraph / Microsoft Agent Framework node | Shipped | `adapters.frameworks.langgraph_node()`, `langgraph_condition()`, `maf_router_executor()` |
| vLLM Semantic Router / gateway parity | Shipped | `adapters.load_semantic_router_config(path_or_dict)` imports a semantic-router `config.yaml` (categories, `model_scores`, `use_reasoning`, `model_config.pricing`, `system_prompt`) as `RouteTarget`s plus category `Rule`s, so an existing gateway config routes through OpenSmartRoute unchanged (2603.04444) |

### 0.7b Two-stage selection

| Item | Status | Notes |
|---|---|---|
| Retrieve-then-rank narrowing | Shipped | `retrieval.Retriever`: BM25 + hashing-dense retrieval with reciprocal-rank fusion, on above `narrow_above` (default 500) targets (Enrich-Retrieve-Rank; SCOUT 2608.23992) |
| `tool_search` / `execute_tool` meta-tools | Shipped | `retrieval.tool_search_target()` and `execute_tool_target()` expose the router as two MCP tools (SCOUT) |
| Field-aware / schema-aware tool matching | Shipped | `discovery.schema_match(request, target)` extracts typed entities (emails, URLs, dates, times, paths, ids, numbers, currency, quoted strings) and scores coverage of the tool's required `input_schema` properties; `discovery.SchemaAwareStrategy` fuses it with the other strategies (SchemaRouter 2608.21375) |
| Cache-preserving tool routing | Shipped | `discovery.CachePreservingSelector(prefix_size, evict_after)` keeps a per-session ordered tool prefix byte-stable across turns, appends new tools and evicts only after inactivity; `prefix_hit_ratio()` reports cache stability (CacheRouter 2608.22708) |
| Submodular skill-set selection | Shipped | `retrieval.select_skill_set()`: greedy benefit minus redundancy under a token budget (Best Prefix Selection 2608.19993) |
| Profile-conditioned skill routing | Shipped | `learning.SkillAffinity`: Beta posterior per (profile bucket, skill); `relevance(profile, skills)` plugs into `select_skill_set(relevance=)` (SkillFeed 2608.28241) |
| Skill graphs / composition | Shipped | `discovery.SkillGraph`: `requires` / `conflicts` / `composes` edges from `SKILL.md` frontmatter (`osr-requires`, `osr-conflicts`, `osr-composes`) plus learned co-usage; `compose(seeds)` closes under dependencies, resolves conflicts by priority and returns a dependency order (2606.18051; CaSKG 2608.25500) |

### 0.7c Shared state

| Item | Status | Notes |
|---|---|---|
| Redis / Postgres `StateStore` | Shipped | `enterprise.stores.RedisStateStore`, `SQLStateStore` (DB-API 2.0), `BatchedStateStore` for write coalescing, `NamespacedStateStore` |
| State schema version and migration | Shipped | `VersionedStateStore` stamps `{"_schema": n}` and runs forward migrations; refuses to load newer state |

**Exit criterion:** a fresh install routes to an MCP server's tools with no YAML written; with 5 000
synthetic tools Match@1 stays within 5 points of the 50-tool figure. **Met.**
`eval.criteria.match_at_1_at_scale()` builds 50 and 5 000 verb-object-qualifier tools
(`synthetic_tool_catalogue()`), routes 200 paraphrased prompts through `RouterBuilder.with_retrieval()`
and measures Match@1 1.000 -> 1.000 (drop 0.000).

---

## v0.8: Operations, risk control and economics (Complete)

**Goal:** make the learners safe to leave running: bounded risk, bounded budgets, honest under drift,
measurable without live traffic.

### 0.8a Risk control and abstention

| Item | Status | Notes |
|---|---|---|
| Conformal escalation thresholds | Shipped | `math.calibration.ConformalCalibrator` (split conformal) behind `RouterBuilder.with_calibration()` (RouteNLP 2604.23577) |
| Set-valued routing with abstention | Shipped | `RouteDecision.candidate_set` = smallest nested set with mis-routing risk <= alpha (RACER 2603.06616; CR2 2605.12001) |
| Learning-to-defer for `TargetKind.HUMAN` | Shipped | `DeferStrategy` (Madras 2018; Mozannar and Sontag 2020; Verma 2023) |
| Uncertainty features from the response | Shipped | `signals.semantic_entropy()` / `response_uncertainty()` (meaning clusters over sampled answers, hedging, optional P(True) judge); `signals.UncertaintyGate` is a drop-in `Cascade` quality gate and `MixtureOfAgents` trigger; `signals.EventTrigger` fires named actions from threshold rules (Kuhn 2023, Kadavath 2022; 2607.13048) |

### 0.8b Non-stationarity and budgets

| Item | Status | Notes |
|---|---|---|
| Forgetting in every learner | Shipped | `decay < 1` for `ThompsonBeta`, `IRTModel`, `BradleyTerry`, `MarkovChain` (row counts) and `RoutingMDP` (exponentially weighted rewards); `discount < 1` for `LinUCB` with an undiscounted ridge floor (D-UCB, Garivier and Moulines 2011); `MarkovStrategy(decay=)` wires both Markov knobs |
| Multi-knapsack `CostAwareBandit` | Shipped | `math.bandits.MultiKnapsackBandit`: several resources (USD, tokens, GPU-seconds, energy), one shadow price per resource by dual ascent, hard sliding-window meters checked **before** commitment, learned optimistic costs, pessimistic rewards, bounded `audit` stream of every pruned arm (Drift-Aware Sparse Routing 2609.00662; Badanidiyuru 2013) |
| Tenant fairness under shared budgets | Shipped | `enterprise.ops.FairShareMiddleware`: dominant-resource fairness over a sliding window |
| Numerically stable LinUCB | Shipped | Sherman-Morrison rank-1 updates; surfaced posterior variance |

### 0.8c Latency, hardware and energy

| Item | Status | Notes |
|---|---|---|
| Queue-aware latency estimate | Shipped | `enterprise.ops.InflightTracker` + `QueueAwareStrategy` (Erlang-C / Kingman) (Latency-aware routing 2607.18253) |
| Energy / carbon as cost dimensions | Shipped | `RouteTarget.cost["wh_per_1k_tokens"]` / `["gco2_per_1k_tokens"]` (or `gco2_per_wh` grid intensity) read by `unit_energy` / `unit_carbon`; `Objective(energy=, carbon=)` weights them in the utility, log-normalised like money; `MultiKnapsackBandit` meters them as resources |
| Hardware-aware cost | Shipped | `math.EnergyModel` fits `Wh = e0 + e_in * prompt + e_out * output` per target by ridge regression from measured samples and converts to gCO2 with a grid factor; `math.HardwareProfile` / `hardware_profile()` give priors for A100, H100, L4, RTX 4090, CPU and NPU when no meter exists; feeds `wh_per_1k_tokens` / `gco2_per_1k_tokens` (2608.28044; HW-Router 2608.14575) |
| Edge-cloud token-aware routing | Shipped | `strategies.EdgeCloudStrategy`: targets tagged `metadata["tier"] = edge | cloud`; edge success is a Thompson posterior per complexity bucket, penalised by decode time against the latency SLO; cloud is penalised by upload time and excluded when the request's data boundary forbids it (Pro-Router 2608.28726, RelayLLM 2601.05167) |

### 0.8d Offline and online evaluation

| Item | Status | Notes |
|---|---|---|
| Off-policy evaluation | Shipped | `eval.ope`: IPS / SNIPS / doubly-robust with weight clipping and effective sample size; `osr ope` (Dudik et al. 2011); `RouteDecision.propensities` logs the softmax routing policy so `LoggedDecision.from_decision` needs no extra plumbing |
| Shadow mode and A/B harness | Shipped | `enterprise.ops.ShadowMiddleware` and `ABTest` with Wald's SPRT for promotion |
| Federated learner merge | Shipped | `merge()` on `ThompsonBeta`, `LinUCB`, `IRTModel`, `BradleyTerry`, `MarkovChain` / `RoutingMDP`, `BanditStrategy` and `TaskTableStrategy` combine sufficient statistics without sharing prompts; `learning.merge_learners(local, remote)` pairs same-named strategies across replicas ("Federate the Router" 2601.22318). `AutoLearner.refresh()` is the writer / reader pattern for replicas that share one `StateStore` |
| Market / auction strategy | Shipped | `strategies.AuctionStrategy`: targets bid `(claimed success, price)`; claims are corrected by each bidder's observed bias and realised rate, the highest corrected surplus wins and pays the second price (EA-RAM 2608.12719) |

**Exit criterion:** conformal thresholds hit their nominal mis-routing rate +/- 2 pp on held-out data;
the knapsack bandit never exceeds a hard budget in a 10^6-step simulation with drift; IPS estimates of a
held-out strategy are within the bootstrap CI of its live result. **Met** (`python scripts/exit_criteria.py --full`):

- `conformal_coverage()`: `ConformalCalibrator(alpha=0.1)` fitted on 1 000 router propensities over
  graded mixed-domain prompts covers 0.909 of 1 000 held-out labels (gap 0.9 pp, mean set size 1.2).
- `knapsack_never_exceeds_cap()`: `MultiKnapsackBandit(on_capped="abstain")` over 10^6 steps with reward
  means reshuffled every 10^5 steps; peak sliding-window spend / cap = 0.9998 (USD) and 0.9996 (tokens),
  zero abstentions, mean reward 0.75. Getting there fixed two real defects: the "release the cheapest
  arm" fallback could breach the cap (peak 1.0165), and shadow prices exploded for token-scale budgets.
- `ope_within_live_ci()`: 2 000 decisions logged under a cost-heavy softmax policy; IPS 0.812, SNIPS
  0.816 and DR 0.816 all fall inside the live value's 95 % bootstrap CI [0.798, 0.832] (live 0.815,
  n_eff 992).

---

## v0.9: Security hardening of the control plane (Complete)

**Goal:** close the gaps listed in [SECURITY.md](https://opensmartroute.ai/docs/SECURITY.md) that heuristics cannot.

| Item | Status | Notes |
|---|---|---|
| Learned gadget / confounder detector | Shipped | `security.gadget.GadgetDetector`; `InputGuard(learned=True)`; trained via `osr train --gadget` (Rerouting LLM Routers) |
| Origin policy for tool parameters | Shipped | `security.provenance.OriginPolicy`: sensitive parameters of state-changing tools must originate from the user turn (ROPE 2608.27496) |
| Signed MCP manifests | Shipped | `adapters.mcp.sign_manifest()` / `verify_manifest()` (HMAC-SHA256 or Ed25519); `osr mcp-manifest` (2601.23132) |
| Resource-amplification limits | Shipped | `security.limits.ResourceLimiter`: per-task caps on steps, tool calls, depth, tokens, cost and wall-clock (Beyond Max Tokens 2601.10955) |
| Safety-routing regression suite | Shipped | `security.safety.run_safety_suite()`; `osr safety --learned-guard` gates CI (When Safety Routing Breaks 2609.01455) |
| Encrypted state at rest | Shipped | `enterprise.stores.EncryptedStateStore` (AES-256-GCM, key rotation) |
| External review pack | Complete | [SECURITY_REVIEW.md](https://opensmartroute.ai/docs/SECURITY_REVIEW.md): scope, trust boundaries, evidence table with reproduction commands, reviewer questions, known gaps, review log |

**Exit criterion:** red-team suite passes in CI (met); published threat-model delta (met:
[SECURITY.md, "Threat-model delta: 0.3 -> 0.4"](https://opensmartroute.ai/docs/SECURITY.md#threat-model-delta-03---04)); a published
review pack that lets an independent party run the review without a briefing (met:
[SECURITY_REVIEW.md](https://opensmartroute.ai/docs/SECURITY_REVIEW.md)).

The original wording also required the independent report itself. That cannot be produced from inside
the repository, so it is now the adoption item it really is: tracked in the v1.0 readiness table below
and closed when a third-party report is added to the review log in `SECURITY_REVIEW.md`.

---

## v1.0: Stable API (Complete)

**Goal:** freeze the public surface.

- All `opensmartroute.*` public symbols frozen; the snapshot test (`tests/public_api.json`) already
  fails on any added or removed name. Deprecation policy: **shipped** in
  [CONTRIBUTING.md, "Public API and deprecation policy"](https://opensmartroute.ai/docs/contributing.md#public-api-and-deprecation-policy)
  with `errors.deprecated()` / `OpenSmartRouteDeprecationWarning` as the mechanism.
- Documented performance envelope: **shipped** in [ARCHITECTURE.md](https://opensmartroute.ai/docs/ARCHITECTURE.md#performance-envelope)
  (p50 / p95 / p99 for 16 / 64 / 256 / 1024 targets, with and without two-stage narrowing;
  `python scripts/bench.py --scale` reproduces it).
- Security review of the control plane against `docs/SECURITY.md` threat model: the review pack is
  **shipped** ([SECURITY_REVIEW.md](https://opensmartroute.ai/docs/SECURITY_REVIEW.md)); the independent report is an adoption item
  (table below).
- Reference deployments: **shipped** as library (`pip install`), sidecar (`deploy/Dockerfile`, `osr serve`)
  and central control plane (`deploy/helm/opensmartroute`; hardened pod security, HPA, PDB, NetworkPolicy,
  ConfigMap-driven catalogue) - see [deploy/README.md](https://opensmartroute.ai/docs/deploy.md); **hosted platform** on
  Azure Container Apps with Azure OpenAI (`platform/`, `infra/`, `azd up`; community and enterprise
  editions, API keys, plans, metering, tenants, audit) - see [platform/README.md](https://github.com/isathish/OpenSmartRoute/blob/main/platform/README.md).
- Public leaderboard runs with reproducible configs: **shipped** -
  [examples/leaderboard/results](https://opensmartroute.ai/docs/leaderboard-results.md) holds the 0.5.0 run
  (learned router vs declarative, static task table and baselines on MT-Bench human, PPE human and
  WebDev Arena, with robustness and held-out calibration; suite-level hold-out, seed 0, SLM SHA-256
  recorded) produced by the recipe in [examples/leaderboard](https://opensmartroute.ai/docs/leaderboard.md). Listing
  on RouterArena, LLMRouterBench and xRouteBench is a submission to those projects and is tracked below.

**Exit criterion (settled with 1.0.0).** 1.0 means what the repository can prove: a frozen public API with a
deprecation path, a published performance envelope, reference deployments, the security controls with a
red-team suite in CI and a review pack, a published leaderboard run, and two consecutive releases without a
breaking change. Adoption evidence - an independent security report, accepted leaderboard listings, two
production users - needs a third party, so it is tracked and reported but does not gate the version.
Nothing is typed by hand: `python scripts/release.py readiness` computes the table below from the
repository (snapshot and tests present, envelope section, deployment files, review pack and CI safety
suite, published run, non-breaking release steps in `CHANGELOG.md` verified against the tagged
`tests/public_api.json`, review-log rows in `SECURITY_REVIEW.md`, `accepted` rows under *Listings* in
the results page, rows in [ADOPTERS.md](https://github.com/isathish/OpenSmartRoute/blob/main/ADOPTERS.md)); `release.py check` refuses any `1.x` version
while a **release** row is open, and the *Release* workflow prints the whole table in its summary.

| Readiness item | Kind | How it is verified | Status |
|---|---|---|---|
| Public API frozen with a deprecation path | release | `tests/test_public_api.py` fails on any removed or renamed name; `errors.deprecated()` | Met |
| Performance envelope published | release | [ARCHITECTURE.md](https://opensmartroute.ai/docs/ARCHITECTURE.md#performance-envelope), `scripts/bench.py --scale` | Met |
| Reference deployments (library, sidecar, control plane, hosted) | release | `deploy/`, `platform/`, live at opensmartroute.ai | Met |
| Security controls, red-team suite in CI, threat-model delta, review pack | release | v0.9 above | Met |
| Leaderboard run published with reproducible config | release | [examples/leaderboard/results](https://opensmartroute.ai/docs/leaderboard-results.md) | Met |
| Two consecutive releases with no breaking change | release | 0.4.0 -> 0.5.0 and 0.5.0 -> 1.0.0, additions only against the tagged `tests/public_api.json` | Met |
| Independent security report | adoption | A row in [SECURITY_REVIEW.md, "Review log"](https://opensmartroute.ai/docs/SECURITY_REVIEW.md#7-review-log) | Open |
| Leaderboard listings accepted | adoption | An `accepted` row under [results, "Listings"](https://opensmartroute.ai/docs/leaderboard-results.md#listings) linking the published run | Open |
| Two independent production users | adoption | Two rows in [ADOPTERS.md](https://github.com/isathish/OpenSmartRoute/blob/main/ADOPTERS.md) (named, or anonymised with consent) | 0 of 2 |

What closes each adoption row:

- **Security report** - a third party runs [SECURITY_REVIEW.md](https://opensmartroute.ai/docs/SECURITY_REVIEW.md) and adds a row to
  its review log with the report link; findings are fixed or accepted first.
- **Listings** - submit the 0.5.0 run (commit, SLM SHA-256, results page) to RouterArena,
  LLMRouterBench and xRouteBench; record the entry URL and `accepted` in the results page.
- **Production users** - two organisations agree to be listed in `ADOPTERS.md`.

After 1.0 the public API changes only through the deprecation policy: a removal needs a
`deprecated()` warning for at least one minor and a `### Removed` entry, which the readiness check
reads as a breaking step and the snapshot test refuses without an explicit snapshot update.

---

## Research track

Every question below now has a runnable answer in the tree; the table records where, so the
question can be re-asked against a real catalogue and dataset rather than argued about.

| Question | Answer in the code | Related work |
|---|---|---|
| Is the remaining router-to-oracle gap real or label noise on *our* pools? | `eval.headroom.routing_headroom(rows, targets)`: oracle vs best single target, split into the label-noise floor (`noise_floor`) and the measurable, recoverable headroom | 2607.03436; LLMRouterBench 2601.07206 |
| Does a static task table already capture most of the value for typical enterprise catalogues? | `eval.headroom.learnability_by_difficulty` buckets rows by oracle-minus-best-single gap; `TaskTableStrategy` is a mandatory `osr eval --baselines` row, so the table's share of the headroom is reported per run | 2608.23023 |
| How few, and how different, must targets be before routing beats the best single model? | `eval.headroom.target_diversity` (mean pairwise disagreement, winner entropy / share) and `min_catalogue(rows, targets, fraction)` (smallest subset keeping 95 % of the oracle) | 2607.09197 |
| Contrastive router training on (query, target) pairs | `learning.ContrastiveRouter` (softmax cross-entropy over `q·e_t / τ` against the *acceptable set* of a row, `acceptable_set(row, slack)`) and `ContrastiveStrategy` with online updates from outcomes | RouterDC (NeurIPS 2024); RouterDC-style objectives in LLMRouter |
| Reward-model-distilled routing labels | `learning.soft_labels(scores, temperature)` and `ContrastiveRouter.fit(objective="distilled")`: train against the softened reward distribution instead of one-hot labels | Zooter |
| Can the whole ensemble be compressed into one small routing model? | `learning.RouterSLM` (hashed dual encoder + temperature calibration + target snapshots in one JSON file) and `distill_router(router, texts)`, which trains it on the router's own propensities; `SLMStrategy` puts the student back in the ensemble and keeps learning online; `osr slm train/eval/predict` | RouteLLM matrix-factorisation router; Zooter; RouterDC |
| Does a router keep improving on its own from live data, without regressing? | `learning.SelfImprover`: champion/challenger cycle over feedback rows (`eval.rows_from_feedback`), Hub datasets (`eval.DatasetCollector`, RouterBench / RouterEval) and synthetic rows; challenger is calibrated on the holdout and promoted only above `OSR_SLM_MIN_GAIN`; JSONL history; `osr improve` | RouterBench 2403.12031; RouterEval 2503.10657 |
| Where do current prices and quality priors come from at scale? | `adapters.ModelCatalogue` (OpenRouter prices, Hugging Face `model-index` benchmarks and popularity → `quality_prior`, Pareto `frontier()`, `cost_benchmark()`) and `adapters.WebKnowledge` (Hugging Face / DuckDuckGo / Brave discovery, risk-scored page text); `osr catalogue` | LLMRouterBench 2601.07206 cost-quality frontier |
| Model-level scaling: does accuracy keep rising with more targets? | `eval.headroom.scaling_curve(rows, targets, sizes)`: oracle / best-single / headroom for random sub-catalogues of each size, exposing diminishing returns | RouterEval; LLMRouterBench diminishing returns |
| Intra-generation escalation vs pre-generation routing | `strategies.SelfEscalation` + `wrap_stream` (stop mid-stream when `loss(stop) < loss(continue)`); compare against `Cascade` on the same dataset | 2608.24087 |
| Routing as an RL problem end-to-end (Router-R1, RLCascadeRouter) vs predict-then-optimise | `learning.PolicyGradientStrategy` (REINFORCE on hashed request features, EWMA baseline, entropy bonus; reward is the *decision* utility `decision_reward`) and `learning.decision_regret(rows, targets, choose, predict)` which reports decision regret next to prediction MAE so the two losses can be compared directly | 2608.15817 |
| Is routing learnable where it is most valuable (web / SWE agents)? | `eval.headroom.learnability_by_difficulty(rows, targets, buckets)`: per-difficulty-bucket oracle, best single, headroom and noise floor; the hardest bucket answers the question for a given pool | 2608.06171; SWE-Router 2607.00053; Agent-as-a-Router 2606.22902 |
| Memory-tier routing for agents | `strategies.MemoryRouter`: tiers with capacity, write/read cost and latency; learned `ImportanceGate` decides what to write, value decays per tick, lowest-value residents are demoted, `recall(query, budget_tokens)` fills a token budget by relevance | BudgetMem 2602.06025; Gated-Memory Routing 2609.00237 |
| Multimodal / modality escalation | `strategies.ModalityStrategy` (penalises text-only targets when the request carries images / audio / video / files that only a text surrogate describes) and `ModalityEscalation` (text-first with a Beta posterior on "surrogate sufficed", escalates to the multimodal target when confidence is low) | LatentRouter 2605.11301; modality escalation 2607.05438; CUA routing 2603.12823 |
| Mechanism design when providers are strategic | `strategies.AuctionStrategy`: bias-corrected claims, second-price payment | EA-RAM 2608.12719; strategy auctions 2602.02751; coalition pricing 2608.07532 |
| Semantic caching as a target | `strategies.SemanticCache.target()` is a `DESTINATION` whose handler serves near-duplicate answers; `SemanticCacheStrategy` scores it from the hit similarity and learns per-band hit quality from outcomes | GPTCache; vLLM semantic router 2603.04444 |
| Routing among human annotators / experts | `strategies.AnnotatorPool` (Dawid-Skene EM over per-domain annotator accuracy), `select_quorum(domain, target_accuracy, budget)` (greedy majority-vote quorum via `quorum_accuracy`) and `HumanRoutingStrategy` for `HUMAN` targets | QUORUM 2608.27974 |
| Speculative / draft-based cascades | `strategies.SpeculativeCascade`: runs draft and strong in parallel when the learned acceptance posterior says speculation is cheaper than either alone (`expected_mode_costs`), cancels the strong call when the draft is accepted | speculative cascades; Differential Reasoning Router 2608.30224 |

Open follow-ups (not scheduled): learned energy/carbon priors per provider region, and replacing the
hashed encoders in `ContrastiveRouter` / `PolicyGradientStrategy` with a real embedding model behind
`Embedder` when a dependency is acceptable.

---

## Out of scope

- Model serving, quantisation, or inference optimisation (use vLLM / TGI).
- Protocol translation between provider APIs (use LiteLLM / aisix in front).
- Token-level / layer-level routing inside a model (MoE, early exit, hybrid attention).
- Prompt management / versioning.
- Being a chat UI.

---

<!-- Page: Contributing (https://opensmartroute.ai/docs/contributing) -->

<!-- OpenSmartRoute: Contributing. Source https://github.com/isathish/OpenSmartRoute/blob/main/CONTRIBUTING.md; HTML https://opensmartroute.ai/docs/contributing -->
# Contributing to OpenSmartRoute

Thanks for helping build an open, intelligent route to the right decision.

## Ground rules

- **Core stays dependency-free.** Anything needing torch, numpy, FastAPI, etc. goes behind an
  optional extra or into `opensmartroute-contrib-*`.
- **Every strategy is inductive.** It must work for a target it has never seen, using only
  the target's declared capabilities/examples.
- **Every decision is explainable.** New strategies return a `rationale` string.
- **Every learner is persistable.** Implement `state()` / `load()`.
- **No raw request text in logs.** Hash + length only.

## Workflow

```bash
python3.12 -m venv .venv && source .venv/bin/activate      # Windows: .venv\Scripts\Activate.ps1
pip install -e '.[dev]'
ruff format src tests scripts
ruff check src tests scripts && mypy && bandit -q -r src -c pyproject.toml
pytest -q
osr -t examples/targets.yaml -r examples/rules.yaml eval examples/eval_dataset.jsonl --min-accuracy 0.85
osr -t examples/targets.yaml safety --learned-guard
python scripts/release.py check
```

CI runs the same commands on Linux (Python 3.10 to 3.13) and Windows (3.12), then builds the wheel,
sdist and container image. Pull requests must keep:

- eval accuracy >= 0.85 on `examples/eval_dataset.jsonl` (`--min-accuracy` exits non-zero otherwise);
- the safety-routing suite green (`osr safety --learned-guard`);
- the public API snapshot (`tests/public_api.json`) unchanged, or updated deliberately in the same PR
  with a CHANGELOG entry;
- `pyproject.toml`, `opensmartroute.__version__` and the top CHANGELOG section in agreement
  (`scripts/release.py check`);
- every exported name with a one-line summary (a docstring, or a same-line comment for constants)
  and `docs/REFERENCE.md` regenerated (`python scripts/api_reference.py`; `tests/test_docs.py`
  fails on a stale reference or an undocumented name; `... report` lists the gaps).

Every user-visible change gets a bullet under `## [Unreleased]` in `CHANGELOG.md`
(Keep a Changelog format: Added / Changed / Deprecated / Removed / Fixed / Security).

If you touch the hot path (`signals`, `policy`, `router`, any `Strategy.score`), run
`python scripts/bench.py` before and after and paste both tables in the PR.
If you touch brand assets, edit `scripts/brand_build.py` and run it with `--export`; never edit
the generated SVG/PNG files by hand.

## Public API and deprecation policy

The public surface is every name in the `__all__` of the packages listed in `tests/public_api.json`
plus the keyword arguments of `Router.__init__`. `tests/test_public_api.py` fails on any addition or
removal, so every change to that surface is a deliberate, reviewed diff of the snapshot.

- **Additions** are allowed in any minor release and get a CHANGELOG `Added` bullet.
- **Behaviour changes** to an existing public symbol need a `Changed` bullet; if the old behaviour
  can be kept behind a parameter, keep it as the default for one minor.
- **Removals and renames** go through a deprecation: the old symbol keeps working for at least one
  minor release, calls `opensmartroute.errors.deprecated(name, since=, removal=, replacement=)`
  (which emits `OpenSmartRouteDeprecationWarning`, a `DeprecationWarning` subclass), is listed
  under `Deprecated` in the CHANGELOG, and is deleted no earlier than the `removal` version named in
  the warning, with a `Removed` bullet. Before 1.0 the minimum notice is one minor; from 1.0 on it is
  two minors, and removals happen only in a major release.
- **State formats** (`state()` / `load()` payloads, `VersionedStateStore` schema) are versioned; a
  newer library must load older state, and loading newer state raises rather than guessing.
- **Settings** (`OSR_<GROUP>_<FIELD>`) follow the same rules as symbols; renamed keys are read under
  both names for the deprecation window.
- `tests/public_api.json` is regenerated from the actual `__all__` lists, never edited by hand, and
  the regenerating PR must show only additions unless it also carries the matching `Removed` entry.

## Adding a strategy

1. Subclass `opensmartroute.strategies.Strategy`, set `name`, implement `score()`.
2. If it learns, implement `update(outcome)` and, if it has a model, `model.state()/load()`.
3. Add a unit test with a deterministic seed.
4. Add a row to the strategy table in `README.md` and the idea-to-module map in `docs/RESEARCH.md`.

## Adding an adapter

Adapters live in `src/opensmartroute/adapters/`. Anything needing a third-party package must be
imported lazily inside the function/class and raise `ConfigurationError` naming the extra to install.
Test transports against a real in-process server (see `tests/test_adapters.py`), not by mocking
`urllib`.

## Adding math

Put it in `src/opensmartroute/math/`, cite the source in the docstring, include the formula,
and add a numerical test against a known value.

## Adding or editing an agent skill

Skills live in `.claude/skills/<name>/SKILL.md` and follow the Agent-Skills format the SDK loads
with `load_skills`. The directory name must equal `name`; every fact in the body must be
verifiable against the code (tests assert the packages load and route). Keep `osr-domains` to
ontology names (`opensmartroute.signals.ontology.DOMAIN_LEXICON` or `general`) and omit
`osr-actions` for broad skills. Validate with `osr skills` and run `tests/test_skills.py`; when a
skill's behaviour changes, update the matching request in `REPRESENTATIVE` too. Every skill is
published on the documentation site at `/docs/skills/<name>`; no web change is needed.

## Editing documentation

The documentation site in `platform/web` (`/docs/<slug>`; see "Documentation site" in
`platform/README.md`) is for end users of the hosted platform, the Python SDK and the pip package. It
renders `docs/*.md` (except `docs/sales/`), the root `README`/`CHANGELOG`/`SECURITY`/this file,
`deploy/README.md`, `spec/ocm/README.md`, the skills, and a REST API reference generated from
`platform/api/openapi.json`. Internal documents (`docs/PLATFORM_PLAN.md`, `docs/GO_TO_MARKET.md`,
`docs/BRAND.md`, `docs/sales/`, `platform/README.md`) stay in the repository and are linked from
`README.md` only. Keep one H1 per file, use H2/H3 for structure (they become the table of contents and
the search index), give fenced code a language and use relative links to other Markdown files. A new
`docs/*.md` needs a link in `README.md` (`tests/test_docs.py`) and, when it is user-facing, an entry in
`platform/web/src/lib/docs/catalogue.ts`; run `npm run build` in `platform/web` to preview
(`npm run dev` for live reload). `docs/REFERENCE.md` is generated; never edit it by hand. After changing
a platform route or model, regenerate the OpenAPI snapshot with
`python platform/api/scripts/export_openapi.py` (CI runs `--check`). Documentation may only name things
that exist: `tests/test_docs_claims.py` verifies every `osr <command> --flag` and every `OSR_*` variable
mentioned in the Markdown against the CLI parser and the code, and
`platform/api/tests/test_platform_docs_claims.py` verifies documented endpoints against the app's routes,
`/dashboard/<page>` paths against the web app, the site catalogue against `docs/` and the CHANGELOG
sections (date, body, compare link). Coding agents should start from `AGENTS.md`.

## Commit style

Conventional Commits (`feat:`, `fix:`, `docs:`, `perf:`, `refactor:`, `test:`, `chore:`).

## Releasing

Releases are cut by GitHub Actions; nobody edits version numbers or tags by hand.

1. **Prepare.** Run the *Prepare release* workflow (Actions tab, `workflow_dispatch`) with a bump
   (`major` / `minor` / `patch`) or an explicit version. It runs `scripts/release.py prepare`, which
   bumps `pyproject.toml` and `__version__`, turns `## [Unreleased]` into a dated section and rewrites
   the compare links, then opens a pull request labelled `release`.
2. **Review.** The PR diff is the release: version bump plus CHANGELOG. CI runs on it like any other PR.
3. **Merge.** Merging a `release`-labelled PR into `main` triggers the *Release* workflow, which
   re-verifies version consistency, runs the test suite, builds the sdist and wheel (with build
   provenance attestations), creates the annotated tag `vX.Y.Z` and the GitHub release with the
   CHANGELOG section as notes, publishes to PyPI through trusted publishing (no API token) and pushes
   `ghcr.io/isathish/opensmartroute:X.Y.Z` and `:latest`.

The *Release* workflow can also be dispatched manually on `main` when the tree is already at a
releasable version (used for the first release, or to retry a failed publish; it refuses to run if the
tag already exists).

One-time repository setup for maintainers:

- PyPI: add a trusted publisher for project `opensmartroute` (owner `isathish`, repository
  `OpenSmartRoute`, workflow `release.yml`, environment `pypi`).
- GitHub: create the `pypi` environment (optionally with required reviewers); allow Actions to create
  pull requests (Settings > Actions > General) so *Prepare release* can open the PR. Optionally add a
  fine-grained `RELEASE_TOKEN` secret (contents and pull-requests write) so CI runs on the release PR
  even though it was opened by automation.
- Tags for historical versions so compare links resolve: `git tag -a v0.3.0 <sha> -m "opensmartroute 0.3.0"`.

`scripts/release.py` has no dependencies beyond the standard library and is covered by
`tests/test_release_tooling.py`.

**1.0.0.** `python scripts/release.py readiness` evaluates the v1.0 readiness table in
[docs/ROADMAP.md](https://opensmartroute.ai/docs/ROADMAP.md) from the repository. *Release* rows (frozen API, performance envelope,
reference deployments, security review pack, published leaderboard run, two consecutive non-breaking
releases verified against the tagged `tests/public_api.json`) block any `1.x` version in `release.py check`;
*adoption* rows (an independent report row in `docs/SECURITY_REVIEW.md`, an `accepted` listing in
`examples/leaderboard/results/README.md`, two rows in `ADOPTERS.md`) are tracked and printed but never
block. The *Release* job prints the table in its summary on every run.

## Code of conduct

We follow the [Contributor Covenant](https://www.contributor-covenant.org/version/2/1/code_of_conduct/).