Skip to content
OpenSmartRoute
TemplateFeaturedv1.0.0

Cost guardrails

Cheap by default, frontier only for genuinely hard requests, with a monthly budget objective.

by OpenSmartRoute(0) 0 installs
Free
Sign in to install

Free account. Installing gives you the manifest plus copy-paste snippets.

See reviews

About

A three-tier setup tuned to spend as little as possible: everything routes to the small model unless the request is hard, in which case the mid or frontier tier steps in. Cost is weighted heavily in the objective. Rename the targets to your own deployments and apply.

Use it

Copy one of these into your project. Installing also returns the manifest and these snippets.

bash
osr stack plan registry://cost-guardrails@1.0.0 --registry https://api.opensmartroute.ai
osr stack apply registry://cost-guardrails@1.0.0 --registry https://api.opensmartroute.ai

Manifest

A declarative stack: models, rules and settings in one file. `osr stack apply` turns it into a running router.

cost-guardrails.stack.jsonjson
{
  "osr": "1",
  "kind": "stack",
  "name": "cost-guardrails",
  "description": "Cheap by default; the frontier tier is reserved for hard requests.",
  "targets": [
    {
      "id": "llm-small",
      "kind": "llm",
      "name": "Small model",
      "description": "Everyday questions, classification, extraction and short drafts",
      "cost": {
        "usd_per_1k_tokens": 0.0002
      },
      "latency_ms": 300,
      "quality_prior": 0.6
    },
    {
      "id": "llm-mid",
      "kind": "llm",
      "name": "Mid model",
      "description": "Moderate reasoning and longer drafting",
      "cost": {
        "usd_per_1k_tokens": 0.003
      },
      "latency_ms": 900,
      "quality_prior": 0.75,
      "capabilities": {
        "min_complexity": 0.35
      }
    },
    {
      "id": "llm-frontier",
      "kind": "llm",
      "name": "Frontier model",
      "description": "Hard multi-step reasoning and long documents",
      "cost": {
        "usd_per_1k_tokens": 0.015
      },
      "latency_ms": 2500,
      "quality_prior": 0.93,
      "capabilities": {
        "min_complexity": 0.7
      }
    }
  ],
  "rules": [
    {
      "name": "hard-only-frontier",
      "when": {
        "min_complexity": 0.7
      },
      "prefer": [
        "llm-frontier"
      ]
    },
    {
      "name": "avoid-frontier-for-easy",
      "when": {
        "max_complexity": 0.3
      },
      "avoid": [
        "llm-frontier"
      ]
    }
  ],
  "objective": {
    "quality": 0.6,
    "cost": 1,
    "latency": 0.1
  }
}

Fetch it by URL: GET /api/v1/registry/cost-guardrails/manifest?version=1.0.0

Reviews

Star ratings from people who tried it. One review per account; edit yours any time.

No reviews yet. Install it, try it, and be the first to rate it.