Does my existing code need to know about the router?

No. The platform serves the OpenAI chat completions, Responses and embeddings APIs and the Anthropic Messages API on one key. Change the base URL and the key; the client library, your prompts and your tests stay as they are. Fields the router does not know are forwarded to the provider unchanged.

What does model="auto" do, and what if I want a specific model?

"auto" lets the router choose per request: it reads the task, refuses what your policy forbids, prices every allowed candidate and picks the best value. A model id ("openai/gpt-4.1") pins that model and still routes among its providers; a target id pins one deployment. osr/auto:cheap, :fast, :quality and :private preset the objective.

What about streaming, tool calls and JSON mode?

stream: true relays the provider's tokens as they arrive in the OpenAI (or Anthropic) event format; the last chunk carries usage and the opensmartroute metadata. tools, tool_choice, response_format, stop, seed and the rest are forwarded unchanged, and a tool call comes back as message.tool_calls as before.

Which keys does it use - mine or yours?

Yours by default: connect your own provider accounts (Azure OpenAI, OpenAI, Anthropic, Bedrock, Vertex, Ollama, ...) and the router executes on them; model usage is billed by the provider, the platform meters requests. Where the deployment sells models by the token you can also pay from prepaid credits.

Where do my prompts go, and are they stored?

A request goes to the chosen provider and comes back; the platform keeps the decision, tokens, cost and outcome. Prompt and answer text are stored only when a workspace turns request logging on. The privacy page lists every field, with the retention of each.

Put the previous base URL and key back. The router keeps no state your application depends on, so the change is one environment variable each way. Self-hosting the open-source package is the third option: the same API at an address you run.

Switch in one line

Keep your code. Change one URL.

OpenSmartRoute speaks the APIs your code already calls - OpenAI chat completions and Responses, the Anthropic Messages API, embeddings. Point the client at the router, set model to "auto" and every request is priced, routed and explained. Rolling back is the same one line.

Create a free account Watch a decision firstFree plan, no card. An API key takes a minute.

Today

OpenAI client · app.pypython

from openai import OpenAI

client = OpenAI(api_key=OPENAI_API_KEY)

reply = client.chat.completions.create(
    model="gpt-4.1",
    messages=[{"role": "user", "content": prompt}],
)
print(reply.choices[0].message.content)

Routed

OpenAI client · app.pypython

from openai import OpenAI

client = OpenAI(base_url="https://api.opensmartroute.ai/v1",
                api_key=OSR_API_KEY)

reply = client.chat.completions.create(
    model="auto",   # the router picks per request
    messages=[{"role": "user", "content": prompt}],
)
print(reply.choices[0].message.content)
print(reply.model)  # the model that answered

What changed: Two arguments: base_url and the key. model="auto" lets the router decide; keep "gpt-4.1" and the call is pinned, still routed among its providers.

What comes back

The same answer, with the decision attached.

A standard completion your parser already reads. The model field names the target that answered; the opensmartroute block carries the request id, the confidence, the alternatives and the cost, so you can report the outcome and audit the choice later.

modelthe target that answered - a catalogue model or a deployment of yours
opensmartroute.request_idwhat you send back with feedback, and what the trace is filed under
opensmartroute.alternativesthe runners-up, in rank order
opensmartroute.cost_usdwhat this answer cost; the ledger compares it with the baseline
opensmartroute.fallback_fromthe targets that failed before this one answered, if any
X-OSR-Target headerthe same target id for clients that only read headers

POST /v1/chat/completions · responsejson

{
  "id": "chatcmpl-…", "object": "chat.completion",
  "model": "llm-small",
  "choices": [{ "message": { "role": "assistant", "content": "…" },
                "finish_reason": "stop" }],
  "usage": { "prompt_tokens": 42, "completion_tokens": 118 },
  "opensmartroute": {
    "request_id": "7f3c…", "target": "llm-small", "kind": "llm",
    "confidence": 0.88, "alternatives": ["llm-mid"],
    "cost_usd": 0.00012, "latency_ms": 412.5, "fallback_from": []
  }
}

For the reviewer

What the pull request changes - and what it does not.

What stays the same

Your client library, its types and its retry logic
Streaming, tool calls, JSON mode, images and audio parts
The shape of the answer: a chat completion, a Responses object or an Anthropic message
Your prompts and your evaluation suite - nothing is rewritten

What you gain

Every request priced against every allowed model before it is sent
The model that answered, the alternatives and the cost in each response
A provider outage becomes a fallback, not an error
A savings ledger and a trace per request in the dashboard

How you go back

Put the old base URL and key back - there is no state in your app to unwind
Pin a model id instead of "auto" to route one call without changing the rest
Keep your provider keys: bring your own and nothing is billed through us
Self-host the same open-source package and point the URL at it

Verify in a minute

Three commands prove the first routed request.

List the models your key can reach, route one request, report the outcome. The dashboard's activity page shows the same request with its trace a moment later.

verify.shbash

# 1. the models your key can route to - "auto" is first
curl -s https://api.opensmartroute.ai/v1/models -H "Authorization: Bearer $OSR_API_KEY" | head -c 400

# 2. one routed request; read "model" and the opensmartroute block
curl -s https://api.opensmartroute.ai/v1/chat/completions \
  -H "Authorization: Bearer $OSR_API_KEY" -H "Content-Type: application/json" \
  -d '{"model":"auto","messages":[{"role":"user","content":"Say hi in five words."}]}'

# 3. tell the router whether the answer was good (the learners move)
curl -s https://api.opensmartroute.ai/api/v1/feedback \
  -H "Authorization: Bearer $OSR_API_KEY" -H "Content-Type: application/json" \
  -d '{"request_id":"<request_id from step 2>","target_id":"<target>","success":true}'

Keep your rules

Candidates, weights and limits ride along in the request.

models narrows the field to the targets you trust; osr.objective weighs quality against price and speed; osr.constraints are never traded off - a data boundary, a region, a latency or price cap removes a candidate before pricing. Other providers ignore the fields.

POST /v1/chat/completions · with routing optionsbash

curl -s https://api.opensmartroute.ai/v1/chat/completions \
  -H "Authorization: Bearer $OSR_API_KEY" -H "Content-Type: application/json" \
  -d '{
    "model": "auto",
    "models": ["llm-frontier", "llm-mid"],
    "osr": {
      "objective":   { "quality": 1.0, "cost": 0.4 },
      "constraints": { "data_boundary": "private", "max_latency_ms": 4000 }
    },
    "messages": [{ "role": "user", "content": "Summarise this contract clause." }]
  }'

The same weights and limits are what the routing lab lets you try without a key. On 150 real prompts the router spent 89% less than always using the frontier model; the benchmark page shows the cases it lost too.

Questions

What comes up in review

Does my existing code need to know about the router?: No. The platform serves the OpenAI chat completions, Responses and embeddings APIs and the Anthropic Messages API on one key. Change the base URL and the key; the client library, your prompts and your tests stay as they are. Fields the router does not know are forwarded to the provider unchanged.
What does model="auto" do, and what if I want a specific model?: "auto" lets the router choose per request: it reads the task, refuses what your policy forbids, prices every allowed candidate and picks the best value. A model id ("openai/gpt-4.1") pins that model and still routes among its providers; a target id pins one deployment. osr/auto:cheap, :fast, :quality and :private preset the objective.
What about streaming, tool calls and JSON mode?: stream: true relays the provider's tokens as they arrive in the OpenAI (or Anthropic) event format; the last chunk carries usage and the opensmartroute metadata. tools, tool_choice, response_format, stop, seed and the rest are forwarded unchanged, and a tool call comes back as message.tool_calls as before.
Which keys does it use - mine or yours?: Yours by default: connect your own provider accounts (Azure OpenAI, OpenAI, Anthropic, Bedrock, Vertex, Ollama, ...) and the router executes on them; model usage is billed by the provider, the platform meters requests. Where the deployment sells models by the token you can also pay from prepaid credits.
Where do my prompts go, and are they stored?: A request goes to the chosen provider and comes back; the platform keeps the decision, tokens, cost and outcome. Prompt and answer text are stored only when a workspace turns request logging on. The privacy page lists every field, with the retention of each.
How do I roll back?: Put the previous base URL and key back. The router keeps no state your application depends on, so the change is one environment variable each way. Self-hosting the open-source package is the third option: the same API at an address you run.

Change the URL on one service today.

Route a slice of real traffic on the free plan, read the ledger and the traces, and decide with your own numbers. Nothing in your application has to know.

Create a free account The five-minute quickstart

Free plan, no card. Fifteen thousand decisions a month with the full trace.

from openai import OpenAI client = OpenAI(api_key=OPENAI_API_KEY) reply = client.chat.completions.create( model="gpt-4.1", messages=[{"role": "user", "content": prompt}], ) print(reply.choices[0].message.content)

from openai import OpenAI client = OpenAI(base_url="https://api.opensmartroute.ai/v1", api_key=OSR_API_KEY) reply = client.chat.completions.create( model="auto", # the router picks per request messages=[{"role": "user", "content": prompt}], ) print(reply.choices[0].message.content) print(reply.model) # the model that answered

{ "id": "chatcmpl-…", "object": "chat.completion", "model": "llm-small", "choices": [{ "message": { "role": "assistant", "content": "…" }, "finish_reason": "stop" }], "usage": { "prompt_tokens": 42, "completion_tokens": 118 }, "opensmartroute": { "request_id": "7f3c…", "target": "llm-small", "kind": "llm", "confidence": 0.88, "alternatives": ["llm-mid"], "cost_usd": 0.00012, "latency_ms": 412.5, "fallback_from": [] } }

# 1. the models your key can route to - "auto" is first curl -s https://api.opensmartroute.ai/v1/models -H "Authorization: Bearer $OSR_API_KEY" | head -c 400 # 2. one routed request; read "model" and the opensmartroute block curl -s https://api.opensmartroute.ai/v1/chat/completions \ -H "Authorization: Bearer $OSR_API_KEY" -H "Content-Type: application/json" \ -d '{"model":"auto","messages":[{"role":"user","content":"Say hi in five words."}]}' # 3. tell the router whether the answer was good (the learners move) curl -s https://api.opensmartroute.ai/api/v1/feedback \ -H "Authorization: Bearer $OSR_API_KEY" -H "Content-Type: application/json" \ -d '{"request_id":"<request_id from step 2>","target_id":"<target>","success":true}'

curl -s https://api.opensmartroute.ai/v1/chat/completions \ -H "Authorization: Bearer $OSR_API_KEY" -H "Content-Type: application/json" \ -d '{ "model": "auto", "models": ["llm-frontier", "llm-mid"], "osr": { "objective": { "quality": 1.0, "cost": 0.4 }, "constraints": { "data_boundary": "private", "max_latency_ms": 4000 } }, "messages": [{ "role": "user", "content": "Summarise this contract clause." }] }'