osr-integrations

Everything below produces RouteTargets or handlers; the router itself has no I/O.

LLM providers (adapters.openai_compat)#

from opensmartroute.adapters import OpenAICompatClient, chat_handler, embedder, judge_fn
openai = OpenAICompatClient()                                   # OPENAI_API_KEY (or *_FILE) from env
ollama = OpenAICompatClient("http://localhost:11434/v1", api_key="ollama")
registry.get("llm-frontier").handler = chat_handler(openai, model="gpt-4o", system_prompt=None)
registry.get("llm-small").handler   = chat_handler(ollama, model="llama3.1")
router = Router(registry,
    strategies=[CapabilityStrategy(), SimilarityStrategy(embedder=embedder(openai, "text-embedding-3-small"))],
    llm_judge=LLMJudgeStrategy(judge_fn(openai, "gpt-4o-mini")), escalate_llm_judge_below=0.5)

chat_handler returns ChatResult(text, model, prompt_tokens, completion_tokens, latency_ms, raw); execution derives cost from tokens x usd_per_1k_tokens. Local embeddings: sentence_transformers_embedder(model) (embeddings extra).

MCP tools (adapters.mcp)#

from opensmartroute.adapters import tools_from_mcp, connect_mcp, tools_from_manifest
targets = tools_from_mcp(tools_list_json, server="github", call=lambda name, args: client.call(name, args),
                         cost_per_call_usd=0.0, latency_ms=300, quality_prior=0.6, max_description_risk=0.6)
client, targets = connect_mcp(["npx", "-y", "@modelcontextprotocol/server-filesystem", "."], server="fs")

Ids are server:tool; inputSchema is kept in metadata; risky descriptions are dropped. Prefer signed manifests in production (sign_manifest / verify_manifest / tools_from_manifest).

Which server to connect in the first place (adapters.mcp_servers):

from opensmartroute.adapters import ServerCard, recommend_servers
cards = [ServerCard.from_dict(d) for d in registry_json]            # name, description, tools, tags, auth,
for rec in recommend_servers("open a pull request", cards, k=2,    #   latency_ms, region, data_boundary
                             constraints=req.constraints, allowed_auth=["none", "token"]):
    rec.server.name, rec.score, rec.rationale, rec.matched_tools

BM25 + hashed-dense fusion over the card text; hard filters on latency, region, data boundary and auth kind run before ranking, so a private request never gets a public-only server recommended.

vLLM semantic-router configs (adapters.semantic_router)#

from opensmartroute.adapters import load_semantic_router_config
imported = load_semantic_router_config("config.yaml")            # or a dict
router = Router(imported.registry, strategies=[RulesStrategy(imported.rules), *default_strategies()])
imported.categories, imported.default_model, imported.warnings

Categories with model_scores become RouteTargets (quality_prior = mean score, cost from model_config.pricing, effort: high when every category uses reasoning) plus one Rule per category; system_prompt lands in the top target's instructions.

A2A agents (adapters.a2a)#

from opensmartroute.adapters import fetch_agent_card, agent_from_card, skills_from_card, a2a_handler
card = fetch_agent_card("https://agents.example.com/travel")          # https only, /.well-known/agent.json
agent = agent_from_card(card, call=a2a_handler(card["url"], token=load_secret("A2A_TOKEN")))
for s in skills_from_card(card): registry.add(s)                       # primary=False, id "<agent>/<skill>"

Agent harnesses (adapters.harness)#

CallableHarness(fn) (fn(task, context, history) -> str | dict | HarnessResult), HTTPHarness(url, api_key_env=..., timeout_s=300) (POST {"task","context","history"}), SubprocessHarness(argv, json_output=True, system_flag="--system"). Wrap with harness_handler(harness) and assign to a kind: agent target. Return HarnessResult(text, success, cost_usd, prompt_tokens, completion_tokens, quality) so outcomes carry real cost/quality.

Frameworks (adapters.frameworks)#

node = langgraph_node(router, execute=False, plan=False)      # state["messages"] -> route_target, route
graph.add_conditional_edges("route", langgraph_condition(by="target", default="__end__"), {...})
executor, handoffs = maf_router_executor(router)               # Microsoft Agent Framework
spec, handle = openai_tool_spec("route_request"), openai_tool_handler(router)   # function calling
decision, result = route_and_execute(router, text, history=..., context=..., execute=True)

Personas and skills#

• Personas: load_personas(path) accepts a directory of *.md, .json, .jsonl or .csv; persona_target(name, prompt, description, domains=..., primary=False). Their instructions are composed into context["system"] as # Persona: <name> when they fill a plan slot.
• Skills: load_skills(root) globs */SKILL.md; load_skill(dir) loads one.

Authoring a SKILL.md the router can select#

---
name: sql-reporting                      # must equal the directory name; ^[a-z0-9]+(-[a-z0-9]+)*$, <= 64
description: Writes warehouse SQL for recurring reports. Use for SQL or analytics questions.   # <= 1024
license: Apache-2.0
metadata:
  osr-domains: "data_analysis coding"    # routing hints (space/comma separated)
  osr-languages: "en"
  osr-tags: "sql"
  osr-quality-prior: "0.82"
  osr-primary: "false"                   # slot-only: layered on a model in the plan
  osr-latency-ms: "0"
  osr-requires: "warehouse-schema"       # optional skill-graph edges, space/comma separated ids
  osr-conflicts: ""
  osr-composes: "chart-builder"
---
# Body = instructions, disclosed only when the skill is selected (progressive disclosure)

Only name/description are used for routing; write the description for the request the skill should attract (it seeds examples, one per sentence). osr-domains/osr-actions must use the signal ontology names (signals.DOMAIN_LEXICON / ACTION_LEXICON, e.g. coding, data_analysis, code_generation, reasoning); an unmatched declared scope is penalised, so omit osr-actions when the skill spans many request types. Validate with osr skills <root>; register with registry.add(skill) or --skills <root> on the CLI; the router fills the plan's skill slot (router.route(req, plan=True)) or scores skills directly via candidates=[...]. osr-requires / osr-conflicts / osr-composes feed discovery.SkillGraph(load_skills(root)); graph.compose(["sql-reporting"]) returns the dependency-ordered skill set and the ids dropped for conflicts, and graph.observe(skill_ids, success) learns co-usage companions.

OpenAI-compatible proxy#

osr -t targets.yaml serve exposes POST /v1/chat/completions; model: "auto" (or "opensmartroute") lets the router pick, a real target id pins it. See the deploy skill.