Imported from sabadia/minerouter (skills/minerouter-operations/SKILL.md). Install upstream with npx skills add sabadia/minerouter --skill minerouter-operations. Copyright stays with the author.

MineRouter — Operations

Observability surfaces

• GET /metrics — Prometheus. Key series (app/core/metrics.py):
  • minerouter_requests_total{category,tier,cache} — request counts.
  • minerouter_tier_resolutions_total{tier} — where routing resolved (embedding/reranker/llm/sticky/sticky_switch/vision/explicit_*/default).
  • minerouter_cache_events_total{result} — cache hit/miss.
  • minerouter_tokens_total{type,model,category,tier} — upstream tokens, attributable to the route that spent them. Cache hits are excluded (they replay already-counted usage).
  • minerouter_cached_prompt_tokens_total{model} — prompt tokens served from the provider's prefix cache. Growth means prefix caching is working; a collapse means something broke it.
  • minerouter_router_llm_tokens_total{type} — tokens spent by the routing classifier itself, kept separate so routing overhead is never mistaken for useful generation.
  • minerouter_cost_usd_total{model,category} and minerouter_baseline_cost_usd_total{category} — actual spend vs. what the priciest model in the same pool would have cost. Requires input_cost_per_1k/output_cost_per_1k on the models.
  • minerouter_upstream_attempts_total{model,outcome} — every attempt including retries and fallbacks. The gap versus minerouter_requests_total is wasted upstream spend.
  • minerouter_upstream_errors_total — failed dispatches surfaced to clients.
  • minerouter_routing_seconds, minerouter_upstream_seconds — latency histograms.
• GET /ready — strategy, embedding backend + dimension, rerank enabled, database connectivity, categories, models.
• Logs — structured (JSON by default; LOG_FORMAT=console for dev). Notable events: request_completed (one line per request: session, category, tier, model, tokens, cached tokens, cost, cache result, stream, latency), routing_decision, session_rerouted, context_ready, database_connect_failed/database_unavailable, redis_connected, routing_state_local_only, llm_router_failed, auth_open, vision_pool_unmarked, tool_pool_unmarked, tool_pool_empty, no_tool_capable_models, upstream_attempt_failed, dispatch_failed, stream_aborted.
• Request log table (request_logs in Postgres, if enabled) — per request: category, tier, model, strategy, score, tokens, cost, latency, cache_hit, stream. Streamed requests are now recorded too — usage is captured from the final stream chunk after the stream completes.

Tuning routing

Session affinity changes the trade-off. Classification now runs roughly once per conversation, not once per turn, so escalating to a higher tier is cheap — while a wrong first decision is expensive, because it sticks for session_affinity.ttl_seconds. The shipped thresholds (0.38 / 0.08) are deliberately strict for this reason: buy accuracy on turn 1.

Use minerouter_tier_resolutions_total to see how often each tier fires:

• Healthy agent traffic is dominated by sticky. If embedding/llm rivals it, conversations are not being recognised — check that clients keep a stable system prompt + first user message, or have them send an explicit X-MineRouter-Session header.
• Too many at llm → make sure rerank.enabled: true (free local tier-b absorbs most of these) and llm_router.memo_ttl_seconds > 0; then add clearer, more distinct category examples. Lower the thresholds only if minerouter_router_llm_tokens_total is genuinely material.
• Frequent sticky_switch → conversations are being re-pinned mid-flight, which costs a cold prefix cache each time. Raise resticky_threshold.
• Wrong categories chosen → improve/disambiguate examples, or raise thresholds so ambiguous cases escalate to the reranker/LLM.
• Everything at default → the LLM router is failing (see below) or thresholds are too strict.

Measuring token savings

Is routing actually saving money? Compare the two cost series (needs per-model pricing in config.yml):

sum(rate(minerouter_cost_usd_total[1h])) / sum(rate(minerouter_baseline_cost_usd_total[1h]))

Below 1.0 means routing is cheaper than always using the priciest model in each pool.

Is the upstream prefix cache working? For agent workloads this dominates everything else:

sum(rate(minerouter_cached_prompt_tokens_total[1h])) / sum(rate(minerouter_tokens_total{type="prompt"}[1h]))

A collapse here means conversations are switching models mid-flight (check minerouter_tier_resolutions_total{tier="sticky_switch"}) or session affinity is off.

What is routing overhead costing? minerouter_router_llm_tokens_total ÷ minerouter_tokens_total — if this is not tiny, lower escalation (see Tuning routing) or raise llm_router.memo_ttl_seconds.

Is retry fan-out burning tokens? minerouter_upstream_attempts_total ÷ minerouter_requests_total should sit near 1.0. Well above that means requests are re-sending the whole conversation across retries/fallbacks — lower dispatch.num_retries / dispatch.max_fallbacks, or fix the failing provider.

• minerouter_cache_events_total{result="hit"} ÷ total = cache hit ratio (each hit is 0 upstream tokens). Note tool-calling and continuation turns are never cached by design.
• Per-model/category cost analysis from request_logs, e.g.:

  SELECT category, model_group, count(*), sum(total_tokens), sum(cost), avg(latency_ms)
  FROM request_logs GROUP BY 1,2 ORDER BY 3 DESC;

Troubleshooting

Graceful-degradation behaviour (by design)

• DB unreachable → gateway routes normally; request logging + pg cache are skipped.
• Redis unreachable → session affinity and the classification memo fall back to in-process stores (routing_state_local_only). Works single-process; not shared across replicas.
• LLM router unreachable in hybrid → falls back to the best embedding guess, not a blind default.
• Provider-unsupported params → dropped automatically (dispatch.drop_unsupported_params), except dispatch.critical_params (tools, tool_choice, response_format), which raise a DispatchError instead. Silently dropping those changes behaviour and costs the caller turns.

See also: minerouter-configuration (all tunables) and minerouter-deployment (probes, scaling).