Loading
90 days of the enterprise, pro and team plans free, no card needed. See the plans
Open-source LLM proxy
LiteLLM is an excellent execution layer: one SDK and proxy for 100+ providers with keys, budgets and fallbacks. OpenSmartRoute adds the decision layer on top: which capability, under which policy, explained, and learned from outcomes.
“LiteLLM documents an execution layer; OpenSmartRoute adds the decision layer in front of it.”
Our reading of public documentation; corrections welcome through the support page.
| Capability | OpenSmartRoute | LiteLLM |
|---|---|---|
| Routes to LLMs | Yes | Yes |
| Routes to agents, skills, tools, workflows and humans | Yes | No |
| Hard policy before scoring (region, data boundary, PII, tenant, cost)Budgets and allow-lists per key; no data-boundary or PII gating | Yes | Partial |
| Per-request trace with ranked candidates and rejections | Yes | No |
| Learns from your outcomes (per target, per tenant) | Yes | No |
| Multi-step plans (persona -> skill -> model) | Yes | No |
| Self-hosted, air-gapped, zero runtime dependenciesSelf-hosted, but with a large dependency tree | Yes | Partial |
| Prompt-injection and steering-gadget defences on the router | Yes | No |
| Output guard on responses (PII, secrets, injection: flag, redact or block)Guardrail plugins (external services) rather than a built-in response screen | Yes | Partial |
| Savings ledger, bill analysis and routing auditEvery request records baseline and routed cost; upload last month's provider export to see what routing would have saved before you switch | Yes | No |
| OpenAI-compatible proxy (chat, embeddings, Responses, moderations, audio, images) | Yes | Yes |
| Anthropic Messages API (/v1/messages) | Yes | Yes |
| Virtual keys: per-key budgets, rate limits, allowed targets, scopes and expiry | Yes | Yes |
| Per-target rate limits, budgets and circuit breakers | Yes | Yes |
| Python and TypeScript SDKsPython SDK; the proxy works with any OpenAI client | Yes | Partial |
| MCP server for IDEs and agentsMCP gateway for tools; the router itself is not an MCP server | Yes | Partial |
| Browser extension and desktop app for the people on the teamPrice, personal-data and writing checks on the device before a prompt is sent, on ChatGPT, Claude, Gemini and in any desktop app | Yes | No |
| Provider marketplace with creditsModels by the token: every catalogue model on offer per provider with the price you pay, one key, a prepaid credits wallet and provider preferences (order, only, ignore, max_price) | Yes | No |
| Request/response observability UI for every callDecision trace, tokens, cost and outcome per request; prompt and answer stored when the workspace turns on request logging (redacted or full) | Yes | Partial |
Call /v1/route first, then hand the chosen model to LiteLLM for execution. Nothing in your provider setup changes.
H = {"Authorization": f"Bearer {OSR_API_KEY}"}
decision = requests.post(f"{OSR}/v1/route", json={"text": prompt, "constraints": {"tenant": tenant}}, headers=H).json()
resp = litellm.completion(model=decision["target"]["model"], messages=messages) # "openai/gpt-4.1-mini"
requests.post(
f"{OSR}/v1/feedback",
json={"request_id": decision["request_id"], "target_id": decision["target"]["id"], "success": True,
"cost_usd": resp._hidden_params["response_cost"]},
headers=H,
)Run the Routing Audit on a week of anonymised logs: savings versus your current mix, policy violations your routing allowed, and where agents or humans would have been the better answer.
Other comparisons: OpenRouter, Portkey, Cloudflare AI Gateway, Bifrost (Maxim AI), Kong AI Gateway, Helicone, Not Diamond, Martian, Unify, Requesty, Building it yourself.
Photo: Mario Gogh
