# Python SDK quickstart Route between models, agents, skills, tools and humans inside your own process in about five minutes. The core package has zero runtime dependencies; everything here runs offline. The [user guide](https://osr-web.kindrock-caaab1a2.centralindia.azurecontainerapps.io/docs/GUIDE.md) continues where this page stops - real providers, plans, learning, the enterprise builder. :::tip[Everything here runs offline] No API keys and no network calls are needed. Sections 2 to 5 build on each other - paste them into one file in order and run it - the router is deterministic, so you get the same decisions shown here. ::: ## 1. Install ::::tabs :::tab[pip] ```bash pip install opensmartroute ``` The core package has **zero runtime dependencies**. ::: :::tab[Extras] ```bash pip install 'opensmartroute[yaml]' # + YAML catalogues and rules pip install 'opensmartroute[server]' # + FastAPI server and OpenAI-compatible proxy pip install 'opensmartroute[fast]' # + numpy-accelerated SLM training pip install 'opensmartroute[all]' # every extra (yaml, server, fast, embeddings, tiktoken, otel, crypto, dev) ``` `osr doctor` tells you what is installed and what a missing extra needs. ::: :::tab[CLI installer] ```bash curl -LsSf https://opensmartroute.ai/install.sh | sh ``` On Windows PowerShell: ```powershell irm https://opensmartroute.ai/install.ps1 | iex ``` Installs the `osr` command line together with the package. ::: :::: ## 2. Describe your targets A target is anything a request can be sent to. Declare what each one is good at, what it costs and how fast it is; the router does the rest. ```python from opensmartroute import Router, TargetRegistry, RouteTarget, TargetKind, Capabilities, Outcome, TargetConstraints registry = TargetRegistry( [ RouteTarget( "llm-small", TargetKind.LLM, capabilities=Capabilities(max_complexity=0.45), cost={"usd_per_1k_tokens": 0.0002}, latency_ms=300, quality_prior=0.55, examples=["Hi, how are you?", "What is the capital of France?"], ), RouteTarget( "llm-frontier", TargetKind.LLM, capabilities=Capabilities(min_complexity=0.5, domains=["math", "coding"]), cost={"usd_per_1k_tokens": 0.015}, latency_ms=2500, quality_prior=0.93, examples=["Prove the theorem step by step."], ), RouteTarget( "llm-onprem", TargetKind.LLM, capabilities=Capabilities(domains=["general"]), constraints=TargetConstraints(regions=["eu"], data_boundary="private", pii_allowed=True), cost={"usd_per_1k_tokens": 0.001}, latency_ms=1200, quality_prior=0.7, ), RouteTarget( "human", TargetKind.HUMAN, capabilities=Capabilities(actions=["escalate"], tags=["safety"]), cost={"usd_per_1k_tokens": 0.5}, latency_ms=300_000, ), ] ) ``` Catalogues also load from YAML or JSON files, MCP `tools/list` payloads, A2A agent cards, SKILL.md and persona directories - see the [user guide](https://osr-web.kindrock-caaab1a2.centralindia.azurecontainerapps.io/docs/GUIDE.md#1-targets). With a `targets.yaml` (and optionally a `rules.yaml`) on disk - the repository ships a pair under `examples/` - the whole setup is one line (YAML files need `pip install 'opensmartroute[yaml]'`, JSON needs nothing): ```python from opensmartroute import Router router = Router.from_files("targets.yaml", "rules.yaml", state_dir=".osr-state") # state_dir keeps what it learns ``` ## 3. Route ```python router = Router(registry) d = router.route("Prove that sqrt(2) is irrational, step by step.") print(d.target.id, f"{d.confidence:.2f}") # llm-frontier 0.89 print(d.trace.explain()) # per-strategy scores and rationales ``` Every decision carries a trace: which signals were extracted, which targets policy rejected and why, how each strategy scored the rest. Nothing is a black box. :::note[The trace is the audit trail] `d.trace.explain()` prints the whole decision - signals, policy rejections and per-strategy scores. Keep it in your logs and every route is reproducible and defensible after the fact. ::: Every `route()` call runs the same five stages - the trace records each one: ```mermaid flowchart TB R["router.route(request)"] --> SIG["Signals: complexity, domains, intent, PII"] SIG --> POL{"Policy: hard constraints + rules"} POL -->|rejected| REJ["trace.policy_rejections"] POL -->|admissible| STR["Strategies score each candidate"] STR --> ENS["Ensemble to utility, weighted by the objective"] ENS --> DEC["Decision: target, confidence, alternatives, trace"] ``` ## 4. Constraints and objectives Hard constraints are filtered before any scoring - they are never traded off against quality. The objective sets the trade-off between quality, cost and latency per request. A note with private patient data only clears the on-prem target; the public models are excluded before scoring begins. :::warning[Constraints are hard, objectives are soft] A `RequestConstraints` violation removes a target from consideration entirely - it is never bought back by a strong quality score. `Objective` weights only rank the targets that already comply. ::: ```python from opensmartroute import RouteRequest, RequestConstraints, Objective req = RouteRequest( "Summarize this patient intake note.", constraints=RequestConstraints(region="eu", data_boundary="private", max_cost_per_1k=0.005), objective=Objective(quality=1.0, cost=0.5, latency=0.1), ) d = router.route(req) print(d.target.id) # llm-onprem print(d.trace.policy_rejections) # {'llm-small': 'data boundary public < required private', ...} ``` ## 5. Close the loop Report how the routed answer went and the router's learners (bandits, IRT, Bradley-Terry) shift future decisions toward what actually works for your traffic: ```python router.learn( Outcome( request_id=d.request_id, target_id=d.target.id, success=True, quality=0.9, cost_usd=0.002, latency_ms=1800, domains=d.trace.signals.domains, ) ) ``` Attach handlers to targets and `router.run(req)` executes the chosen target or plan and records the outcomes itself - wiring real providers is the [user guide's fourth chapter](https://osr-web.kindrock-caaab1a2.centralindia.azurecontainerapps.io/docs/GUIDE.md#4-real-providers). ## 6. The same thing from the command line Without `--targets` the `route`, `estimate`, `serve`, `targets` and `mcp` commands use a built-in starter catalogue (`llm-small`, `llm-frontier`, `llm-onprem`, with two rules), so the first command works straight after the install; `osr stack init --out stack.json` writes that catalogue to disk to edit, and `-t stack.json -r stack.json` routes through your copy. The repository's `examples/` folder holds the richer YAML pair used below. ```bash osr route "Prove that sqrt(2) is irrational" # built-in starter catalogue, no files needed osr -t examples/targets.yaml -r examples/rules.yaml route "I want a refund for order #123" --plan osr -t examples/targets.yaml -r examples/rules.yaml estimate "Summarise this contract" # tokens, cost, quality per candidate osr -t examples/targets.yaml -r examples/rules.yaml eval examples/eval_dataset.jsonl --frontier export OSR_LLM_BASE_URL=https://api.openai.com/v1 OSR_LLM_API_KEY=sk-... # the provider `serve` executes on osr -t examples/targets.yaml -r examples/rules.yaml serve # http://127.0.0.1:8000/docs osr -t examples/targets.yaml mcp # MCP server for your IDE, marketplace tools included ``` Routing needs no provider. Execution does: `OSR_LLM_BASE_URL` names any OpenAI-compatible endpoint (OpenAI, Azure OpenAI, vLLM, Ollama, OpenRouter, ...) and `OSR_LLM_API_KEY` its key; without them the served `/v1/chat/completions` answers 503 because no target has a handler, and routing still works. `osr serve` exposes the same `/route`, `/feedback` and `/mcp` endpoints as the hosted platform and the same `/v1` surface: OpenAI `chat/completions` (streaming, tool calling), `responses`, `embeddings`, `moderations`, `audio/speech`, `audio/transcriptions`, `images/generations` and the Anthropic `messages` API - point the official OpenAI or Anthropic SDK at it with `model="osr/auto"` (or `"auto"`), an MCP client at `/mcp`, or the TypeScript client (`npm install @opensmartroute/sdk`, `apiPrefix: ""`). Every model call is compacted first (repeated lines, stray whitespace and stale history go; the saving is reported as `tokens_saved`), `--marketplace-refresh 86400` keeps the served catalogue in step with the marketplace, and `--guard --redact-pii --output-guard redact --health --audit audit.jsonl` turns on the hardening the [deploy guide](https://osr-web.kindrock-caaab1a2.centralindia.azurecontainerapps.io/docs/deploy.md) describes. ## FAQ ::::accordions :::accordion[Do I need API keys or a network connection?] No. The core package is offline and deterministic - it decides *where* a request should go. You only add provider credentials when you attach real handlers and call `router.run()`. ::: :::accordion[How does the router pick a target?] It extracts signals (complexity, domains, intent, PII), filters targets that fail the hard constraints, scores the rest with an ensemble of strategies, and combines the scores into a utility weighted by your `Objective`. `d.trace.explain()` shows every step. ::: :::accordion[Can it route to more than models?] Yes. A `RouteTarget` can be an LLM, an agent, a skill, a tool or a human - the `TargetKind`. The same scoring picks the best target of any kind, and `RequestConstraints(allowed_kinds=...)` limits it when you need to. ::: :::accordion[Does it learn from outcomes?] Yes. `router.learn(Outcome(...))` feeds the bandit, IRT and Bradley-Terry learners so future decisions shift toward what actually works on your traffic. It is optional and off until you call it. ::: :::: ## Where next - [User guide](https://osr-web.kindrock-caaab1a2.centralindia.azurecontainerapps.io/docs/GUIDE.md) - targets, rules, providers, plans, learning, the CLI, latency numbers. - [SDK guide](https://osr-web.kindrock-caaab1a2.centralindia.azurecontainerapps.io/docs/SDK.md) - decorators, settings, error handling, threading, extension points. - [Enterprise builder](https://osr-web.kindrock-caaab1a2.centralindia.azurecontainerapps.io/docs/ENTERPRISE.md) - middleware, telemetry, stores, shadow and A/B rollout. - [Deploy with Docker and Helm](https://osr-web.kindrock-caaab1a2.centralindia.azurecontainerapps.io/docs/deploy.md) - run `osr serve` on your own infrastructure. - [TypeScript SDK](https://osr-web.kindrock-caaab1a2.centralindia.azurecontainerapps.io/docs/sdk-typescript.md) - call `osr serve` or the platform from Node, Deno, Bun, edge and browsers. - [Platform quickstart](https://osr-web.kindrock-caaab1a2.centralindia.azurecontainerapps.io/docs/QUICKSTART_PLATFORM.md) - the hosted version of everything above.