# Cost estimates, recommended models and the MCP server Three ways to ask the router "what will this cost and what should I use?" before spending tokens: a quote API and page, a recommended-models digest, and a [Model Context Protocol](https://modelcontextprotocol.io) server that puts all of it inside VS Code, Cursor, Claude, Windsurf or any agent framework. The same pieces exist in the SDK (`opensmartroute.estimate`, `opensmartroute.mcp_server`, `osr estimate`, `osr mcp`) and on the hosted platform (`POST /api/v1/estimate`, `GET /api/v1/models/recommended`, `/mcp`, the `/estimate` page and *Dashboard -> Integrations*). ## 1. Quotes A quote routes the request exactly as `route()` would - signals, policy, strategies, learned quality - and then prices every ranked candidate instead of executing one: - **Tokens.** Input tokens come from the text (word / digit / punctuation / CJK / code heuristic, `estimate_tokens`, calibrated against billed gpt-4.1 prompt tokens on twelve text types - mean ratio 1.02, every type within about 20 %) plus the history (`estimate_messages_tokens`, 4 tokens of framing per message). Output tokens are the caller's `output_tokens`, else the signals' predicted answer length, else `DEFAULT_OUTPUT_TOKENS` (256); `output_tokens_source` says which. - **Prices.** `usd_per_1k_input` / `usd_per_1k_output` on the target when declared, else the blended `usd_per_1k_tokens`; a fixed `usd_per_call` is added for agents, tools and humans. On the platform a target that resolves to a reference model is priced from the live catalogue (`priced_from: "catalogue"`), so quotes follow vendor list prices. - **Picks.** `recommended` is the router's decision; `best_quality` the highest quality estimate; `cheapest` and `fastest` are chosen among candidates within `quality_tolerance` (0.1) of the best that fit their context window, so a free-but-useless target never wins. `savings_usd` is the gap between the dearest candidate and the recommended one. - **Nothing is executed and the text is never stored.** ```python from opensmartroute.estimate import estimate q = estimate(router, "Summarise this contract in five bullets", output_tokens=300) print(q.input_tokens, q.output_tokens, q.output_tokens_source) # 9 300 caller print(q.recommended.name, q.recommended.cost_usd) # Small model 0.000062 for t in q.targets: # every candidate, ranked print(t.rank, t.target_id, f"${t.cost_usd:.6f}", t.quality_estimate, t.latency_ms, t.fits_context) print(q.cheapest.target_id, q.best_quality.target_id, q.fastest.target_id, q.savings_usd) q.to_dict() # the JSON the API returns ``` `estimate(router, request, *, output_tokens=None, kinds=None, prices=None, quality_tolerance=0.1, decision=None)` accepts a `RouteRequest` or a string, a `kinds` filter and a `prices(target) -> (usd_per_1k_in, usd_per_1k_out) | None` hook for live price feeds; pass `decision=` to quote what an earlier `route()` already ranked. ```bash osr -t targets.yaml estimate "Translate this contract into German" --monthly 100000 osr -t targets.yaml estimate "..." --output-tokens 500 --cost-weight 1.0 --json ``` ### `POST /api/v1/estimate` Same body as `POST /api/v1/route` (`text`, `history`, `objective`, `constraints`, `kinds`) plus `output_tokens` and `monthly_requests`. Without a key the endpoint is rate limited per client (20 per minute, 8 000 characters) and ignores tenant rules; with a key it is metered as `estimate`, applies your plan and tenants and returns `X-Request-Id`. ```bash curl -s https:///api/v1/estimate -H "Authorization: Bearer $OSR_KEY" \ -H "Content-Type: application/json" \ -d '{"text": "Summarise this contract...", "monthly_requests": 50000}' ``` ```json { "request_id": "…", "input_tokens": 9, "output_tokens": 256, "output_tokens_source": "signals", "total_tokens": 265, "confidence": 0.71, "savings_usd": 0.00209, "recommended": {"target_id": "llm-small", "name": "Small model", "vendor": "openai", "model": "openai/gpt-4.1-nano", "input_cost_usd": 0.0000009, "output_cost_usd": 0.0001, "cost_usd": 0.000103, "latency_ms": 400, "quality_estimate": 0.62, "fits_context": true, "priced_from": "catalogue", "rationale": {"semantic": "…", "cost": "…"}}, "cheapest": {"…": "…"}, "best_quality": {"…": "…"}, "fastest": {"…": "…"}, "targets": [{"…": "…"}], "signals": {"complexity": 0.31, "domains": ["legal"], "task_type": "summarise", "contains_pii": false, "…": "…"}, "policy_rejections": {}, "monthly": {"requests": 50000, "recommended_usd": 5.15, "cheapest_usd": 5.15, "best_quality_usd": 109.6, "dearest_usd": 109.6, "savings_usd": 104.45, "tokens": 13250000}, "anonymous": false } ``` The `/estimate` page is the same call with a form: paste a prompt, choose what to optimise for, set the expected answer length and monthly volume, and read the four picks, the signals the router saw and the full candidate table with input / output cost bars. ## 2. Recommended models `GET /api/v1/models/recommended` (no key) quotes a representative prompt per use case - `chat`, `code`, `reasoning`, `summarise`, `extraction`, `pii` - against the live catalogue and returns, for each, the `recommended`, `cheapest`, `best_quality` and `fastest` pick with cost per request, quality estimate, latency, 30-day traffic, the reference model's intelligence index and the reasons (`why`). A `leaderboard` counts how often each target was the router's pick. The digest is recomputed at most once a minute and follows the deployment's targets, prices and learned quality rather than a hand-maintained list. It renders on `/models` ("Recommended per use case") and on `/estimate`. ## 3. MCP server An MCP client - VS Code, Cursor, Claude, Windsurf or an agent framework - calls the router's tools over JSON-RPC and gets a decision (or a quote, or the executed answer) back: ```mermaid sequenceDiagram participant C as MCP client (Cursor / Claude / agent) participant S as MCPServer(router) participant R as Router C->>S: tools/list S-->>C: route, estimate, recommend, explain, list_targets, feedback, ask C->>S: tools/call route "translate this contract" S->>R: route(request) R-->>S: decision + trace S-->>C: target, confidence, alternatives, reasons Note over C,R: ask also runs the chosen target and returns the answer, tokens and cost ``` ### What it exposes | Tool | Purpose | Side effects | |---|---|---| | `route` | Pick the best target: decision, confidence, alternatives, signals, ranked candidates with reasons. Takes everything `Router.route` does - `objective` weights (quality, cost, latency, energy, carbon, `quality_floor`), hard `constraints` (region, data boundary, tenant, PII, cost and latency caps, allow / deny lists), `kinds`, `plan`, `top_k`, `exclude` (an agent handing the request back), `context`, `history` and `session_id`; with a marketplace hook also `marketplace: [slug, ...]` - listings considered as candidates for this request only, nothing installed | none | | `estimate` | The quote above for a request | none | | `recommend` | One-paragraph recommendation for a task; `priority` = balanced, cost, quality or speed | none | | `explain` | Human-readable trace: signals, policy filtering, per-strategy breakdown | none | | `list_targets` | The catalogue (optionally one `kind`) with prices, latency, quality prior, boundary | none | | `feedback` | Report an outcome (`request_id`, `target_id`, `success`, `quality`, `cost_usd`, `latency_ms`) | learners update | | `ask` | Route **and** run the chosen target / plan (same `objective`, `constraints`, `history`, `context`, `session_id` as `route`); returns the answer, tokens, cost and the plan's steps | executes | | `marketplace_search`, `marketplace_get` | Search listings; fetch one with its manifest and snippets (`install: true` also loads it into a local router's catalogue) | installs on request | Resources: `osr://targets` (the catalogue as JSON) and `osr://stats` (feedback statistics). The server speaks MCP `2025-06-18` (also accepts `2025-03-26` and `2024-11-05`) over JSON-RPC 2.0: `initialize`, `ping`, `tools/list`, `tools/call`, `resources/list`, `resources/read`, `prompts/list`, `logging/setLevel` (validated, applied to the package logger); notifications get no reply; unknown methods return `-32601`, unknown tools `-32602`, and every tool failure - invalid arguments, *no admissible target*, a provider that is down, a marketplace listing that does not exist - comes back as an `isError` result the model can read and act on, never as a protocol error. `ask` and `marketplace_*` appear only when the server can honour them: `osr serve` and the platform provide both; `osr mcp` (local, stdio) offers `ask` when something in the catalogue can run - an LLM with a chat handler (`OSR_LLM_BASE_URL`) or a bound manifest endpoint such as an A2A agent or MCP server synced from the marketplace - and the marketplace tools through `MarketplaceClient.mcp_hook` (`--no-marketplace` leaves them out); a bare `MCPServer(router)` exposes the read-only tools and `feedback`. ### Hosted platform `GET /mcp` describes the server (tools, resources, protocol version, connection snippets) without a key. `POST /mcp` takes one JSON-RPC message or a batch with the same `Authorization: Bearer ` (or `X-API-Key`) header as the REST API; notifications return `202`. Every routed call carries the workspace tenant and is metered as `mcp` against the plan's quotas; `feedback` needs the feedback feature and `ask` the plan feature (Pro and above). Quota, plan and routing errors surface as tool errors such as `429: rate limit exceeded`. *Dashboard -> Integrations* renders ready-to-paste configuration for each client with the workspace's key. HTTP transport (the client talks to `https:///mcp` directly): ```jsonc // VS Code: .vscode/mcp.json {"servers": {"opensmartroute": {"type": "http", "url": "https:///mcp", "headers": {"Authorization": "Bearer osr_..."}}}} // Cursor: .cursor/mcp.json {"mcpServers": {"opensmartroute": {"url": "https:///mcp", "headers": {"Authorization": "Bearer osr_..."}}}} // Windsurf: ~/.codeium/windsurf/mcp_config.json {"mcpServers": {"opensmartroute": {"serverUrl": "https:///mcp", "headers": {"Authorization": "Bearer osr_..."}}}} ``` ```bash claude mcp add --transport http opensmartroute https:///mcp --header "Authorization: Bearer osr_..." ``` Stdio bridge for clients that only launch local processes (Claude Desktop and older clients): the CLI forwards each line to the platform with your key. ```json {"mcpServers": {"opensmartroute": {"command": "osr", "args": ["mcp", "--url", "https:///mcp", "--api-key", "osr_..."]}}} ``` #### Streamable HTTP: sessions, streams and notifications The endpoint implements the Streamable HTTP transport of the MCP specification, not only request / reply: * **Sessions.** The reply to `initialize` carries an `Mcp-Session-Id`; send it back on later calls. The id is signed with the deployment secret and bound to the workspace, so every replica accepts it without shared state; an id of another workspace, an expired one (24 hours) or one that was ended answers `404` - re-initialize. `DELETE /mcp` with the header ends the session (`204`). * **Streamed replies.** Send `Accept: text/event-stream` on `POST /mcp` and the answer is a server-sent-events stream: `notifications/progress` for a `_meta.progressToken` (`ask` reports *routing*, *running on ``*, *recording*), `notifications/message` log lines at the level set with `logging/setLevel`, then the JSON-RPC result as the last `message` event. Without that header the reply is plain JSON as before. * **Notification stream.** `GET /mcp` with `Accept: text/event-stream` and your credential opens the server's own stream: `notifications/tools/list_changed` and `notifications/resources/list_changed` when the catalogue, your installed listings or the tools change, a `: ping` comment every 15 seconds, closed after `max_s` seconds (default and ceiling `OSR_PLATFORM_MCP_STREAM_MAX_S`, one hour). Every frame has an `id`; reconnect with `Last-Event-ID` and the stream starts with a `list_changed` so a change during the gap is not missed. `initialize` advertises `listChanged: true` for tools and resources because of it. * **Guards.** A browser `Origin` must be one of the deployment's (the web app, the API's own origin, `OSR_PLATFORM_CORS_ORIGINS`, loopback) - the DNS-rebinding protection the specification asks for; non-browser clients send none. An `MCP-Protocol-Version` header must name a revision the server speaks (`2025-06-18`, `2025-03-26`, `2024-11-05`), else `400`. Bodies are capped at 256 KB. #### Signing in with your identity provider (OAuth 2.1, on-behalf-of) Besides workspace API keys, `/mcp` accepts a bearer token your organization's identity provider issued **for this resource** - the OAuth 2.1 resource-server flow MCP clients implement (VS Code, Claude, Cursor ...): the client is refused with `401` and `WWW-Authenticate: Bearer resource_metadata="https:///.well-known/oauth-protected-resource/mcp"`, reads that document (RFC 9728: `resource`, `authorization_servers`, `scopes_supported`), signs the person in at the authorization server and comes back with a JWT. The platform verifies the signature against the issuer's JWKS (RS256 / PS256 / ES256 families, keys refreshed on rotation), the issuer, the audience (`https:///mcp`, or the app's client id / `api://`), expiry and not-before, then maps the token to the platform user: a linked single sign-on identity whose subject is the token's `sub` or `oid` first, else the user whose address the token names (`preferred_username`, `email`, `upn`; `OSR_PLATFORM_MCP_OAUTH_EMAIL_MATCH=false` turns that off). Someone the platform does not know gets `403` with a pointer to sign in on the site once. The call then runs in the person's workspace, metered and audited like a keyed one, with their role. The endpoint is reachable both on the API host and on the web origin (`https:///mcp`, proxied with the session, protocol-version and stream headers intact); the metadata, the challenge and the accepted audience name whichever URL the client used, and *Dashboard -> Integrations* shows the key-less configuration when the deployment has an identity provider. Configuration defaults from the `microsoft` single sign-on provider (`OSR_PLATFORM_SSO_MICROSOFT_*`): its tenant names the issuer (`https://login.microsoftonline.com//v2.0`; a multi-tenant app keeps Entra's `{tenantid}` placeholder and matches any tenant), its client id the audience, its credentials the on-behalf-of client. Any OIDC issuer works explicitly: | Variable | Meaning | |---|---| | `OSR_PLATFORM_MCP_OAUTH_ISSUERS` | Comma-separated issuers whose tokens are accepted (discovery at `/.well-known/openid-configuration`) | | `OSR_PLATFORM_MCP_OAUTH_AUDIENCES` | Audiences accepted besides `/mcp` | | `OSR_PLATFORM_MCP_OBO_CLIENT_ID`, `OSR_PLATFORM_MCP_OBO_CLIENT_SECRET` | The confidential client the on-behalf-of exchange authenticates as | | `OSR_PLATFORM_MCP_OBO_TOKEN_URL` | Token endpoint for the exchange (default: the token's issuer's) | | `OSR_PLATFORM_MCP_OAUTH_EMAIL_MATCH` | Whether a verified address claim may identify the user (default `true`) | | `OSR_PLATFORM_MCP_STREAM_MAX_S` | Longest `GET /mcp` notification stream (default 3600) | **On-behalf-of.** A model behind the organization's own identity - Azure OpenAI or Azure AI Foundry with Entra authentication, a corporate gateway - can be a provider whose credential is `obo:` instead of a key (`obo:https://cognitiveservices.azure.com/.default`), in the deployment file, the operator console or a workspace's own providers. When a call that arrived with an identity provider's token reaches such a provider, the platform exchanges that token for one with the scope - Entra's `jwt-bearer` grant with `requested_token_use=on_behalf_of` for a Microsoft issuer, the RFC 8693 token exchange for any other - and calls the model with `Authorization: Bearer `: the person's own permissions apply, no shared key exists, and the exchange is cached per user and scope until shortly before it expires and recorded in the audit log (`mcp.obo_exchange`, no token material). A request that came with an API key has nobody to act for: an `obo:` provider is unavailable to it and the fallback loop moves on to the next executable target. The health probe of such a provider checks reachability without a credential. ```jsonc // VS Code: no key in the file - the editor signs you in at the authorization server {"servers": {"opensmartroute": {"type": "http", "url": "https:///mcp"}}} ``` ### Self-hosted and local `osr serve` mounts the same server at `POST /mcp` (and `GET /mcp` for the description) next to `/route` and `/v1`, with `ask` backed by the configured providers - and the same Streamable HTTP transport as the hosted platform: a signed `Mcp-Session-Id` on `initialize` (bound to the access token it was issued to; `DELETE /mcp` ends it), event-stream replies with `notifications/progress` and log messages when the client accepts `text/event-stream`, a `GET /mcp` notification stream that announces `tools/list_changed` / `resources/list_changed` when the catalogue moves (a marketplace sync, a `/targets` write; `max_s` bounds it, ceiling `OSR_SERVER_MCP_STREAM_MAX_S`), the protocol-version check and the browser-origin guard (loopback always passes; `OSR_SERVER_MCP_ORIGINS` names other origins). Without a service, the CLI runs the server over stdio straight from a catalogue - useful for a laptop IDE setup; set `OSR_LLM_BASE_URL` (and `OSR_LLM_API_KEY`) and `ask` runs the routed model for real, exactly as `osr serve` would: ```bash osr -t targets.yaml -r rules.yaml mcp # newline-delimited JSON-RPC on stdin/stdout osr -t targets.yaml mcp --list-tools # what the IDE will see osr -t targets.yaml mcp --marketplace-url https://api.example.test # marketplace_* against another marketplace osr -t targets.yaml mcp --no-marketplace # local tools only osr mcp --url https:///mcp --api-key osr_... # bridge to a hosted platform osr mcp --remote --list-tools # tools of the platform you signed in to (osr login) ``` ```json {"servers": {"opensmartroute": {"type": "stdio", "command": "osr", "args": ["-t", "targets.yaml", "-r", "rules.yaml", "mcp"]}}} ``` When the IDE's environment does not have `osr` on `PATH` (a project virtualenv, `uv run`), use the interpreter instead: `"command": "python", "args": ["-m", "opensmartroute", "-t", "targets.yaml", "mcp"]`. Both transports are exercised in the test suite with the official `mcp` Python client: initialize, `tools/list`, `tools/call`, `resources/list` over stdio against `python -m opensmartroute ... mcp`, and over Streamable HTTP against `osr serve`'s `POST /mcp` with a bearer token. ### In Python ```python from opensmartroute.mcp_server import MCPServer, serve_stdio server = MCPServer(router, execute=my_execute, marketplace=my_marketplace, prices=my_prices) server.tools() # tool specs (tools/list) server.call("estimate", {"text": "…"}) # {"content": [...], "structuredContent": {...}} reply = server.handle( { "jsonrpc": "2.0", "id": 1, "method": "tools/call", "params": {"name": "recommend", "arguments": {"task": "…", "priority": "cost"}}, } ) serve_stdio(server) # block on stdin/stdout ``` `execute(request) -> dict` enables `ask`; `marketplace(action, args)` with `action` in `search` / `get` / `target` (the last returns a `RouteTarget` for a slug and powers the `route` tool's `marketplace` candidates) enables the marketplace tools; `prices` is the quote hook above. `RemoteMCP(url, api_key)` and `bridge_stdio(url, api_key)` are the client side used by `osr mcp --url`. Everything is standard library only; `osr serve` needs the `server` extra (FastAPI). ### Importing other MCP servers as targets The router is also an MCP *client*: a server's tools, resources and prompts become routable targets. ```python from opensmartroute.adapters import connect_mcp, connect_mcp_http client, targets = connect_mcp(["npx", "-y", "@modelcontextprotocol/server-filesystem", "."], resources=True) http, remote = connect_mcp_http("https://tools.example.com/mcp", api_key="...", resources=True, prompts=True) for t in [*targets, *remote]: registry.upsert(t) ``` `connect_mcp` spawns a stdio server; `connect_mcp_http` (`adapters.HttpMCPClient`) speaks the Streamable HTTP transport (JSON or SSE replies, `Mcp-Session-Id`) to remote servers over https. Tools become `tool` targets whose handler runs `tools/call`; with `resources=True` every resource becomes a read-only `tool` target that reads it (`resources_from_mcp`), with `prompts=True` every prompt becomes an instructions-only `skill` slot whose handler renders the template (`prompts_from_mcp`). Tool descriptions are still scored for injection risk at import time and signed manifests still verify as before. ### Things to ask your assistant - "Estimate what this prompt will cost and which model to use." - "Recommend the cheapest model that can refactor this file." - "Explain how OpenSmartRoute would route this request." - "Search the marketplace for a SQL reporting skill and show me its manifest." - "Route and answer this with the best model, then record that the answer was good."