# API reference Every module of the `opensmartroute` package and every public name it exports, generated from the source. The first line of each docstring is the summary; `help()` on a name gives the full contract. Narrative documentation: [GUIDE.md](https://osr-web.kindrock-caaab1a2.centralindia.azurecontainerapps.io/docs/GUIDE.md) (usage), [SDK.md](https://osr-web.kindrock-caaab1a2.centralindia.azurecontainerapps.io/docs/SDK.md) (stability, extension points), [ARCHITECTURE.md](https://osr-web.kindrock-caaab1a2.centralindia.azurecontainerapps.io/docs/ARCHITECTURE.md), [MATH.md](https://osr-web.kindrock-caaab1a2.centralindia.azurecontainerapps.io/docs/MATH.md), [ENTERPRISE.md](https://osr-web.kindrock-caaab1a2.centralindia.azurecontainerapps.io/docs/ENTERPRISE.md), [SECURITY.md](https://osr-web.kindrock-caaab1a2.centralindia.azurecontainerapps.io/docs/SECURITY.md), [RESEARCH.md](https://osr-web.kindrock-caaab1a2.centralindia.azurecontainerapps.io/docs/RESEARCH.md). Stability: names re-exported from `opensmartroute` (the top-level package) are frozen and follow SemVer. Sub-module names are public but may change in a minor release with a CHANGELOG entry. ## Modules - [`opensmartroute`](#opensmartroute) - OpenSmartRoute — an open, intelligent route to the right decision, solution, or destination. (92 names) - [`opensmartroute.__main__`](#opensmartroute__main__) - ``python -m opensmartroute ...`` is the ``osr`` command (IDE MCP configs, containers without the script on PATH). (0 names) - [`opensmartroute.adapters`](#opensmartrouteadapters) - Adapters connect OpenSmartRoute to real providers and infrastructure. (87 names) - [`opensmartroute.adapters.a2a`](#opensmartrouteadaptersa2a) - Import A2A (Agent-to-Agent protocol) **Agent Cards** as ``TargetKind.AGENT`` targets. (5 names) - [`opensmartroute.adapters.anthropic_client`](#opensmartrouteadaptersanthropic_client) - Native Anthropic **Messages API** client - route TO Claude, not only accept Claude-dialect requests. (8 names) - [`opensmartroute.adapters.catalogue`](#opensmartrouteadapterscatalogue) - Live model catalogue: collect model cards (price, context, modalities, benchmarks) from public sources. (11 names) - [`opensmartroute.adapters.frameworks`](#opensmartrouteadaptersframeworks) - Drop-in nodes for agent frameworks. (7 names) - [`opensmartroute.adapters.handlers`](#opensmartrouteadaptershandlers) - Executors for non-LLM targets: HTTP endpoints, MCP tools and asynchronous queues. (7 names) - [`opensmartroute.adapters.harness`](#opensmartrouteadaptersharness) - Agent-harness adapters: route to a *runtime*, not just a model. (7 names) - [`opensmartroute.adapters.marketplace`](#opensmartrouteadaptersmarketplace) - Pull skills, agents, tools and templates from an OpenSmartRoute marketplace into a router. (8 names) - [`opensmartroute.adapters.mcp`](#opensmartrouteadaptersmcp) - Import MCP (Model Context Protocol) tools as ``TargetKind.TOOL`` targets. (15 names) - [`opensmartroute.adapters.mcp_servers`](#opensmartrouteadaptersmcp_servers) - MCP *server* recommendation (MCP-Zero 2506.01056; ToolRet 2603.06467). (3 names) - [`opensmartroute.adapters.mock_client`](#opensmartrouteadaptersmock_client) - A provider that answers deterministically without a network: the ``mock`` provider kind. (3 names) - [`opensmartroute.adapters.openai_compat`](#opensmartrouteadaptersopenai_compat) - OpenAI-compatible HTTP client (stdlib only). (10 names) - [`opensmartroute.adapters.optional`](#opensmartrouteadaptersoptional) - Optional adapters that need extra dependencies. Everything is lazily imported so the. (3 names) - [`opensmartroute.adapters.personas`](#opensmartrouteadapterspersonas) - Import persona catalogues as ``TargetKind.PERSONA`` targets. (4 names) - [`opensmartroute.adapters.semantic_router`](#opensmartrouteadapterssemantic_router) - Import a vLLM *semantic-router* configuration (vllm-project/semantic-router). (3 names) - [`opensmartroute.adapters.skills`](#opensmartrouteadaptersskills) - Load Agent-Skills ``SKILL.md`` packages as ``TargetKind.SKILL`` targets. (4 names) - [`opensmartroute.adapters.websearch`](#opensmartrouteadapterswebsearch) - Web knowledge for the self-improving router: stdlib HTTP fetch, search providers, page text. (20 names) - [`opensmartroute.agents`](#opensmartrouteagents) - ``osr run ``: launch a coding agent or AI client with the router as its model endpoint. (5 names) - [`opensmartroute.aio`](#opensmartrouteaio) - Async façade. Routing itself is CPU-bound and sub-millisecond, so we run it in. (1 names) - [`opensmartroute.anthropic`](#opensmartrouteanthropic) - Anthropic **Messages API** (``POST /v1/messages``) on top of the chat-completions proxy. (9 names) - [`opensmartroute.branding`](#opensmartroutebranding) - OpenSmartRoute naming conventions: one place for every brand-bound identifier. (28 names) - [`opensmartroute.cli`](#opensmartroutecli) - ``osr`` command-line interface. (5 names) - [`opensmartroute.client`](#opensmartrouteclient) - Client for the hosted platform and a self-hosted ``osr serve``: route, quote, report, chat - standard library only. (10 names) - [`opensmartroute.compress`](#opensmartroutecompress) - Prompt compaction: fewer tokens into the model without changing what it is asked. (4 names) - [`opensmartroute.config`](#opensmartrouteconfig) - Configuration loading: targets and rules from JSON or YAML. (3 names) - [`opensmartroute.core`](#opensmartroutecore) - see module (17 names) - [`opensmartroute.core.registry`](#opensmartroutecoreregistry) - Target registry: the catalogue of everything a request may be routed to. (1 names) - [`opensmartroute.core.types`](#opensmartroutecoretypes) - Core data model for OpenSmartRoute. (17 names) - [`opensmartroute.credentials`](#opensmartroutecredentials) - Credentials for the ``osr`` CLI: where the access token lives and how it is obtained. (18 names) - [`opensmartroute.data`](#opensmartroutedata) - Data files shipped inside the package. (0 names) - [`opensmartroute.discovery`](#opensmartroutediscovery) - Tool discovery beyond text similarity. (5 names) - [`opensmartroute.enterprise`](#opensmartrouteenterprise) - Enterprise integration layer: ports (hexagonal architecture), middleware and telemetry. (22 names) - [`opensmartroute.enterprise.grounding`](#opensmartrouteenterprisegrounding) - Live web context in front of the model: search the web for requests about *now* and hand the hits to the target. (4 names) - [`opensmartroute.enterprise.ops`](#opensmartrouteenterpriseops) - Operational controls: shadow / A-B routing, tenant fairness and queue-aware latency. (7 names) - [`opensmartroute.enterprise.savings`](#opensmartrouteenterprisesavings) - Savings ledger - the always-on savings report that backs the ROI story and the dashboard. (3 names) - [`opensmartroute.enterprise.stores`](#opensmartrouteenterprisestores) - Production state-store backends and wrappers. (8 names) - [`opensmartroute.errors`](#opensmartrouteerrors) - Exception hierarchy. Every error raised by the SDK derives from :class:`OpenSmartRouteError`. (13 names) - [`opensmartroute.estimate`](#opensmartrouteestimate) - Token, cost and latency estimates *before* a request is sent anywhere. (13 names) - [`opensmartroute.eval`](#opensmartrouteeval) - RouterBench-style evaluation harness. (21 names) - [`opensmartroute.eval.agentic`](#opensmartrouteevalagentic) - tau-bench-style agentic task evaluation: does per-step routing beat the best single agent?. (4 names) - [`opensmartroute.eval.audit`](#opensmartrouteevalaudit) - Routing Audit - shadow-mode replay of logged LLM traffic to quantify what the router would change. (6 names) - [`opensmartroute.eval.baselines`](#opensmartrouteevalbaselines) - Baselines every router must beat, plus oracle ceilings and the sampling noise floor. (15 names) - [`opensmartroute.eval.collect`](#opensmartrouteevalcollect) - Collect routing datasets from the Hugging Face Hub, your own feedback log and synthetic seeds. (32 names) - [`opensmartroute.eval.criteria`](#opensmartrouteevalcriteria) - Offline realisations of the release exit criteria. (15 names) - [`opensmartroute.eval.datasets`](#opensmartrouteevaldatasets) - Adapters from public routing benchmarks to :class:`EvalRow`. (7 names) - [`opensmartroute.eval.frontier`](#opensmartrouteevalfrontier) - Three-objective frontier and ablations. (3 names) - [`opensmartroute.eval.headroom`](#opensmartrouteevalheadroom) - Routing headroom: when does routing pay, and how much catalogue does it need?. (7 names) - [`opensmartroute.eval.ope`](#opensmartrouteevalope) - Off-policy evaluation from logged routing decisions. (6 names) - [`opensmartroute.eval.robustness`](#opensmartrouteevalrobustness) - Robustness and fairness checks for a router. (6 names) - [`opensmartroute.execution`](#opensmartrouteexecution) - Plan-aware execution: turn a :class:`RouteDecision` into a real answer. (17 names) - [`opensmartroute.feedback`](#opensmartroutefeedback) - Feedback store: append-only outcome log that closes the learning loop. (1 names) - [`opensmartroute.gemini`](#opensmartroutegemini) - Google Gemini API dialect: ``generateContent`` / ``streamGenerateContent`` over the routed chat path. (9 names) - [`opensmartroute.inference`](#opensmartrouteinference) - Run open-weight models on the CPU and RAM you have: footprint arithmetic, host planning, Ollama control. (25 names) - [`opensmartroute.inference.footprint`](#opensmartrouteinferencefootprint) - What an open-weight model needs from a CPU host: memory for weights and KV cache, and the tokens per second. (14 names) - [`opensmartroute.inference.ollama`](#opensmartrouteinferenceollama) - Talk to an Ollama server's control API: which models it has, their exact shape, which are resident, load and unload. (4 names) - [`opensmartroute.inference.planner`](#opensmartrouteinferenceplanner) - Place models on hosts and write the server settings that make them fit and stay warm. (7 names) - [`opensmartroute.jsonrepair`](#opensmartroutejsonrepair) - Repair the JSON a model almost produced. (3 names) - [`opensmartroute.knowledge`](#opensmartrouteknowledge) - Knowledge management and context engineering: a knowledge graph the router learns into and reads from. (21 names) - [`opensmartroute.knowledge.context`](#opensmartrouteknowledgecontext) - Context engineering: what the model should know for *this* request, chosen and trimmed to a token budget. (6 names) - [`opensmartroute.knowledge.graph`](#opensmartrouteknowledgegraph) - The knowledge graph: typed facts with provenance, in memory, in one JSONL file, exportable to the open formats. (7 names) - [`opensmartroute.knowledge.ingest`](#opensmartrouteknowledgeingest) - Turning what the router sees into facts: catalogue, conversations, decisions, outcomes and live web hits. (8 names) - [`opensmartroute.learning`](#opensmartroutelearning) - Auto-learning strategies built on :mod:`opensmartroute.math`. (71 names) - [`opensmartroute.learning.attention`](#opensmartroutelearningattention) - A pure-Python transformer block for the routing SLM's query encoder. (2 names) - [`opensmartroute.learning.autopilot`](#opensmartroutelearningautopilot) - Self-operation: the routing SLM runs its own improvement loop inside the live process. (2 names) - [`opensmartroute.learning.coldstart`](#opensmartroutelearningcoldstart) - Cold start for new targets and self-improving target descriptions. (8 names) - [`opensmartroute.learning.contrastive`](#opensmartroutelearningcontrastive) - Contrastive and reward-distilled router training (RouterDC, NeurIPS 2024; Zooter 2311.08692). (5 names) - [`opensmartroute.learning.credit`](#opensmartroutelearningcredit) - Delayed, task-level credit assignment for agentic trajectories. (2 names) - [`opensmartroute.learning.embed`](#opensmartroutelearningembed) - Pretrained transformer embeddings as frozen features for the routing SLM. (3 names) - [`opensmartroute.learning.handoff`](#opensmartroutelearninghandoff) - Permanent-handoff policy from censored teacher signals (TACIT-Switch 2608.27911). (3 names) - [`opensmartroute.learning.implicit`](#opensmartroutelearningimplicit) - Implicit feedback: outcomes inferred from the answer itself and from what the user does next. (5 names) - [`opensmartroute.learning.kge`](#opensmartroutelearningkge) - A learned link predictor for the knowledge graph (optional extra ``tensorflow`` to train; numpy to use). (1 names) - [`opensmartroute.learning.multiturn`](#opensmartroutelearningmultiturn) - Multi-turn routing with history-target joint embeddings (MTRouter 2604.23530). (3 names) - [`opensmartroute.learning.neural`](#opensmartroutelearningneural) - TensorFlow / Keras models that sharpen two things the router does on every request (optional extra ``tensorflow``). (7 names) - [`opensmartroute.learning.personal`](#opensmartroutelearningpersonal) - Few-shot personalisation (GMTRouter 2511.08590; SkillFeed 2608.28241). (4 names) - [`opensmartroute.learning.policy_gradient`](#opensmartroutelearningpolicy_gradient) - End-to-end policy-gradient routing (Router-R1 2506.09033; RLCascadeRouter 2608.15817). (4 names) - [`opensmartroute.learning.quant`](#opensmartroutelearningquant) - Weight quantisation for the routing SLM's model file. (5 names) - [`opensmartroute.learning.self_improve`](#opensmartroutelearningself_improve) - Self-improvement loop: refresh the catalogue, gather evidence, train a challenger, promote it only if better. (3 names) - [`opensmartroute.learning.slm`](#opensmartroutelearningslm) - The OpenSmartRoute routing SLM: a small, self-contained model that picks the target for a prompt. (16 names) - [`opensmartroute.learning.window`](#opensmartroutelearningwindow) - The reading window of the routing SLM over a long prompt. (6 names) - [`opensmartroute.math`](#opensmartroutemath) - Mathematical toolkit behind OpenSmartRoute's decisions. (46 names) - [`opensmartroute.math.bandits`](#opensmartroutemathbandits) - Multi-armed and contextual bandits for online routing decisions. (8 names) - [`opensmartroute.math.calibration`](#opensmartroutemathcalibration) - Calibration and distribution-free risk control for routing confidence. (6 names) - [`opensmartroute.math.decision`](#opensmartroutemathdecision) - Multi-objective decision helpers and queueing theory for real-time routing. (9 names) - [`opensmartroute.math.dirichlet`](#opensmartroutemathdirichlet) - Dirichlet probe over host hidden states (ProbeDirichlet, RouterXBench 2602.11877). (2 names) - [`opensmartroute.math.energy`](#opensmartroutemathenergy) - Hardware-aware energy characterisation (HW-Router 2608.14575; 2608.28044). (3 names) - [`opensmartroute.math.estimators`](#opensmartroutemathestimators) - Streaming estimators, drift detection and calibration. (11 names) - [`opensmartroute.math.irt`](#opensmartroutemathirt) - Item Response Theory for routing (IRT-Router, ACL 2025). (3 names) - [`opensmartroute.math.markov`](#opensmartroutemathmarkov) - Markov chains and MDPs for conversational / multi-step routing. (2 names) - [`opensmartroute.math.preference`](#opensmartroutemathpreference) - Bradley–Terry pairwise preference model (RouteLLM, Prompt-to-Leaderboard). (2 names) - [`opensmartroute.mcp_server`](#opensmartroutemcp_server) - Model Context Protocol server: the router as a set of tools for any IDE or agent. (15 names) - [`opensmartroute.media`](#opensmartroutemedia) - Speech and image adapters for the OpenAI-compatible proxy (``osr serve`` and the hosted platform). (20 names) - [`opensmartroute.migrate`](#opensmartroutemigrate) - Import another gateway's configuration as a routing catalogue. (6 names) - [`opensmartroute.observability`](#opensmartrouteobservability) - Tracing and observability: every step of routing, execution and learning as a structured event. (20 names) - [`opensmartroute.ocm`](#opensmartrouteocm) - Open Capability Manifest (OCM) - a vendor-neutral description of any routable capability. (12 names) - [`opensmartroute.policy`](#opensmartroutepolicy) - Policy layer: hard constraints that are never traded off against utility. (20 names) - [`opensmartroute.realtime`](#opensmartrouterealtime) - Real-time operational controls: health, circuit breaking, rate & budget limits. (11 names) - [`opensmartroute.responses`](#opensmartrouteresponses) - OpenAI **Responses API** (``POST /v1/responses``) on top of the chat-completions proxy. (7 names) - [`opensmartroute.retrieval`](#opensmartrouteretrieval) - Retrieval-based candidate narrowing for very large target pools (ToolRet / Skill-RAG). (12 names) - [`opensmartroute.router`](#opensmartrouterouter) - The Router: signals -> policy -> strategies -> ensemble -> decision (+plan). (3 names) - [`opensmartroute.sdk`](#opensmartroutesdk) - OpenSmartRoute SDK: decorator-driven registration of routing components. (34 names) - [`opensmartroute.security`](#opensmartroutesecurity) - Security controls for the routing control plane. (33 names) - [`opensmartroute.security.gadget`](#opensmartroutesecuritygadget) - Learned confounder-gadget detector (Rerouting LLM Routers, Shafran et al. 2025). (6 names) - [`opensmartroute.security.injection`](#opensmartroutesecurityinjection) - Instruction-injection detection for *text that is not the user's request*. (4 names) - [`opensmartroute.security.limits`](#opensmartroutesecuritylimits) - Resource-amplification limits ("Beyond Max Tokens"). (8 names) - [`opensmartroute.security.moderation`](#opensmartroutesecuritymoderation) - ``POST /v1/moderations`` over the router's own guards. (5 names) - [`opensmartroute.security.output`](#opensmartroutesecurityoutput) - Response-side guard: what a target answered is screened before it leaves the router. (8 names) - [`opensmartroute.security.provenance`](#opensmartroutesecurityprovenance) - Origin (provenance) policy for tool parameters – ROPE-style control-flow integrity. (12 names) - [`opensmartroute.security.safety`](#opensmartroutesecuritysafety) - Safety-routing regression suite ("When Safety Routing Breaks"). (4 names) - [`opensmartroute.server`](#opensmartrouteserver) - Optional FastAPI server exposing the router over HTTP. (23 names) - [`opensmartroute.settings`](#opensmartroutesettings) - Typed, environment-overridable settings: the only home for OpenSmartRoute's tunable constants. (21 names) - [`opensmartroute.signals`](#opensmartroutesignals) - Signal extraction: cheap, deterministic features computed from the request. (50 names) - [`opensmartroute.signals.corpus`](#opensmartroutesignalscorpus) - Natural-language prompts per task type: how people actually ask, not how the ontology templates phrase it. (1 names) - [`opensmartroute.signals.events`](#opensmartroutesignalsevents) - Event- and workflow-driven signals for agentic inputs. (7 names) - [`opensmartroute.signals.models`](#opensmartroutesignalsmodels) - Learned signal models: hashed n-gram linear models with no dependencies. (7 names) - [`opensmartroute.signals.ontology`](#opensmartroutesignalsontology) - Task ontology: families -> types -> subtypes, with an orthogonal *domain* axis. (4 names) - [`opensmartroute.signals.uncertainty`](#opensmartroutesignalsuncertainty) - Uncertainty signals that come from *outside the query text*. (12 names) - [`opensmartroute.stack`](#opensmartroutestack) - Declarative *stacks*: one document that describes a whole routing setup. (13 names) - [`opensmartroute.strategies`](#opensmartroutestrategies) - see module (67 names) - [`opensmartroute.strategies.aggregate`](#opensmartroutestrategiesaggregate) - Routing / aggregation switch (Mixture-of-Agents, Wang et al. 2024; JiSi 2601.01330). (5 names) - [`opensmartroute.strategies.auction`](#opensmartroutestrategiesauction) - Error-aware reverse auction across providers (EA-RAM 2608.12719). (5 names) - [`opensmartroute.strategies.bandit`](#opensmartroutestrategiesbandit) - Online-learning strategy: contextual Thompson-sampling bandit. (1 names) - [`opensmartroute.strategies.base`](#opensmartroutestrategiesbase) - Strategy interface. (3 names) - [`opensmartroute.strategies.capability`](#opensmartroutestrategiescapability) - Capability-fit strategy: match request signals to a target's declared capabilities. (3 names) - [`opensmartroute.strategies.cascade`](#opensmartroutestrategiescascade) - Cascade execution (FrugalGPT / Router-R1 multi-round / AutoMix POMDP). (5 names) - [`opensmartroute.strategies.defer`](#opensmartroutestrategiesdefer) - Defer-to-human and effort ("think or not") strategies. (3 names) - [`opensmartroute.strategies.edge`](#opensmartroutestrategiesedge) - Edge-cloud token-aware routing (Pro-Router 2608.28726; RelayLLM 2601.05167). (1 names) - [`opensmartroute.strategies.elastic`](#opensmartroutestrategieselastic) - Token-budget-aware routing to elastic / many-in-one models (Nemotron Elastic 2511.16664; Star 2605.07182). (3 names) - [`opensmartroute.strategies.escalation`](#opensmartroutestrategiesescalation) - Bayesian self-escalation during generation (2608.24087). (5 names) - [`opensmartroute.strategies.human`](#opensmartroutestrategieshuman) - Routing among human annotators and experts (QUORUM 2608.27974; Dawid-Skene 1979). (5 names) - [`opensmartroute.strategies.llm_judge`](#opensmartroutestrategiesllm_judge) - Generative routing: let an LLM act as the router (Router-R1 / LLM-as-judge). (2 names) - [`opensmartroute.strategies.memory`](#opensmartroutestrategiesmemory) - Memory-tier routing for agents (BudgetMem 2602.06025; Gated-Memory Routing 2609.00237). (5 names) - [`opensmartroute.strategies.modality`](#opensmartroutestrategiesmodality) - Multimodal routing and modality escalation (LatentRouter 2605.11301; modality escalation. (4 names) - [`opensmartroute.strategies.probe`](#opensmartroutestrategiesprobe) - Hidden-state routing with a Dirichlet probe (ProbeDirichlet, RouterXBench 2602.11877). (2 names) - [`opensmartroute.strategies.progress`](#opensmartroutestrategiesprogress) - Agentic trajectories: per-step routing and multi-round execution. (5 names) - [`opensmartroute.strategies.protocol`](#opensmartroutestrategiesprotocol) - Collaboration-protocol selection (2608.14927). (7 names) - [`opensmartroute.strategies.rules`](#opensmartroutestrategiesrules) - Declarative rule-based routing (Arch-Router-style domain/action preferences). (2 names) - [`opensmartroute.strategies.semantic_cache`](#opensmartroutestrategiessemantic_cache) - Semantic caching as a routing target (GPTCache; vLLM semantic router 2603.04444). (4 names) - [`opensmartroute.strategies.session`](#opensmartroutestrategiessession) - Session affinity: keep a conversation with the target that is already serving it. (2 names) - [`opensmartroute.strategies.similarity`](#opensmartroutestrategiessimilarity) - Similarity-based routing (UniRoute / GraphRouter flavour). (3 names) - [`opensmartroute.strategies.speculative`](#opensmartroutestrategiesspeculative) - Speculative (draft-based) cascades (speculative cascades, Narasimhan et al. 2024; Differential. (3 names) - [`opensmartroute.strategies.spread`](#opensmartroutestrategiesspread) - Weighted spread across interchangeable deployments (LiteLLM "simple-shuffle", OpenRouter provider weights). (1 names) - [`opensmartroute.strategies.task_table`](#opensmartroutestrategiestask_table) - Static task table strategy (SCX Router, 2609.02292). (1 names) ## `opensmartroute` OpenSmartRoute — an open, intelligent route to the right decision, solution, or destination. | Name | Kind | Summary | |---|---|---| | `APIError` | re-export of [`opensmartroute.errors.APIError`](#opensmartrouteerrors) | The platform or ``osr serve`` answered a call with an error status; ``status``, ``detail`` and the. | | `AsyncRouter` | re-export of [`opensmartroute.aio.AsyncRouter`](#opensmartrouteaio) | ``await``-able wrapper around a Router or EnterpriseRouter: route / learn / execute / run off the event loop. | | `AuthenticationError` | re-export of [`opensmartroute.errors.AuthenticationError`](#opensmartrouteerrors) | The CLI has no valid access token for the platform / server, or a sign-in was denied or timed out. | | `BanditStrategy` | re-export of [`opensmartroute.strategies.bandit.BanditStrategy`](#opensmartroutestrategiesbandit) | Thompson-sampling Beta bandit per (context, target); context = dominant domain (+ plan role). | | `Capabilities` | re-export of [`opensmartroute.core.types.Capabilities`](#opensmartroutecoretypes) | Declarative description of what a target is good at. | | `CapabilityStrategy` | re-export of [`opensmartroute.strategies.capability.CapabilityStrategy`](#opensmartroutestrategiescapability) | Declarative fit: domain / action overlap, complexity band, language, modality and quality prior. | | `Cascade` | re-export of [`opensmartroute.strategies.cascade.Cascade`](#opensmartroutestrategiescascade) | Execute ranked targets in planner order (cheapest / MDP / POMDP), stopping when the quality gate passes. | | `CascadePlanner` | re-export of [`opensmartroute.strategies.cascade.CascadePlanner`](#opensmartroutestrategiescascade) | Finite-horizon MDP over an ordered cascade with a stop action after each step. | | `ComponentRegistry` | re-export of [`opensmartroute.sdk.ComponentRegistry`](#opensmartroutesdk) | Blueprints for every routing component, with decorators that register into it. | | `ConfigurationError` | re-export of [`opensmartroute.errors.ConfigurationError`](#opensmartrouteerrors) | Invalid catalogue, rules, SKILL.md, settings or a missing optional dependency. | | `ContextEngineer` | re-export of [`opensmartroute.knowledge.context.ContextEngineer`](#opensmartrouteknowledgecontext) | Builds the engineered context (module docstring). ``budget_tokens`` / ``max_facts`` / ``max_web_hits``. | | `ContextMiddleware` | re-export of [`opensmartroute.knowledge.context.ContextMiddleware`](#opensmartrouteknowledgecontext) | Run the :class:`ContextEngineer` in front of every routed request and teach the graph from the traffic. | | `DeferStrategy` | re-export of [`opensmartroute.strategies.defer.DeferStrategy`](#opensmartroutestrategiesdefer) | Learning-to-defer: scores human targets by risk, PII, escalation intent, frustration and model uncertainty. | | `EffortStrategy` | re-export of [`opensmartroute.strategies.defer.EffortStrategy`](#opensmartroutestrategiesdefer) | Match a target's reasoning ``effort_level`` to ``signals.reasoning_need``; penalise over- and under-thinking. | | `Event` | re-export of [`opensmartroute.observability.Event`](#opensmartrouteobservability) | One captured span or event: flat, JSON-friendly, never carries request text. | | `EventSink` | re-export of [`opensmartroute.observability.EventSink`](#opensmartrouteobservability) | Receiver port: override :meth:`emit`; live bridges may also implement :meth:`span_start` / :meth:`span_end`. | | `ExecutionError` | re-export of [`opensmartroute.errors.ExecutionError`](#opensmartrouteerrors) | A target handler failed while executing a plan. | | `ExecutionResult` | re-export of [`opensmartroute.execution.ExecutionResult`](#opensmartrouteexecution) | What happened when a decision was executed. | | `ExecutionStep` | re-export of [`opensmartroute.execution.ExecutionStep`](#opensmartrouteexecution) | One executed plan slot: role, target, latency and whether it succeeded. ``stage`` orders the steps. | | `FeedbackStore` | re-export of [`opensmartroute.feedback.FeedbackStore`](#opensmartroutefeedback) | Append-only :class:`Outcome` log (in memory or JSONL file) with per-target statistics. | | `KnowledgeGraph` | re-export of [`opensmartroute.knowledge.graph.KnowledgeGraph`](#opensmartrouteknowledgegraph) | Facts indexed three ways, upserted on their key, persisted as JSONL (see the module docstring). | | `KnowledgeStrategy` | re-export of [`opensmartroute.knowledge.context.KnowledgeStrategy`](#opensmartrouteknowledgecontext) | Scores candidates by what the graph knows worked on this kind of task: the ``performed_on`` facts of each. | | `LLMJudgeStrategy` | re-export of [`opensmartroute.strategies.llm_judge.LLMJudgeStrategy`](#opensmartroutestrategiesllm_judge) | LLM-as-router with optional **score calibration**. | | `NoRouteError` | re-export of [`opensmartroute.errors.NoRouteError`](#opensmartrouteerrors) | No target satisfied the hard constraints. | | `Objective` | re-export of [`opensmartroute.core.types.Objective`](#opensmartroutecoretypes) | What the caller wants to optimise. Weights are relative. | | `OpenSmartRoute` | re-export of [`opensmartroute.client.OpenSmartRoute`](#opensmartrouteclient) | A client for the platform (``https://opensmartroute.ai`` or your own deployment) and for ``osr serve``. | | `OpenSmartRouteError` | re-export of [`opensmartroute.errors.OpenSmartRouteError`](#opensmartrouteerrors) | Base class for all SDK errors. | | `Outcome` | re-export of [`opensmartroute.core.types.Outcome`](#opensmartroutecoretypes) | Feedback about how a routed request actually went. | | `PlanSlot` | re-export of [`opensmartroute.core.types.PlanSlot`](#opensmartroutecoretypes) | One filled slot of a multi-target plan (persona -> skill -> model). | | `Policy` | re-export of [`opensmartroute.policy.Policy`](#opensmartroutepolicy) | Ordered chain of :data:`PolicyRule`; returns the first rejection reason or ``None``. | | `ProgressRouter` | re-export of [`opensmartroute.strategies.progress.ProgressRouter`](#opensmartroutestrategiesprogress) | Route each step of a task with trajectory context. | | `RankedTarget` | re-export of [`opensmartroute.core.types.RankedTarget`](#opensmartroutecoretypes) | A scored candidate: utility, ensemble quality estimate and the per-strategy breakdown. | | `RateLimitError` | re-export of [`opensmartroute.errors.RateLimitError`](#opensmartrouteerrors) | 429: the rate limit, the monthly quota or a budget is exhausted; ``retry_after`` says when to try again. | | `RemoteDecision` | re-export of [`opensmartroute.client.RemoteDecision`](#opensmartrouteclient) | What ``route()`` returns: the decision as the server sent it (``raw``) with the fields people read first. | | `RequestConstraints` | re-export of [`opensmartroute.core.types.RequestConstraints`](#opensmartroutecoretypes) | Hard constraints on the request (never traded off). | | `RouteDecision` | re-export of [`opensmartroute.core.types.RouteDecision`](#opensmartroutecoretypes) | The answer to ``route()``: chosen target, confidence, alternatives, optional plan, trace and propensities. | | `RoutePlan` | re-export of [`opensmartroute.core.types.RoutePlan`](#opensmartroutecoretypes) | A composed route (MasRouter-style): several targets working together, executed in stages. | | `RouteRequest` | re-export of [`opensmartroute.core.types.RouteRequest`](#opensmartroutecoretypes) | The customer need. | | `RouteTarget` | re-export of [`opensmartroute.core.types.RouteTarget`](#opensmartroutecoretypes) | A routable destination: an LLM, agent, skill, persona, tool, workflow or human. | | `RouteTrace` | re-export of [`opensmartroute.core.types.RouteTrace`](#opensmartroutecoretypes) | Everything needed to explain a decision. | | `Router` | re-export of [`opensmartroute.router.Router`](#opensmartrouterouter) | Signals -> policy -> strategies -> ensemble -> decision (+ optional plan). | | `RouterSLM` | re-export of [`opensmartroute.learning.slm.RouterSLM`](#opensmartroutelearningslm) | Small routing model: dual encoder + target catalogue snapshot + calibration, in one JSON file. | | `Rule` | re-export of [`opensmartroute.strategies.rules.Rule`](#opensmartroutestrategiesrules) | If all `when` conditions match, boost `prefer` targets and penalise `avoid`. | | `RulesStrategy` | re-export of [`opensmartroute.strategies.rules.RulesStrategy`](#opensmartroutestrategiesrules) | Applies declarative :class:`Rule` preferences (prefer / avoid / pin) when their ``when`` conditions match. | | `SLMStrategy` | re-export of [`opensmartroute.learning.slm.SLMStrategy`](#opensmartroutelearningslm) | Ensemble member backed by a :class:`RouterSLM`: scores are its probabilities, it keeps learning online. | | `SecurityError` | re-export of [`opensmartroute.errors.SecurityError`](#opensmartrouteerrors) | Request rejected by an input guard (prompt injection, oversize, etc.). | | `SelfImprover` | re-export of [`opensmartroute.learning.self_improve.SelfImprover`](#opensmartroutelearningself_improve) | Closed loop that keeps a :class:`RouterSLM` current with the model market and its own traffic. | | `Settings` | re-export of [`opensmartroute.settings.Settings`](#opensmartroutesettings) | All tunables, grouped by consumer. Immutable; derive variants with :meth:`replace`. | | `Signals` | re-export of [`opensmartroute.core.types.Signals`](#opensmartroutecoretypes) | Cheap deterministic features extracted from a request. | | `SimilarityStrategy` | re-export of [`opensmartroute.strategies.similarity.SimilarityStrategy`](#opensmartroutestrategiessimilarity) | Embed the request and each target's examples / description; score by best and top-k mean similarity. | | `Span` | re-export of [`opensmartroute.observability.Span`](#opensmartrouteobservability) | An open unit of work; a context manager that records duration, status and nested events. | | `StateStoreError` | re-export of [`opensmartroute.errors.StateStoreError`](#opensmartrouteerrors) | A learner-state store failed to load, save or migrate. | | `Strategy` | re-export of [`opensmartroute.strategies.base.Strategy`](#opensmartroutestrategiesbase) | Scores each candidate target in [0, 1] and explains why. | | `StrategyScore` | re-export of [`opensmartroute.core.types.StrategyScore`](#opensmartroutecoretypes) | One strategy's opinion about one target. | | `TargetConstraints` | re-export of [`opensmartroute.core.types.TargetConstraints`](#opensmartroutecoretypes) | Where / for whom a target may be used. Checked by the policy layer. | | `TargetKind` | re-export of [`opensmartroute.core.types.TargetKind`](#opensmartroutecoretypes) | Kinds of things a request can be routed to. | | `TargetRegistry` | re-export of [`opensmartroute.core.registry.TargetRegistry`](#opensmartroutecoreregistry) | In-memory catalogue of :class:`RouteTarget` by id: add / upsert / remove, filtered listing, (de)serialisation. | | `TargetUnavailableError` | re-export of [`opensmartroute.errors.TargetUnavailableError`](#opensmartrouteerrors) | A remote target or catalogue source could not be reached. | | `TaskTableStrategy` | re-export of [`opensmartroute.strategies.task_table.TaskTableStrategy`](#opensmartroutestrategiestask_table) | Static ``task_type -> target -> quality`` table with family and prior fallbacks; learns from outcomes. | | `Tracer` | re-export of [`opensmartroute.observability.Tracer`](#opensmartrouteobservability) | Opens spans, records events and fans them out to sinks; ``sample_rate`` < 1 traces a share of requests. | | `ValidationError` | re-export of [`opensmartroute.errors.ValidationError`](#opensmartrouteerrors) | A request, outcome or target failed validation. | | `__version__` | constant | | | `agent` | re-export of [`opensmartroute.sdk.agent`](#opensmartroutesdk) | @agent(id, ...): shorthand for an ``agent`` target. | | `aroute` | re-export of [`opensmartroute.sdk.aroute`](#opensmartroutesdk) | await aroute(text): the async form of route(). | | `arun` | re-export of [`opensmartroute.sdk.arun`](#opensmartroutesdk) | await arun(text): the async form of run(); coroutine handlers are awaited. | | `components` | re-export of [`opensmartroute.sdk.components`](#opensmartroutesdk) | : The process-wide registry that the top-level decorators (``opensmartroute.strategy`` ...) bind to. | | `configure` | re-export of [`opensmartroute.settings.configure`](#opensmartroutesettings) | Install process-wide settings. ``configure()`` with no arguments re-reads the environment;. | | `configure_tracing` | re-export of [`opensmartroute.observability.configure_tracing`](#opensmartrouteobservability) | Install sinks on the process-wide tracer. With no sinks, use the ones named by settings. | | `feedback` | re-export of [`opensmartroute.sdk.feedback`](#opensmartroutesdk) | feedback(decision, success=, quality=): learn from a decision in one call. | | `get_settings` | re-export of [`opensmartroute.settings.get_settings`](#opensmartroutesettings) | The process-wide :class:`Settings` (environment overlay applied once, lazily). | | `get_tracer` | re-export of [`opensmartroute.observability.get_tracer`](#opensmartrouteobservability) | The process-wide tracer (disabled until :func:`configure_tracing` adds sinks). | | `guard` | re-export of [`opensmartroute.sdk.guard`](#opensmartroutesdk) | @guard: ``fn(request) -> reason \| RouteRequest \| None`` runs before routing. | | `handler` | re-export of [`opensmartroute.sdk.handler`](#opensmartroutesdk) | @handler(*target_ids): bind code to targets declared in a catalogue. | | `human` | re-export of [`opensmartroute.sdk.human`](#opensmartroutesdk) | @human(id, ...): shorthand for a ``human`` target. | | `llm` | re-export of [`opensmartroute.sdk.llm`](#opensmartroutesdk) | @llm(id, ...): shorthand for an ``llm`` target. | | `load_rules` | re-export of [`opensmartroute.config.load_rules`](#opensmartrouteconfig) | Load a rules file (top-level list or ``rules:`` key) into a :class:`RulesStrategy`. | | `load_targets` | re-export of [`opensmartroute.config.load_targets`](#opensmartrouteconfig) | Load a catalogue file (top-level list or ``targets:`` key) into a :class:`TargetRegistry`. | | `middleware` | re-export of [`opensmartroute.sdk.middleware`](#opensmartroutesdk) | @middleware: register a Middleware class or ``fn(request, next_route)``. | | `on_decision` | re-export of [`opensmartroute.sdk.on_decision`](#opensmartroutesdk) | @on_decision: ``fn(request, decision)`` after every route. | | `on_error` | re-export of [`opensmartroute.sdk.on_error`](#opensmartroutesdk) | @on_error: ``fn(request, error)`` when a route fails. | | `on_outcome` | re-export of [`opensmartroute.sdk.on_outcome`](#opensmartroutesdk) | @on_outcome: ``fn(outcome)`` for every outcome learned. | | `persona` | re-export of [`opensmartroute.sdk.persona`](#opensmartroutesdk) | @persona(id, ...): shorthand for a slot-only ``persona`` target. | | `policy_rule` | re-export of [`opensmartroute.sdk.policy_rule`](#opensmartroutesdk) | @policy_rule: register ``fn(target, request, signals) -> reason \| None``. | | `route` | re-export of [`opensmartroute.sdk.route`](#opensmartroutesdk) | route(text): one request through the process-wide registry's router. | | `rule` | re-export of [`opensmartroute.sdk.rule`](#opensmartroutesdk) | rule(name, prefer=, avoid=, pin=, **when) or @rule(...) over a predicate. | | `run` | re-export of [`opensmartroute.sdk.run`](#opensmartroutesdk) | run(text): route + execute + learn through the process-wide registry. | | `signal` | re-export of [`opensmartroute.sdk.signal`](#opensmartroutesdk) | @signal: register a SignalExtractor class or ``fn(request, signals) -> mapping \| value``. | | `skill` | re-export of [`opensmartroute.sdk.skill`](#opensmartroutesdk) | @skill(id, ...): shorthand for a ``skill`` target. | | `strategy` | re-export of [`opensmartroute.sdk.strategy`](#opensmartroutesdk) | @strategy(weight=, name=): register a Strategy class or ``fn(request, signals, candidates)``. | | `target` | re-export of [`opensmartroute.sdk.target`](#opensmartroutesdk) | @target(id, kind, ...): the decorated callable becomes a RouteTarget handler. | | `telemetry` | re-export of [`opensmartroute.sdk.telemetry`](#opensmartroutesdk) | @telemetry: register a Telemetry sink class or factory. | | `tool` | re-export of [`opensmartroute.sdk.tool`](#opensmartroutesdk) | @tool(id, ...): shorthand for a ``tool`` target. | ## `opensmartroute.__main__` ``python -m opensmartroute ...`` is the ``osr`` command (IDE MCP configs, containers without the script on PATH). No public names. ## `opensmartroute.adapters` Adapters connect OpenSmartRoute to real providers and infrastructure. | Name | Kind | Summary | |---|---|---| | `AgentHarness` | re-export of [`opensmartroute.adapters.harness.AgentHarness`](#opensmartrouteadaptersharness) | Structural interface of an agent runtime: ``run(task, context=, history=) -> HarnessResult``. | | `AnthropicClient` | re-export of [`opensmartroute.adapters.anthropic_client.AnthropicClient`](#opensmartrouteadaptersanthropic_client) | The Messages API behind the :class:`OpenAICompatClient` surface: ``chat`` / ``stream_chat`` take. | | `CallableHarness` | re-export of [`opensmartroute.adapters.harness.CallableHarness`](#opensmartrouteadaptersharness) | Wrap an in-process agent: ``fn(task, context, history) -> str \| dict \| HarnessResult``. | | `ChatResult` | re-export of [`opensmartroute.adapters.openai_compat.ChatResult`](#opensmartrouteadaptersopenai_compat) | Result of a chat completion: text, model, token counts, latency, tool calls, finish reason and raw response. | | `HTTPHarness` | re-export of [`opensmartroute.adapters.harness.HTTPHarness`](#opensmartrouteadaptersharness) | POST the task as JSON to an agent endpoint. | | `HarnessResult` | re-export of [`opensmartroute.adapters.harness.HarnessResult`](#opensmartrouteadaptersharness) | What an agent harness returns: text, success, token / cost / latency usage and optional self-graded quality. | | `HttpMCPClient` | re-export of [`opensmartroute.adapters.mcp.HttpMCPClient`](#opensmartrouteadaptersmcp) | Stdlib MCP client for the **Streamable HTTP** transport (``POST `` per JSON-RPC message). | | `InMemoryQueue` | re-export of [`opensmartroute.adapters.handlers.InMemoryQueue`](#opensmartrouteadaptershandlers) | Reference :class:`Queue`: FIFO in memory, resolves into :class:`~opensmartroute.Outcome`. | | `Listing` | re-export of [`opensmartroute.adapters.marketplace.Listing`](#opensmartrouteadaptersmarketplace) | A marketplace search hit (what ``GET /api/v1/registry`` returns per item). | | `MarketplaceClient` | re-export of [`opensmartroute.adapters.marketplace.MarketplaceClient`](#opensmartrouteadaptersmarketplace) | Search a marketplace and load its listings as routing targets. | | `MarketplaceSync` | re-export of [`opensmartroute.adapters.marketplace.MarketplaceSync`](#opensmartrouteadaptersmarketplace) | Keep a registry in step with the marketplace on a schedule (the ``osr serve`` side of the daily refresh). | | `MockClient` | re-export of [`opensmartroute.adapters.mock_client.MockClient`](#opensmartrouteadaptersmock_client) | Deterministic in-process provider; see the module docstring for what it answers. | | `ModelCard` | re-export of [`opensmartroute.adapters.catalogue.ModelCard`](#opensmartrouteadapterscatalogue) | What the catalogue knows about one model: identity, price, limits, evidence of quality, provenance. | | `ModelCatalogue` | re-export of [`opensmartroute.adapters.catalogue.ModelCatalogue`](#opensmartrouteadapterscatalogue) | Merged, persisted model cards from every source; the SLM's view of the target universe. | | `OpenAICompatClient` | re-export of [`opensmartroute.adapters.openai_compat.OpenAICompatClient`](#opensmartrouteadaptersopenai_compat) | Stdlib-only client for the OpenAI chat / embeddings API (OpenAI, Azure, vLLM, Ollama, LiteLLM) with retries. | | `OpenTelemetrySink` | re-export of [`opensmartroute.adapters.optional.OpenTelemetrySink`](#opensmartrouteadaptersoptional) | Live bridge from the tracer to OpenTelemetry: every OpenSmartRoute span becomes an OTel span. | | `OpenTelemetryTelemetry` | re-export of [`opensmartroute.adapters.optional.OpenTelemetryTelemetry`](#opensmartrouteadaptersoptional) | Emits one span per decision and counters/histograms via the OTel API. | | `PendingResult` | re-export of [`opensmartroute.adapters.handlers.PendingResult`](#opensmartrouteadaptershandlers) | Immediate answer of a queued target: the request was accepted and will be answered later. | | `Queue` | re-export of [`opensmartroute.adapters.handlers.Queue`](#opensmartrouteadaptershandlers) | Structural interface of an asynchronous queue target (ticketing, human tier, workflow run). | | `QueuedItem` | re-export of [`opensmartroute.adapters.handlers.QueuedItem`](#opensmartrouteadaptershandlers) | One request waiting in (or resolved from) a queue. | | `SearchHit` | re-export of [`opensmartroute.adapters.websearch.SearchHit`](#opensmartrouteadapterswebsearch) | One search result: where it came from, what it says, and when it was seen. | | `SemanticRouterImport` | re-export of [`opensmartroute.adapters.semantic_router.SemanticRouterImport`](#opensmartrouteadapterssemantic_router) | Result of importing a vLLM semantic-router config: registry, rules, default model, categories, warnings. | | `ServerCard` | re-export of [`opensmartroute.adapters.mcp_servers.ServerCard`](#opensmartrouteadaptersmcp_servers) | Description of an MCP server (tools, tags, auth, latency, cost, region, data boundary) for recommendation. | | `ServerRecommendation` | re-export of [`opensmartroute.adapters.mcp_servers.ServerRecommendation`](#opensmartrouteadaptersmcp_servers) | A ranked server from ``recommend_servers`` with its fused score, rationale and matched tool names. | | `StdioMCPClient` | re-export of [`opensmartroute.adapters.mcp.StdioMCPClient`](#opensmartrouteadaptersmcp) | Tiny JSON-RPC-over-stdio MCP client (newline-delimited). Thread-safe, blocking. | | `SubprocessHarness` | re-export of [`opensmartroute.adapters.harness.SubprocessHarness`](#opensmartrouteadaptersharness) | Run a CLI agent: task on stdin, answer on stdout, exit code 0 = success. | | `SyncReport` | re-export of [`opensmartroute.adapters.marketplace.SyncReport`](#opensmartrouteadaptersmarketplace) | What :meth:`MarketplaceClient.sync` did to a registry. | | `WebKnowledge` | re-export of [`opensmartroute.adapters.websearch.WebKnowledge`](#opensmartrouteadapterswebsearch) | Fan a query out to search providers, de-duplicate by URL and cache the hits as JSON. | | `a2a_handler` | re-export of [`opensmartroute.adapters.a2a.a2a_handler`](#opensmartrouteadaptersa2a) | Return a handler that sends the request text to an A2A agent and returns its text. | | `agent_from_card` | re-export of [`opensmartroute.adapters.a2a.agent_from_card`](#opensmartrouteadaptersa2a) | Build an ``agent`` RouteTarget from an A2A agent card (name, skills, tags, input modes). | | `attach_chat_handlers` | re-export of [`opensmartroute.adapters.openai_compat.attach_chat_handlers`](#opensmartrouteadaptersopenai_compat) | Give every LLM target a :func:`chat_handler` on ``client``; returns the ids that got one. | | `brave_search` | re-export of [`opensmartroute.adapters.websearch.brave_search`](#opensmartrouteadapterswebsearch) | Brave Search API web results; the key is read from ``BRAVE_API_KEY`` (or ``BRAVE_API_KEY_FILE``). | | `card_to_target` | re-export of [`opensmartroute.adapters.catalogue.card_to_target`](#opensmartrouteadapterscatalogue) | A :class:`RouteTarget` for a model card; risky third-party descriptions are replaced by the name. | | `chat_handler` | re-export of [`opensmartroute.adapters.openai_compat.chat_handler`](#opensmartrouteadaptersopenai_compat) | Adapter for ``RouteTarget.handler``: turns a RouteRequest into a chat call. | | `connect_mcp` | re-export of [`opensmartroute.adapters.mcp.connect_mcp`](#opensmartrouteadaptersmcp) | Spawn a stdio MCP server, list its tools (and optionally resources and prompts) and return. | | `connect_mcp_http` | re-export of [`opensmartroute.adapters.mcp.connect_mcp_http`](#opensmartrouteadaptersmcp) | Connect to a Streamable-HTTP MCP server, import its tools (and optionally resources and prompts) as. | | `duckduckgo_search` | re-export of [`opensmartroute.adapters.websearch.duckduckgo_search`](#opensmartrouteadapterswebsearch) | DuckDuckGo instant-answer API (abstract + related topics). Keyless; shallow but good for definitions. | | `embed_handler` | re-export of [`opensmartroute.adapters.openai_compat.embed_handler`](#opensmartrouteadaptersopenai_compat) | Adapter for an embeddings-capable ``RouteTarget.handler``: embeds ``context["embeddings_input"]`` (or the text). | | `embedder` | re-export of [`opensmartroute.adapters.openai_compat.embedder`](#opensmartrouteadaptersopenai_compat) | Adapter for ``SimilarityStrategy(embedder=...)``. | | `enrich_description` | re-export of [`opensmartroute.adapters.mcp.enrich_description`](#opensmartrouteadaptersmcp) | Return ``(routing_description, examples, capabilities)`` for a tool. | | `fetch_agent_card` | re-export of [`opensmartroute.adapters.a2a.fetch_agent_card`](#opensmartrouteadaptersa2a) | Download an agent card over HTTPS (plain HTTP is refused); raises TargetUnavailableError on failure. | | `fetch_bytes` | re-export of [`opensmartroute.adapters.websearch.fetch_bytes`](#opensmartrouteadapterswebsearch) | GET ``url`` over https with a timeout and a body cap; transport errors become ``TargetUnavailableError``. | | `fetch_feed` | re-export of [`opensmartroute.adapters.websearch.fetch_feed`](#opensmartrouteadapterswebsearch) | Fetch an RSS / Atom feed over https and return its articles (see :func:`parse_feed`). | | `fetch_huggingface_models` | re-export of [`opensmartroute.adapters.catalogue.fetch_huggingface_models`](#opensmartrouteadapterscatalogue) | Model cards from the Hugging Face Hub search (downloads, likes, tags, ``model-index`` benchmarks). | | `fetch_json` | re-export of [`opensmartroute.adapters.websearch.fetch_json`](#opensmartrouteadapterswebsearch) | GET a JSON document (see :func:`fetch_bytes`); malformed bodies raise ``TargetUnavailableError``. | | `fetch_leaderboard_quality` | re-export of [`opensmartroute.adapters.catalogue.fetch_leaderboard_quality`](#opensmartrouteadapterscatalogue) | ``{hub model id: {benchmark: accuracy}}`` from the Open LLM Leaderboard table (official rows, unflagged). | | `fetch_openrouter_models` | re-export of [`opensmartroute.adapters.catalogue.fetch_openrouter_models`](#opensmartrouteadapterscatalogue) | Model cards from OpenRouter's public listing (prices per token, context, modalities, tool support). | | `fetch_page_text` | re-export of [`opensmartroute.adapters.websearch.fetch_page_text`](#opensmartrouteadapterswebsearch) | Fetch a page and return ``{url, title, text, risk}``; ``risk`` is the injection/gadget risk of the text. | | `harness_handler` | re-export of [`opensmartroute.adapters.harness.harness_handler`](#opensmartrouteadaptersharness) | Adapter for ``RouteTarget.handler``: a RouteRequest becomes a harness task. | | `html_to_text` | re-export of [`opensmartroute.adapters.websearch.html_to_text`](#opensmartrouteadapterswebsearch) | ``(title, text)`` of an HTML document with scripts/styles removed and whitespace collapsed. | | `http_handler` | re-export of [`opensmartroute.adapters.handlers.http_handler`](#opensmartrouteadaptershandlers) | ``RouteTarget.handler`` that POSTs the request to ``url`` and returns a :class:`HarnessResult`. | | `huggingface_search` | re-export of [`opensmartroute.adapters.websearch.huggingface_search`](#opensmartrouteadapterswebsearch) | Search the Hugging Face Hub (``what`` = ``models`` or ``datasets``), ranked by downloads. No key needed. | | `judge_fn` | re-export of [`opensmartroute.adapters.openai_compat.judge_fn`](#opensmartrouteadaptersopenai_compat) | Adapter for ``LLMJudgeStrategy(llm=...)``: prompt in, completion text out. | | `langgraph_condition` | re-export of [`opensmartroute.adapters.frameworks.langgraph_condition`](#opensmartrouteadaptersframeworks) | Edge selector for ``add_conditional_edges``: routes on target id (``by='target'``) or kind. | | `langgraph_node` | re-export of [`opensmartroute.adapters.frameworks.langgraph_node`](#opensmartrouteadaptersframeworks) | Return a LangGraph-compatible node ``state -> dict`` (partial state update). | | `load_marketplace_targets` | re-export of [`opensmartroute.adapters.marketplace.load_marketplace_targets`](#opensmartrouteadaptersmarketplace) | Shortcut: ``MarketplaceClient(base_url, api_key=api_key).targets(q, kind=kind, tag=tag, limit=limit)``. | | `load_personas` | re-export of [`opensmartroute.adapters.personas.load_personas`](#opensmartrouteadapterspersonas) | Load personas from a directory of markdown files or a JSON/JSONL/CSV catalogue. | | `load_semantic_router_config` | re-export of [`opensmartroute.adapters.semantic_router.load_semantic_router_config`](#opensmartrouteadapterssemantic_router) | Convert a vLLM semantic-router ``model_config`` / ``categories`` document into targets and rules. | | `load_skill` | re-export of [`opensmartroute.adapters.skills.load_skill`](#opensmartrouteadaptersskills) | Load one skill directory (must contain ``SKILL.md``). | | `load_skills` | re-export of [`opensmartroute.adapters.skills.load_skills`](#opensmartrouteadaptersskills) | Load every ``*/SKILL.md`` under ``root`` (one level deep, sorted by name). | | `maf_router_executor` | re-export of [`opensmartroute.adapters.frameworks.maf_router_executor`](#opensmartrouteadaptersframeworks) | Agent Framework style: ``executor(message, ctx) -> target id \| response``; ``handoffs`` maps. | | `manifest_from_targets` | re-export of [`opensmartroute.adapters.mcp.manifest_from_targets`](#opensmartrouteadaptersmcp) | Reverse: dump MCP-shaped tool dicts (for signing / publishing a catalogue). | | `mcp_server_handler` | re-export of [`opensmartroute.adapters.mcp.mcp_server_handler`](#opensmartrouteadaptersmcp) | ``RouteTarget.handler`` for a target that stands for a whole remote MCP server (an ``mcp`` endpoint in an. | | `mcp_tool_handler` | re-export of [`opensmartroute.adapters.handlers.mcp_tool_handler`](#opensmartrouteadaptershandlers) | ``RouteTarget.handler`` that invokes one MCP tool via ``call(name, arguments)``. | | `model_key` | re-export of [`opensmartroute.adapters.catalogue.model_key`](#opensmartrouteadapterscatalogue) | Vendor-agnostic key for matching model names across sources: lowercase, no vendor prefix, no punctuation. | | `openai_tool_handler` | re-export of [`opensmartroute.adapters.frameworks.openai_tool_handler`](#opensmartrouteadaptersframeworks) | Callable behind ``openai_tool_spec``: ``(text, objective?, kinds?) -> RouteDecision.to_dict()``. | | `openai_tool_spec` | re-export of [`opensmartroute.adapters.frameworks.openai_tool_spec`](#opensmartrouteadaptersframeworks) | OpenAI function-calling tool definition that lets a model ask the router for a target. | | `parse_feed` | re-export of [`opensmartroute.adapters.websearch.parse_feed`](#opensmartrouteadapterswebsearch) | Articles of an RSS 2.0 / RDF / Atom document as ``kind="article"`` hits, newest first. | | `persona_from_markdown` | re-export of [`opensmartroute.adapters.personas.persona_from_markdown`](#opensmartrouteadapterspersonas) | Parse a ``*.agent.md`` / ``*.chatmode.md`` / front-matter markdown file into a persona target. | | `persona_target` | re-export of [`opensmartroute.adapters.personas.persona_target`](#opensmartrouteadapterspersonas) | Build a ``persona`` RouteTarget whose ``instructions`` is the system prompt (non-primary by default). | | `personas_from_records` | re-export of [`opensmartroute.adapters.personas.personas_from_records`](#opensmartrouteadapterspersonas) | Persona targets from JSON / CSV-style records (``name`` + ``prompt``/``system`` keys); others skipped. | | `prompts_from_mcp` | re-export of [`opensmartroute.adapters.mcp.prompts_from_mcp`](#opensmartrouteadaptersmcp) | Convert ``prompts/list`` output into instructions-only ``skill`` targets (``primary=False``). | | `quality_from_benchmarks` | re-export of [`opensmartroute.adapters.catalogue.quality_from_benchmarks`](#opensmartrouteadapterscatalogue) | Mean of normalised benchmark scores (percentages are divided by 100); None when there are none. | | `quality_from_popularity` | re-export of [`opensmartroute.adapters.catalogue.quality_from_popularity`](#opensmartrouteadapterscatalogue) | Weak prior in [0.35, 0.75] from log-scaled downloads and likes (popularity is not quality; it is a hint). | | `queue_handler` | re-export of [`opensmartroute.adapters.handlers.queue_handler`](#opensmartrouteadaptershandlers) | ``RouteTarget.handler`` that enqueues the request and returns a :class:`PendingResult`. | | `recommend_servers` | re-export of [`opensmartroute.adapters.mcp_servers.recommend_servers`](#opensmartrouteadaptersmcp_servers) | Rank MCP servers for a task; constraint violations are excluded, not down-weighted. | | `resources_from_mcp` | re-export of [`opensmartroute.adapters.mcp.resources_from_mcp`](#opensmartrouteadaptersmcp) | Convert ``resources/list`` output into read-only ``tool`` targets whose handler reads the resource. | | `route_and_execute` | re-export of [`opensmartroute.adapters.frameworks.route_and_execute`](#opensmartrouteadaptersframeworks) | Route ``text`` and optionally execute the plan; returns ``(decision, ExecutionResult \| None)``. | | `searxng_search` | re-export of [`opensmartroute.adapters.websearch.searxng_search`](#opensmartrouteadapterswebsearch) | A self-hosted SearXNG instance (``SEARXNG_URL`` or ``base_url``; its JSON API must be enabled): real web. | | `sentence_transformers_embedder` | re-export of [`opensmartroute.adapters.optional.sentence_transformers_embedder`](#opensmartrouteadaptersoptional) | Semantic embedder for ``SimilarityStrategy``. Requires ``opensmartroute[embeddings]``. | | `sign_manifest` | re-export of [`opensmartroute.adapters.mcp.sign_manifest`](#opensmartrouteadaptersmcp) | Wrap a tool list in a signed manifest. ``algorithm`` = ``hmac-sha256`` (key = shared. | | `skill_from_markdown` | re-export of [`opensmartroute.adapters.skills.skill_from_markdown`](#opensmartrouteadaptersskills) | Parse one SKILL.md (Agent-Skills front matter + body) into a ``skill`` RouteTarget; strict validation. | | `skills_from_card` | re-export of [`opensmartroute.adapters.a2a.skills_from_card`](#opensmartrouteadaptersa2a) | One non-primary ``skill`` target per skill declared on an A2A agent card (``/``). | | `tools_from_manifest` | re-export of [`opensmartroute.adapters.mcp.tools_from_manifest`](#opensmartrouteadaptersmcp) | Import tools from a (signed) manifest; ``kw`` goes to :func:`tools_from_mcp`. | | `tools_from_mcp` | re-export of [`opensmartroute.adapters.mcp.tools_from_mcp`](#opensmartrouteadaptersmcp) | Convert ``tools/list`` output into targets. ``call(name, arguments)`` becomes the handler. | | `verify_manifest` | re-export of [`opensmartroute.adapters.mcp.verify_manifest`](#opensmartrouteadaptersmcp) | Constant-time verification. ``key`` = shared secret (HMAC) or 32-byte Ed25519 public key. | | `wikipedia_search` | re-export of [`opensmartroute.adapters.websearch.wikipedia_search`](#opensmartrouteadapterswebsearch) | Wikipedia article search (MediaWiki API, no key): title, first snippet and the article URL. Encyclopaedic,. | ## `opensmartroute.adapters.a2a` Import A2A (Agent-to-Agent protocol) **Agent Cards** as ``TargetKind.AGENT`` targets. | Name | Kind | Summary | |---|---|---| | `TERMINAL_TASK_STATES` | constant | : A2A task states after which no more work happens (``tasks/get`` polling stops). | | `a2a_handler` | function `(url: str, token: str \| None=None, timeout_s: float=60.0, method: str='message/send', poll_interval_s: float=1.0)` | Return a handler that sends the request text to an A2A agent and returns its text. | | `agent_from_card` | function `(card: dict[str, Any], *, call: Callable[[str, RouteRequest], Any] \| None=None, cost_per_1k_tokens: float=0.0, latency_ms: float=2000.0, quality_prior: float=0.65, languages: list[str] \| None=None)` | Build an ``agent`` RouteTarget from an A2A agent card (name, skills, tags, input modes). | | `fetch_agent_card` | function `(base_url: str, timeout_s: float=10.0, path: str='/.well-known/agent.json')` | Download an agent card over HTTPS (plain HTTP is refused); raises TargetUnavailableError on failure. | | `skills_from_card` | function `(card: dict[str, Any], agent_id: str \| None=None)` | One non-primary ``skill`` target per skill declared on an A2A agent card (``/``). | ## `opensmartroute.adapters.anthropic_client` Native Anthropic **Messages API** client - route TO Claude, not only accept Claude-dialect requests. | Name | Kind | Summary | |---|---|---| | `ANTHROPIC_VERSION` | constant | : The API version header every request carries (the newest stable revision the translation targets). | | `CACHE_MAX_BREAKPOINTS` | constant | : Breakpoints the Messages API accepts per request. | | `CACHE_MIN_CHARS` | constant | : cache breakpoint is worth placing on it; tools and the conversation prefix are marked whenever they exist. | | `DEFAULT_MAX_TOKENS` | constant | : The Messages API requires ``max_tokens``; used when neither the caller nor the proxy set one. | | `THINKING_BUDGETS` | constant | : Extended-thinking budgets per ``reasoning_effort`` value. | | `AnthropicClient` | class | The Messages API behind the :class:`OpenAICompatClient` surface: ``chat`` / ``stream_chat`` take. | | `cache_breakpoints` | function `(body: dict[str, Any])` | Place Anthropic prompt-cache breakpoints on a Messages body the caller sent without any: the last tool. | | `to_messages_request` | function `(messages: list[dict[str, Any]], model: str, *, temperature: float=0.0, max_tokens: int \| None=None, **kw: Any)` | A chat-completions call as a Messages API body (see the module docstring for the mapping). | ## `opensmartroute.adapters.catalogue` Live model catalogue: collect model cards (price, context, modalities, benchmarks) from public sources. | Name | Kind | Summary | |---|---|---| | `LEADERBOARD_DATASET` | constant | Open LLM Leaderboard v2 results table on the Hub. | | `OPENROUTER_MODELS` | constant | public model + pricing listing (no key). | | `ModelCard` | class | What the catalogue knows about one model: identity, price, limits, evidence of quality, provenance. | | `ModelCatalogue` | class | Merged, persisted model cards from every source; the SLM's view of the target universe. | | `card_to_target` | function `(card: ModelCard, *, default_latency_ms: float=1500.0, settings: Settings \| None=None)` | A :class:`RouteTarget` for a model card; risky third-party descriptions are replaced by the name. | | `fetch_huggingface_models` | function `(query: str='', *, limit: int=50, pipeline: str='text-generation', timeout_s: float \| None=None, settings: Settings \| None=None)` | Model cards from the Hugging Face Hub search (downloads, likes, tags, ``model-index`` benchmarks). | | `fetch_leaderboard_quality` | function `(*, limit: int=5000, dataset: str=LEADERBOARD_DATASET, timeout_s: float \| None=None, settings: Settings \| None=None)` | ``{hub model id: {benchmark: accuracy}}`` from the Open LLM Leaderboard table (official rows, unflagged). | | `fetch_openrouter_models` | function `(*, url: str=OPENROUTER_MODELS, timeout_s: float \| None=None, settings: Settings \| None=None)` | Model cards from OpenRouter's public listing (prices per token, context, modalities, tool support). | | `model_key` | function `(model_id: str)` | Vendor-agnostic key for matching model names across sources: lowercase, no vendor prefix, no punctuation. | | `quality_from_benchmarks` | function `(benchmarks: dict[str, float])` | Mean of normalised benchmark scores (percentages are divided by 100); None when there are none. | | `quality_from_popularity` | function `(downloads: int, likes: int)` | Weak prior in [0.35, 0.75] from log-scaled downloads and likes (popularity is not quality; it is a hint). | ## `opensmartroute.adapters.frameworks` Drop-in nodes for agent frameworks. | Name | Kind | Summary | |---|---|---| | `langgraph_condition` | function `(by: str='target', default: str='__end__')` | Edge selector for ``add_conditional_edges``: routes on target id (``by='target'``) or kind. | | `langgraph_node` | function `(router: Router, *, execute: bool=False, plan: bool=False, context_key: str='route_context', on_no_route: str='__no_route__')` | Return a LangGraph-compatible node ``state -> dict`` (partial state update). | | `last_user_text` | function `(state: dict[str, Any])` | (last user utterance, prior history as role/content dicts). | | `maf_router_executor` | function `(router: Router, *, execute: bool=False)` | Agent Framework style: ``executor(message, ctx) -> target id \| response``; ``handoffs`` maps. | | `openai_tool_handler` | function `(router: Router)` | Callable behind ``openai_tool_spec``: ``(text, objective?, kinds?) -> RouteDecision.to_dict()``. | | `openai_tool_spec` | function `(name: str='route_request')` | OpenAI function-calling tool definition that lets a model ask the router for a target. | | `route_and_execute` | function `(router: Router, text: str, *, history: list[dict[str, str]] \| None=None, context: dict[str, Any] \| None=None, objective: Objective \| None=None, execute: bool=False, plan: bool=False, learn: bool=True)` | Route ``text`` and optionally execute the plan; returns ``(decision, ExecutionResult \| None)``. | ## `opensmartroute.adapters.handlers` Executors for non-LLM targets: HTTP endpoints, MCP tools and asynchronous queues. | Name | Kind | Summary | |---|---|---| | `InMemoryQueue` | class | Reference :class:`Queue`: FIFO in memory, resolves into :class:`~opensmartroute.Outcome`. | | `PendingResult` | class | Immediate answer of a queued target: the request was accepted and will be answered later. | | `Queue` | class | Structural interface of an asynchronous queue target (ticketing, human tier, workflow run). | | `QueuedItem` | class | One request waiting in (or resolved from) a queue. | | `http_handler` | function `(url: str, *, api_key: str \| None=None, api_key_env: str \| None=None, timeout_s: float=300.0, headers: Mapping[str, str] \| None=None, payload: Callable[[str, Mapping[str, Any], list[dict[str, str]]], dict[str, Any]] \| None=None, body: Mapping[str, Any] \| None=None)` | ``RouteTarget.handler`` that POSTs the request to ``url`` and returns a :class:`HarnessResult`. | | `mcp_tool_handler` | function `(call: Callable[[str, dict[str, Any]], Any], tool: str, input_schema: dict[str, Any] \| None=None, arguments: Callable[[RouteRequest], dict[str, Any]] \| None=None)` | ``RouteTarget.handler`` that invokes one MCP tool via ``call(name, arguments)``. | | `queue_handler` | function `(queue: Queue, target_id: str \| None=None)` | ``RouteTarget.handler`` that enqueues the request and returns a :class:`PendingResult`. | ## `opensmartroute.adapters.harness` Agent-harness adapters: route to a *runtime*, not just a model. | Name | Kind | Summary | |---|---|---| | `AgentHarness` | class | Structural interface of an agent runtime: ``run(task, context=, history=) -> HarnessResult``. | | `CallableHarness` | class | Wrap an in-process agent: ``fn(task, context, history) -> str \| dict \| HarnessResult``. | | `HTTPHarness` | class | POST the task as JSON to an agent endpoint. | | `HarnessResult` | class | What an agent harness returns: text, success, token / cost / latency usage and optional self-graded quality. | | `SubprocessHarness` | class | Run a CLI agent: task on stdin, answer on stdout, exit code 0 = success. | | `coerce_result` | function `(out: Any)` | Normalise ``str \| dict \| HarnessResult \| object-with-text`` into a HarnessResult. | | `harness_handler` | function `(harness: AgentHarness)` | Adapter for ``RouteTarget.handler``: a RouteRequest becomes a harness task. | ## `opensmartroute.adapters.marketplace` Pull skills, agents, tools and templates from an OpenSmartRoute marketplace into a router. | Name | Kind | Summary | |---|---|---| | `DEFAULT_MARKETPLACE` | constant | the hosted marketplace API. | | `LISTING_KINDS` | constant | : Listing kinds the marketplace catalogue knows (search filter values). | | `ROUTABLE_KINDS` | constant | : Listing kinds whose manifest is an Open Capability Manifest the router can load as a target. | | `Listing` | class | A marketplace search hit (what ``GET /api/v1/registry`` returns per item). | | `MarketplaceClient` | class | Search a marketplace and load its listings as routing targets. | | `MarketplaceSync` | class | Keep a registry in step with the marketplace on a schedule (the ``osr serve`` side of the daily refresh). | | `SyncReport` | class | What :meth:`MarketplaceClient.sync` did to a registry. | | `load_marketplace_targets` | function `(base_url: str=DEFAULT_MARKETPLACE, q: str='', *, kind: str \| None=None, tag: str \| None=None, limit: int=24, api_key: str \| None=None)` | Shortcut: ``MarketplaceClient(base_url, api_key=api_key).targets(q, kind=kind, tag=tag, limit=limit)``. | ## `opensmartroute.adapters.mcp` Import MCP (Model Context Protocol) tools as ``TargetKind.TOOL`` targets. | Name | Kind | Summary | |---|---|---| | `HttpMCPClient` | class | Stdlib MCP client for the **Streamable HTTP** transport (``POST `` per JSON-RPC message). | | `MCPTool` | class | A tool as listed by an MCP server: name, description, JSON input schema and annotations. | | `StdioMCPClient` | class | Tiny JSON-RPC-over-stdio MCP client (newline-delimited). Thread-safe, blocking. | | `canonical_json` | function `(obj: Any)` | Deterministic JSON encoding (sorted keys, no whitespace) used for manifest digests and signatures. | | `connect_mcp` | function `(command: list[str], server: str='', *, resources: bool=False, prompts: bool=False, **kw: Any)` | Spawn a stdio MCP server, list its tools (and optionally resources and prompts) and return. | | `connect_mcp_http` | function `(url: str, api_key: str \| None=None, server: str='', *, resources: bool=False, prompts: bool=False, timeout_s: float=60.0, **kw: Any)` | Connect to a Streamable-HTTP MCP server, import its tools (and optionally resources and prompts) as. | | `enrich_description` | function `(name: str, description: str, input_schema: dict[str, Any] \| None, server: str='', enricher: Callable[[str], list[str]] \| None=None)` | Return ``(routing_description, examples, capabilities)`` for a tool. | | `manifest_digest` | function `(tools: list[dict[str, Any]])` | SHA-256 hex digest of a tool list in canonical JSON. | | `manifest_from_targets` | function `(targets: Iterable[RouteTarget])` | Reverse: dump MCP-shaped tool dicts (for signing / publishing a catalogue). | | `prompts_from_mcp` | function `(payload: Any, server: str='', get: Callable[[str, dict[str, Any]], Any] \| None=None, *, quality_prior: float=0.6)` | Convert ``prompts/list`` output into instructions-only ``skill`` targets (``primary=False``). | | `resources_from_mcp` | function `(payload: Any, server: str='', read: Callable[[str], Any] \| None=None, *, latency_ms: float=200.0, quality_prior: float=0.6)` | Convert ``resources/list`` output into read-only ``tool`` targets whose handler reads the resource. | | `sign_manifest` | function `(tools: list[dict[str, Any]], key: bytes, *, server: str='', algorithm: str='hmac-sha256', key_id: str='')` | Wrap a tool list in a signed manifest. ``algorithm`` = ``hmac-sha256`` (key = shared. | | `tools_from_manifest` | function `(manifest: dict[str, Any], key: bytes \| None, *, require_signature: bool=True, max_age_s: float \| None=None, **kw: Any)` | Import tools from a (signed) manifest; ``kw`` goes to :func:`tools_from_mcp`. | | `tools_from_mcp` | function `(payload: Any, server: str='', call: Callable[[str, dict[str, Any]], Any] \| None=None, enricher: Callable[[str], list[str]] \| None=None, cost_per_call_usd: float=0.0, latency_ms: float=300.0, quality_prior: float=0.6, max_description_risk: float=0.6, guard: InputGuard \| None=None)` | Convert ``tools/list`` output into targets. ``call(name, arguments)`` becomes the handler. | | `verify_manifest` | function `(manifest: dict[str, Any], key: bytes, max_age_s: float \| None=None)` | Constant-time verification. ``key`` = shared secret (HMAC) or 32-byte Ed25519 public key. | ## `opensmartroute.adapters.mcp_servers` MCP *server* recommendation (MCP-Zero 2506.01056; ToolRet 2603.06467). | Name | Kind | Summary | |---|---|---| | `ServerCard` | class | Description of an MCP server (tools, tags, auth, latency, cost, region, data boundary) for recommendation. | | `ServerRecommendation` | class | A ranked server from ``recommend_servers`` with its fused score, rationale and matched tool names. | | `recommend_servers` | function `(request: RouteRequest \| str, servers: Iterable[ServerCard \| dict[str, Any]], k: int=3, constraints: RequestConstraints \| None=None, allowed_auth: Iterable[str] \| None=None, min_score: float=0.0)` | Rank MCP servers for a task; constraint violations are excluded, not down-weighted. | ## `opensmartroute.adapters.mock_client` A provider that answers deterministically without a network: the ``mock`` provider kind. | Name | Kind | Summary | |---|---|---| | `MOCK_SCHEME` | constant | a base URL that selects this client. | | `MockClient` | class | Deterministic in-process provider; see the module docstring for what it answers. | | `is_mock` | function `(kind: str \| None, base_url: str \| None)` | True when a provider spec selects the mock client (``kind: mock`` or a ``mock://`` base URL). | ## `opensmartroute.adapters.openai_compat` OpenAI-compatible HTTP client (stdlib only). | Name | Kind | Summary | |---|---|---| | `PASSTHROUGH_FIELDS` | constant | Chat-completion fields the proxy passes through to the upstream untouched (``request.context["openai"]``). | | `UPSTREAM_REQUEST_ID_HEADERS` | constant | : Response headers providers use for their own request id, in the order they are looked up. | | `ChatResult` | class | Result of a chat completion: text, model, token counts, latency, tool calls, finish reason and raw response. | | `OpenAICompatClient` | class | Stdlib-only client for the OpenAI chat / embeddings API (OpenAI, Azure, vLLM, Ollama, LiteLLM) with retries. | | `attach_chat_handlers` | function `(targets: Iterable[Any], client: OpenAICompatClient, *, model_key: str='model', default_model: str \| None=None, overwrite: bool=False)` | Give every LLM target a :func:`chat_handler` on ``client``; returns the ids that got one. | | `chat_handler` | function `(client: OpenAICompatClient, model: str, system_prompt: str \| None=None, **defaults: Any)` | Adapter for ``RouteTarget.handler``: turns a RouteRequest into a chat call. | | `embed_handler` | function `(client: OpenAICompatClient, model: str)` | Adapter for an embeddings-capable ``RouteTarget.handler``: embeds ``context["embeddings_input"]`` (or the text). | | `embedder` | function `(client: OpenAICompatClient, model: str, batch: int=64)` | Adapter for ``SimilarityStrategy(embedder=...)``. | | `judge_fn` | function `(client: OpenAICompatClient, model: str, max_tokens: int=400)` | Adapter for ``LLMJudgeStrategy(llm=...)``: prompt in, completion text out. | | `upstream_request_id` | function `(headers: Any)` | The provider's request id from a response's headers (:data:`UPSTREAM_REQUEST_ID_HEADERS`), or ``None``. | ## `opensmartroute.adapters.optional` Optional adapters that need extra dependencies. Everything is lazily imported so the. | Name | Kind | Summary | |---|---|---| | `OpenTelemetrySink` | class | Live bridge from the tracer to OpenTelemetry: every OpenSmartRoute span becomes an OTel span. | | `OpenTelemetryTelemetry` | class | Emits one span per decision and counters/histograms via the OTel API. | | `sentence_transformers_embedder` | function `(model_name: str='sentence-transformers/all-MiniLM-L6-v2', device: str \| None=None, normalize: bool=True)` | Semantic embedder for ``SimilarityStrategy``. Requires ``opensmartroute[embeddings]``. | ## `opensmartroute.adapters.personas` Import persona catalogues as ``TargetKind.PERSONA`` targets. | Name | Kind | Summary | |---|---|---| | `load_personas` | function `(path: str \| Path, **kw: Any)` | Load personas from a directory of markdown files or a JSON/JSONL/CSV catalogue. | | `persona_from_markdown` | function `(text: str, *, path: Path \| None=None, **kw: Any)` | Parse a ``*.agent.md`` / ``*.chatmode.md`` / front-matter markdown file into a persona target. | | `persona_target` | function `(name: str, prompt: str, description: str='', *, id_prefix: str='persona:', domains: list[str] \| None=None, tags: list[str] \| None=None, languages: list[str] \| None=None, primary: bool=False, quality_prior: float=0.6, source: str='', extra: dict[str, Any] \| None=None)` | Build a ``persona`` RouteTarget whose ``instructions`` is the system prompt (non-primary by default). | | `personas_from_records` | function `(records: Iterable[dict[str, Any]], **kw: Any)` | Persona targets from JSON / CSV-style records (``name`` + ``prompt``/``system`` keys); others skipped. | ## `opensmartroute.adapters.semantic_router` Import a vLLM *semantic-router* configuration (vllm-project/semantic-router). | Name | Kind | Summary | |---|---|---| | `DEFAULT_DOMAIN_MAP` | constant | semantic-router's MMLU-style categories -> OpenSmartRoute ontology domains. | | `SemanticRouterImport` | class | Result of importing a vLLM semantic-router config: registry, rules, default model, categories, warnings. | | `load_semantic_router_config` | function `(source: str \| Path \| dict[str, Any], domain_map: dict[str, list[str]] \| None=None, rule_weight: float=0.8)` | Convert a vLLM semantic-router ``model_config`` / ``categories`` document into targets and rules. | ## `opensmartroute.adapters.skills` Load Agent-Skills ``SKILL.md`` packages as ``TargetKind.SKILL`` targets. | Name | Kind | Summary | |---|---|---| | `load_skill` | function `(skill_dir: str \| Path, **kw: Any)` | Load one skill directory (must contain ``SKILL.md``). | | `load_skills` | function `(root: str \| Path, **kw: Any)` | Load every ``*/SKILL.md`` under ``root`` (one level deep, sorted by name). | | `parse_frontmatter` | function `(text: str)` | Split ``---`` frontmatter from the body. Returns ``(frontmatter, body)``. | | `skill_from_markdown` | function `(text: str, *, path: Path \| None=None, cost_per_1k_tokens: float=0.0)` | Parse one SKILL.md (Agent-Skills front matter + body) into a ``skill`` RouteTarget; strict validation. | ## `opensmartroute.adapters.websearch` Web knowledge for the self-improving router: stdlib HTTP fetch, search providers, page text. | Name | Kind | Summary | |---|---|---| | `BRAVE_API` | constant | Brave web search (needs BRAVE_API_KEY). | | `DATASETS_SERVER` | constant | Hugging Face datasets-server REST root (/rows). | | `DDG_API` | constant | DuckDuckGo instant-answer JSON endpoint (no key). | | `HF_API` | constant | Hugging Face Hub REST root (models, datasets, search). | | `SEARXNG_URL_ENV` | constant | base URL of a self-hosted SearXNG instance (JSON API); registers the provider. | | `WIKIPEDIA_API` | constant | MediaWiki search API (no key; ``lang`` = edition). | | `SearchHit` | class | One search result: where it came from, what it says, and when it was seen. | | `SearchProvider` | constant | (query, limit) -> hits. | | `WebKnowledge` | class | Fan a query out to search providers, de-duplicate by URL and cache the hits as JSON. | | `brave_search` | function `(query: str, limit: int=10, *, api_key_env: str='BRAVE_API_KEY', timeout_s: float \| None=None, settings: Settings \| None=None)` | Brave Search API web results; the key is read from ``BRAVE_API_KEY`` (or ``BRAVE_API_KEY_FILE``). | | `duckduckgo_search` | function `(query: str, limit: int=10, *, timeout_s: float \| None=None, settings: Settings \| None=None)` | DuckDuckGo instant-answer API (abstract + related topics). Keyless; shallow but good for definitions. | | `fetch_bytes` | function `(url: str, *, timeout_s: float \| None=None, max_bytes: int \| None=None, headers: Mapping[str, str] \| None=None, settings: Settings \| None=None)` | GET ``url`` over https with a timeout and a body cap; transport errors become ``TargetUnavailableError``. | | `fetch_feed` | function `(url: str, *, max_entries: int \| None=None, settings: Settings \| None=None)` | Fetch an RSS / Atom feed over https and return its articles (see :func:`parse_feed`). | | `fetch_json` | function `(url: str, *, timeout_s: float \| None=None, headers: Mapping[str, str] \| None=None, settings: Settings \| None=None)` | GET a JSON document (see :func:`fetch_bytes`); malformed bodies raise ``TargetUnavailableError``. | | `fetch_page_text` | function `(url: str, *, max_chars: int=20000, timeout_s: float \| None=None, settings: Settings \| None=None)` | Fetch a page and return ``{url, title, text, risk}``; ``risk`` is the injection/gadget risk of the text. | | `html_to_text` | function `(html: str)` | ``(title, text)`` of an HTML document with scripts/styles removed and whitespace collapsed. | | `huggingface_search` | function `(query: str, limit: int=10, *, what: str='models', pipeline_tag: str \| None='text-generation', timeout_s: float \| None=None, settings: Settings \| None=None)` | Search the Hugging Face Hub (``what`` = ``models`` or ``datasets``), ranked by downloads. No key needed. | | `parse_feed` | function `(data: bytes \| str, *, source: str='feed', max_entries: int=50)` | Articles of an RSS 2.0 / RDF / Atom document as ``kind="article"`` hits, newest first. | | `searxng_search` | function `(query: str, limit: int=10, *, base_url: str \| None=None, timeout_s: float \| None=None, settings: Settings \| None=None)` | A self-hosted SearXNG instance (``SEARXNG_URL`` or ``base_url``; its JSON API must be enabled): real web. | | `wikipedia_search` | function `(query: str, limit: int=10, *, lang: str='en', timeout_s: float \| None=None, settings: Settings \| None=None)` | Wikipedia article search (MediaWiki API, no key): title, first snippet and the article URL. Encyclopaedic,. | ## `opensmartroute.agents` ``osr run ``: launch a coding agent or AI client with the router as its model endpoint. | Name | Kind | Summary | |---|---|---| | `AGENTS` | constant | the coding agents `osr run` knows how to launch, in menu order. | | `Agent` | class | One tool the router can sit behind, and how it is told where to go. | | `agent_environment` | function `(agent: Agent, base_url: str, key: str, model: str='auto')` | The variables to set for ``agent`` (without launching anything - what ``--print-env`` shows). | | `find_agent` | function `(agent_id: str)` | Return the agent with this id; ``ConfigurationError`` names the known ids when there is none. | | `launch` | function `(agent: Agent, base_url: str, key: str, extra_args: list[str], *, model: str='auto', environ: dict[str, str] \| None=None, execvpe: Any=None)` | Replace this process with the agent's command, the router wired in through the environment. | ## `opensmartroute.aio` Async façade. Routing itself is CPU-bound and sub-millisecond, so we run it in. | Name | Kind | Summary | |---|---|---| | `AsyncRouter` | class | ``await``-able wrapper around a Router or EnterpriseRouter: route / learn / execute / run off the event loop. | ## `opensmartroute.anthropic` Anthropic **Messages API** (``POST /v1/messages``) on top of the chat-completions proxy. | Name | Kind | Summary | |---|---|---| | `ERROR_TYPES` | constant | : HTTP status -> Anthropic error ``type``. | | `MESSAGE_ROLES` | constant | : Messages ``role`` values (Anthropic has no system role - ``system`` is a top-level field). | | `STOP_REASONS` | constant | : ``finish_reason`` of a chat completion -> Messages ``stop_reason``. | | `MessageStreamTranslator` | class | Push-based translation of ``chat.completion.chunk`` SSE bytes into Messages API streaming events. | | `chat_to_message` | function `(completion: dict[str, Any], request: dict[str, Any] \| None=None)` | Shape a finished ``chat.completion`` as an Anthropic ``message`` object. | | `count_tokens` | function `(body: dict[str, Any], estimator: Any=None)` | ``POST /v1/messages/count_tokens``: ``{"input_tokens": n}`` for ``system`` + ``messages`` + ``tools``. | | `error_body` | function `(status: int, message: str, *, kind: str \| None=None)` | The ``{"type": "error", "error": {"type", "message"}}`` envelope of the Messages API. | | `message_events` | function `(events: Iterable[bytes], request: dict[str, Any] \| None=None)` | Translate a ``chat.completion.chunk`` SSE byte stream into Messages API streaming events. | | `messages_to_chat` | function `(body: dict[str, Any], *, default_model: str='auto')` | Translate an Anthropic Messages request body into a ``chat.completion`` request body. | ## `opensmartroute.branding` OpenSmartRoute naming conventions: one place for every brand-bound identifier. | Name | Kind | Summary | |---|---|---| | `API_KEY_ENV` | constant | ``OSR_API_KEY`` - access token the CLI sends (overrides the credentials file). | | `API_URL_ENV` | constant | ``OSR_API_URL`` - platform / server URL the CLI talks to. | | `BRAND` | constant | product name, one word. | | `CLI` | constant | console script name. | | `CONFIG_DIR_ENV` | constant | ``OSR_CONFIG_DIR`` - overrides the per-user config folder. | | `CONFIG_DIR_NAME` | constant | per-user config folder name (``~/.config/opensmartroute``, ``%APPDATA%\opensmartroute``). | | `ENTRY_POINT_GROUP` | constant | importlib.metadata entry-point group for plugins. | | `ENV_PREFIX` | constant | ``OSR_`` - every settings environment variable starts with this. | | `ERROR_CODE_PREFIX` | constant | ``OSR_`` - machine-readable error codes (``OSR_NO_ROUTE``). | | `INSTALL_SCRIPT_PS1` | constant | Windows installer: ``irm .../install.ps1 \| iex``. | | `INSTALL_SCRIPT_SH` | constant | Linux / macOS installer: ``curl -fsSL .../install.sh \| sh``. | | `LOCAL_TOKEN_PREFIX` | constant | self-hosted ``osr serve`` access tokens (``osr_local_...``). | | `MARKETPLACE_URL` | constant | hosted marketplace API (``osr marketplace``, MarketplaceClient). | | `METADATA_PREFIX` | constant | ``osr-`` - SKILL.md / persona front-matter keys (``osr-domains``). | | `PACKAGE` | constant | Python package / distribution / logger root. | | `REPOSITORY` | constant | public home of the release files (installers, bundles); the source repository is private. | | `SHORT_NAME` | constant | short form used for env-var and error-code prefixes. | | `SKILLS_DIR` | constant | : Default Agent-Skills root (``*/SKILL.md``); the location Claude Code discovers project skills in. | | `STATE_DIR` | constant | default learner-state directory. | | `TOKEN_PREFIX` | constant | hosted-platform API keys (``osr_live_...``). | | `WEBSITE` | constant | public website; also the default hosted-platform URL of ``osr login``. | | `env_key` | function `(*parts: str)` | ``env_key("routing", "softmax_temperature") -> "OSR_ROUTING_SOFTMAX_TEMPERATURE"``. | | `error_code` | function `(kind: str)` | ``error_code("no_route") -> "OSR_NO_ROUTE"``. | | `logger` | function `(component: str \| None=None)` | ``logger() -> "opensmartroute"``; ``logger("enterprise") -> "opensmartroute.enterprise"``. | | `metadata_key` | function `(field: str)` | ``metadata_key("quality_prior") -> "osr-quality-prior"`` (SKILL.md / persona frontmatter). | | `platform_url` | function `(environ: Mapping[str, str] \| None=None)` | The platform / server URL the CLI talks to: ``OSR_API_URL`` or the hosted platform (:data:`WEBSITE`). | | `user_agent` | function `()` | ``"opensmartroute/"`` for outbound HTTP clients. | | `version` | function `()` | The package version (``opensmartroute.__version__``), resolved at call time. | ## `opensmartroute.cli` ``osr`` command-line interface. | Name | Kind | Summary | |---|---|---| | `STARTER_COMMANDS` | constant | work without --targets (starter catalogue). | | `build_router` | function `(targets: str \| None, rules: str \| None=None, state: str \| None=None, models: str \| None=None, skills: str \| None=None, slm: str \| None=None, remember_requests: int=0)` | Assemble the CLI's router from targets / rules / state / models / SKILL.md / SLM paths (every command). | | `EXTRAS` | constant | Optional extras -> the modules they install; `osr doctor` reports which are present without importing them. | | `doctor_report` | function `(targets: str \| None=None, rules: str \| None=None, profile: str='default')` | What ``osr doctor`` prints: interpreter, package, extras, bundled model, config dir, sign-in, catalogue. | | `main` | function `(argv: list[str] \| None=None)` | Entry point of the ``osr`` console script; returns the process exit status. | ## `opensmartroute.client` Client for the hosted platform and a self-hosted ``osr serve``: route, quote, report, chat - standard library only. | Name | Kind | Summary | |---|---|---| | `AUTO_MODEL` | constant | the model name that lets the router choose on the OpenAI-compatible endpoint. | | `PLATFORM_API_PREFIX` | constant | where the REST routes live on the platform; ``osr serve`` has them at the root. | | `PRIVATE_HOST_SUFFIXES` | constant | : Host-name suffixes that mean "inside the network": a key may travel to them over plain http. | | `RETRY_STATUSES` | constant | answers that are safe to retry (nothing was served). | | `SERVER_API_PREFIX` | constant | the prefix an ``osr_local_...`` token selects. | | `HttpResponse` | constant | : What the transport returns: status, response headers, raw body (or an iterator of lines when streaming). | | `OpenSmartRoute` | class | A client for the platform (``https://opensmartroute.ai`` or your own deployment) and for ``osr serve``. | | `RemoteDecision` | class | What ``route()`` returns: the decision as the server sent it (``raw``) with the fields people read first. | | `Transport` | constant | : iterator of raw lines when ``stream`` is true. Tests inject a fake; the default is urllib. | | `is_private_host` | function `(host: str)` | True for loopback and private addresses and for names that only resolve inside a network (no dot,. | ## `opensmartroute.compress` Prompt compaction: fewer tokens into the model without changing what it is asked. | Name | Kind | Summary | |---|---|---| | `TOKENS_SAVED_KEY` | constant | context key carrying the estimated saving (ExecutionResult.tokens_saved, proxy). | | `compact_text` | function `(text: str, *, dedupe_lines: bool=True, max_blank_lines: int=1, min_dedupe_chars: int=24)` | Whitespace- and repetition-compacted ``text``; fenced code blocks are returned untouched. | | `compress_request` | function `(request: RouteRequest, settings: Settings \| None=None)` | ``(request, tokens_saved)``: text, history and ``context["system"]`` compacted per ``CompressionSettings``. | | `trim_history` | function `(history: list[dict[str, Any]], budget_tokens: int, *, keep_last_turns: int=2)` | ``history`` with the oldest non-system turns dropped until it fits ``budget_tokens`` (0 = unchanged). | ## `opensmartroute.config` Configuration loading: targets and rules from JSON or YAML. | Name | Kind | Summary | |---|---|---| | `load_document` | function `(path: str \| Path)` | Read a JSON or YAML file (YAML needs the ``yaml`` extra); raises ConfigurationError on any problem. | | `load_rules` | function `(path: str \| Path)` | Load a rules file (top-level list or ``rules:`` key) into a :class:`RulesStrategy`. | | `load_targets` | function `(path: str \| Path)` | Load a catalogue file (top-level list or ``targets:`` key) into a :class:`TargetRegistry`. | ## `opensmartroute.core` | Name | Kind | Summary | |---|---|---| | `PLAN_MODES` | re-export of [`opensmartroute.core.types.PLAN_MODES`](#opensmartroutecoretypes) | : merges their outputs in slot order, and calls the tools of one round concurrently. | | `Capabilities` | re-export of [`opensmartroute.core.types.Capabilities`](#opensmartroutecoretypes) | Declarative description of what a target is good at. | | `Objective` | re-export of [`opensmartroute.core.types.Objective`](#opensmartroutecoretypes) | What the caller wants to optimise. Weights are relative. | | `Outcome` | re-export of [`opensmartroute.core.types.Outcome`](#opensmartroutecoretypes) | Feedback about how a routed request actually went. | | `PlanSlot` | re-export of [`opensmartroute.core.types.PlanSlot`](#opensmartroutecoretypes) | One filled slot of a multi-target plan (persona -> skill -> model). | | `RankedTarget` | re-export of [`opensmartroute.core.types.RankedTarget`](#opensmartroutecoretypes) | A scored candidate: utility, ensemble quality estimate and the per-strategy breakdown. | | `RequestConstraints` | re-export of [`opensmartroute.core.types.RequestConstraints`](#opensmartroutecoretypes) | Hard constraints on the request (never traded off). | | `RouteDecision` | re-export of [`opensmartroute.core.types.RouteDecision`](#opensmartroutecoretypes) | The answer to ``route()``: chosen target, confidence, alternatives, optional plan, trace and propensities. | | `RoutePlan` | re-export of [`opensmartroute.core.types.RoutePlan`](#opensmartroutecoretypes) | A composed route (MasRouter-style): several targets working together, executed in stages. | | `RouteRequest` | re-export of [`opensmartroute.core.types.RouteRequest`](#opensmartroutecoretypes) | The customer need. | | `RouteTarget` | re-export of [`opensmartroute.core.types.RouteTarget`](#opensmartroutecoretypes) | A routable destination: an LLM, agent, skill, persona, tool, workflow or human. | | `RouteTrace` | re-export of [`opensmartroute.core.types.RouteTrace`](#opensmartroutecoretypes) | Everything needed to explain a decision. | | `Signals` | re-export of [`opensmartroute.core.types.Signals`](#opensmartroutecoretypes) | Cheap deterministic features extracted from a request. | | `StrategyScore` | re-export of [`opensmartroute.core.types.StrategyScore`](#opensmartroutecoretypes) | One strategy's opinion about one target. | | `TargetConstraints` | re-export of [`opensmartroute.core.types.TargetConstraints`](#opensmartroutecoretypes) | Where / for whom a target may be used. Checked by the policy layer. | | `TargetKind` | re-export of [`opensmartroute.core.types.TargetKind`](#opensmartroutecoretypes) | Kinds of things a request can be routed to. | | `TargetRegistry` | re-export of [`opensmartroute.core.registry.TargetRegistry`](#opensmartroutecoreregistry) | In-memory catalogue of :class:`RouteTarget` by id: add / upsert / remove, filtered listing, (de)serialisation. | ## `opensmartroute.core.registry` Target registry: the catalogue of everything a request may be routed to. | Name | Kind | Summary | |---|---|---| | `TargetRegistry` | class | In-memory catalogue of :class:`RouteTarget` by id: add / upsert / remove, filtered listing, (de)serialisation. | ## `opensmartroute.core.types` Core data model for OpenSmartRoute. | Name | Kind | Summary | |---|---|---| | `TargetKind` | class | Kinds of things a request can be routed to. | | `Capabilities` | class | Declarative description of what a target is good at. | | `TargetConstraints` | class | Where / for whom a target may be used. Checked by the policy layer. | | `RouteTarget` | class | A routable destination: an LLM, agent, skill, persona, tool, workflow or human. | | `EFFORT_LEVELS` | constant | Named reasoning-effort levels -> numeric effort in [0, 1] (``RouteTarget.effort``). | | `Objective` | class | What the caller wants to optimise. Weights are relative. | | `RequestConstraints` | class | Hard constraints on the request (never traded off). | | `RouteRequest` | class | The customer need. | | `Signals` | class | Cheap deterministic features extracted from a request. | | `StrategyScore` | class | One strategy's opinion about one target. | | `RankedTarget` | class | A scored candidate: utility, ensemble quality estimate and the per-strategy breakdown. | | `RouteTrace` | class | Everything needed to explain a decision. | | `PlanSlot` | class | One filled slot of a multi-target plan (persona -> skill -> model). | | `PLAN_MODES` | constant | : merges their outputs in slot order, and calls the tools of one round concurrently. | | `RoutePlan` | class | A composed route (MasRouter-style): several targets working together, executed in stages. | | `RouteDecision` | class | The answer to ``route()``: chosen target, confidence, alternatives, optional plan, trace and propensities. | | `Outcome` | class | Feedback about how a routed request actually went. | ## `opensmartroute.credentials` Credentials for the ``osr`` CLI: where the access token lives and how it is obtained. | Name | Kind | Summary | |---|---|---| | `CREDENTIALS_FILE` | constant | file name inside :func:`config_dir`. | | `DEFAULT_PROFILE` | constant | profile used when ``--profile`` is not given. | | `DEVICE_CODE_PATH` | constant | RFC 8628 device-authorization endpoint of the platform. | | `DEVICE_TOKEN_PATH` | constant | noqa: S105 # nosec B105 - RFC 8628 token endpoint (a URL path). | | `ME_PATH` | constant | platform: who am I (workspace, plan, edition). | | `TOKEN_BYTES` | constant | entropy of :func:`generate_token`. | | `WHOAMI_PATH` | constant | self-hosted ``osr serve``: is this token accepted. | | `Credential` | class | One saved sign-in: where (``url``), what (``token``) and what the platform said about it. | | `CredentialStore` | class | Profiles in ``/credentials.json`` (owner-only permissions on POSIX). | | `PlatformClient` | class | Minimal JSON client for the platform / server API used by the CLI (stdlib only, injectable transport). | | `Transport` | constant | : ``(method, url, headers, body, timeout) -> (status, json)``; tests inject a fake instead of urllib. | | `apply_identity` | function `(cred: Credential, info: Mapping[str, Any])` | Copy what :func:`whoami` learned (kind, workspace, plan, edition) onto ``cred``. | | `config_dir` | function `(environ: Mapping[str, str] \| None=None)` | Per-user configuration directory (``OSR_CONFIG_DIR`` > ``%APPDATA%`` > ``$XDG_CONFIG_HOME`` > ``~/.config``). | | `device_login` | function `(url: str \| None=None, *, client_name: str \| None=None, open_browser: bool=True, out: Callable[[str], None]=print, transport: Transport \| None=None, sleep: Callable[[float], None]=time.sleep, timeout_s: float \| None=None)` | Sign in to the hosted platform with the device authorization grant and return the credential. | | `generate_token` | function `(prefix: str=LOCAL_TOKEN_PREFIX)` | A fresh random access token (``osr_local_<43 chars>``) for a self-hosted ``osr serve``. | | `redact` | function `(token: str, keep: int=6)` | ``osr_live_abc123...`` - the prefix plus a few characters, never the whole token. | | `token_kind` | function `(token: str)` | ``"platform"`` for ``osr_live_`` keys, ``"server"`` for ``osr_local_`` tokens, else ``"unknown"``. | | `whoami` | function `(cred: Credential, *, transport: Transport \| None=None, timeout: float=30.0)` | Validate ``cred`` against its server and describe the identity behind it. | ## `opensmartroute.data` Data files shipped inside the package. No public names. ## `opensmartroute.discovery` Tool discovery beyond text similarity. | Name | Kind | Summary | |---|---|---| | `CachePreservingSelector` | class | Keep the serialized tool list prefix-stable across turns of a session. | | `SchemaAwareStrategy` | class | Scores tools by how many of their ``input_schema`` parameters the request can fill (see ``schema_match``). | | `SkillGraph` | class | Dependency / conflict / composition graph over skill targets. | | `extract_entities` | function `(text: str)` | Typed entity mentions found in ``text`` (kind -> values). | | `schema_match` | function `(request: RouteRequest \| str, target: RouteTarget)` | Return ``(coverage, filled, missing)`` for the target's input schema. | ## `opensmartroute.enterprise` Enterprise integration layer: ports (hexagonal architecture), middleware and telemetry. | Name | Kind | Summary | |---|---|---| | `AuditSink` | class | Tamper-evident audit trail port (hash-chained). | | `CacheMiddleware` | class | LRU decision cache keyed on (text, constraints, objective, route options). TTL in seconds. | | `EnterpriseRouter` | class | Router + middleware chain + telemetry + auto-learning + audit. Thread-safe façade. | | `FileAuditSink` | class | Append-only JSONL, each line ``{prev, hash, record}`` with. | | `FileStateStore` | class | One JSON file per key under ``root``; keys are hashed so they can't traverse paths. | | `InMemoryStateStore` | class | Thread-safe dict-backed :class:`StateStore`; values are deep-copied through JSON on read and write. | | `LiveContext` | re-export of [`opensmartroute.enterprise.grounding.LiveContext`](#opensmartrouteenterprisegrounding) | What one grounding retrieved: the query, the hits (as dicts), when, how long, and the signal that asked. | | `LoggingTelemetry` | class | Structured JSON logs; never logs raw request text (only a hash + length). | | `MetricsTelemetry` | class | In-process counters, outcome tallies and a latency histogram. | | `Middleware` | class | Chain-of-responsibility hook around routing: ``__call__(request, next_) -> RouteDecision``. | | `RouteFn` | constant | the "next" callable a middleware wraps. | | `RouterBuilder` | class | Fluent builder that validates configuration and wires all enterprise pieces. | | `SavingsEntry` | re-export of [`opensmartroute.enterprise.savings.SavingsEntry`](#opensmartrouteenterprisesavings) | One routed request in the ledger. | | `SavingsLedger` | re-export of [`opensmartroute.enterprise.savings.SavingsLedger`](#opensmartrouteenterprisesavings) | Telemetry sink that keeps a per-request baseline-vs-routed cost ledger (see module docs). | | `SavingsReport` | re-export of [`opensmartroute.enterprise.savings.SavingsReport`](#opensmartrouteenterprisesavings) | Aggregate savings and the quality they were bought at. | | `StateStore` | class | Key/value persistence port for learner state, caches, breaker state. | | `Telemetry` | class | Observer port with no-op defaults. Implement for OpenTelemetry, Prometheus, Datadog…. | | `TenantMiddleware` | class | Enforces that a tenant is present and applies per-tenant defaults/limits. | | `TimeoutMiddleware` | class | Soft deadline: raise if routing itself exceeded ``budget_ms`` (should never happen. | | `WebGroundingMiddleware` | re-export of [`opensmartroute.enterprise.grounding.WebGroundingMiddleware`](#opensmartrouteenterprisegrounding) | Search the web for requests about current facts and put the hits in front of the model (module docstring). | | `harden` | function `(router: Router, *, guard: bool=False, redact_pii: bool=False, learned_guard: bool=False, output_guard: str \| Any='off', health: bool=False, latency_slo_ms: float \| None=None, audit: str \| Path \| AuditSink \| None=None, audit_outcomes: bool=False, origin_policy: Any=False, limits: Any=False, web_grounding: Any=False, knowledge: Any=False, state_store: StateStore \| str \| None=None, save_every: int=1, middleware: list[Middleware] \| None=None, telemetry: list[Telemetry] \| None=None)` | Wrap an already built :class:`Router` in the production controls (what ``osr serve --guard ...`` does). | | `search_query` | re-export of [`opensmartroute.enterprise.grounding.search_query`](#opensmartrouteenterprisegrounding) | The search query for a request: its first sentence or line, whitespace-normalised, filler words dropped. | ## `opensmartroute.enterprise.grounding` Live web context in front of the model: search the web for requests about *now* and hand the hits to the target. | Name | Kind | Summary | |---|---|---| | `GROUNDING_HEADER` | constant | : Opens the block appended to the system prompt; ``{when}`` is the retrieval time in UTC. | | `LiveContext` | class | What one grounding retrieved: the query, the hits (as dicts), when, how long, and the signal that asked. | | `WebGroundingMiddleware` | class | Search the web for requests about current facts and put the hits in front of the model (module docstring). | | `search_query` | function `(text: str, max_chars: int=200)` | The search query for a request: its first sentence or line, whitespace-normalised, filler words dropped. | ## `opensmartroute.enterprise.ops` Operational controls: shadow / A-B routing, tenant fairness and queue-aware latency. | Name | Kind | Summary | |---|---|---| | `SPRT` | class | Wald SPRT for two Bernoulli success rates: H0 p=p0 vs H1 p=p0+delta. | | `ABTest` | class | Traffic split + SPRT comparison of candidate vs control outcomes. | | `FairShareMiddleware` | class | Dominant Resource Fairness across tenants over a sliding window. | | `InflightTracker` | class | Per-target in-flight counters + arrival/service statistics for queueing estimates. | | `QueueAwareStrategy` | class | Scores targets by *current* time-to-first-token = catalogue latency + queueing wait vs. | | `ShadowMiddleware` | class | Run ``candidate`` beside production. ``mode='shadow'``: log only. ``mode='ab'``: serve. | | `TenantUsage` | class | Sliding-window ledger of one tenant's requests, cost and tokens for :class:`FairShareMiddleware`. | ## `opensmartroute.enterprise.savings` Savings ledger - the always-on savings report that backs the ROI story and the dashboard. | Name | Kind | Summary | |---|---|---| | `SavingsEntry` | class | One routed request in the ledger. | | `SavingsLedger` | class | Telemetry sink that keeps a per-request baseline-vs-routed cost ledger (see module docs). | | `SavingsReport` | class | Aggregate savings and the quality they were bought at. | ## `opensmartroute.enterprise.stores` Production state-store backends and wrappers. | Name | Kind | Summary | |---|---|---| | `BatchedStateStore` | class | Write-behind buffer. Reads are served from the pending map first. | | `EncryptedStateStore` | class | AES-256-GCM envelope encryption. ``key`` is 32 raw bytes, or read (base64/hex/raw). | | `Migration` | constant | state-document transform applied by VersionedStateStore. | | `NamespacedStateStore` | class | Prefix every key with ``/`` so several routers or tenants can share one backing store. | | `RedisStateStore` | class | StateStore on any redis-py-compatible client (``get`` / ``set`` / ``delete``), JSON values, optional TTL. | | `SQLStateStore` | class | DB-API 2.0 key/value store (PostgreSQL, SQLite, MySQL). | | `VersionedStateStore` | class | Schema-versioned envelope with forward migrations. | | `open_state_store` | function `(url: str, *, encrypt_key_env: str \| None=None)` | Build a store from a URL (``osr serve --state-store``, ``OSR_SERVER_STATE_STORE``). | ## `opensmartroute.errors` Exception hierarchy. Every error raised by the SDK derives from :class:`OpenSmartRouteError`. | Name | Kind | Summary | |---|---|---| | `OpenSmartRouteError` | class | Base class for all SDK errors. | | `ConfigurationError` | class | Invalid catalogue, rules, SKILL.md, settings or a missing optional dependency. | | `ValidationError` | class | A request, outcome or target failed validation. | | `NoRouteError` | class | No target satisfied the hard constraints. | | `TargetUnavailableError` | class | A remote target or catalogue source could not be reached. | | `ExecutionError` | class | A target handler failed while executing a plan. | | `SecurityError` | class | Request rejected by an input guard (prompt injection, oversize, etc.). | | `StateStoreError` | class | A learner-state store failed to load, save or migrate. | | `AuthenticationError` | class | The CLI has no valid access token for the platform / server, or a sign-in was denied or timed out. | | `APIError` | class | The platform or ``osr serve`` answered a call with an error status; ``status``, ``detail`` and the. | | `RateLimitError` | class | 429: the rate limit, the monthly quota or a budget is exhausted; ``retry_after`` says when to try again. | | `OpenSmartRouteDeprecationWarning` | class | Emitted by :func:`deprecated`; filter with ``warnings.simplefilter`` on this class. | | `deprecated` | function `(name: str, *, since: str, removal: str, replacement: str \| None=None, stacklevel: int=3)` | Announce a deprecation according to the stability policy in SDK.md. | ## `opensmartroute.estimate` Token, cost and latency estimates *before* a request is sent anywhere. | Name | Kind | Summary | |---|---|---| | `DEFAULT_OUTPUT_TOKENS` | constant | : Output length assumed when neither the caller nor the signals say (a short answer). | | `MESSAGE_OVERHEAD_TOKENS` | constant | : Tokens a chat API adds per message for role / separators (OpenAI-style framing). | | `PriceHook` | constant | : ``(usd_per_1k_input, usd_per_1k_output)`` for a target, or ``None`` to fall back to its declared cost. | | `RequestEstimate` | class | A quote for one request across every candidate, with named picks. | | `TargetEstimate` | class | The quote for one candidate target. | | `build_token_counter` | function `(spec: str)` | The counter a tokenizer ``spec`` names (``EstimateSettings.tokenizer``): ``tiktoken:`` counts with. | | `estimate` | function `(router: Router, request: RouteRequest \| str, *, output_tokens: int \| None=None, kinds: list[str] \| None=None, prices: PriceHook \| None=None, quality_tolerance: float=0.1, decision: RouteDecision \| None=None)` | Quote ``request`` against every candidate the router would consider - nothing is executed. | | `estimate_messages_tokens` | function `(messages: list[dict[str, Any]])` | Token estimate for an OpenAI-style message list (content plus per-message framing). | | `estimate_tokens` | function `(text: str)` | Token count of ``text``: the installed :data:`token_counter` when there is one (installed by. | | `heuristic_tokens` | function `(text: str)` | The dependency-free tokenizer approximation (what :func:`estimate_tokens` uses unless a counter is installed). | | `set_token_counter` | function `(counter: Callable[[str], int] \| None)` | Install (or, with ``None``, remove) the process-wide token counter behind :func:`estimate_tokens`; the. | | `target_prices` | function `(target: RouteTarget)` | ``(usd per 1k input tokens, usd per 1k output tokens)`` from the target's declared cost. | | `token_counter` | constant | : :class:`~opensmartroute.learning.neural.TokenCountModel` trained on billed counts (``None`` = the heuristic). | ## `opensmartroute.eval` RouterBench-style evaluation harness. | Name | Kind | Summary | |---|---|---| | `AuditReport` | re-export of [`opensmartroute.eval.audit.AuditReport`](#opensmartrouteevalaudit) | Aggregate of a shadow replay; ``to_markdown`` renders the Routing Audit deliverable. | | `AuditRow` | re-export of [`opensmartroute.eval.audit.AuditRow`](#opensmartrouteevalaudit) | One logged request: what happened, and what the router would have done. | | `DatasetCollector` | re-export of [`opensmartroute.eval.collect.DatasetCollector`](#opensmartrouteevalcollect) | Cache-backed corpus builder: collect sources, add feedback / synthetic rows, dedupe, split. | | `DatasetSource` | re-export of [`opensmartroute.eval.collect.DatasetSource`](#opensmartrouteevalcollect) | One Hub dataset split to collect: repo id, config, split, the preset that parses it, an optional model map. | | `EvalResult` | class | Aggregate routing metrics over a dataset: accuracy, cost, latency, confidence, calibration, coverage. | | `EvalRow` | class | One labelled prompt: expected / acceptable targets or per-target quality scores (RouterBench style). | | `area_under_frontier` | function `(points: list[dict[str, float]])` | Trapezoidal area under accuracy(cost) — analogous to RouterBench's AIQ. | | `calibration_report` | function `(router: Router, rows: list[EvalRow], objective: Objective \| None=None)` | Confidence calibration of the router on labelled rows: ECE, Brier, reliability bins and. | | `collect_dataset` | re-export of [`opensmartroute.eval.collect.collect_dataset`](#opensmartrouteevalcollect) | Download ``source`` from the Hub and parse it into rows. | | `cost_quality_frontier` | function `(router: Router, rows: list[EvalRow], cost_weights: list[float] \| None=None)` | Sweep the cost weight to trace the accuracy-vs-cost curve (RouterBench Fig. 1 style). | | `evaluate` | function `(router: Router, rows: list[EvalRow], objective: Objective \| None=None)` | Route every row and score accuracy, realised quality, cost, latency, ECE / Brier and conformal coverage. | | `fetch_hf_rows` | re-export of [`opensmartroute.eval.collect.fetch_hf_rows`](#opensmartrouteevalcollect) | One page (max 100) of records from the datasets-server ``/rows`` endpoint, flattened to ``{column: value}``. | | `load_audit_log` | re-export of [`opensmartroute.eval.audit.load_audit_log`](#opensmartrouteevalaudit) | Read a JSONL traffic log (``text`` or ``prompt``/``messages`` per line). | | `load_dataset` | function `(path: str \| Path)` | Read a JSONL evaluation dataset (``text`` or ``prompt`` plus the optional EvalRow keys); a line that is. | | `model_quality` | re-export of [`opensmartroute.eval.collect.model_quality`](#opensmartrouteevalcollect) | Data-derived quality prior per model in [0, 1]: pairwise rows (two scored models) are fitted with. | | `routing_audit` | re-export of [`opensmartroute.eval.audit.routing_audit`](#opensmartrouteevalaudit) | Replay ``rows`` through ``router`` in shadow mode and aggregate an :class:`AuditReport`. | | `rows_from_feedback` | re-export of [`opensmartroute.eval.collect.rows_from_feedback`](#opensmartrouteevalcollect) | Rows from :class:`Outcome` records. ``texts`` maps ``request_id`` to the prompt (outcomes carry no text);. | | `summarize_audit` | re-export of [`opensmartroute.eval.audit.summarize_audit`](#opensmartrouteevalaudit) | Aggregate :class:`AuditRow` records into an :class:`AuditReport`. | | `synthetic_followup_rows` | re-export of [`opensmartroute.eval.collect.synthetic_followup_rows`](#opensmartrouteevalcollect) | Conversation rows built from labelled ``rows``. | | `synthetic_rows` | re-export of [`opensmartroute.eval.collect.synthetic_rows`](#opensmartrouteevalcollect) | Ontology seed prompts (templates and, with ``natural``, the everyday phrasings - see. | | `synthetic_slot_rows` | re-export of [`opensmartroute.eval.collect.synthetic_slot_rows`](#opensmartrouteevalcollect) | Ontology seed prompts scored for a plan slot: every ``role`` target of the catalogue (personas, skills,. | ## `opensmartroute.eval.agentic` tau-bench-style agentic task evaluation: does per-step routing beat the best single agent?. | Name | Kind | Summary | |---|---|---| | `AgentTask` | class | A multi-step task with the measured per-step success probability and latency of each agent. | | `load_agentic_tasks` | function `(path: str \| Path)` | Read tasks from JSONL: ``{"task_id", "steps", "success", "latency_ms", "cost_usd"?, "domain"?}``. | | `synthetic_agentic_tasks` | function `(n: int=200, *, seed: int=0, hard_share: float=0.3)` | Three agents (fast / balanced / strong) and ``n`` tasks of 3-5 steps whose steps are mostly. | | `task_routing_frontier` | function `(registry: TargetRegistry, tasks: Sequence[AgentTask], *, retries: int=1, accuracy_tolerance: float=0.02, latency_ratio_target: float=0.9, objective: Objective \| None=None, router_factory: Callable[[TargetRegistry], Router] \| None=None, seed: int=0)` | Replay ``tasks`` step by step through :class:`ProgressRouter` (each agent's handler draws the. | ## `opensmartroute.eval.audit` Routing Audit - shadow-mode replay of logged LLM traffic to quantify what the router would change. | Name | Kind | Summary | |---|---|---| | `DEFAULT_COMPLETION_TOKENS` | constant | : Default assumed output length when a log row carries no ``completion_tokens``. | | `AuditReport` | class | Aggregate of a shadow replay; ``to_markdown`` renders the Routing Audit deliverable. | | `AuditRow` | class | One logged request: what happened, and what the router would have done. | | `load_audit_log` | function `(path: str \| Path)` | Read a JSONL traffic log (``text`` or ``prompt``/``messages`` per line). | | `routing_audit` | function `(router: Router, rows: list[dict[str, Any]], *, baseline: str \| None=None, monthly_requests: int \| None=None, plan: bool=False, keep_rows: bool=True)` | Replay ``rows`` through ``router`` in shadow mode and aggregate an :class:`AuditReport`. | | `summarize_audit` | function `(rows: list[AuditRow], *, monthly_requests: int \| None=None, keep_rows: bool=True)` | Aggregate :class:`AuditRow` records into an :class:`AuditReport`. | ## `opensmartroute.eval.baselines` Baselines every router must beat, plus oracle ceilings and the sampling noise floor. | Name | Kind | Summary | |---|---|---| | `Policy` | constant | a baseline: (row, targets) -> chosen target id. | | `PolicyResult` | class | Metrics of one baseline policy on a dataset (same fields as the router's EvalResult headline). | | `baseline_suite` | function `(rows: list[EvalRow], targets: Sequence[RouteTarget], train_rows: list[EvalRow] \| None=None, quality_floor: float=0.7, seed: int=0)` | Run every baseline and both oracles; ``train_rows`` fit the task table. | | `best_prior_policy` | function `(row: EvalRow, targets: Sequence[RouteTarget])` | Always the target with the highest declared ``quality_prior``. | | `cheapest_policy` | function `(row: EvalRow, targets: Sequence[RouteTarget])` | Always the lowest unit cost (ties broken by latency). | | `compare_policies` | function `(router_hits: Sequence[bool], baselines: Sequence[PolicyResult], *, seed: int=0, n_boot: int=2000, level: float=0.95)` | The router against each baseline **on the same rows**: accuracy delta with a paired percentile-bootstrap. | | `evaluate_policy` | function `(name: str, policy: Policy, rows: list[EvalRow], targets: Sequence[RouteTarget])` | Run a baseline :data:`Policy` over the rows and aggregate accuracy, quality, cost and latency. | | `fit_task_table` | function `(rows: list[EvalRow], targets: Sequence[RouteTarget])` | ``task_type -> target`` with the highest mean quality on the training rows. | | `held_out_split` | function `(rows: Sequence[EvalRow], seed: int=0, train_fraction: float=0.5)` | A seeded shuffle split into ``(train, test)``: the task table (and anything else fitted) learns from. | | `most_expensive_policy` | function `(row: EvalRow, targets: Sequence[RouteTarget])` | Always the highest unit cost - the "just use the frontier model" baseline. | | `multi_sample_oracle` | function `(rows: list[EvalRow], targets: Sequence[RouteTarget])` | Mean quality of the oracle that picks by the *mean over all samples* per target. | | `noise_floor` | function `(rows: list[EvalRow], targets: Sequence[RouteTarget], seed: int=0)` | How much accuracy is lost to sampling noise alone. | | `oracle_policy` | function `(quality_floor: float \| None=None)` | Per-row best target. With ``quality_floor`` -> cheapest target reaching the floor. | | `random_policy` | function `(seed: int=0)` | Uniformly random target (seeded). | | `static_task_table_policy` | function `(table: dict[str, str])` | Look the row's ``task_type`` up in a table from :func:`fit_task_table` (``"*"`` = default). | ## `opensmartroute.eval.collect` Collect routing datasets from the Hugging Face Hub, your own feedback log and synthetic seeds. | Name | Kind | Summary | |---|---|---| | `ARENA_TIERS` | constant | everything else in those datasets is a 7B-14B chat model -> "small". | | `DATASETS_SERVER` | re-export of [`opensmartroute.adapters.websearch.DATASETS_SERVER`](#opensmartrouteadapterswebsearch) | Hugging Face datasets-server REST root (/rows). | | `DEFAULT_SOURCES` | constant | public ones. | | `FOLLOWUP_PROMPTS` | constant | : task-neutral: "translate it" or "summarise that" name a task of their own and belong to a different target. | | `HISTORY_FILE` | constant | the self-improver's report log; lives in the cache dir but is not a dataset. | | `KNOWN_SOURCES` | constant | pairwise human / judge preference battles - the winner is the label. | | `PAIRWISE` | constant | preset name for battle datasets: prompt / conversation, model_a, model_b, winner. | | `PARSERS` | constant | collector presets that are not wide / nested tables. | | `REWARD_BENCH` | constant | preset name for RewardBench: prompt, chosen_model, rejected_model, subset. | | `ROUTELLM_GOOD_ENOUGH` | constant | RouteLLM's threshold: a weak-model score >= 4 means the cheap model was good enough. | | `ROUTELLM_GPT4` | constant | preset name for RouteLLM's gpt4_dataset: prompt + GPT-4-judged Mixtral score 1-5. | | `ROUTELLM_STRONG` | constant | the strong model of RouteLLM's gpt4_dataset (its answers are the reference). | | `ROUTELLM_WEAK` | constant | the weak model whose answer GPT-4 scores 1-5 against the reference. | | `ULTRAFEEDBACK` | constant | preset name for UltraFeedback: instruction + N completions with a model and score. | | `DatasetCollector` | class | Cache-backed corpus builder: collect sources, add feedback / synthetic rows, dedupe, split. | | `DatasetSource` | class | One Hub dataset split to collect: repo id, config, split, the preset that parses it, an optional model map. | | `Parser` | constant | record -> row (or None to skip). | | `collect_dataset` | function `(source: DatasetSource, *, timeout_s: float \| None=None, settings: Settings \| None=None)` | Download ``source`` from the Hub and parse it into rows. | | `fetch_hf_rows` | function `(dataset: str, *, config: str='default', split: str='train', offset: int=0, length: int=100, timeout_s: float \| None=None, settings: Settings \| None=None)` | One page (max 100) of records from the datasets-server ``/rows`` endpoint, flattened to ``{column: value}``. | | `from_pairwise_row` | function `(rec: dict[str, Any], model_map: dict[str, str] \| None=None, source: DatasetSource \| None=None)` | A battle record -> row with ``scores`` 1 / 0 for winner / loser (0.5 each on a tie). Understands the. | | `from_reward_bench_row` | function `(rec: dict[str, Any], source: DatasetSource \| None=None)` | A RewardBench record (``prompt``, ``chosen_model``, ``rejected_model``, ``subset``) -> a pairwise row. | | `from_routellm_gpt4_row` | function `(rec: dict[str, Any], source: DatasetSource \| None=None)` | A ``routellm/gpt4_dataset`` record (``prompt`` + ``mixtral_score`` 1-5) -> the strong model scores 1.0 and. | | `from_ultrafeedback_row` | function `(rec: dict[str, Any], source: DatasetSource \| None=None)` | An UltraFeedback record (``instruction`` + ``completions[{model, overall_score}]``) -> a scored row with. | | `iter_hf_rows` | function `(source: DatasetSource, *, offset: int=0, timeout_s: float \| None=None, settings: Settings \| None=None)` | Page through ``source`` from raw record ``offset`` until ``source.limit`` records or the split is exhausted. | | `model_quality` | function `(rows: Iterable[EvalRow], *, min_n: int=20, epochs: int=5, seed: int=0)` | Data-derived quality prior per model in [0, 1]: pairwise rows (two scored models) are fitted with. | | `rows_from_feedback` | function `(outcomes: Iterable[Outcome] \| FeedbackStore, texts: dict[str, str] \| None=None, *, min_quality: float=0.5, role: str \| None=None)` | Rows from :class:`Outcome` records. ``texts`` maps ``request_id`` to the prompt (outcomes carry no text);. | | `rows_from_records` | function `(records: Iterable[dict[str, Any]], source: DatasetSource)` | Parse raw records with the source's preset (``text_key`` is tried first when set). | | `synthetic_followup_rows` | function `(rows: Sequence[EvalRow], *, per_row: int=1, switches: int=0, seed: int=0, assistant: str='(answer)')` | Conversation rows built from labelled ``rows``. | | `synthetic_rows` | function `(targets: Sequence[RouteTarget], *, per_template: int=2, seed: int=0, router: Any \| None=None, natural: bool \| int=True, natural_phrases: Mapping[str, Sequence[str]] \| None=None)` | Ontology seed prompts (templates and, with ``natural``, the everyday phrasings - see. | | `synthetic_slot_rows` | function `(targets: Sequence[RouteTarget], role: str, *, per_template: int=2, seed: int=0)` | Ontology seed prompts scored for a plan slot: every ``role`` target of the catalogue (personas, skills,. | | `tier_model_map` | function `(tiers: Mapping[str, str], models: Iterable[str]=())` | ``model_map`` for a :class:`DatasetSource`: every known battle model (plus ``models``) -> the target id. | | `tier_of` | function `(model: str)` | Tier of a battle-dataset model name: the :data:`ARENA_TIERS` entry, else the name rules, else the parameter. | ## `opensmartroute.eval.criteria` Offline realisations of the release exit criteria. | Name | Kind | Summary | |---|---|---| | `CriterionResult` | class | One measured exit criterion. | | `bootstrap_ci` | function `(values: Sequence[float], *, n_boot: int=1000, level: float=0.95, seed: int=0)` | Percentile bootstrap confidence interval for the mean of ``values``. | | `cold_start_ratio` | function `(n_outcomes: int=200, *, domain: str='legal', n_eval: int=200, seed: int=0, target_ratio: float=0.9, generalist_quality: float=0.6, explore_rate: float=0.1, max_requests: int=20000)` | Accuracy on ``domain`` prompts of a target declared with only ``id`` / ``kind`` / ``cost``. | | `conformal_coverage` | function `(alpha: float=0.1, *, n_cal: int=1000, n_test: int=1000, label_noise: float=0.5, seed: int=0, tolerance: float=0.02)` | Fit :class:`ConformalCalibrator` on the real router's propensities over labelled prompts and. | | `domain_expert_registry` | function `(domains: Sequence[str]=_DOMAINS, *, strip: str \| None=None, generalist: bool=False)` | One hand-configured expert per domain. ``strip=`` replaces that expert with a. | | `effort_token_savings` | function `(registry: TargetRegistry, rows: Sequence[EvalRow], *, quality_tolerance: float=0.02, token_ratio_target: float=0.7, objective: Objective \| None=None, router_factory: Callable[[TargetRegistry], Router] \| None=None)` | Route rows that carry per-target ``scores`` *and* ``tokens`` and compare the tokens spent. | | `knapsack_never_exceeds_cap` | function `(steps: int=1000000, *, window: int=1000, drift_every: int=100000, seed: int=0)` | Drive :class:`MultiKnapsackBandit` (``on_capped="abstain"``) for ``steps`` pulls with. | | `match_at_1` | function `(router: Router \| EnterpriseRouter, prompts: Sequence[tuple[str, str]])` | Fraction of ``(text, expected_target_id)`` pairs the router gets right at rank 1. | | `match_at_1_at_scale` | function `(small: int=50, large: int=5000, *, n_prompts: int=200, seed: int=0, max_drop: float=0.05, narrow_above: int=32, narrow_to: int=24)` | Match@1 on a ``small`` catalogue versus a ``large`` one with retrieve-then-rank narrowing. | | `multi_round_vs_best_single` | function `(registry: TargetRegistry, rows: Sequence[EvalRow], *, threshold: float=0.8, max_rounds: int=3, failures_before_switch: int=1, objective: Objective \| None=None, cost_ratio_target: float=0.6, router_factory: Callable[[TargetRegistry], Router] \| None=None)` | Run :class:`MultiRoundExecutor` over rows that carry per-target ``scores`` (RouterBench. | | `ope_within_live_ci` | function `(n_log: int=2000, *, seed: int=0, logging_objective: Objective \| None=None, target_objective: Objective \| None=None, logging_temperature: float=0.3)` | Log decisions from a cost-seeking router (actions *sampled* from its propensities), estimate. | | `run_all` | function `(*, quick: bool=True, seed: int=0)` | Every offline criterion. ``quick`` shrinks the expensive simulations (10^5 steps, 1 000 tools,. | | `synthetic_effort_rows` | function `(n: int=400, *, seed: int=0)` | One reasoning model exposed as two effort siblings (``reasoner@fast`` with ``effort="low"``. | | `synthetic_scored_rows` | function `(n: int=300, *, seed: int=0)` | A three-tier catalogue (small / medium / large) and RouterBench-style rows with a quality. | | `synthetic_tool_catalogue` | function `(n: int, *, seed: int=0)` | ``n`` distinct tools (verb x object x qualifier, up to 5 000 unique combinations) and one. | ## `opensmartroute.eval.datasets` Adapters from public routing benchmarks to :class:`EvalRow`. | Name | Kind | Summary | |---|---|---| | `PRESETS` | constant | Named layouts accepted by ``load_benchmark`` / ``osr eval --preset``. | | `NestedPreset` | class | Layout of a "nested" benchmark: a per-model mapping under ``container_keys`` holding score / cost / samples. | | `WidePreset` | class | Column layout of a "wide" benchmark: one row per prompt, ```` columns for scores / costs. | | `from_nested_row` | function `(row: dict[str, Any], preset: NestedPreset, model_map: dict[str, str] \| None=None)` | Convert one nested-format record to an :class:`EvalRow`, keeping repeated samples when present. | | `from_wide_row` | function `(row: dict[str, Any], preset: WidePreset, model_map: dict[str, str] \| None=None)` | Convert one wide-format record to an :class:`EvalRow` (None when it has no prompt or scores). | | `load_benchmark` | function `(path: str \| Path, preset: str \| WidePreset \| NestedPreset='routerbench', model_map: dict[str, str] \| None=None, limit: int \| None=None)` | Load a benchmark file (.jsonl / .json / .csv) into :class:`EvalRow` objects. | | `models_in` | function `(rows: Iterable[EvalRow])` | Distinct target ids that carry scores in the rows, in first-seen order. | ## `opensmartroute.eval.frontier` Three-objective frontier and ablations. | Name | Kind | Summary | |---|---|---| | `ablation_report` | function `(router: Router, rows: list[EvalRow], objective: Objective \| None=None)` | Leave-one-out over strategies (and extractors, when explicitly set on the router). | | `frontier3` | function `(router: Router, rows: list[EvalRow], cost_weights: list[float] \| None=None, latency_weights: list[float] \| None=None)` | Sweep cost / latency objective weights and return the Pareto-optimal (quality, cost, latency) points. | | `hypervolume` | function `(points: list[dict[str, float]], ref: tuple[float, float, float] \| None=None)` | Dominated hypervolume of the Pareto points in normalised (quality, cost, latency) space. | ## `opensmartroute.eval.headroom` Routing headroom: when does routing pay, and how much catalogue does it need?. | Name | Kind | Summary | |---|---|---| | `DiversityReport` | class | How differently the targets score rows: pairwise disagreement, winner entropy and winner share. | | `HeadroomReport` | class | Oracle vs best-single quality on a dataset, with the label-noise floor that says whether the gap is real. | | `learnability_by_difficulty` | function `(rows: Sequence[EvalRow], targets: Sequence[RouteTarget], buckets: int=4)` | Routing gain (oracle - best single) per difficulty bucket. Difficulty of a row is. | | `min_catalogue` | function `(rows: Sequence[EvalRow], targets: Sequence[RouteTarget], fraction: float=0.95)` | Greedy forward selection: the smallest ordered subset whose oracle reaches ``fraction`` of the. | | `routing_headroom` | function `(rows: Sequence[EvalRow], targets: Sequence[RouteTarget], seed: int=0)` | Oracle minus best-single quality with the label-noise floor for context. | | `scaling_curve` | function `(rows: Sequence[EvalRow], targets: Sequence[RouteTarget], sizes: Sequence[int] \| None=None, *, trials: int=20, seed: int=0)` | Mean oracle / best-single quality and headroom on random subsets of each size. | | `target_diversity` | function `(rows: Sequence[EvalRow], targets: Sequence[RouteTarget])` | How different the targets are on this data: pairwise score disagreement and winner spread. | ## `opensmartroute.eval.ope` Off-policy evaluation from logged routing decisions. | Name | Kind | Summary | |---|---|---| | `LoggedDecision` | class | One logged routing event for off-policy evaluation: text, action taken, its propensity and observed reward. | | `OPEResult` | class | IPS, self-normalised IPS and doubly-robust value estimates with effective sample size and clip count. | | `ips_confidence_interval` | function `(logs: list[LoggedDecision], weights: list[float], z: float=1.96)` | Normal-approximation CI for the IPS estimate given per-sample weights. | | `mean_reward_model` | function `(logs: list[LoggedDecision])` | Simplest DR reward model: mean logged reward per action, global mean for unseen actions. | | `off_policy_evaluate` | function `(router: Router, logs: list[LoggedDecision], objective: Objective \| None=None, max_weight: float=20.0, reward_model: Callable[[LoggedDecision, str], float] \| None=None, deterministic: bool=False)` | Estimate the value of ``router`` on logged traffic. | | `target_propensities` | function `(router: Router, lg: LoggedDecision, objective: Objective \| None=None)` | The evaluated router's ``P(target \| x)`` for a logged context (empty when it has no route). | ## `opensmartroute.eval.robustness` Robustness and fairness checks for a router. | Name | Kind | Summary | |---|---|---| | `coreset` | function `(rows: list[EvalRow], k: int, embedder: Callable[[list[str]], list[list[float]]] \| None=None, seed: int=0)` | Greedy k-center (farthest-first) subset of ``rows`` for a diverse evaluation set. | | `diversity` | function `(rows: list[EvalRow], embedder: Callable[[list[str]], list[list[float]]] \| None=None, sample: int=300)` | Mean pairwise cosine distance of the row texts (sampled), 0 = all identical. | | `paraphrase_robustness` | function `(router: Router, rows: list[EvalRow], n: int=4, objective: Objective \| None=None, seed: int=0)` | Share of rule-based paraphrases that route to the same target as the original (``robustness``). | | `paraphrases` | function `(text: str, n: int=4, seed: int=0)` | ``n`` deterministic surface-level paraphrases of ``text`` (never returns the original). | | `profile_swap_fairness` | function `(router: Router, rows: list[EvalRow], profiles: Sequence[dict[str, Any]] \| None=None, objective: Objective \| None=None)` | Decision agreement across user profiles. ``dependence`` = fraction of rows whose. | | `repeat_flip_rate` | function `(router: Router, rows: list[EvalRow], k: int=5, objective: Objective \| None=None)` | Route each row ``k`` times; ``flip_rate`` = share of rows whose decision is not identical every time. | ## `opensmartroute.execution` Plan-aware execution: turn a :class:`RouteDecision` into a real answer. | Name | Kind | Summary | |---|---|---| | `ON_DELTA_KEY` | constant | Handler keyword that receives streamed text deltas (also mirrored into ``request.context``). | | `PRELUDE_ROLES` | constant | Roles executed *before* the primary target, in this order. | | `SSE_DONE` | constant | the terminal event of an OpenAI chat stream. | | `SSE_KEEPALIVE` | constant | an SSE comment: keeps proxies and clients on the line while a model is silent. | | `TOOL_ROLE` | constant | Plan role whose targets are offered to the model as callable functions. | | `ChatStream` | class | Relay one chat completion as OpenAI ``chat.completion.chunk`` events while the handler is still generating. | | `ExecutionResult` | class | What happened when a decision was executed. | | `ExecutionStep` | class | One executed plan slot: role, target, latency and whether it succeeded. ``stage`` orders the steps. | | `aexecute` | async function `(decision: RouteDecision, request: RouteRequest, learn: LearnFn \| None=None, *, min_slot_confidence: float=0.0, task_id: str \| None=None, **kw: Any)` | Async twin of :func:`execute`; awaits coroutine handlers. | | `completion_chunks` | function `(completion: dict[str, Any], chunk_chars: int \| None=None)` | Serve a finished OpenAI ``chat.completion`` as ``chat.completion.chunk`` events. | | `cost_tokens` | function `(result: ExecutionResult)` | ``(total tokens, USD)`` of an execution: the handler's token count and the sum of every outcome's cost. | | `execute` | function `(decision: RouteDecision, request: RouteRequest, learn: LearnFn \| None=None, *, min_slot_confidence: float=0.0, task_id: str \| None=None, preferred_over: str \| None=None, **kw: Any)` | Run the plan (persona -> skill -> primary) and record outcomes via ``learn``. | | `execute_with_fallback` | function `(decision: RouteDecision, request: RouteRequest, candidates: list[RouteTarget] \| None=None, fallbacks: bool=True, learn: LearnFn \| None=None, *, router: Any=None, committed: Callable[[], bool] \| None=None, on_attempt: Callable[[RouteDecision], None] \| None=None, max_fallbacks: int \| None=None, settings: Settings \| None=None, **handler_kw: Any)` | Run the decision; when its handler fails, re-route among the remaining executable targets and retry. | | `repair_completion` | function `(completion: dict[str, Any])` | Repair the assistant text of a finished chat completion in place when it is almost JSON. | | `sse_event` | function `(payload: dict[str, Any])` | One server-sent event carrying ``payload`` as JSON (``data: {...}\n\n``). | | `tool_name` | function `(target: RouteTarget)` | The function name a tool target is offered under (OpenAI: ``^[A-Za-z0-9_-]{1,64}$``). | | `tool_spec` | function `(target: RouteTarget)` | OpenAI function-calling spec of a tool target: its MCP ``input_schema`` when it has one, else a single ``input``. | ## `opensmartroute.feedback` Feedback store: append-only outcome log that closes the learning loop. | Name | Kind | Summary | |---|---|---| | `FeedbackStore` | class | Append-only :class:`Outcome` log (in memory or JSONL file) with per-target statistics. | ## `opensmartroute.gemini` Google Gemini API dialect: ``generateContent`` / ``streamGenerateContent`` over the routed chat path. | Name | Kind | Summary | |---|---|---| | `FINISH_REASONS` | constant | : Gemini ``finishReason`` for an OpenAI ``finish_reason``. | | `STATUS_NAMES` | constant | : gRPC-style ``status`` names Google error envelopes carry, per HTTP status. | | `GenerateStreamTranslator` | class | Push-based translation of ``chat.completion.chunk`` SSE bytes into ``streamGenerateContent?alt=sse``. | | `chat_to_generate` | function `(completion: dict[str, Any], request: dict[str, Any] \| None=None)` | Shape a finished ``chat.completion`` as a ``GenerateContentResponse``. | | `count_tokens` | function `(body: dict[str, Any], model: str, estimator: Any=None)` | ``:countTokens`` - ``{"totalTokens": n}`` from the router's own token estimate (no provider call). | | `decode_inline` | function `(inline: dict[str, Any])` | The bytes of an ``inlineData`` part (helper for callers that store or inspect uploads). | | `error_body` | function `(status: int, message: str)` | Google's ``{"error": {"code", "message", "status"}}`` envelope. | | `generate_events` | function `(events: Iterable[bytes], request: dict[str, Any] \| None=None, model: str='auto')` | Translate a ``chat.completion.chunk`` SSE byte stream into ``streamGenerateContent`` SSE frames. | | `generate_to_chat` | function `(body: dict[str, Any], model: str, *, default_model: str='auto')` | Translate a ``generateContent`` request into a ``chat.completion`` request body. | ## `opensmartroute.inference` Run open-weight models on the CPU and RAM you have: footprint arithmetic, host planning, Ollama control. | Name | Kind | Summary | |---|---|---| | `DEFAULT_URL` | re-export of [`opensmartroute.inference.ollama.DEFAULT_URL`](#opensmartrouteinferenceollama) | where ``ollama serve`` listens by default. | | `GIB` | re-export of [`opensmartroute.inference.footprint.GIB`](#opensmartrouteinferencefootprint) | bytes in a GiB (container memory limits are written as ``28Gi``). | | `KNOWN_TAGS` | re-export of [`opensmartroute.inference.footprint.KNOWN_TAGS`](#opensmartrouteinferencefootprint) | Only a fallback for planning without a server; ``/api/show`` gives the exact figures. | | `KV_FALLBACKS` | re-export of [`opensmartroute.inference.planner.KV_FALLBACKS`](#opensmartrouteinferenceplanner) | The order the planner tries KV precisions when a model does not fit at the requested one. | | `KV_TYPE_BYTES` | re-export of [`opensmartroute.inference.footprint.KV_TYPE_BYTES`](#opensmartrouteinferencefootprint) | Bytes per KV cache element by ``OLLAMA_KV_CACHE_TYPE`` (the quantised types carry block scales). | | `QUANT_BITS` | re-export of [`opensmartroute.inference.footprint.QUANT_BITS`](#opensmartrouteinferencefootprint) | Bits per weight of GGUF file types: measured file size x 8 / parameter count (llama.cpp quantize output). | | `Footprint` | re-export of [`opensmartroute.inference.footprint.Footprint`](#opensmartrouteinferencefootprint) | Memory one loaded model takes at a given context, parallelism and KV precision. | | `Host` | re-export of [`opensmartroute.inference.footprint.Host`](#opensmartrouteinferencefootprint) | A machine (or container) that serves models: its memory, cores and, when known, bandwidth and store speed. | | `ModelSpec` | re-export of [`opensmartroute.inference.footprint.ModelSpec`](#opensmartrouteinferencefootprint) | An open-weight model as the planner sees it: size, quantisation and the attention shape behind its KV cache. | | `OllamaClient` | re-export of [`opensmartroute.inference.ollama.OllamaClient`](#opensmartrouteinferenceollama) | Control-plane client for one Ollama server: tags, show, ps, load, unload, exact :class:`ModelSpec` rows. | | `Placement` | re-export of [`opensmartroute.inference.planner.Placement`](#opensmartrouteinferenceplanner) | The models one host keeps resident, what they use and the server environment that pins them. | | `Plan` | re-export of [`opensmartroute.inference.planner.Plan`](#opensmartrouteinferenceplanner) | Placements per host plus the models that fit nowhere (with the reason). | | `ResidentModel` | re-export of [`opensmartroute.inference.ollama.ResidentModel`](#opensmartrouteinferenceollama) | One row of ``/api/ps``: a model currently loaded, where it sits and when it expires. | | `Throughput` | re-export of [`opensmartroute.inference.footprint.Throughput`](#opensmartrouteinferencefootprint) | Tokens per second a host gives a model, and how long a cold load takes. | | `api_root` | re-export of [`opensmartroute.inference.ollama.api_root`](#opensmartrouteinferenceollama) | ``http://host:11434/v1`` or ``http://host:11434/`` -> ``http://host:11434`` (validated http/https). | | `footprint` | re-export of [`opensmartroute.inference.footprint.footprint`](#opensmartrouteinferencefootprint) | Memory ``spec`` needs when loaded with ``context`` tokens per slot in ``parallel`` slots (defaults: settings). | | `format_bytes` | re-export of [`opensmartroute.inference.footprint.format_bytes`](#opensmartrouteinferencefootprint) | Human size in binary units (``2.4 GiB``), the way ``ollama ps`` and container limits read. | | `kv_type_bytes` | re-export of [`opensmartroute.inference.footprint.kv_type_bytes`](#opensmartrouteinferencefootprint) | Bytes per KV cache element for an ``OLLAMA_KV_CACHE_TYPE`` value. | | `ollama_env` | re-export of [`opensmartroute.inference.planner.ollama_env`](#opensmartrouteinferenceplanner) | ``OLLAMA_*`` variables that make a server hold exactly these models with these slots. | | `parse_memory` | re-export of [`opensmartroute.inference.footprint.parse_memory`](#opensmartrouteinferencefootprint) | ``"28Gi"`` / ``"8 GiB"`` / ``"16GB"`` / ``"512Mi"`` / a number of bytes -> bytes (Kubernetes and SI units). | | `plan` | re-export of [`opensmartroute.inference.planner.plan`](#opensmartrouteinferenceplanner) | First-fit-decreasing packing of ``models`` onto ``hosts`` at the requested context and parallelism. | | `quant_bits` | re-export of [`opensmartroute.inference.footprint.quant_bits`](#opensmartrouteinferencefootprint) | Bits per weight of a GGUF quantisation name (``q4_k_m``, ``Q8_0``, ``F16``, ``MXFP4``); unknown ``Qn`` -> n.5. | | `recommend` | re-export of [`opensmartroute.inference.planner.recommend`](#opensmartrouteinferenceplanner) | The largest configuration of ``spec`` that fits ``budget`` bytes, or None when even the weights do not. | | `render` | re-export of [`opensmartroute.inference.planner.render`](#opensmartrouteinferenceplanner) | A plain-text report of a plan: per host the resident models, memory used / free and the ``OLLAMA_*`` lines. | | `throughput` | re-export of [`opensmartroute.inference.footprint.throughput`](#opensmartrouteinferencefootprint) | Bandwidth-bound decode, compute-bound prefill and cold-load time of ``spec`` on ``host``. | ## `opensmartroute.inference.footprint` What an open-weight model needs from a CPU host: memory for weights and KV cache, and the tokens per second. | Name | Kind | Summary | |---|---|---| | `GIB` | constant | bytes in a GiB (container memory limits are written as ``28Gi``). | | `KNOWN_TAGS` | constant | Only a fallback for planning without a server; ``/api/show`` gives the exact figures. | | `KV_TYPE_BYTES` | constant | Bytes per KV cache element by ``OLLAMA_KV_CACHE_TYPE`` (the quantised types carry block scales). | | `QUANT_BITS` | constant | Bits per weight of GGUF file types: measured file size x 8 / parameter count (llama.cpp quantize output). | | `Footprint` | class | Memory one loaded model takes at a given context, parallelism and KV precision. | | `Host` | class | A machine (or container) that serves models: its memory, cores and, when known, bandwidth and store speed. | | `ModelSpec` | class | An open-weight model as the planner sees it: size, quantisation and the attention shape behind its KV cache. | | `Throughput` | class | Tokens per second a host gives a model, and how long a cold load takes. | | `footprint` | function `(spec: ModelSpec, *, context: int \| None=None, parallel: int \| None=None, kv_type: str \| None=None, settings: Settings \| None=None)` | Memory ``spec`` needs when loaded with ``context`` tokens per slot in ``parallel`` slots (defaults: settings). | | `format_bytes` | function `(n: int \| float)` | Human size in binary units (``2.4 GiB``), the way ``ollama ps`` and container limits read. | | `kv_type_bytes` | function `(kv_type: str)` | Bytes per KV cache element for an ``OLLAMA_KV_CACHE_TYPE`` value. | | `parse_memory` | function `(text: str \| int \| float)` | ``"28Gi"`` / ``"8 GiB"`` / ``"16GB"`` / ``"512Mi"`` / a number of bytes -> bytes (Kubernetes and SI units). | | `quant_bits` | function `(quant: str)` | Bits per weight of a GGUF quantisation name (``q4_k_m``, ``Q8_0``, ``F16``, ``MXFP4``); unknown ``Qn`` -> n.5. | | `throughput` | function `(spec: ModelSpec, host: Host, *, context_tokens: int=0, kv_type: str \| None=None, settings: Settings \| None=None)` | Bandwidth-bound decode, compute-bound prefill and cold-load time of ``spec`` on ``host``. | ## `opensmartroute.inference.ollama` Talk to an Ollama server's control API: which models it has, their exact shape, which are resident, load and unload. | Name | Kind | Summary | |---|---|---| | `DEFAULT_URL` | constant | where ``ollama serve`` listens by default. | | `OllamaClient` | class | Control-plane client for one Ollama server: tags, show, ps, load, unload, exact :class:`ModelSpec` rows. | | `ResidentModel` | class | One row of ``/api/ps``: a model currently loaded, where it sits and when it expires. | | `api_root` | function `(base_url: str)` | ``http://host:11434/v1`` or ``http://host:11434/`` -> ``http://host:11434`` (validated http/https). | ## `opensmartroute.inference.planner` Place models on hosts and write the server settings that make them fit and stay warm. | Name | Kind | Summary | |---|---|---| | `KV_FALLBACKS` | constant | The order the planner tries KV precisions when a model does not fit at the requested one. | | `Placement` | class | The models one host keeps resident, what they use and the server environment that pins them. | | `Plan` | class | Placements per host plus the models that fit nowhere (with the reason). | | `ollama_env` | function `(footprints: Sequence[Footprint], *, keep_alive: str \| None=None, settings: Settings \| None=None)` | ``OLLAMA_*`` variables that make a server hold exactly these models with these slots. | | `plan` | function `(models: Iterable[ModelSpec], hosts: Sequence[Host], *, context: int \| None=None, parallel: int \| None=None, kv_type: str \| None=None, shrink: bool=False, settings: Settings \| None=None)` | First-fit-decreasing packing of ``models`` onto ``hosts`` at the requested context and parallelism. | | `recommend` | function `(spec: ModelSpec, budget: int, *, context: int \| None=None, parallel: int \| None=None, kv_type: str \| None=None, settings: Settings \| None=None)` | The largest configuration of ``spec`` that fits ``budget`` bytes, or None when even the weights do not. | | `render` | function `(p: Plan, *, settings: Settings \| None=None)` | A plain-text report of a plan: per host the resident models, memory used / free and the ``OLLAMA_*`` lines. | ## `opensmartroute.jsonrepair` Repair the JSON a model almost produced. | Name | Kind | Summary | |---|---|---| | `completion_tail` | function `(streamed: str)` | For a stream whose content so far is ``streamed``: the suffix that completes it into valid JSON when the. | | `repair_json` | function `(text: str)` | ``(repaired, changed)`` - ``repaired`` parses with :func:`json.loads` when any repair was possible; when the. | | `wants_json` | function `(response_format: Any)` | True when an OpenAI ``response_format`` asks for JSON (``json_object`` or ``json_schema``). | ## `opensmartroute.knowledge` Knowledge management and context engineering: a knowledge graph the router learns into and reads from. | Name | Kind | Summary | |---|---|---| | `CONTEXT_HEADER` | re-export of [`opensmartroute.knowledge.context.CONTEXT_HEADER`](#opensmartrouteknowledgecontext) | : Opens the engineered block; the sections follow as ``##