# Platform guide The hosted platform runs OpenSmartRoute as a service. A workspace has an API key and a plan; each request to the API is routed to a target (model, agent, skill, persona or tool) and answered with the decision, its trace and, on request, the target's output. An OpenAI-compatible endpoint accepts existing chat-completion clients. This page covers calling the platform: authentication, routing, execution, the OpenAI-compatible endpoint, feedback, plans, organizations, tenants, governance (workspace policy and budgets), observability (probes, metrics, per-request traces and the event stream), caching, the dashboard and the MCP server. The [marketplace](https://osr-web.kindrock-caaab1a2.centralindia.azurecontainerapps.io/docs/MARKETPLACE.md) - shared agents, skills, personas, prompts and stack templates - has its own page. To run the router on your own infrastructure see [Deploy with Docker and Helm](https://osr-web.kindrock-caaab1a2.centralindia.azurecontainerapps.io/docs/deploy.md); to use it as a Python library see the [user guide](https://osr-web.kindrock-caaab1a2.centralindia.azurecontainerapps.io/docs/GUIDE.md) and the [SDK guide](https://osr-web.kindrock-caaab1a2.centralindia.azurecontainerapps.io/docs/SDK.md). Every endpoint below is listed in the REST API reference (`/docs/api`) and in `/openapi.json`. ## 1. Authentication 1. Sign up at `/platform/signup` with Microsoft, Google, GitHub, GitLab, LinkedIn (the providers the deployment enables - `GET /api/v1/info` lists them under `sso`; the LinkedIn app the growth engine posts with doubles as the sign-in provider unless `OSR_PLATFORM_LINKEDIN_SSO=false`) or an email address - with a password if you want to sign in by email later (`OSR_PLATFORM_PASSWORD_LOGIN`, on by default). Signup creates a personal workspace on the **free** plan and issues an API key. With Google configured the site also shows Google's One Tap prompt ("Sign in to ... with google.com") to visitors who are not signed in; picking an account posts the ID token to `POST /api/v1/auth/{provider}/credential` `{credential, next}` (Google only for now), which has Google validate it and answers with the same browser session as the redirect callback - no page change, a first sign-up still shows the key once. The Google OAuth client must list the site's origin (`OSR_PLATFORM_WEB_URL`) under *Authorized JavaScript origins* or Google refuses to show the prompt. 2. The key is shown once. Store it in a secret manager. Keys are created, renamed, rotated and revoked in the dashboard (`/platform/dashboard/keys`) or with `POST /api/v1/keys`, `PATCH /api/v1/keys/{key_id}` `{name}`, `POST /api/v1/keys/{key_id}/rotate` (a new secret for the same key, shown once; the old one stops working immediately) and `DELETE /api/v1/keys/{key_id}` - all admin role, or the lead of the team a key was minted for (`team` on `POST /api/v1/keys`, [Teams](#teams)). A key does everything its workspace can unless you limit it: `scopes` (`read` - every GET; `route` - decisions without execution: `/route`, `/estimate`, MCP ranking tools; `execute` - anything that can spend money: `execute: true`, `/v1/*`, the MCP `ask` tool; `feedback`; `manage` - every other write) and `expires_in_days` (1-3650) on `POST /api/v1/keys` or `PATCH /api/v1/keys/{key_id}` `{scopes}`; `GET /api/v1/keys/scopes` lists them. A scoped key that steps outside its scopes is refused with `403` and `X-Key-Scope: ` before anything is metered; an expired key answers `401 API key expired` until it is revoked (rotation keeps the expiry). `limits` narrows what a key may spend and reach: `daily_budget_usd` / `monthly_budget_usd` cap the key's own spend (checked like the workspace and tenant budgets - `429` with the `X-Budget-*` headers once reached; routing without execution still works), `requests_per_minute` caps its rate below the plan's (`429` with `Retry-After` and `X-Key-Limit: requests_per_minute`), `tokens_per_minute` caps the tokens it may send and receive in a sliding minute (charged with the request's bound - the prompt plus the output allowance - before any provider is called, so a burst is refused rather than billed: `429` with `Retry-After` and `X-Key-Limit: tokens_per_minute`), `allowed_ips` pins the key to client addresses or CIDR blocks (at most 32; the address is the one past the trusted proxies, never a header the client wrote; a call from elsewhere answers `403` with `X-Key-Limit: allowed_ips`) and `allow_targets` lists the target ids the key may route to (the router never leaves the list, a pinned `model` outside it answers `403` with `X-Key-Limit: allow_targets`, and a caller's own `allow_targets` can only narrow it further). `required_headers` names headers every request with the key must carry (a project tag, a ticket id, a cost centre - at most ten; a request without them answers `428` with `X-Key-Limit: required_headers`), and `mcp_tools` lists the connected MCP servers' tools the key may reach (`server:tool` or `server:*`; section 12). `PATCH /api/v1/keys/{key_id}` `{limits}` replaces them (`{}` clears); `GET /api/v1/keys/scopes` lists the limit keys; every change is audited (`key.limited`). Give keys that leave your hands - CI, a partner, a demo - the smallest scopes, an expiry and a budget. 3. Send the key with every metered request in either header, and - when you want the row attributed to a cost centre without a tenant - the project tag: ```http Authorization: Bearer osr_... X-API-Key: osr_... X-OSR-Project: checkout-service ``` `X-OSR-Project` (letters, digits, `.`, `_`, `-`; up to 64) is recorded on the usage row as `project` and broken out in `GET /api/v1/usage` as `by_project` - cheaper than a tenant when all that is wanted is a line in the report. The Google GenAI SDKs' `x-goog-api-key` header is accepted as the key too (section 4). Public endpoints (`/api/v1/info`, `/api/v1/models`, `/api/v1/llms`, `/api/v1/rankings`, `/api/v1/catalogue`, `/api/v1/estimate`, `/api/v1/stats/public`, `/api/v1/registry`) need no key. ### Email and password `POST /api/v1/auth/password/login` `{email, password}` returns the same browser session as a single sign-on callback (`session_token`, `user`, `account`, `plan`); the dashboard stores it and sends it as a bearer token. Passwords are at least 8 characters (no composition rules) and are stored as salted scrypt hashes. A signed-in user sets or changes theirs with `POST /api/v1/auth/password` `{password, current_password}` - the current one is required once set and every other browser is signed out - and removes it with `DELETE /api/v1/auth/password` as long as a provider identity remains. Sign-in attempts are limited per address and per email; `has_password` on the user object says which users have one. `/platform/dashboard/account` has the form. **Two-factor authentication.** A person with a password can add an authenticator app (TOTP, RFC 6238: six digits every 30 s, one step of drift accepted) under *Account › Two-factor authentication*: `POST /api/v1/auth/mfa/setup` `{current_password}` returns the secret and the `otpauth://` URI, `POST /api/v1/auth/mfa/enable` `{code}` confirms it and returns eight single-use recovery codes (shown once; other browsers are signed out), `POST /api/v1/auth/mfa/recovery-codes` `{code}` replaces them and `DELETE /api/v1/auth/mfa` `{current_password, code}` switches the factor off - each change is e-mailed as a security notice (`two_factor_changed`). From then on `POST /api/v1/auth/password/login` answers `mfa_required` with a five-minute signed `challenge` instead of a session, and `POST /api/v1/auth/mfa/verify` `{challenge, code}` with the authenticator code (or an unused recovery code) issues it; the sign-in page shows the code step. A password-reset link proves the inbox, not the second factor, so `POST /api/v1/auth/password/reset` saves the password and answers the same challenge (`password_changed: true`). Identity providers bring their own second factor and are not asked again; API keys are the workspace, not a person, and never are. `mfa_enabled`, `mfa_enabled_at` and `recovery_codes_left` are on the user object; an operator resets a locked-out person with `PATCH /api/v1/admin/users/{id}` `{"mfa_reset": true}` (sessions end, the person is told). History: `user.mfa_enabled`, `user.mfa_disabled`, `user.mfa_recovery_used`, `user.mfa_recovery_codes_replaced`, `user.login_mfa_failed`, `user.mfa_reset`. ### Account lifecycle: confirmation, recovery, deletion Anyone can create an account with any email address. What happens around it: | Step | How | Endpoints | |---|---|---| | Email confirmation | Signup sends a link to `/platform/verify-email?token=...` (valid three days). Signing in through a provider that vouches for the address, accepting an invitation sent to it or completing a password reset also confirms it. A provider vouches when it sends `email_verified` (Google, any OIDC issuer, Entra's optional `xms_edov`) or when it only reports addresses it verified or owns - GitHub, Microsoft, GitLab and LinkedIn, as the shared button or as an organization's own connection of that kind; a generic OIDC issuer that sends no such claim leaves the address unconfirmed and the confirmation link goes out. `email_verified` on the user object. **Confirmation is required** (`OSR_PLATFORM_REQUIRE_EMAIL_VERIFICATION`, on by default): until it is done every call but `/api/v1/me`, `/api/v1/onboarding`, `/api/v1/auth/*` and `/api/v1/legal` answers `403` with `X-Email-Verification: required` - for the person's session and for the workspace's API keys alike (a key stands on an owner's confirmed address), on the OpenAI-compatible `/v1/*` proxy and the MCP server as on the REST API, and a public endpoint that personalises its answer for a key (`/api/v1/estimate`, the site assistant) treats an unconfirmed key as no key at all - anonymous limits, never the workspace's; `GET /api/v1/me` says so in `email_verification_required` and the dashboard shows the confirmation step in place of its pages, with *Send the link again*. Signups from domains reserved for testing (`example.com`, `example.org`, `*.test`, `*.local`, `*.invalid`, `*.localhost`) are refused unless `OSR_PLATFORM_SIGNUP_TEST_DOMAINS` is on (throw-away CI stacks only), and a user who never confirms is removed after `OSR_PLATFORM_UNVERIFIED_ACCOUNT_TTL_DAYS` (14) with the personal workspace only they belonged to. | `POST /api/v1/auth/verify` `{token}`, `POST /api/v1/auth/verify/send` (signed in, once a minute) | | Forgot password | `/platform/forgot-password` emails a single-use link to `/platform/reset-password?token=...` (one hour). The answer is always `202` so it never reveals whether an address has an account; limited per address and per email. Resetting revokes every session, confirms the address, signs the person in and sends a notice. | `POST /api/v1/auth/password/forgot` `{email}`, `POST /api/v1/auth/password/reset` `{token, password}` | | Invitations | `POST /api/v1/workspace/invites` emails the invitee a link to `/platform/invite/` (fourteen days) and returns the same link once to the inviter. | see [Organizations](#7-organizations) | | Account history | The person's own sign-ins, password and email changes and memberships, newest first. | `GET /api/v1/me/audit` | | Download my data | One JSON document with everything held about the person and the current workspace: profile, identities, sessions, memberships and history; the workspace with plan, members, teams and their rosters, key prefixes, tenants, policy, SSO, usage, the request log, reported outcomes and the kept evaluation reports (`evaluations`) of up to a year. Prompt text is stored only when the workspace policy turns request logging on (`log_payloads`); whatever was kept is included as `request_logs`. *Download my data* on `/platform/dashboard/account`. | `GET /api/v1/me/export?days=` | | Delete account | *Delete my account* on `/platform/dashboard/account`: type the email (and the password when one is set). Sessions, linked identities and memberships go, workspaces only this user belonged to are disabled and their keys revoked; an organization with other members needs another owner first. A notice is emailed. | `POST /api/v1/me/delete` `{confirm, password}` | | Delete organization | *Delete organization* on `/platform/dashboard/workspace`, owners only: type the organization's name or slug (and the password when one is set). Its keys, members, invitations, tenants, policy, SSO connection, usage and outcomes go at once and a live subscription is cancelled immediately - nothing more is billed, the paid remainder is not refunded automatically. Every member keeps their personal workspace and their session moves to it; the deletion is on the audit trail (`account.deleted`). Personal workspaces go with the user. | `POST /api/v1/workspace/delete` `{confirm, password}` | | Getting started | The checklist of a workspace: e-mail confirmed, API key created, first request routed, first outcome reported (the playground's one-click *Good* / *Poor*), a model provider connected and - for organizations - a teammate invited; each step carries the dashboard link and the API call that completes it, `next` is the first open one and `complete` the share done. The dashboard overview shows it until every step is done. Until the first request is routed the card also asks *What is in your way?* - one click from a short list (no time yet, no provider to connect, not clear how, a different integration, something failed, still comparing, a feature missing) and an optional line; the answer reaches the operator next to the workspace and the thanks link to the page that removes the obstacle. | `GET /api/v1/onboarding`, `GET /api/v1/journey/blocker`, `POST /api/v1/journey/answer` | Signup, confirmation, password recovery, invitation links and this checklist form the `onboarding` domain, which a deployment can run as its own service (see [Services](#services)). Every one of these events - plus sign-ins, failed sign-ins, role changes, removals and operator actions - is written to the account audit log. Workspace admins read their workspace's entries with `GET /api/v1/workspace/audit`; operators see everything at `/platform/admin/audit` (`GET /api/v1/admin/audit-log`) and per user on `/platform/admin/users/`, where they can also issue a reset link, re-send or force the email confirmation, disable, sign out everywhere or delete the user. ### Sign-in locations and alerts Every browser session and every account event records the address it came from (`ip`). An address alone tells a person little - *203.0.113.9* does not say whether that was their own phone on holiday or somebody else - so the platform can turn it into a place (`location`: country, region, city and a label such as *Lisbon, Portugal*) from one of two sources the operator chooses; without either, nothing is resolved and the address alone is shown: | Source | Setting | What it gives | |---|---|---| | A header the ingress sets | `OSR_PLATFORM_GEOIP_HEADER` - the header's name, `CF-IPCountry` behind Cloudflare, `X-Country-Code` behind a gateway that resolves it | The country. Free, nothing leaves the platform. | | A lookup service | `OSR_PLATFORM_GEOIP_SERVICE` - a preset (`ipinfo`, `ipapi`, `ip-api`, `ipwho`) or a URL template with `{ip}` and `{token}`; `OSR_PLATFORM_GEOIP_TOKEN` when the service wants a key | City level. One HTTPS call per new address, two seconds at most (a slow answer never holds a sign-in), cached in the database for thirty days and shared by every replica. Private and loopback addresses are never looked up (*local network*). | Both settings are also under *Runtime configuration* in the console, so a deployment can switch them on without a redeploy. Choose a service whose terms fit your use: sending a visitor's address to a third party is a processing of personal data, and your privacy notice should name it. **Sign-in alerts.** A sign-in from a device the account has not been seen on in ninety days, or - once places are resolved - from a country none of its recent sessions came from, is announced to the person by e-mail (`new_sign_in`, a security message that cannot be switched off) and in their notifications, with the device, the method, the place and the address; the account history records `user.new_device` or `user.new_location`. A person who wants to hear about every sign-in chooses *Every sign-in* under *Sign-in alerts* on `/platform/dashboard/account` (`PUT /api/v1/me/login-alerts` `{mode}`, `new` or `always`; `login_alerts` on the user object). The very first session of an account is not announced - the welcome covers it. **What operators see.** The user page shows each session's place next to its address and a *Sign-in places* card: where the account was signed in from in the last ninety days, with the addresses, the devices, how many sign-ins and the first and last one - a traveller shows one device in several places, a takeover a new place on a new device. The deployment audit log carries the place of every event, filters by country (`country=DE`, `--` for events without a place), searches it with `q` and exports it in the CSV. **Forgetting.** Addresses and places are kept with the row for `OSR_PLATFORM_NETWORK_DATA_RETENTION_DAYS` (365; `0` = as long as the event) and then removed by the retention sweep - the event stays, the address and the place go. *Forget addresses* on the user page (`DELETE /api/v1/admin/users/{id}/network-data`) does the same for one person at once, for a privacy request; it is audited as `user.network_data_cleared` with the operator's own address and place, like every operator action. **Operators too.** A console sign-in records its address and place on the operator's session (shown under *Operators*), `operator.login`, `operator.login_failed`, `operator.login_mfa_failed` and `operator.logout` are in the audit log with the place, and every operator action carries the address and place of the console session it came from. A console sign-in from a browser or a country the operator has not used in ninety days is written as `operator.new_device` / `operator.new_location` and filed in the console's bell for every operator to see. ### Email: transports, templates, welcome, newsletter and reports Every message the platform sends is composed once as a branded HTML card (privacy and terms links in the footer) with a plain-text alternative: the welcome message (greeting, a four-step checklist, dashboard and quick-start links - sent by signup, or by the first sign-in of a person who arrived another way: a shared or an organization's identity provider, an operator-created user, a password-reset link; never twice), address confirmation (sent by signup; the first sign-in sends it too when the address is still unconfirmed and no link ever went out), password reset, invitation, account deleted, password changed, the operator test message, usage reports, alerts, support-ticket mail, product news and newsletter issues. Each is written to the outbox first and then delivered by whichever transport is configured (the first configured one in this order wins). Every message follows the same design so a reader knows what they are holding before they read a word: the wordmark, a coloured band and a small label naming the kind of message (*Security* in red, *Alert* in amber, *Account* in ink, *Getting started* and *Report* in blue, *Release notes* and *Product news* in green, *Newsletter* in violet, *Support* and *Feedback* in teal), one heading, short sentences that read at an eighth-grade level (`tests/test_email_readability.py` fails a template that does not), one clear action, a plain-text alternative, and a footer that says why the person is receiving it and where to change that. A message that could worry someone - a new sign-in, failed attempts, a change to the account - always says what happened, whether anything is lost, what to do if it was not them, and that a reply reaches a person. | Transport | Setting | Notes | |---|---|---| | Microsoft 365 (Microsoft Graph) | `OSR_PLATFORM_M365_TENANT_ID`, `OSR_PLATFORM_M365_CLIENT_ID`, `OSR_PLATFORM_M365_CLIENT_SECRET` | Mail leaves from the Exchange Online mailbox named by `OSR_PLATFORM_MAIL_FROM` (display name `OSR_PLATFORM_MAIL_FROM_NAME`, default the brand) with `POST /users/{sender}/sendMail`, so DKIM, SPF and the tenant's retention apply as for any other mailbox; nothing is saved to Sent Items. Set-up: in Entra ID register an application, add the **application** permission `Mail.Send` (Microsoft Graph) and grant admin consent, create a client secret, and give the platform the tenant id (GUID or verified domain), client id and secret; the tenant id and client id are validated at start-up and a malformed value disables the transport with the reason in `GET /api/v1/admin/mail`. Restrict the app to the one sender with an Exchange *application access policy* (`New-ApplicationAccessPolicy -PolicyScopeGroupId -AccessRight RestrictAccess`). Tokens (client credentials) are cached until a minute before expiry and refreshed once on a `401`. | | Azure Communication Services Email | `OSR_PLATFORM_ACS_CONNECTION_STRING` | `endpoint=https://..communication.azure.com/;accesskey=...`; the platform signs requests itself (HMAC-SHA256, no SDK). `OSR_PLATFORM_MAIL_FROM` must be a sender of a linked domain, e.g. `DoNotReply@.azurecomm.net` for an Azure-managed domain or `no-reply@yourdomain` once a custom domain is verified. | | SMTP | `OSR_PLATFORM_SMTP_URL` | `smtp://user:pass@host:587` (STARTTLS) or `smtps://user:pass@host:465`. | | Outbox only | none | Nothing is lost: operators read the links at `/platform/admin/mail` (`GET /api/v1/admin/mail`) and hand them over; `mail` in `GET /api/v1/info` tells the web app which case applies. | **Delivery guarantees.** Sending is at-least-once with bounded retries. Every attempt is recorded on the outbox row (`attempts`, `error`, `next_attempt_at`); a failure is retried on a back-off ladder (30 s, 2 min, 10 min, 30 min, 2 h - six attempts in all) by a sweeper thread in the API, one leader per Redis lease in a cluster. A row whose sending process disappeared mid-send is retried after five minutes. Once the ladder is exhausted the row stays in the outbox as failed for an operator to resend by hand (`POST /api/v1/admin/mail/{id}/resend` always tries once more). ACS accepts a message with `202` and delivers asynchronously; the platform polls the operation status briefly and records a terminal `Failed` (unverified sender, suppressed recipient) as an error instead of a false "sent", keeping the operation id as `provider_id`. `GET /api/v1/admin/mail` returns a `delivery` block (transport `ok`, last error, rows `retrying`, sweeper state) next to `summary.retrying` and `summary.abandoned`; `GET /readyz` reports the same under `mail` (informational, never fails readiness). **The gallery: every message before a customer sees it.** Operators manage what the platform says from `/platform/admin/mail`: `GET /api/v1/admin/mail/templates` lists every kind of message with its category, whether people can switch it off and how many went out; `GET /api/v1/admin/mail/templates/{kind}` renders it with sample data (subject, the HTML card, the plain-text alternative) and the page shows it as a customer would see it; `POST /api/v1/admin/mail/templates/{kind}/send` `{to}` sends that sample to the operator's own inbox (stored in the outbox under the kind with a `[Sample]` subject; audit `mail.sample`) to check the words and the design in a real mail client. Together with the outbox (read, resend, delete), the transport test, the suppression list, each user's switches (`/platform/admin/users/`), the report schedule (`/platform/admin/reports`) and the campaign switches (`PATCH /api/v1/admin/campaigns/{campaign}` `{enabled}`, on `/platform/admin/growth`), every message the platform sends can be seen, tested, held back or re-sent from the console. **Bounces and the suppression list.** An address the transport rejects for good (SMTP `550`-`554`, a `5.1.x` status, "user unknown", "mailbox unavailable", an ACS `Suppressed`/`Bounced` status) is not retried: the row is abandoned at once with the reason and the address goes on the **suppression list** (`email_suppressions`, reason `bounce`), so no later message - of any category, newsletter included - is composed for it; such a skip is counted as `suppressed` in `delivery`. Delivery reports keep the list honest after acceptance: with `OSR_PLATFORM_MAIL_EVENTS_TOKEN` set, `POST /api/v1/email/events/acs?token=…` receives Azure Communication Services' Event Grid events (subscribe the endpoint to `EmailDeliveryReportReceived` and `EmailEngagementTrackingReportReceived`; the handshake is answered automatically, a wrong token is `401`, and the endpoint answers `404` without the setting) and stamps each outbox row with `delivery_status` / `delivery_at` (`delivered`, `bounced`, `spam`, `quarantined`, `expired`, `failed`); a bounce, quarantine or spam report also suppresses the address, a soft failure (`expired`) for thirty days only. Operators see and edit the list at `/platform/admin/mail` - `GET /api/v1/admin/mail/suppressions` (`q`, `limit`), `POST /api/v1/admin/mail/suppressions` (`email`, `reason`, `detail` - for complaints and requests made out of band) and `DELETE /api/v1/admin/mail/suppressions/{email}` to allow an address again; `GET /api/v1/admin/mail` carries the list size by reason under `suppressions`, whether delivery reports are on under `delivery_reports`, and the last thirty days of reported outcomes under `delivery.delivery_30d`. `GET /metrics` exposes the same as `osr_mail_delivered_total`, `osr_mail_failed_total`, `osr_mail_suppressed_total`, `osr_mail_reports_total`, `osr_mail_retrying`, `osr_mail_suppression_list` and `osr_mail_delivery_30d{status=…}`. **The sender's address.** `OSR_PLATFORM_MAIL_POSTAL_ADDRESS` (one line, e.g. `Acme Ltd, 1 Main St, Springfield`) is printed in the footer of every message, HTML and text, as CAN-SPAM and most national rules expect of commercial mail; the console's mail page shows what is configured (`postal_address` in `GET /api/v1/admin/mail`). **What people receive, and how they say no.** Mail falls into categories. *Security* (address confirmation, password changes, sign-in and deletion notices, an API key created or revoked - to every owner, naming who did it and the key's prefix) and *account* (welcome, invitations, payment problems, plan changes - the owners hear when the plan moves up or down, with the new limits) are always sent. *Usage reports*, *alerts* (quota, budget and error-rate alerts that reach the owners when a workspace has no alert channel), *support* (ticket confirmations and answers), *product news* and *getting started* (one reminder with the next step when a workspace has not routed a request a day after signing up - sent once per workspace, to signups of the last seven days only, by the `accounts` service on the digest cadence) are optional: every such message carries a **Manage e-mail preferences** link and an **Unsubscribe** link in the footer plus the `List-Unsubscribe` / `List-Unsubscribe-Post` headers (RFC 8058), so Gmail, Outlook and Apple Mail show their own unsubscribe button (Graph accepts only `x-` custom headers, so on the Microsoft 365 transport the footer links do the job). The links carry a signed token (HMAC of the user id, valid for six months, signed with `OSR_PLATFORM_SESSION_SECRET`), so they work without signing in: `/platform/email/preferences?token=…` shows the switches (`GET`/`PUT /api/v1/email/preferences`), and a mail client's one-click POST goes to `POST /api/v1/email/unsubscribe?token=…&category=…`. Signed in, the same switches - categories, the newsletter subscription and, for owners and admins, the workspace's report schedule - sit under **Account › E-mail** (`GET`/`PUT /api/v1/me/email-preferences`). A message to someone who switched its category off is skipped before it is composed and counted as `skipped` in `GET /api/v1/admin/mail`; every change is written to the audit log (`user.email_preferences`, `user.email_unsubscribed`). **Everyone's own clock.** Scheduled mail is timed by the recipient's time zone, not the server's. The sign-up form and the newsletter form send the browser's IANA zone (`timezone` on `POST /api/v1/signup` and `POST /api/v1/newsletter/subscribe`); a signed-in person who never named one gets the browser's zone the first time they open **Account › E-mail** (and is told so), and the same card lets anyone pick another zone or clear it (`timezone` on `PUT /api/v1/me/email-preferences` and on the token variant; `schedule` in the answer names the zone, its current offset and the hours below; operators can set it with `PATCH /api/v1/admin/users/{id}`). Until a zone is known, UTC applies. What follows the clock: usage digests go out at `OSR_PLATFORM_REPORTS_HOUR` (default 8) o'clock local over the recipient's own completed day, week or month; the newsletter issue at `OSR_PLATFORM_NEWSLETTER_HOUR` (default 9) local and the evening issue at `OSR_PLATFORM_NEWSLETTER_EVENING_HOUR` (default 18) local; the getting-started reminder, the upgrade offer, release notes and a trial-ending notice that is not urgent only between 08:00 and 20:00 local (a deferred recipient is picked up by the next sweep, unclaimed; the console's *Run now* / *Send reminders now* buttons do not wait). Alerts, confirmations, invitations, receipts and support answers are never held back. **Newsletter.** `POST /api/v1/newsletter/subscribe` (the form on `/newsletter` - the landing page that says what the daily issue contains and how the address is treated, with the latest posts - and in the site footer of every marketing page; off with `OSR_PLATFORM_NEWSLETTER=false`, which hides both forms) stores the address, sends a branded confirmation and always answers `202`; `POST /api/v1/newsletter/confirm` completes the double opt-in and sends the welcome issue with its unsubscribe link. Every newsletter message carries a one-click unsubscribe (`POST /api/v1/newsletter/unsubscribe`, token as JSON from the web page or as `?token=` from a mail client's `List-Unsubscribe` POST). A signed-in user with a verified address joins straight from the preferences card (no confirmation round trip); an unverified one gets the usual confirmation. **Registered people read it by default** (`OSR_PLATFORM_NEWSLETTER_SIGNUP_DEFAULT`, on): the signup form says so next to the terms, `POST /api/v1/signup` files the address as a *pending* reader (source `signup`, the browser's time zone) without a second confirmation mail, and the account's own proof of the address - the confirmation link (`POST /api/v1/auth/verify`), a password reset, an accepted invitation, an identity provider that vouches for it or an operator marking it verified - confirms the reader with a consent record naming that statement. The welcome mail says the newsletter comes with the account and where to turn it off; the switch on **Account › E-mail** shows *after e-mail confirmation* until then. An address that unsubscribed is never re-enrolled, a reader who confirmed through the form keeps their row, and `GET /api/v1/info` reports `newsletter_signup_default` so the site can word the form accordingly. **The blog asks readers in.** A visitor who is not signed in and reads on (a few seconds on `/blog` or a post, or a third of the page) sees a sheet at the foot of the page - *Keep reading with a free account* - with the deployment's identity providers (`sso` from `GET /api/v1/info`, one click), an e-mail field that lands pre-filled on `/platform/signup`, and the sign-in link, each carrying the page to come back to (`next`, same-origin paths only, checked again by the API's `safe_next`); the text says the newsletter comes with the account when `newsletter_signup_default` is on. The post is never held back: *Not now* keeps the sheet away for a week, a subscription from any newsletter form in that browser or a signed-in browser hides it for good, and it waits for the cookie banner's answer. After signing in the welcome page's button reads *Back to the post* / *Continue where you left off* instead of *Go to the dashboard*. Operators manage subscribers and campaigns at `/platform/admin/newsletter`: add an address by hand, export CSV, write a campaign in plain text (blank lines separate paragraphs, `## ` starts a heading), preview the rendered email, send a test to one address, then send once to every active subscriber. The daily issue follows each reader's clock: the issue of a calendar day is composed from the unmailed posts when the first reader's zone reaches `OSR_PLATFORM_NEWSLETTER_HOUR` and every later zone receives that issue at its own hour - one issue per reader per day, each post mailed once (`/platform/admin/content` shows the hour; its *Send now* delivers to everyone who has not had today's issue whatever their local time, `force` composes a fresh issue and repeats the day). **Morning and evening.** Daily readers get a second issue once their clock passes `OSR_PLATFORM_NEWSLETTER_EVENING_HOUR` (default 18, local): it carries the posts written since the morning issue ("N new posts since this morning", subject *evening: ...*), is composed the same way - once, when the first zone reaches the hour - and is skipped on a day with nothing new. `-1`, or an hour not after the morning one, sends the morning issue only; the hour is editable at `/platform/admin/settings`. Weekly readers are not affected. **How much gets written.** However many feeds deliver, the writer turns at most `OSR_PLATFORM_CONTENT_POSTS_PER_DAY` (default 6, 0 = no limit) entries into posts in any 24 hours, newest first and spread evenly over the day's polls, so the blog grows steadily and both issues have news; the entries left over wait for a later poll. **Daily or weekly, by topic, and the reader's own page.** Every newsletter message links to `/newsletter/preferences?token=…` (the same signed token as the unsubscribe link; `GET`/`PUT /api/v1/newsletter/preferences`) where a reader picks `frequency` `daily` (default) or `weekly`, sets or clears the `timezone`, chooses the `topics` to receive (the blog categories `models`, `llm`, `agents`, `research`, `ai`; an empty list or all of them means everything), sees what is stored about them, and can erase the record outright (`DELETE /api/v1/newsletter/preferences` - the row is gone, not marked). A reader who follows some topics receives the day's issue cut to those posts, with a subject and opener written for what they get; a day with nothing in their topics sends them no e-mail (the issue is still noted as theirs, so it is never repeated). `POST /api/v1/newsletter/subscribe` accepts `topics` too - the form on a blog category page subscribes to that category - and the blog's RSS takes the same narrowing: `/blog/feed.xml?category=agents` or `?tag=…` is a channel of its own. Weekly readers get one digest on `OSR_PLATFORM_NEWSLETTER_WEEKLY_DAY` (0 = Monday … 6 = Sunday, default 0) at the newsletter hour of their own zone, composed from the week's posts ("This week: …"); a day without posts sends nothing. Consent is recorded with the confirmation: the moment, the IP address, the browser and the exact statement the reader agreed to (`consent` on the subscriber in the operator listing), so a double opt-in can be shown later. `GET /api/v1/admin/content/issues` reports for each issue how many distinct readers opened it and how many clicked (`engagement.opens`, `engagement.clickers`; the single-issue endpoint adds the clicks per link). The counting uses a one-pixel image (`GET /api/v1/newsletter/o/{issue}/{reader}/{sig}.gif`) and a click redirect (`GET /api/v1/newsletter/c/{issue}/{reader}/{sig}?u=…`, `302` to the post, `404` on a bad signature) keyed by a hash of the subscriber id, kept for 180 days and never joined to anything else; `OSR_PLATFORM_NEWSLETTER_TRACKING=false` turns both off and the issues go out with direct links. Every copy also carries *View this issue in your browser* above the card: `/newsletter/issues/{issue}` (`GET /api/v1/newsletter/issues/{issue}`) shows the same posts the mail carried, as the blog lists them, for a client that mangles the HTML - a public page keyed by the random issue id, never by the reader, so it is not tracked and holds nothing about anyone. **What an issue - and every other message - reads like.** Everything the platform writes to a person is held to plain language: sentences under about twenty words, everyday words, one idea per sentence, one action per message, and a line that says why the person is receiving it. Readability is measured (Flesch-Kincaid grade level, longest sentence, a short jargon list) and every release is gated so no customer-facing message reads above grade 9 - roughly an eighth grader on the first pass. The newsletter issue opens with how many posts there are and how long they take to read; each card carries the title, one line on what happened, up to three **key points** (`highlights` on the post: three or four concrete facts from the source - a number, a model name, a date, a limit - so a reader who stops there knows the news), a **Why it matters** line the writer produces (`why` on the post, also on `GET /api/v1/blog/{slug}`), the reading time, the source and the link; the issue ends by asking the reader what they want more of (replies reach the operators). The blog page shows the key points and *Why it matters* above the article; operators edit both (`PATCH /api/v1/admin/content/posts/{post_id}`, `why` and `highlights`). The writer's house skill asks for the same plain language, the key points, a `## Why it matters` section and - when the source allows it - a `## What to do` section in every post, and records the draft's readability grade next to the routing decision. **Every post is a long-form article of the blog's own**, not a rewritten excerpt: the writer reads the source page (up to `OSR_PLATFORM_CONTENT_ARTICLE_SOURCE_CHARS` characters, default 12,000), asks the model for the brief first - title, summary, key points, *why it matters*, an outline of six to nine sections, tags and keywords, one small JSON object - then writes the article from that outline in plain Markdown, aiming at `OSR_PLATFORM_CONTENT_ARTICLE_WORDS` words (default 2,000) and never under `OSR_PLATFORM_CONTENT_ARTICLE_MIN_WORDS` (default 1,500): a draft that stops short is continued once from where it ended, covering only the sections still missing. **Every post ends its argument with a `## How OpenSmartRoute helps` section**, in front of *What to do*: what a team that routes its requests through the router gains from this particular story and what it would set up - written from a fixed list of product facts and the facts of the story, nothing else; on the site a card under that section takes the reader to the playground, the catalogue and the set-up guide. The brief plans it, the article pass writes it, and a post whose article skipped it gets the section in one more metered pass (`passes` shows `product`); a post that fell back to the excerpt or the brief's own text carries a short hand-written version angled by the story's tags (`routing.product_section` says `model` or `template`); posts written before the section existed receive it on the next poll, a few at a time, written by the model for each story - `POST /api/v1/admin/content/posts/product-sections` (*Add product sections* on the console's Posts tab) does up to a hundred at once, `model: false` uses the hand-written text only. Both passes are metered requests of the editorial workspace and both are graded for the learners; the post's `routing` carries them (`passes`) and the measured length (`length`: words, target, minimum, whether it came in short). When the article pass cannot produce at least 400 words the post keeps the brief's own text or the attributed excerpt and says so (`reason`). The three lengths are operator-editable at `/platform/admin/settings` and take effect on the next post. **No picture, no post**: with `OSR_PLATFORM_CONTENT_REQUIRE_IMAGE` (default `true`) a story becomes a post only when it has a real picture that answers as an image - the feed's own, or what the article page advertises (`og:image`, `twitter:image`, ``, the JSON-LD `image`, the first photo inside `
`; logos and share banners never count) - otherwise the entry is marked `no-image` and left alone, and the issue only ever carries posts with a picture, so nobody opens a digest with a blank or broken image. `POST /api/v1/admin/content/posts/images` keeps probing older posts for a picture (*Fetch missing images* on the console's Posts tab). The platform keeps its own copy of every post's picture: when a post is written its image is downloaded (16 MB cap; sources that answer `application/octet-stream` are sniffed for JPEG, PNG, GIF and WebP), brought to web size - no wider than 1,600 pixels, encoded as WebP, transparency kept, animated GIFs left as they are - and stored; each poll copies up to 25 more of the older posts, newest first, and `POST /api/v1/admin/content/posts/images/store` copies up to 200 on demand (*Copy pictures* on the Posts tab). The site serves the copy at `/blog/{slug}/image` (`GET /api/v1/blog/{slug}/image`: a week of cache, `ETag`; the web app keeps each picture for a day so the API answers once per replica, not once per reader), so pages, the RSS enclosure and the newsletter never embed a third-party host and a source that later disappears does not leave a broken picture; until the copy exists the address redirects to the source, and a source that cannot be copied is tried again after a week. The post's `image.source` keeps the original address. Registering a feed (`POST /api/v1/admin/content/feeds`) looks at the URL first: a feed is named after its own title unless a name is given; a blog's home page yields the feed it advertises in its `` (``) or at one of the conventional paths (`/feed`, `/feed.xml`, `/rss.xml`, `/rss`, `/atom.xml`, `/index.xml`); a page that is reachable but has no feed is refused with the reason; a URL that does not answer is registered with the error on its row and tried again on the next refresh. `POST /api/v1/admin/content/feeds/preview` is the same look without registering anything - the title, the site, the newest entries and whether the feed is already followed. `POST /api/v1/admin/content/feeds/seed` adds every curated default feed the deployment does not follow yet (*Add the curated feeds* on the Feeds tab); a release that adds curated feeds offers them to a running deployment once on its next start (a feed an operator deleted is never offered again). The curated list covers about seventy-five sources in four groups - the model vendors and platforms (OpenAI, Google, Hugging Face, Microsoft Foundry, NVIDIA, AWS, Databricks, Together, Replicate, Qwen, Mistral, Ollama, PyTorch, Cloudflare, Weights & Biases, Weaviate, Qdrant, Lightning AI, GitHub), the research labs (DeepMind, Google Research, Microsoft Research, Meta, Apple, Amazon Science, BAIR, EleutherAI, Ai2, MIT, arXiv cs.AI / cs.CL / cs.LG), practitioners and independent analysis (Simon Willison, Sebastian Raschka, Ethan Mollick, Lilian Weng, Hamel Husain, Eugene Yan, Andrej Karpathy, Chip Huyen, Jay Alammar, Phil Schmid, Sebastian Ruder, Interconnects, Latent Space, The Gradient, AI Snake Oil, SemiAnalysis, Last Week in AI, Towards Data Science, KDnuggets) and the press (Ars Technica, The Verge, Wired, MIT Technology Review, TechCrunch, The Decoder, MarkTechPost, InfoQ, SiliconANGLE, ZDNET, The Register, Unite.AI, Stratechery); every URL is verified to fetch and parse, and the topic filter keeps the mixed sources to posts about models and agents. **How a post credits the router.** Under every article's byline the post page carries the decision that wrote it, in three tabs. *Decision* names the target and the model, the confidence, the skill layered on the prompt, the targets also considered or fallen back from, and the router's own reason. *Waterfall* draws one bar per pass - the brief, the article, and the continuation when there was one - scaled to the slowest, with what each pass cost, how long it took and how many words it had produced by then, so a reader sees where the time and the money went. *Evaluation* shows the grade each pass was credited with - the same outcome the learners were taught, now kept on the post itself as `grade` rather than only on the learner - together with the finished text's Flesch-Kincaid reading level (`readability_grade`, 9 reads like a ninth-grade textbook) and its length against the minimum, which says plainly when a post came up short. A tab is only offered when the post carries the data behind it, so a post written before a measurement existed simply shows fewer. Every panel sits in the HTML whether or not it is the open tab, so a reader without JavaScript and a search engine still see the facts, and the tabs move with the arrow keys. A post that no model wrote says so in one line instead. **Lifecycle campaigns.** Four messages the platform sends by itself (`campaigns.py`; the `accounts` service runs the sweep on the digest cadence, the *Lifecycle campaigns* card on `/platform/admin/growth` shows what is due, what went out and runs it now - `GET /api/v1/admin/campaigns`, `POST /api/v1/admin/campaigns/run` with an optional `campaigns: [...]` list). Each is one message per workspace, claimed in `campaign_sends` before it is composed, so replicas never double-send and nobody is asked twice: | Campaign | Who | Category | |---|---|---| | `trial_ending` | The owners of a workspace whose coupon trial of a paid plan ends within three days: what happens next and where to keep the plan. | *account* (always sent) | | `upgrade_offer` | Free workspaces at least a week old that routed 25 or more requests: what the paid plans add, with the live launch coupon (`OSR_PLATFORM_LAUNCH_COUPON`) when there is one. | *product news* (opt-out honoured) | | `release_news` | Every active user when the platform version changes: *What's new in* followed by the brand and the version (outbox kind `release_notes`), written for two readers at once. First the plain words: *Nothing to do on your side*, an *In short* list of the three headline changes, then each changelog group under a label anyone understands - **New** (things you could not do before), **Changed** (things that work differently now), **Fixed**, **Security**, **On the way out**, **Removed** - with the bullet's bold lead as the title and its first sentence as the summary, at most twelve items with the count of the rest. Last, *For the technical reader*: the version's entry on `/docs/changelog`, the API reference and the diff of every commit. The first sweep on a deployment records the running version without mailing, so a backlog is never announced; a version without a section is recorded silently. | *product news* (opt-out honoured) | | `apps_reminder` | The owners of a workspace that has routed through the API for three days (and less than sixty) without the browser extension or the desktop app ever reporting in: one note on what each app does for the people on the team - the pre-send card, private data blanked, writing fixed in place; the shortcut in any app - with the download pages, and that the workspace only ever sees counts, never text. | *getting started* (opt-out honoured) | | `first_value` | The owners of a workspace (younger than thirty days) the first time a model really answered one of its requests - *Your first answer came through*: the target the router chose, requests routed and answered, the amount kept against the dearest model the workspace used, and the one ask that improves the next answer - mark it *Good* or *Poor* in the playground. Sent once, on the next sweep after the answer. | *getting started* (opt-out honoured) | | `key_expiring` | The owners of a workspace with an active API key whose end date falls within the next seven days, provided the key lived longer than that (a key minted for three days needs no reminder) - *API key … stops working in n days* (outbox kind `api_key_expiring`): the key's name and prefix, the date in the owner's time zone, and what to do - create a key with a later end date, move the apps, revoke this one. Once per key; a key ending within a day waits for nobody's morning. | *security* (always sent) | **The apps report in.** The first batch of activity an app (`browser-extension`, `desktop`) records for a workspace sends *Your browser extension / desktop app is connected* (kind `app_connected`, *account*: always sent) to the person who connected it - the signed-in user, else the owners: what it does, that the workspace records counts only, the Apps page, the app's documentation, and what to do if they did not install it. Once per app and workspace; audit `app.connected`. Every campaign has a switch: `PATCH /api/v1/admin/campaigns/{campaign}` `{enabled: false}` (the *On* column of the console's card) stops the sweep and *Run now* from sending it deployment-wide - the run reports `disabled` for it and `release_news` keeps recording the running version, so switching it back on never announces a backlog. On by default; audit `campaign.disabled` / `campaign.enabled`. The release announcement does not wait for the digest cadence: the `accounts` service checks every `OSR_PLATFORM_RELEASE_NEWS_INTERVAL_S` (default 600 s, `0` = digest cadence and the console only) whether the running version differs from the last one announced, so the mail follows a deploy within minutes; the autonomy board lists it as the `release-news` job (`GET /api/v1/admin/autonomy`, run it with `POST /api/v1/admin/autonomy/{job}/run` and `job=release-news`). **Release notes everywhere else.** The same changelog feeds `GET /api/v1/releases` (public: every released version newest first with date, sections, highlights and the changelog anchor, plus `current` - the version this deployment runs; `?limit=`) and `GET /api/v1/releases/{version}`. Signed-in people see the releases they have not read yet on the dashboard overview (*What's new*, one card per visit until they press *Got it*): `GET /api/v1/me/whats-new` lists the releases newer than the user's acknowledged version up to the running one (a first-time reader sees the running release only, never history), `POST /api/v1/me/whats-new/seen` records the running version on the user, so the card stays away on every device. The footer's version badge asks the running API for `platform_version` after the page loads, so a cached page or a docs page built before the last deploy still names the version that is live, and links to the changelog. Recipients are always people who signed up here. The platform never buys, rents, scrapes or guesses addresses and sends nothing to anyone who has not either created an account (transactional and opted-in product mail) or confirmed a newsletter subscription (the daily issue): that is what keeps deliverability - and the platform - on the right side of the law. Every run is audited (`campaign.trial_ending`, `campaign.upgrade_offer`, `campaign.release_news`, `campaign.apps_reminder`, `campaigns.run` from the console) and the messages appear in the outbox with their kind. **Scheduled reports.** `reports.py` runs inside the API: every `OSR_PLATFORM_REPORTS_INTERVAL_S` (default hourly, `0` = on demand only) it checks, for each owner of each workspace, whether a daily (yesterday), weekly (Monday) or monthly (1st) window has completed **on that owner's clock** and it is past `OSR_PLATFORM_REPORTS_HOUR` there, and sends them the workspace's **value report** for that local window; the operators in `OSR_PLATFORM_REPORT_RECIPIENTS` get the deployment-wide digest for every period (UTC). The workspace report leads with what routing saved - the executed requests' provider cost against the baseline model's list price for the same tokens, the same formula as `GET /api/v1/savings` and the Impact page; a workspace that only asked for decisions sees *N requests routed* and how to get to the saving instead - then the figures each captioned against the previous window (requests routed, answered, provider cost, tokens), where the requests came from (browser extension, desktop app, CLI, API / SDK / playground - the `X-OSR-App` header), the models that answered, what the guard protected in the window (personal data redacted, requests blocked, answers withheld, boundary refusals, injection attempts; *tracing off* when there are no trace events), the workspace's activity (keys created, rotated or revoked, members joining and leaving, providers connected, policy and plan changes, support tickets) and a button to the Impact report. The subject carries the headline (`weekly report for Acme · $142.60 saved`). `GET /api/v1/workspace/reports` returns the same composition as `preview` (usage, `previous`, `savings`, `guard`, `by_app`, `activity`) for the caller's own zone (`timezone`, `hour`). Owners and admins pick the schedule (`daily`, `weekly` - the default -, `monthly` or `off`) with `PUT /api/v1/workspace/reports` or under **Account › E-mail**; each owner can also switch the *Usage reports* category off personally. A window is recorded per recipient in `report_deliveries` and per workspace in `report_runs` and never sent twice, so several API replicas stay idempotent. `/platform/admin/reports` shows the scheduler, past digests and sends one now (`POST /api/v1/admin/reports/send`, `POST /api/v1/admin/reports/tick`). **New registrations.** Every time somebody registers - through `POST /api/v1/signup` or the first sign-in with an identity provider, which is where an account is created - the operators in `OSR_PLATFORM_SIGNUP_RECIPIENTS` get one message: who registered, the workspace, how they signed in, whether the provider vouched for the address, the acquisition `source` and `campaign` when the signup carried them, and a link to that person's page in the console. The list falls back to `OSR_PLATFORM_SUPPORT_RECIPIENTS` and then `OSR_PLATFORM_REPORT_RECIPIENTS`; with none of them set nothing is sent. A later sign-in by somebody who already has an account is not a registration and is quiet. `OSR_PLATFORM_NOTIFY_SIGNUPS=false` switches the message off for a deployment that would rather read the signup count in the daily digest and the console's activation funnel. **Support tickets.** A user raises a concern under `/platform/dashboard/support` or with `POST /api/v1/support/tickets` (`subject`, `body`, `category` question / bug / billing / account / feature / security / other, `priority` low / normal / high / urgent, optional `request_id` of the routed request it is about). The answer - and a confirmation e-mail to the requester - carry the ticket **reference**, `OSR-00042`, which identifies the ticket everywhere (`GET /api/v1/support/tickets/{key}` takes the reference or the id). Operators are told at once: an e-mail to `OSR_PLATFORM_SUPPORT_RECIPIENTS` (falls back to `OSR_PLATFORM_REPORT_RECIPIENTS`) and an entry in the console inbox. The conversation lives on the ticket: `POST /api/v1/support/tickets/{key}/messages` adds the requester's reply (an `answered` or `resolved` ticket goes back to `open`), `POST /api/v1/support/tickets/{key}/close` ends it, and `GET /api/v1/support/tickets/{key}/stream` follows it live (server-sent events: the ticket with its conversation now and again on every change, a keep-alive every 15 s, a new connection every ten minutes - the ticket page uses it, so an answer appears without a refresh); `GET /api/v1/support/tickets` lists the workspace's tickets with counts per status (`open` = waiting for support, `answered` = waiting for the requester, `resolved`, `closed`). Every public answer from the support team and every status change is e-mailed to the requester with the reference in the subject and the message quoted, and appears in the workspace inbox (rule `support`); internal notes never leave the console. A workspace may have 25 tickets waiting at a time; every step is in the audit log (`support.ticket_opened`, `support.ticket_answered`, `support.ticket_replied`, `support.ticket_updated`, `support.ticket_closed`, `support.note_added`). **Product feedback.** Every internal page has a *Send feedback* button in the dashboard header (and the page `/platform/dashboard/feedback`): a bug, a feature idea, an improvement, praise or anything else about the page the person is on. The browser attaches what an operator needs to reproduce it and the person cannot easily describe - the page and URL (query strings dropped), the browser and viewport, the platform version and plan, the last console errors and warnings and the last API calls with their status and `X-Request-Id` (never a request body, never a credential; anything that looks like a key or a bearer token is redacted in the browser and again on the API) - and up to three screenshots: captured from the screen (`getDisplayMedia`, the browser asks what to share), pasted or uploaded, at most 2 MB each, accepted by their bytes (PNG, JPEG, WebP, GIF). The item gets a **reference**, `FB-00012`, the person receives a confirmation e-mail that quotes it, and operators are told (e-mail to `OSR_PLATFORM_SUPPORT_RECIPIENTS`, console inbox). `POST /api/v1/product-feedback` (`kind` bug / idea / improvement / praise / other, `title`, `body`, `page`, `url`, optional `request_id`, `diagnostics`, `attachments` as data URLs) does the same from any client; `GET /api/v1/product-feedback` lists the workspace's items with counts per status, `GET /api/v1/product-feedback/{key}` returns one with its conversation, diagnostics and screenshot list, `POST /api/v1/product-feedback/{key}/comments` adds to it and `GET /api/v1/product-feedback/{key}/attachments/{attachment_id}` serves a screenshot (workspace-scoped, `nosniff`, sandboxed). Operators triage in the console (`/platform/admin/feedback`): status `new` -> `reviewing` -> `planned` -> `in_progress` -> `shipped`, or `declined`; an answer to the person; internal notes; assignment. **Every status change and every public answer is e-mailed to the person** with the reference in the subject and the note quoted (category `feedback`, switchable under **Account › E-mail**) and appears in the workspace inbox (rule `feedback`), so nobody has to come back and check; a shipped or declined item takes no more comments - new feedback that mentions the reference does. A workspace may have 50 open items at a time; every step is in the audit log (`feedback.submitted`, `feedback.commented`, `feedback.answered`, `feedback.updated`, `feedback.note_added`). **Membership and access notices.** People are told about what happens to their account, not only the owners: whoever sent an invitation hears when it is accepted (`member_joined`), a member whose role changes or who is removed from a workspace receives a note saying who did it (`member_role_changed`, `member_removed`; a removed member's sessions on that workspace end and their personal workspace is untouched), and an operator who signs a person out everywhere, disables or re-enables their sign-in triggers a security / account notice to that person (`sessions_revoked`, `account_disabled`, `account_enabled`). These are `account` / `security` mail and always go out. The owners also hear when an API key is rotated (`api_key_rotated`: the old secret stopped working at once), as they do when one is created or revoked, and when what an active key may do changes (`api_key_changed`: the new scopes, budgets, rate or allowed targets in plain words, the secret untouched; a rename alone is not news). The owners hear when a member leaves by themselves (`member_left`: who, and that their keys and work stay), and every member - the owner who did it included - hears when an owner deletes the organization (`workspace_deleted`: the keys stopped working, a paid plan was cancelled, everyone was moved to their own personal workspace, and whom to contact if nobody meant it). The workspace inbox mirrors the roster and the keys for everyone in the dashboard: `workspace.member_left`, `workspace.transferred` (ownership handed over), `workspace.key_rotated`, `workspace.key_scoped`, `workspace.key_limited` beside the existing `key_created`, `key_revoked`, `member_role` and `member_removed` notices; and a request the router handed to a human queue is a `workspace.handoff_pending` notice (warning) until a person answers it (`workspace.handoff_resolved`), so a team sees a waiting hand-off without subscribing a channel. When no channel of the workspace listens for `handoff.*` events, the owners are also mailed (`handoff_waiting`, category *alerts*: the request's first lines, why the router chose a person, how many wait, the link to answer) - at most once an hour per person, so a busy queue never floods a mailbox; a channel subscribed to hand-off events keeps the mailbox quiet. **Sign-in notices.** The account's owner hears about the things that decide who can get in, always (`security`): | Notice (outbox `kind`) | When | What it says | |---|---|---| | `new_sign_in` | A sign-in mints a session from a browser + system (`Chrome on Windows`, `Safari on iPhone` - the same wording as the account page) the account was not seen on in the last 90 days, or - when the deployment resolves places - from a country none of those sessions came from; or any sign-in at all for a person who chose *Every sign-in* (`login_alerts = always`). The very first session and a password-reset sign-in are quiet. | When (in the person's time zone), how (`email and password`, `Google`, the organization connection's label), the device, the place and the IP address the session was opened from; *Review sessions* (`/platform/dashboard/account`) and the reset link. Audit `user.new_device`, `user.new_location` or `user.sign_in_notice`. | | `sign_in_attempts` | Eight password attempts for one address inside a minute paused password sign-in (`429`). At most once a day per address; an address without an account gets the same `429` and no mail. | Nobody got in; wait or reset the password; turn on two-factor authentication. Audit `user.login_attempts_notice`. | | `identity_linked` / `identity_unlinked` | A provider that vouched for the address signed in to an existing account and was linked to it (`user.identity_linked`); the person unlinked a provider on the account page (`DELETE /api/v1/auth/identities/{provider}`, `user.identity_unlinked`). | Which sign-in method now opens - or no longer opens - the account; review methods and sessions. | | `recovery_code_used` | A sign-in used one of the two-factor recovery codes instead of the authenticator (`user.mfa_recovery_used`). | That the code is used up and how many are left; make new ones under Account, or change the password and sign out everywhere if it was not you. | | `recovery_codes_replaced` | The person made a fresh set of recovery codes (`POST /api/v1/auth/mfa/recovery-codes`, `user.mfa_recovery_codes_replaced`); the bell files `security.recovery_codes_replaced` too. | The old codes stopped working at once; keep the new ones safe; change the password and sign out everywhere if it was not you. | | `password_removed` | The person switched password sign-in off (`DELETE /api/v1/auth/password`, `user.password_removed`; refused while no provider is linked). | Which linked providers open the account from now on; set a new password right away if it was not you. | Every browser session records the client address it was opened from (`ip` in `GET /api/v1/auth/sessions`, shown under the device on the account page and in the operator's session table) and, when the deployment resolves places, where that was (`location`; see [Sign-in locations and alerts](#sign-in-locations-and-alerts)); older sessions have none. The account page's history (`GET /api/v1/me/audit`) names each event in plain words - *signed in from a new device*, *signed in from a new country*, *sign-in method added* - with the device, provider, place and address behind it. ### Command line sign-in Install the CLI (`curl -LsSf https://opensmartroute.ai/install.sh | sh`, or `irm https://opensmartroute.ai/install.ps1 | iex` on Windows) and run `osr login`. The CLI shows an eight-character code and opens `/platform/cli/authorize`; sign in there, check that the code and computer name match, choose the key name and approve. The platform mints a new workspace API key for that machine and the CLI stores it in `~/.config/opensmartroute/credentials.json`. This is the OAuth 2.0 device authorization grant (RFC 8628) and works the same for the community and enterprise editions: | Step | Endpoint | Notes | |---|---|---| | CLI asks for a code | `POST /api/v1/auth/device/code` `{client_name}` | anonymous, rate limited per IP; returns `device_code`, `user_code`, `verification_uri[_complete]`, `expires_in` (900 s), `interval` (5 s) | | Browser looks it up | `GET /api/v1/auth/device/{user_code}` | signed in; shows client name, status and expiry | | Browser decides | `POST /api/v1/auth/device/approve` / `deny` `{user_code, name}` | approving needs the `admin` role and a free key slot in the plan | | CLI polls | `POST /api/v1/auth/device/token` `{device_code}` | `authorization_pending`, `slow_down`, `access_denied`, `expired_token` as RFC 8628 errors; success returns `access_token` (the `osr_live_` key), `account`, `plan`, `edition` | Only approve codes from a terminal you started yourself. Keys minted this way appear on `/platform/dashboard/keys` with the chosen name and can be revoked like any other. Pasting a key works too (`osr login --token osr_live_...`, or `--with-token` from stdin in CI); `osr whoami` shows the workspace, plan and edition behind the stored credential, `osr token create|list|revoke` manage keys and `osr mcp --remote` bridges an IDE to this workspace's MCP server without copying the key around. ## 2. Route a request `POST /api/v1/route` returns a decision: the chosen target, the confidence, ranked alternatives and the signals extracted from the request. The target is not called unless `execute` is set. `target.provider` is the `provider/model` this workspace would run it on and `target.model` the reference `vendor/model` id behind it (both `null` for agents, skills, tools and humans), so a caller that executes elsewhere - LiteLLM, its own OpenAI client - can hand the chosen model straight to it and report the outcome with `/api/v1/feedback`. ```bash curl -s "$OSR_URL/api/v1/route" \ -H "Authorization: Bearer $OSR_API_KEY" \ -H "Content-Type: application/json" \ -d '{"text": "Prove that sqrt(2) is irrational.", "top_k": 3}' ``` ```json { "request_id": "7f3c...", "target": {"id": "llm-frontier", "kind": "llm", "name": "Frontier model", "provider": "azure/gpt-4.1", "model": "openai/gpt-4.1"}, "confidence": 0.81, "alternatives": [{"id": "llm-mid", "kind": "llm", "utility": 0.62}], "signals": {"complexity": 0.74, "domains": ["math"], "reasoning_need": 0.9, "contains_pii": false, "...": "..."}, "ranked": [{"id": "llm-frontier", "utility": 0.79, "quality_estimate": 0.91, "breakdown": {"...": "..."}}], "policy_rejections": {}, "explanation": "llm-frontier: high reasoning need, math domain, quality weight dominates ...", "executable": true, "elapsed_ms": 1.9 } ``` Request fields: | Field | Type | Meaning | |---|---|---| | `text` | string, required | The user request. May be empty when `history` carries the conversation or `context.event` / `context.workflow` names an event (see `context`); otherwise `422`. | | `history` | list of `{role, content}` | Prior turns; the router reads them for intent, task type and session affinity. | | `context` | object | Free-form facts (`app`, `session_id`, `user_locale`, ...). `event` (`order.shipped`, `payment.failed`, ...) or `workflow` routes a system event without any text - the event name supplies the domain and intent and rules can pin on it (`when: { event: order.* }`). Keys starting with `_` are dropped. | | `objective` | object | Relative weights: `quality` (default 1.0), `cost` (0.15), `latency` (0.05), plus `quality_floor`, `energy`, `carbon`. `cost_weight` is accepted as a synonym of `cost`. | | `constraints` | object | Hard constraints, never traded off: see below. | | `profile` | object | Opaque user/tenant features (locale, expertise, preferences). Hashed into the learners; never logged raw. | | `kinds` | list | Restrict to target kinds: `llm`, `agent`, `skill`, `persona`, `tool`, `human`. | | `top_k` | int, default 3 | How many alternatives to return. | | `plan` | bool | Build a persona → skill → model plan instead of choosing a single target (**pro** plan and above). | | `plan_mode` | `sequential` \| `parallel` | How the plan's companions run. `sequential` (the deployment default): skills run one after another, each seeing the previous one's output, and the model calls its tools in turn. `parallel`: every skill runs on the original request at the same time and their outputs are merged in slot order; the model's tool calls of one round run concurrently. Also accepted as `context.plan_mode`; anything else is `400`. The answer carries `plan_mode` and `plan_stages` - the ordered stages `prepare` (personas and skills), `answer` (the target and the model that runs an instructions-only skill or persona) and `tools`, each with `parallel` and its slots - and an executed result's `steps` carry `stage` (0 prepare, 1 answer, 2+ tool rounds) and `parallel`. | | `execute` | bool | Route and run the chosen target or plan; the output is returned in `result` (**pro** and above). | Constraints (`constraints` object): | Key | Effect | |---|---| | `max_cost_per_1k` | Drop targets whose price per 1k tokens is above this. | | `max_latency_ms` | Drop targets slower than this. | | `preferred_max_latency_ms` | Soft target: targets whose observed p90 latency is above it are penalised, not dropped. | | `region` | Only targets served from this region. | | `data_boundary` | Only targets at least this strict (`public` < `private` < `on_prem`). | | `contains_pii` | Force the PII handling on or off (`null` lets the sensitivity signal decide). | | `allow_targets` / `deny_targets` | Explicit allow / deny lists of target ids. | | `allowed_kinds` | Same as `kinds`. | | `require_tools` | Only targets that can call tools. | **Your marketplace installs are in the pool.** Every agent, skill, persona, tool, subagent or command your workspace installed from the [marketplace](https://osr-web.kindrock-caaab1a2.centralindia.azurecontainerapps.io/docs/MARKETPLACE.md) is added to the routing pool of your requests: a plan picks an installed skill or persona for its slot when it fits better than the catalogue's, and an installed primary competes for the answer itself. An installed agent or tool whose manifest declares a public `https` endpoint (`a2a`, `http` or `mcp`) is bound to it and runs when it wins an executed request; one without a callable endpoint is left out of executed requests so the decision never lands on something that cannot run (skills and personas always can - a plan runs their instructions through its model). `GET /api/v1/targets` lists them with `source: marketplace` next to the catalogue's `source: catalogue`, `executable` says whether the platform can run each one (`metadata.endpoint_unbound` gives the reason when not); another workspace never sees them. ### Quote before routing `POST /api/v1/estimate` takes the same body and returns the price of each candidate without calling anything: `targets[]` with `cost_usd`, `input_tokens`, `output_tokens`, vendor and model, plus the router's `recommended` pick and the `cheapest`, `best_quality` and `fastest` candidates. `output_tokens` overrides the assumed reply length; `monthly_requests` adds a `monthly` projection (`recommended_usd`, `cheapest_usd`, `dearest_usd`, `savings_usd`). Without a key it is rate limited per client and ignores tenant rules; with a key it is metered as `estimate` and applies your plan and tenants. `GET /api/v1/models/recommended` returns the router's picks per use case from live quotes. Details, the SDK form and the `/estimate` page: [MCP.md](https://osr-web.kindrock-caaab1a2.centralindia.azurecontainerapps.io/docs/MCP.md). ## 3. Execute the chosen target With `"execute": true` the platform runs the chosen target (or plan) with the configured providers and returns the output, the steps that ran and the metered cost: ```bash curl -s "$OSR_URL/api/v1/route" -H "Authorization: Bearer $OSR_API_KEY" -H "Content-Type: application/json" \ -d '{"text": "Summarise this contract clause for a non-lawyer: ...", "execute": true, "objective": {"cost": 0.4}}' ``` The response adds `result` with `text`, `target_id`, `steps[]` (`role`, `target_id`, `ok`, `latency_ms`, `note` - `offered` / `called` for the tools the plan gave the model, `-N tokens` for the compaction step), `tokens`, `cost_usd`, `tokens_saved` (estimated prompt tokens the compaction removed) and `fallback_from` (the target ids whose handler failed before another executable target answered - normally `[]`). With `execute: true` the decision is made among the targets that can actually run here: models and tools without a handler are left out of the pool, while skills, personas and agents stay in because a plan runs them through its model slot. Without `execute` the decision is unrestricted and `executable` on each target tells you what would run. ### Built-in tools Three kinds of `tool` target execute without a chat model. The catalogue's **calculator** (`tool-calculator`, or any tool with `metadata.builtin: calculator`) is answered by the platform itself, deterministically and without a provider: arithmetic in symbols or words (`What is 17% of 2,450?`, `12 * 37 plus 5 percent`, `sqrt(2) ^ 2`) and unit conversion for length, mass, volume, time, data, speed and temperature (`Convert 5 miles to kilometers.`). It costs no tokens and reports `builtin/calculator` as its provider (built-in tools are not listed among the upstream providers). The **web search** tool (`tool-web-search`, or `metadata.builtin: web-search`) searches the live web for a lookup, the latest news or a price - DuckDuckGo and Wikipedia without a key, a self-hosted SearXNG instance when `SEARXNG_URL` is set, Brave with `BRAVE_API_KEY` - and answers the top results as Markdown links with their snippets and the engines used (`builtin/web-search`, no tokens); the instruction is stripped from the query (`Search the web for the latest EU AI Act news and cite sources` searches for `the latest EU AI Act news`), hits whose title or snippet reads like an injected instruction are dropped, and a request with no result fails over to another executable target like any handler error. **Image tools** - a tool whose capabilities name the `image` action or modality, such as `tool-image-gen` - execute once the console maps them to a provider that serves `POST /images/generations` (OpenAI, Azure OpenAI, compatible gateways); the answer is the picture as Markdown (`!prompt (`docs/url`)`, or an inline `data:` URL when the provider returns base64), so chat clients and the playground render it, and each image counts as one completion token against the target's declared cost. A request that asks for a picture on a deployment without an image provider is routed to another executable target rather than failing with "has no handler". ### Model providers A target executes when it is mapped to a model on an **OpenAI-compatible endpoint** - OpenAI, Azure OpenAI, OpenRouter, Mistral, Groq, Together, Fireworks, DeepSeek, a local Ollama, vLLM or LiteLLM, anything that speaks `/v1/chat/completions` - or on **Anthropic's own API**: a provider whose kind is `anthropic` (or whose base URL is on `anthropic.com`) is spoken natively over the Messages API (`x-api-key`, `anthropic-version`, `POST /v1/messages`; tools, tool results, images, streaming and extended thinking translated both ways), so routing to Claude needs no OpenAI-compatible gateway in between. Two sources define the mapping and are merged (the console wins on the same provider name or target): * the **file** the deployment mounts, `OSR_PLATFORM_PROVIDERS` (YAML/JSON path or inline JSON) with `providers.` (`base_url`, `api_key` or `api_key_env`, optional `api_key_header`, `extra_headers`, `timeout_s`, `max_retries`, `concurrency` and `queue_timeout_s` - see *Capacity* below) and `models.` (`provider`, `model`, optional `system_prompt` and request defaults). `{"kind": "mock"}` is a provider that needs no endpoint and no key: it answers deterministically in-process (the prompt echoed through `reply`, a template over `{model}`, `{prompt}` and the call count; JSON when `response_format` asks; a tool call when the prompt names a tool), streams word by word, embeds with a stable vector per text and fails on cue (`fail_when` - a substring of the prompt -, `fail_every`, `latency_ms`) - the provider for tests, demos and CI pipelines, metered like any other. At start-up the file is imported into the console additively: providers the store does not know yet and targets that are not mapped yet are added, rows an operator edited are left alone. A provider the file created and later stopped naming is disabled (its offers and mappings stay, and it is enabled again the day the file names it) - a deployment that drops a cloud provider takes its credential with it, and left enabled every call to it would fail with *authentication failed*. Providers an operator created are never touched. * the **operator console**, `/platform/admin/providers`: add an endpoint from a preset, store the key inline or name the environment variable that holds it, *Check* it (`GET /models` with its credentials records reachability, latency and the upstream model ids, which the mapping form then offers), map targets to upstream models, disable or delete. Every write re-binds the handlers in the API at once, and every other API replica re-reads the store within `OSR_PLATFORM_PROVIDERS_RELOAD_S` (30 s) - no restart, no redeploy. The providers owner also runs that check on its own every `OSR_PLATFORM_PROVIDERS_CHECK_S` (300 s; 0 = only on click) for every enabled provider - one replica per interval - so the health the public catalogue, the workspace Providers page and the alerts show is never older than that. The platform's providers are what **every workspace executes on by default**: a fresh workspace needs nothing connected - its Providers page lists the included models under *Included with the platform* (per provider, with health and the target -> upstream model pairs), `GET /api/v1/workspace/providers` returns them as `platform_providers`, and the getting-started step *Models ready* is ticked. A mapping's request defaults travel with every call: the deployment's Ollama mappings carry `reasoning_effort: none` on every chat target except the reasoning-by-design ones (`research-agent`, `gpt-oss`), because a thinking model asked for a short answer otherwise spends the caller's whole `max_tokens` thinking and returns an empty message - a request that sends its own `reasoning_effort` overrides the default. `GET /api/v1/providers` is the public, OpenRouter-style view of what a deployment executes on, rendered by the site at `/providers`: one row per executable target with `provider`, `model`, the recognised reference model, `input_usd_per_1m` / `output_usd_per_1m` (from the reference catalogue when the upstream model is known, else the target's declared cost), context window, tool and reasoning support, circuit-breaker `health` and seven-day traffic; plus each provider's host, kind, health and model count. Credentials never appear: the console shows only whether a key is set and its last four characters (`api_key_set`, `api_key_hint`). Keys stored through the console live in the platform database next to the organization SSO secrets; prefer `api_key_env` when the host injects secrets. Endpoints: `GET|POST /api/v1/admin/providers`, `GET|PATCH|DELETE /api/v1/admin/providers/{id}`, `POST /api/v1/admin/providers/{id}/check`, `PUT|DELETE /api/v1/admin/providers/models/{target_id}`. Targets without a mapping still route (decisions, plans, traces, quotes) and report `executable: false`; `GET /api/v1/providers` also counts the `builtin` tools that run without a provider. #### Bring your own provider (workspace keys) Every workspace (feature `byok`, on all plans of the default catalogue) connects its **own** endpoints and keys at `/platform/dashboard/providers` - Azure OpenAI / Azure AI Foundry, Amazon Bedrock (`bedrock-runtime.. amazonaws.com/openai/v1` with a Bedrock API key), Google Gemini (`generativelanguage.googleapis.com/v1beta/openai`) or Vertex AI (`-aiplatform.googleapis.com/v1/projects//locations//endpoints/openapi`), OpenAI, Anthropic (spoken natively over its Messages API - no gateway needed), Mistral, Groq, xAI, Cohere, OpenRouter, an in-house Ollama, vLLM or LiteLLM gateway, or any other OpenAI-compatible URL - and maps catalogue targets to models on them (Azure: the deployment name; Bedrock: the model id). From then on every request of that workspace that executes - `POST /api/v1/route` with `execute: true`, `/v1/chat/completions`, the MCP `ask` tool - runs the chosen target on the workspace's endpoint with the workspace's credentials. The router, the trace, quotas, budgets and the learners are unchanged; only where the call goes differs, so the usage lands on the workspace's own cloud bill and prompts stay inside the data boundary it chose. Rules: a workspace mapping wins over the platform's for the same target; a target only the workspace mapped is executable for that workspace alone (`GET /api/v1/targets` and `GET /v1/models` are per workspace); unmapped targets keep the platform's providers, so a workspace can bring one Azure deployment and leave the rest to the platform. Disabling a provider withdraws its mappings; deleting it removes them. Responses name the answering endpoint: `opensmartroute.provider` on chat completions and `result.provider` on `/api/v1/route` (`/`). Credentials are stored AES-256-GCM encrypted under `OSR_PLATFORM_SECRET_KEY` (falling back to the session secret; set it explicitly on multi-replica deployments), bound to the workspace, never returned by the API (`api_key_set`, `api_key_hint` only) and never resolved from the API container's environment - a workspace cannot borrow the deployment's keys by leaving the field empty. A key may also be given **by reference**, so the secret itself never enters the platform database: `keyvault:///[/]` (or the secret's `https://.vault.azure.net/secrets/...` URL) is read from Azure Key Vault with the API container's managed identity when the client is built (the identity needs the *Key Vault Secrets User* role; the value is cached for five minutes, so a rotation in the vault reaches the router without a redeploy), `env://NAME` reads a variable of the API container (the operator's store of last resort), and `identity://` needs no secret at all: the container's managed identity mints an access token for the resource (`https://cognitiveservices.azure.com` for Azure OpenAI and Azure AI Foundry, `https://ai.azure.com`) and sends it as the bearer credential - keyless, rotation-free; give the identity the *Cognitive Services OpenAI User* role on the resource (the console's *Azure OpenAI / Foundry (managed identity)* preset fills the reference in, and *Hugging Face Inference Providers* is a preset too). The platform's own file providers may use every form in `api_key`. A reference is not secret: the console shows it whole (`api_key_hint`, `api_key_reference: true`), and a vault that refuses the read is what *Check* reports. The endpoint must be a **public** host: base URLs on loopback, private, link-local or reserved addresses, bare hostnames and `.internal` / `.local` names are refused (`400 base_url must be a public endpoint`), at save time and again before *Check*, so a workspace cannot aim the platform's network position at its database, Redis, the model servers or the cloud metadata service; credentials embedded in the URL are refused too. `OSR_PLATFORM_BYOK_ALLOW_PRIVATE=true` lifts the host rule on a self-hosted deployment whose workspaces are trusted and whose providers live on the internal network (the development compose stack sets it). Members read the page; adding, changing, rotating (`api_key`) or removing (`clear_api_key`) a key, mapping and deleting need the `admin` role (API keys act as the workspace). Every change is in the workspace audit log (`provider.created`, `provider.updated` with `key_rotated`, `provider.mapped`, `provider.unmapped`, `provider.deleted` - never the key). *Check* calls the endpoint's `/models` with the stored key and offers the returned ids on the provider card and in the mapping dialog; the dashboard runs it right after connecting or editing a provider unless told not to. Endpoints: `GET|POST /api/v1/workspace/providers`, `GET|PATCH|DELETE /api/v1/workspace/providers/{id}`, `POST /api/v1/workspace/providers/{id}/check`, `PUT|DELETE /api/v1/workspace/providers/models/{target_id}` (`provider_id`, `model`, optional `system_prompt` and request `defaults`). Up to 20 providers per workspace; a change is in force on the next request on every API replica and service (the workspace's revision is compared per request, so a removed provider is never used again). #### Self-hosted models The deployment runs its own model servers next to the cloud providers: the [model server](https://osr-web.kindrock-caaab1a2.centralindia.azurecontainerapps.io/docs/MODELS.md) (the OpenSmartRoute model host - Ollama and OpenAI APIs over llama.cpp, CPU or CUDA; the Ollama image remains a supported alternative, `ollamaImage` / `OSR_OLLAMA_IMAGE` picks) with a curated set of open weights on a shared file share (`ollamaEnabled`, on by default on Azure; the compose stacks run the same set, pulled by `OSR_OLLAMA_MODELS`). Every model is stored and mapped under its ollama.com tag; where that tag's blob is Ollama's own conversion that a standard llama.cpp cannot read (Qwen3.5, gpt-oss, GLM-4.7-Flash, GLM-OCR, TranslateGemma), `ollamaModelSources` names the stock GGUF on Hugging Face the model server pulls instead (`qwen3.5:4b=hf.co/unsloth/Qwen3.5-4B-GGUF:Q4_K_M`), so the whole set is hosted by the OpenSmartRoute model server without the Ollama image; `ollamaModelImages` stays for a per-model image exception. On CPU profiles **every model runs as its own always-warm model server** (one model pinned in memory per server), so a request never waits for another target's weights to be re-read from the share - on a shared CPU server that eviction churn, not the inference, is the latency. The set is the strongest open model per job - by the published benchmarks, not by name - that answers a short request in a few seconds on its hardware, so the on-prem and specialist targets of the catalogue execute without data leaving the deployment: | Target | Model | Job | |---|---|---| | `llm-onprem` | `qwen3.5:4b` | Private chat, reasoning, extraction and drafting: Qwen 3.5, 201 languages, image input, tool calls, thinking, 256K context. | | `llm-small`, `llm-mid`, `llm-frontier` | `qwen3.5:4b` on CPU; `gpt-oss:20b` / `glm-4.7-flash` for the two upper tiers on a dedicated or GPU profile; `gpt-oss:120b` / `deepseek-v4-flash` through Ollama Cloud with a key | The cloud tiers of the catalogue. Every request stays on open weights; the dedicated-node and Ollama Cloud sets lift the upper tiers where they apply, and an operator who wants closed models connects them in the console or switches `azureOpenAiEnabled` on. | | `writer-small` | `qwen3.5:4b` | Content - posts, emails, product copy - and translation. | | `support-agent` | `qwen3.5:4b` | Fast support replies and classification. | | `coder-small`, `coding-agent` | `qwen2.5-coder:7b` (`glm-4.7-flash` on a dedicated or GPU profile) | Code generation, explanation and repair; GLM-4.7-Flash is the 30B-A3B mixture-of-experts that leads the 30B class on SWE-bench Verified and τ²-Bench. | | `vision-ocr` | `glm-ocr` | OCR and document understanding - text, tables, formulas, figures, seals - #1 on OmniDocBench 1.5 with 0.9B parameters. | | `skill-translate` | `translategemma:4b` | Translation between 55 languages (Gemma 3 based). | | `research-agent` | `deepseek-r1:8b` (dedicated or GPU profiles) | DeepSeek-R1-0528 distilled into Qwen3-8B: visible chain-of-thought for research and analysis - thinking answers take their time by design. | | `embed-small` | `qwen3-embedding:4b` | Multilingual text embeddings (2560 dimensions, 100+ languages) for `POST /v1/embeddings`; one forward pass answers fast even on CPU. | | `gpt-oss` | `gpt-oss:20b` (dedicated or GPU profiles) | OpenAI's open-weight reasoning model - a mixture-of-experts whose ~3B active parameters decode at CPU speed once the weights sit in RAM. | The frontier open weights no single node hosts - Kimi K2.7 Code, DeepSeek-V4-Flash, gpt-oss-120b - answer through **Ollama Cloud** (`https://ollama.com/v1`, the same OpenAI-compatible surface) when `ollamaCloudApiKey` (`OSR_OLLAMA_CLOUD_API_KEY`, an API key from ollama.com/settings/keys) is set: `ollamaCloudTargets` maps `coding-agent`, `research-agent`, `gpt-oss`, `llm-mid` and `llm-frontier` to them and replaces the local mapping of those targets; without the key they stay on the local models. Any other ollama.com model is one console mapping away (provider `ollama-cloud`). `ollamaWorkloadProfile` picks the hardware. `Consumption` (the default) serves the small set: one always-warm 4 vCPU / 8 GiB app per model, each answering one generation at a time and scaling out to `ollamaMaxReplicas` (default 4, `OSR_OLLAMA_MAX_REPLICAS`) on bursts. A dedicated size (`E4`, `E8`, ... - E-series memory fits the 19 GB GLM-4.7-Flash) adds the `ollamaLargeTargets` models, **one node per model**: only models that earn a whole node are in that set - the mixture-of-experts coder and `gpt-oss:20b`, whose ~3B active parameters decode at CPU speed once the weights sit in RAM, and the visible-reasoning `deepseek-r1:8b`, whose answers are slow by design. A node costs the same whether it holds one model or four, and Ollama runs each loaded model in its own runner, so `ollamaNodeGroups` packs models that fit together onto one always-warm server per node with every model resident and preloaded (`OLLAMA_MAX_LOADED_MODELS` = the group size): a short OCR or embedding call runs beside a long reasoning generation instead of waiting for it, and each grouped model keeps its own provider name (`ollama-`) so console mappings and health rows do not move. What limits a group is memory - the resident working sets must fit the node (about 22 GiB of an E4's 28; the per-model `WorkingSetBytes` metric is the figure to add up). The default keeps the two 22 GiB models (`glm-4.7-flash`, `gpt-oss:20b`) on nodes of their own and packs `deepseek-r1:8b` with `translategemma:4b`, `glm-ocr` and `qwen3-embedding:4b`; `qwen3.5:4b`, the busiest model, stays a Consumption replica that scales out on bursts - three nodes and one replica instead of four nodes and four replicas, every model as warm as before. `OSR_OLLAMA_NODE_GROUPS` (a JSON array of `{"name": ..., "models": [...]}`) overrides the grouping; a model named by no group keeps the per-model layout. A model server that drops out of the template when a model moves into a group is retired by the post-provision hook (`OSR_OLLAMA_PRUNE=false` keeps it). Dense 9-12B chat upgrades are deliberately not served on CPU (a dense model that size cannot answer a short request in seconds there); they return with a serverless GPU (`Consumption-GPU-NC8as-T4`, `Consumption-GPU-NC24-A100`; needs GPU quota in a region that offers it), where one shared server keeps several models resident and decodes 8 / 32 requests at once per replica (the T4 at 8K context with the KV cache in `q8_0`, the A100 the whole set at 16K), scaling out to `ollamaMaxReplicas`; the replicas share the model store and pull weights under a lock. `ollamaTargets` / `ollamaLargeTargets` change the mapping; the model tags are pulled at start-up and kept on the share, and `GET /api/v1/providers` shows which targets the servers answer for. **Azure OpenAI is opt-in**: `azureOpenAiEnabled` (`OSR_AZURE_OPENAI_ENABLED`, default `false`) creates the account and the `openAiModels` deployments and hands the `azure` provider every target the Ollama sets leave unmapped - which also gives image generation (`tool-image-gen` → `gpt-image-2`) and, as the fallback for `transcribe-small`, `gpt-transcribe` a home; off, those two wait for a console mapping (the self-hosted speech server covers transcription when `voiceEnabled`). Ollama serves text, chat, code, vision (image input) and embeddings. **Sizing a model server.** The numbers behind those profiles are arithmetic, not folklore, and the SDK does it for you (`opensmartroute.inference`, `osr capacity`). A model in memory is its weights (parameters × bits per weight: a `Q4_K_M` file is about 4.85 bits, `Q8_0` 8.5, `F16` 16, `gpt-oss`'s `MXFP4` 4.25) plus a KV cache the server reserves for `OLLAMA_CONTEXT_LENGTH × OLLAMA_NUM_PARALLEL` tokens (2 × layers × KV heads × head size × 2 bytes per token in `f16`; `q8_0` halves it), plus 5-20 % of runtime buffers. On a CPU, generation is memory-bandwidth bound - every token re-reads the active weights, so a dense 8B `Q4_K_M` model (4.8 GB) decodes at about 4 tokens/s on a 20 GB/s node while a 30B-A3B mixture reads only its ~3B active parameters and runs like a 3B model once its 18 GB sits in RAM - and prompt evaluation is compute bound (about 2 FLOPs per active parameter per token). A cold load streams the weights from the model store, which on a network share is the 40-150 s that used to dominate answers. `osr capacity show qwen3.5:4b glm-4.7-flash --memory 28Gi --cores 4` prints the footprint, whether it fits after the OS headroom, tokens/s and the warm and cold latency of a typical request; `osr capacity plan --memory 28Gi [--hosts N] [--shrink]` packs a set onto hosts largest first, shrinks a model that does not fit (coarser KV cache, shorter context, fewer slots) when asked, names what fits nowhere and prints the `OLLAMA_MAX_LOADED_MODELS` / `OLLAMA_NUM_PARALLEL` / `OLLAMA_CONTEXT_LENGTH` / `OLLAMA_KV_CACHE_TYPE` / `OLLAMA_KEEP_ALIVE` / `OLLAMA_MAX_QUEUE` lines that pin exactly that set (`--strict` exits 2 when a model is unplaced, for CI). With `--url http://:11434` the shapes come from the server's `/api/show` (exact layer and head counts, file size, trained context) instead of the tag heuristic, and `osr capacity ps --url ...` lists what it holds in memory right now, on CPU or GPU, resident or expiring. Defaults - 8K context, one slot, `f16`, 12 % headroom, 20 GB/s, 40 GFLOP/s per core, 0.3 GB/s cold reads - are `OSR_INFERENCE_*` settings. The test suite runs the curated set above through the planner against the 8 GiB Consumption app and the 14 / 28 GiB dedicated nodes, so a model that stops fitting its app fails CI before it fails a deployment. **Warm models, not cold loads.** The measured latency of a self-hosted answer was never the inference - it was the cold load, so the platform now removes it twice over. Every model server loads its set into memory as soon as the weights are pulled (`OSR_OLLAMA_PRELOAD=1` on the per-model CPU apps and the A100, where the whole set stays resident) and keeps it there (`OLLAMA_KEEP_ALIVE=-1`). Independently, the providers owner asks every self-hosted Ollama provider what it holds (`/api/ps`) on each health pass and, with `OSR_PLATFORM_PROVIDERS_KEEP_WARM_MAX` > 0 (Azure sets it to 1 on CPU profiles and to the resident count on a GPU), loads any mapped model that is not resident - but only when the provider maps at most that many distinct models, so a server with more models than memory is never made to evict them in turn. `GET /api/v1/providers` shows the result: `providers[].loaded` lists what each server holds and `models[].warm` says whether a target's upstream model answers from memory (`null` when the endpoint cannot say); the operator's *Check* records the same. `osr capacity ps --url` is the command-line view of the same fact. Voice is a second self-hosted server: `voiceEnabled` (`OSR_VOICE_ENABLED`, on by default) adds `osr-voice`, a [Speaches](https://speaches.ai) container (faster-whisper speech to text, Kokoro text to speech, OpenAI-compatible `/v1/audio/*`, CPU, weights preloaded into its own share) and maps the `transcribe-small` / `speak-small` targets to it; `voiceSttModel` (`OSR_VOICE_STT_MODEL`, default Whisper large-v3-turbo - the accuracy of large-v3 with a 4-layer decoder, on 4 vCPU / 8 GiB; `Systran/faster-whisper-small` runs on 2 vCPU / 4 GiB) and `voiceTtsModel` pick the models - the compose stacks do the same with `COMPOSE_PROFILES=llm,voice`. With voice off, map `speak-small` to OpenAI or Azure OpenAI in the console (`gpt-4o-mini-tts`); the OpenAI Realtime API (a WebSocket session, not a request the router can score) and video generation always need a cloud provider called directly. The router decides which target answers either way. **The router ranks on what the deployment measured and bills, not on the catalogue's word.** A CPU-hosted 4B model writes about five tokens a second: a one-line classification in ten seconds, an article in minutes. The reference catalogue declared it at 1.2 s and at five times the price of the small cloud model, so the router, the quotes and the latency SLO all read a fiction - production routed almost nothing to the six self-hosted servers while the platform's own writers paid the cloud model. Every routing process now calibrates its executable models every `OSR_PLATFORM_CALIBRATION_INTERVAL_S` seconds (300; 0 leaves the declared figures) from the last `OSR_PLATFORM_CALIBRATION_DAYS` (7) of answered requests: the median latency becomes the target's `latency_ms`, and when enough answers carry a token count and the time grows with it, a least-squares line splits it into a fixed part and a per-token part (`latency_per_token_ms`), so the router expects the length of *each* answer - the slow box keeps short requests and loses long drafts, and `max_latency_ms` rejects on the same expectation. The price becomes the reference card of the model the mapping names (the card the estimate and the usage pricing already bill) when the catalogue has one; a local model no card prices keeps its declared price, which is what the deployment bills. The declared figures stay under the target's `metadata.declared`, the pass records itself under `metadata.calibrated` (samples, fit, where the price came from), `GET /api/v1/targets` shows both, the rank trace carries the per-request estimate and `GET /api/v1/learning` -> `calibration` lists what the last pass set. Five answered requests are the least a target needs before its measurement replaces the catalogue. #### Capacity: admission control and overflow A model server has a fixed number of generations it can run at once - one on a CPU replica, 8 or 32 on a GPU - while a burst can bring thousands of chat completions in the same second. The platform therefore never queues requests on the model server. Every provider carries `concurrency` (the console field or the file key; `0` = unbounded, the default for cloud endpoints) and `queue_timeout_s` (file key, default 20 s): that many calls run on the endpoint at once, the next ones wait up to the timeout for a slot, and a call that gets none is **re-routed** by the fallback loop to the next executable target - a chat pinned to `llm-onprem` with the self-hosted server saturated answers from a cloud tier and says so in `opensmartroute.fallback_from`; `osr.fallbacks: false` or a candidate list that is entirely saturated returns `503` with `Retry-After` instead. The re-route is a routing decision like the first one, so every policy still applies: a request with `constraints.data_boundary: on_prem` (or a tenant / workspace policy that sets it) is **never** spilled to a cloud target, even when the caller listed cloud models as fallbacks - it waits for a slot and then answers `503`. The Azure template sets the Ollama provider's `concurrency` to replicas x parallel slots automatically; the compose stacks set it next to `OSR_OLLAMA_NUM_PARALLEL`. The gates are per process and visible in `GET /api/v1/admin/health` (`admission[]`: `concurrency`, `inflight`, `waiting`, `admitted`, `rejected`, `peak`, `avg_hold_ms`) and `GET /api/v1/admin/providers`. The API side holds `OSR_PLATFORM_MAX_INFLIGHT` requests per process (default 10 000): route handlers run on the worker-thread pool, which is widened to that number at start-up (the framework default is 40 threads, i.e. 40 concurrent completions per replica), and the HTTP server answers `503` beyond it rather than queueing sockets. In cluster mode the API and the `openai` service scale on in-flight requests (`apiMaxReplicas`), so ten thousand concurrent requests are spread over replicas, admitted to the model servers up to their capacity and spilled over to the cloud tiers for the rest - the routing decision, trace, budget and quota logic is identical on both paths. **Stopping without dropping a request.** A replica that receives its stop signal (a deploy, a scale-in, a node move) does not close its listener at once: `GET /readyz` answers `503` with `status: draining` so the load balancer moves new requests to the other replicas, while this one keeps accepting for `OSR_PLATFORM_DRAIN_S` seconds (default 10 - the readiness probe's period times its failure threshold); then the listener closes and the requests still in flight - a long streamed completion included - get `OSR_PLATFORM_GRACEFUL_SHUTDOWN_S` seconds (default 120) to finish before the process stops. `/healthz` stays `200` throughout: the process is healthy, it is leaving. The Azure template keeps at least `apiMinReplicas` API replicas (default 2) in cluster mode so one can leave while the other answers, sets the container's termination grace period to cover both windows and probes readiness every 5 seconds. Set `OSR_PLATFORM_DRAIN_S=0` on a laptop to stop at once. ### Response headers | Header | On | Meaning | |---|---|---| | `X-Request-Id` | every response | The id to quote in feedback and support requests; echoed back when you send one. The same id (and your W3C `traceparent`, when you send one) is forwarded to the provider that answers an executed request, so one id follows a request from your logs through the platform into the provider's. | | `X-OSR-Data-Protection` | `/api/v1/route`, `/v1/chat/completions` | The data-protection mode that acted on the request (`detect`, `mask`, `block`), only when personal or health data was found (section 9). | | `Server-Timing` | every response | `app;dur=` - time spent inside the platform. | | `X-OSR-Target` | `/v1/chat/completions`, `/v1/embeddings`, `/v1/audio/*`, `/v1/images/generations` | The target id that answered. | | `X-OSR-Confidence` | `/v1/chat/completions` | The router's confidence in that choice. | | `X-OSR-Charge-USD` | `/api/v1/route` (executed), `/v1/chat/completions` | What the workspace is charged for the request under the price version in force (see *Usage pricing*). | | `X-Wallet-Balance`, `X-Wallet-Available` | `402` | The credits balance of a wallet-billed workspace in cents, and what is left after the charges not yet settled and the holds of requests in flight. | | `X-OSR-Provider`, `X-OSR-Fallback-From` | `/v1/audio/speech` | The `provider/model` that produced the audio and the targets that failed before it (the body is audio, so the decision travels in headers). | | `Retry-After` | `429`, `503` | Seconds until the per-minute window, the monthly quota or the budget period resets; on `503`, when every candidate model server was at capacity, the estimated wait for a slot. | | `X-Quota-Limit`, `X-Quota-Used`, `X-Quota-Reset` | `429` (quota) | Your plan's requests per month, how many the workspace used, and the unix time the month rolls over. | | `X-Budget-Limit`, `X-Budget-Used`, `X-Budget-Period` | `429` (budget) | The exhausted workspace, tenant or key budget in USD (used includes the holds of requests in flight) and its period (`daily` / `monthly`). | | `X-Upgrade: true` | `403`, `429` (quota) | The feature exists but is not in your plan, or a bigger plan would keep routing. | | `X-Edition: enterprise` | `403` | Your plan includes the feature (tenants, statistics, audit) but this deployment runs the community image, which cannot serve it. | | `X-Key-Scope` | `403` | The API key was created with scopes and lacks the named one (`read`, `route`, `execute`, `feedback`, `manage`). | | `ETag`, `Cache-Control` | public catalogue reads | Conditional requests: send `If-None-Match` and receive `304` when nothing changed. | ## 4. OpenAI-compatible endpoint Point an OpenAI SDK at `$OSR_URL/v1` and set `model` to `auto`. The router chooses the target per request, runs it and returns a standard `chat.completion` whose `model` field is the target that answered. Decision metadata is added in an `opensmartroute` object that other clients ignore. ```python from openai import OpenAI client = OpenAI(base_url=f"{OSR_URL}/v1", api_key=OSR_API_KEY) reply = client.chat.completions.create( model="auto", messages=[{"role": "user", "content": "Write a haiku about routing."}], ) print(reply.model) # e.g. "llm-small" - the target the router chose print(reply.choices[0].message.content) ``` ```json "opensmartroute": { "request_id": "7f3c...", "target": "llm-small", "kind": "llm", "confidence": 0.88, "plan": [], "alternatives": ["llm-mid"], "cost_usd": 0.00012, "latency_ms": 412.5, "fallback_from": [], "tokens_saved": 0 } ``` `plan` lists the persona, skill and tool targets the router composed with the model (tool slots are offered to the model as functions and run by the platform when the model calls them, unless the request brought its own `tools`); `tokens_saved` is the estimated prompt saving of the compaction step (whitespace, repeated lines and stale history removed before the model was called - only present when it saved anything). Options outside the OpenAI schema go in the request body and are ignored by other providers: | Field | Meaning | |---|---| | `model` | `auto` / `osr/auto` (default) lets the router choose; `osr/auto:cheap` / `:fast` / `:quality` / `:private` / `:agentic` / `:llm` preset the objective, a data-boundary constraint or the allowed kinds (explicit `osr.*` options win); a target id pins that target. | | `models` | Candidate list: the router chooses among them and falls back down the list when a target fails. | | `osr.objective` / `osr.constraints` | Same as on `/api/v1/route`. | | `osr.tenant` | Tenant slug (or the `X-OSR-Tenant` header); applies the tenant's constraints (**enterprise**). | | `osr.plan` | `true`/`false` to force or suppress plan composition; omitted = whatever the plan tier allows. | | `osr.plan_mode` | `sequential` / `parallel`: how the plan's skills and tool calls run (see `plan_mode` on `POST /api/v1/route`). | | `osr.fallbacks` | `false` disables the automatic retry on the next best target (default `true`). | | `stream` | `true` streams server-sent events in the OpenAI format: each provider token is relayed as it arrives; the last chunk carries `usage` and the `opensmartroute` metadata. Quota, routing and provider errors before the first token are ordinary HTTP errors; a fallback to another target happens only before the first token. | | `tools`, `tool_choice`, `response_format`, `stop`, `n`, `seed`, `logprobs`, `top_logprobs`, `parallel_tool_calls` | Forwarded to the provider unchanged. A tool call comes back as `message.tool_calls` with `finish_reason: "tool_calls"`; send the `tool` turn back in `messages` (with `tool_call_id`) and the model finishes the answer. Unknown fields are passed through too. | Message `content` may be a list of parts: text parts form the request, `image_url` / `input_audio` / `file` parts reach the provider unchanged and make the router prefer multimodal targets (`ModalitySignal`). `GET /v1/models` lists `auto`, `osr/auto`, its `:cheap` / `:fast` / `:quality` / `:private` / `:agentic` / `:llm` variants, every executable target and every catalogue model on offer by the token (OpenRouter's shape with the price this workspace pays; `model: "openai/gpt-4.1"` routes among its endpoints, `"openai/gpt-4.1@azure"` pins one - see *Models by the token* under section 6). `POST /v1/embeddings` serves the OpenAI embeddings API on the same key: `model: "auto"` routes among the targets tagged `embeddings` (`capabilities.tags: [embeddings]`, or `metadata.embeddings: true`) that a provider maps; a target id pins one. Requests are metered like chat completions and the `opensmartroute` metadata block names the target and its cost. Without an embeddings-capable target the endpoint answers `501`. ### Prompt registry The instruction a request uses can live in the workspace instead of in the calling code. `/api/v1/prompts` keeps prompts by `name` with immutable numbered versions (`POST /api/v1/prompts` adds version 1 of a new name or the next version of an existing one), **labels** that deploy a version (`production`, `staging`; a label names one version, so `PUT /api/v1/prompts/{name}/{version}/labels` moves it - that is the release), `{{variable}}` placeholders with optional defaults (`{{tone|calm}}`; the variables a version uses are recorded with it) and a preview (`POST /api/v1/prompts/{name}/render` with `version` or `label` and `variables`) that renders without routing. Two kinds: a **text** prompt becomes the system instruction, in front of the request's own; a **chat** prompt (`[{role, content}]`) puts its system turns there and prepends its user / assistant turns to the history as examples. A request names one in `osr.prompt` (chat completions) or `prompt` (`/api/v1/route`): `{"name": "support-reply", "label": "production", "variables": {"company": "Acme"}}`; `version` pins a number, neither label nor version means the newest. A missing variable without a default is a `400` naming it; an unknown prompt or label a `404`, before anything is routed or metered. The resolved `name@version` is on the response (`X-OSR-Prompt`) and in the request's context, so the trace and the request log show which wording acted; the guard and the data-protection pass see the rendered instruction as the model will. `GET /api/v1/prompts` lists names with newest version, version count and labels; `GET /api/v1/prompts/{name}` the versions with content; `DELETE` removes a version or a name; the workspace export carries every version. Up to 200 prompts of 100 versions each; writes take the `manage` scope and are audited (`prompt.version_created`, `prompt.labels_changed`, `prompt.version_deleted`, `prompt.deleted`). The usage row of a request that named a prompt keeps the tag under `facts.prompt`, and `GET /api/v1/prompts/{name}/stats` (`?days=`, 1-90, default 30) compares the versions on it: per version the requests, the answered share, tokens, cost, median latency and the outcomes reported on them (success rate, mean quality) - the figures a prompt change is judged by, next to each version on the dashboard. Dashboard: *Prompts*. ### Conversations and memory The desktop app, the browser extension's side panel and the playground share one chat path, so the platform gives all of them the same three things at once. Two `osr` options on `POST /v1/chat/completions` turn them on: | Option | Effect | |---|---| | `osr.conversation: true` | Start a conversation the platform keeps. The history the client sends with this first message is imported once; the `conversation` object of the answer's `opensmartroute` block names the conversation (`id`). | | `osr.conversation_id` | Continue that conversation: send **only the new message**. The platform prepends the last 12 turns verbatim and, as a system note, a rolling summary of everything older (extractive - one line per turn that left the window, never invented), so a thread never outgrows the model's context and the same thread continues from any client of the workspace. An id the caller does not own is `404`. | | `osr.memory` | Place the long-term memory in front of the request and learn from this message (`true` / `false`; unset = the subject's switch, on by default). Works with or without a conversation. | Every answer then carries a `conversation` object in its `opensmartroute` block: `id`, `new`, `turns`, `topic_shift` (the question shared no vocabulary or domain with the recent turns and was not a follow-up - a client offers *Move to a new chat*), `resumed` and `idle_s` (the conversation had been idle for hours), `remembered` (facts learned from this message, with their kind), `forgot` (memories removed on request), `memories_used` and `memory` (the switch). The conversation id is also the `X-OSR-Conversation-Id` response header. **Long-term memory is rules, not a model.** Facts a person states about themselves are extracted from their own messages - *my name is …*, *I work at …*, *I'm a data engineer*, *I'm based in …*, *we use Python and dbt*, *I prefer short answers*, *always answer in French*, *I'm working on …*, *remember that …* - and stored per subject as `name`, `role`, `employer`, `location`, `language`, `preference`, `uses`, `project` or `note` (one value for name, location and language; at most 200 memories, the least used dropped first). Before a request the identity facts, the memories whose words the request mentions and the freshest notes (12 at most) are written into the system message as *What you know about the person you are helping*; *forget that I work at Acme* removes the matching memory. Nothing is inferred, summarised by a model or shared across subjects. **Whose memory.** A personal workspace is one person: its memory and conversations belong to the workspace, whatever key or device is used. Inside an organization each signed-in user has their own; an API key has its own (`key`), because the platform cannot tell who holds a shared key. `GET /api/v1/memory` says which (`subject`: `workspace`, `user` or `key`). | Endpoint | Purpose | |---|---| | `GET /api/v1/conversations` | The caller's conversations, newest first (`?client=desktop`, `?q=` searches titles and summaries). | | `GET /api/v1/conversations/{conversation_id}` | One conversation: title, summary, the stored turns (`turns_list` with target, request id and cost per answer, so feedback keeps working after a thread is reopened). | | `PATCH /api/v1/conversations/{conversation_id}` | Rename (`{title}`); the desktop and the extension show the same title. | | `POST /api/v1/conversations/{conversation_id}/split` | A topic change the person confirmed: the turns from `from_seq` on become their own conversation. | | `DELETE /api/v1/conversations/{conversation_id}`, `DELETE /api/v1/conversations` | Delete one, or every conversation of the subject. | | `GET /api/v1/memory` | The switch, the subject and every memory (kind, content, uses, disabled). | | `PUT /api/v1/memory/settings` | `{enabled}` - off: nothing is placed and nothing is learned (stateless chat). | | `POST /api/v1/memory` | Add a fact by hand (`{content, kind?}`; `409` when already known). | | `PATCH /api/v1/memory/{memory_id}` | Edit (`content`) or pause (`disabled`) a memory; a paused fact is never re-learned. | | `DELETE /api/v1/memory/{memory_id}`, `DELETE /api/v1/memory` | Forget one, or everything. | Conversations and memories are deleted with the workspace. The dashboard's Memory page (`/platform/dashboard/memory`) lists both; the desktop app's Settings and the extension's settings page show the memory list with the same switch. ### Knowledge graph and context engineering Memory remembers what was said; the **knowledge graph** keeps *facts* - typed, sourced, weighted, expiring triples in one graph per workspace (the SDK's [`opensmartroute.knowledge`](https://osr-web.kindrock-caaab1a2.centralindia.azurecontainerapps.io/docs/GUIDE.md#knowledge-graph-and-context-engineering), stored in the platform database). It fills itself from the traffic it serves: what a person states about themselves (`person:` - `works_at`, `has_role`, `located_in`, `speaks`, `prefers`, `uses`, `works_on`, `noted`, `mentions`), what their requests were about and where they were routed (`asked_about`, `routed_to`), what a live web search cited (`cites`, expiring) and what the workspace adds by hand. Before every routed chat the graph's context engineer chooses, ranks and trims what the model should know into a token budget (600 tokens by default, per subject) and accounts for every line; the answer's `opensmartroute.knowledge` block says how many tokens were placed, which sections, what was dropped and which facts were learned from the message. The personal section is placed by the graph only when conversation memory is off - otherwise memory already put those facts in front of the model and the graph adds what memory cannot: the workspace's shared facts that match the request. A person's facts are never shown to another subject of the same organization. **Self-managing.** The process that owns the `knowledge` domain (the API, or the `knowledge` service in cluster mode) runs a maintenance pass every `OSR_PLATFORM_KNOWLEDGE_MAINTAIN_S` (an hour by default; one replica per interval): unconfirmed facts lose confidence and are dropped under the floor, expired citations go, and when TensorFlow is on the image a TransE link predictor ([`KnowledgeEmbedding`](https://osr-web.kindrock-caaab1a2.centralindia.azurecontainerapps.io/docs/SDK.md#knowledge-graph-and-context-engineering)) is refitted over each active workspace's shared facts and stored - `GET /api/v1/knowledge/suggest` answers from it (*which domain is this target probably good at that nobody declared*), falling back to frequency over peers without it. | Endpoint | Purpose | |---|---| | `GET /api/v1/knowledge` | The switch and budget, the subject's entity, graph statistics (facts, entities, per predicate, per source), the trained model's report, the predicate vocabulary and the export formats. | | `PUT /api/v1/knowledge/settings` | `{enabled?, budget_tokens?}` - off: nothing is placed and nothing is learned; the budget (0-4000) bounds the engineered context. | | `GET /api/v1/knowledge/facts` | The facts the caller may see - their own personal facts and the workspace's shared ones (`?q=` word search, `?predicate=`, `?personal=true|false`, `?limit=`). | | `POST /api/v1/knowledge/facts` | Add a fact by hand (`{predicate, object, subject?, confidence?}`; the subject defaults to the person, a shared fact names any non-person entity such as `target:llm-small`). | | `DELETE /api/v1/knowledge/facts` | Delete by `subject`, `predicate` and/or `object` (query parameters; another person's facts are refused). | | `DELETE /api/v1/knowledge` | Delete every personal fact of the subject. | | `POST /api/v1/knowledge/forget` | `{phrase}` - *forget that I work at Acme* drops the personal facts the phrase names. | | `POST /api/v1/knowledge/context` | `{text, budget_tokens?}` - preview the engineered context the graph would place for a message: the text, every line placed with its score, tokens and source, and what was dropped and why. | | `GET /api/v1/knowledge/suggest` | `?subject=&predicate=&k=` - likely objects for a fact the graph does not hold, from the trained link predictor when there is one. | | `GET /api/v1/knowledge/export` | `?format=jsonld|ntriples|graphml|cypher` (`&personal=false` leaves the personal facts out) - the graph for rdflib / Jena / GraphDB, Gephi / NetworkX or Neo4j / Memgraph. | | `POST /api/v1/knowledge/train` | Fit the link predictor now (`409` without TensorFlow, `400` with fewer than 24 shared facts). | The knowledge graph is deleted with the workspace. The dashboard's Memory page (`/platform/dashboard/memory`) shows it under *Knowledge graph*: the switch and the context budget, the facts the person may see in plain words with search and delete, *Add a fact*, *Forget that ...*, a preview of the context the graph would place for a message, the export in the four formats and - when the image has TensorFlow - *Train now*. ### Responses API `POST /v1/responses` serves the OpenAI **Responses API** - the surface the OpenAI Agents SDK and newer client code speak - over the same routed chat path and the same key: `input` (a string or a list of message, `function_call` and `function_call_output` items), `instructions`, flat `tools`, `tool_choice`, `max_output_tokens`, `text.format` and the `osr` routing options are translated to a chat completion; the answer is a `response` object (`output` items of type `message` / `function_call`, `usage.input_tokens` / `output_tokens`, the `opensmartroute` block) and `stream: true` emits the typed events (`response.created`, `response.output_text.delta`, `response.function_call_arguments.delta`, `response.completed`). Conversation state is kept per workspace: every response is remembered for `OSR_PLATFORM_RESPONSES_STORE_TTL_S` seconds (default one hour; `store: false` opts a response out, the answer's `store` says whether it was kept) and a follow-up turn may name it as `previous_response_id` - its turns come first, the new `input` after. The state lives on Redis when configured, else in the platform database (`responses` table), so every replica sees it and it survives a restart either way; a workspace can never resolve another workspace's id, and expired rows are dropped by the finance pass. An unknown or expired id is refused with `400` telling the caller to send the full `input`, never silently ignored. Only function tools are proxied: a built-in tool (`web_search`, `file_search`, `code_interpreter`, ...) is refused with `400` rather than dropped, so a completed response never implies a tool ran that did not. Metered like a chat completion. ### Anthropic Messages API `POST /v1/messages` serves the Anthropic **Messages API** - the surface of the Anthropic SDKs, Claude Code and the agent frameworks built on them - over the same routed chat path and the same key: point the client's base URL at the platform and send the API key as `Authorization: Bearer` or `x-api-key`. `system` (string or `text` blocks), `messages` with `text` / `image` (base64 or URL) / `tool_use` / `tool_result` blocks, `tools` with `input_schema`, `tool_choice` (`auto` / `any` / `tool` / `none`, `disable_parallel_tool_use`), `max_tokens`, `stop_sequences`, `metadata.user_id` and the `osr` routing options are translated to a chat completion; the answer is a `message` object (`content` blocks of type `text` / `tool_use`, `stop_reason` `end_turn` / `max_tokens` / `tool_use` / `stop_sequence`, `usage.input_tokens` / `output_tokens`, the `opensmartroute` block) and `stream: true` emits the Anthropic events (`message_start`, `content_block_start`, `content_block_delta` with `text_delta` / `input_json_delta`, `content_block_stop`, `message_delta`, `message_stop`). Errors use the `{"type": "error", "error": {"type", "message"}}` envelope. `model` is `auto` or a target id as everywhere else. Extended thinking is accepted and ignored; PDF `document` blocks and server-side tools are refused with `400`. Metered like a chat completion. `POST /v1/messages/count_tokens` answers `{"input_tokens": n}` from the router's own token estimate - no provider call, nothing metered, the `route` scope suffices. ### Google Gemini API `POST /v1beta/models/{model}:{action}` serves the Google **Gemini API** - the surface of the Google GenAI SDKs and the Gemini CLI - over the routed chat path and the same key, `action` being `generateContent`, `streamGenerateContent` or `countTokens`: point the client's `api_endpoint` / `base_url` at the platform and send the key as `x-goog-api-key` (or `Authorization: Bearer`). `contents` with `text`, `inlineData` / `fileData` (images), `functionCall` and `functionResponse` parts, `systemInstruction`, `tools` with `functionDeclarations` (Gemini's upper-case schema types become JSON Schema), `toolConfig` (`AUTO` / `ANY` / `NONE`, `allowedFunctionNames`), `generationConfig` (`temperature`, `maxOutputTokens`, `stopSequences`, `topP`, `responseMimeType: application/json`) and an `osr` block of routing options translate to a chat completion; the answer is a `GenerateContentResponse` (`candidates[0].content.parts` of `text` / `functionCall`, `finishReason` `STOP` / `MAX_TOKENS` / `SAFETY`, `usageMetadata`, `modelVersion`, the `opensmartroute` block). `:streamGenerateContent?alt=sse` emits the same objects as `data:` frames, the last one carrying `finishReason` and `usageMetadata`; `:countTokens` answers `{"totalTokens": n}` from the router's estimate (nothing metered, the `route` scope suffices); `GET /v1beta/models` lists what the key may name. Errors use Google's `{"error": {"code", "message", "status"}}` envelope. `model` is `auto` or a target id (the SDKs' `models/` prefix is accepted); the same routes answer under `/v1`. `cachedContent` and Google's built-in tools (search grounding, code execution) are refused with `400`. Metered like a chat completion. ### Rerank `POST /v1/rerank` (`query`, `documents` as strings or `{"text": ...}` objects - up to 1 000 -, optional `top_n`, `return_documents`) answers in the Cohere / Jina shape: `results[]` ordered by `relevance_score` (0-1) with each document's original `index` and, when asked, its text; `model` is `auto` (the embeddings-capable target the router picks) or a target id. The query and the documents are embedded in one call and ranked by cosine similarity, so any embeddings provider gives you a reranker; one metered `rerank` row per call. ### JSON repair A request whose `response_format` asks for JSON (`json_object` or `json_schema`) gets its answer repaired when the model left it almost right: a Markdown fence or a sentence around the value, a trailing comma, a string or a bracket left open when `max_tokens` ran out. The smallest edit that makes the text parse is applied - values are never invented, a key cut before its value is dropped - and `opensmartroute.json_repaired: true` marks the answer. On a stream the missing suffix goes out as one more content delta before the finish chunk, so the concatenated deltas parse; a repair that would have to change bytes the client already holds is not attempted. ### Moderation `POST /v1/moderations` (`input`: a string or a list of strings / `text` parts) answers in the OpenAI moderation shape - `results[].flagged`, `categories`, `category_scores` - from the router's own guards rather than a hosted safety classifier: `prompt_injection` (the instruction-injection lexicon), `router_steering` (sentences that talk to the router about the request), `confounder_gadget` (token soup appended to inflate complexity), `pii` (reported under `categories.pii`; `flag_pii: true` also raises `flagged`), `oversize` and `control_characters`. The `opensmartroute` block of each result lists the reasons and the PII types found. No provider call, nothing metered; the `route` scope suffices. Use it to screen prompts before spending on a model. ### Voice and images The same key and the same router serve the OpenAI audio and image APIs, so `client.audio.transcriptions.create`, `client.audio.speech.create` and `client.images.generate` work unchanged with `model: "auto"`: | Endpoint | Routes among | Answer | |---|---|---| | `POST /v1/audio/transcriptions` (multipart `file` + `model`, optional `language`, `prompt`, `temperature`, `response_format` `json` / `verbose_json` / `text` / `srt` / `vtt`) | targets tagged `transcription` (`capabilities.tags`, or `metadata.transcription: true`) - the catalogue's `transcribe-small` | The transcription object (`text`; `language`, `duration`, `segments` with `verbose_json`; plain text for the text formats) plus `opensmartroute`. Files up to 25 MB; the audio never enters the routed text, only the file name and your `prompt` do. | | `POST /v1/audio/speech` (`input` up to 4 096 characters, optional `voice`, `response_format` `mp3` / `opus` / `aac` / `flac` / `wav` / `pcm`, `speed`, `instructions`) | targets tagged `speech` (or `metadata.speech: true`) - `speak-small` | The audio bytes with the media type of the format; the decision is in `X-OSR-Target`, `X-OSR-Provider` and `X-OSR-Fallback-From`. | | `POST /v1/images/generations` (`prompt`, optional `n`, `size`, `quality`, `style`, `response_format` `url` / `b64_json`) | executable image tools (`actions: [image]`) - `tool-image-gen` | The OpenAI images object (`created`, `data[].url` or `b64_json`, `revised_prompt`) plus `opensmartroute`. | A target id instead of `auto` pins one target (`400` when it cannot do that job, `404` when unknown); without any capable executable target the endpoint answers `501` and says which tag to add. The voice and image targets execute once they are mapped to a provider that serves the corresponding OpenAI path - OpenAI, Azure OpenAI, the self-hosted Speaches server (`voiceEnabled` / the `voice` compose profile) for audio - and are metered under the endpoints `transcriptions`, `speech` and `images`; a transcription target may declare `metadata.usd_per_minute` so the cost follows the audio duration the provider reports. Uploads must be an audio container (`flac`, `m4a`, `mp3`, `mp4`, `mpeg`, `mpga`, `oga`, `ogg`, `opus`, `wav`, `webm`; 25 MB), the file name is reduced to its base name and the declared content type to an audio or video type; the audio bytes never enter the routed text, the request memory, the trace or the logs. Calls to the media providers never follow redirects, cap the answer at 64 MB, forward a fixed whitelist of options, and only relay an `audio/*` body as speech. `osr.objective`, `osr.constraints`, `osr.tenant` and `osr.fallbacks` apply to speech and images as on chat completions; a transcription (a form upload) carries no routing options, the workspace policy and the `X-OSR-Tenant` header still apply. Embeddings, transcription and speech targets are never chosen for a chat completion routed as `auto`; an image tool may be, in which case the chat answer is the picture as Markdown. `POST /v1/route` and `POST /v1/feedback` are aliases of `POST /api/v1/route` and `POST /api/v1/feedback` (same body, same plan gates) for gateways that forward only the `/v1` prefix. ### TypeScript client `@opensmartroute/sdk` (published on the npm registry with every release - `npm install @opensmartroute/sdk`) is a zero-dependency TypeScript client for the endpoints above - Node 20+, Deno, Bun, edge runtimes and browsers, `fetch` based. The same client talks to the hosted platform and to a self-hosted `osr serve` (`apiPrefix: ""`, because the server mounts the routes at the root). Its version follows the platform release and its types are generated from that release's OpenAPI document; the full guide is the [TypeScript SDK](https://osr-web.kindrock-caaab1a2.centralindia.azurecontainerapps.io/docs/sdk-typescript.md) page. ```ts import { OpenSmartRoute, RateLimitError } from "@opensmartroute/sdk"; const osr = new OpenSmartRoute({ baseUrl: "https://opensmartroute.ai", apiKey: process.env.OSR_API_KEY! }); const decision = await osr.route({ text: "Refund order #123", objective: { cost: 0.6 }, top_k: 3 }); await osr.feedback({ request_id: decision.request_id, target_id: decision.target.id, success: true, quality: 0.9 }); const reply = await osr.chat.completions.create({ model: "auto", messages: [{ role: "user", content: "Hi" }] }); console.log(reply.model, reply.opensmartroute.confidence); // typed `OsrMetadata` const stream = await osr.chat.completions.create({ messages: [{ role: "user", content: "Hi" }], stream: true }); for await (const chunk of stream) process.stdout.write(chunk.choices[0]?.delta.content ?? ""); const local = new OpenSmartRoute({ baseUrl: "http://localhost:8000", apiKey: "my-token", apiPrefix: "" }); ``` | Method | Endpoint | |---|---| | `route(req)`, `feedback(outcome)`, `estimate(req)` | `POST /api/v1/route`, `/feedback`, `/estimate` | | `targets()`, `me()`, `quota()`, `info()`, `health()`, `whoami()` | `GET /api/v1/targets`, `/me`, `/quota`, `/info`, `/healthz`, `/whoami` (`osr serve`) | | `chat.completions.create(body)` | `POST /v1/chat/completions`; `stream: true` returns an `AsyncIterable` of `chat.completion.chunk` objects parsed from the SSE stream (`finalCompletion()` assembles the answer) | | `embeddings.create(body)`, `models.list()` | `POST /v1/embeddings`, `GET /v1/models` | | `responses.create(body)` | `POST /v1/responses`; `stream: true` returns the typed Responses events (`finalResponse()` assembles the answer, `responseText()` reads it) | | `messages.create(body)`, `messages.countTokens(body)` | `POST /v1/messages` (Anthropic Messages API); `stream: true` returns the Anthropic events (`finalMessage()` assembles the answer, `messageText()` reads it); `POST /v1/messages/count_tokens` | | `audio.speech.create(body)`, `audio.transcriptions.create({ file, ... })`, `images.generate(body)` | `POST /v1/audio/speech` (bytes + headers), `/v1/audio/transcriptions` (multipart), `/v1/images/generations` | | `rest.get/post/put/patch/del(path, { path, query, body })` | every route of this API, typed from the OpenAPI document (`rest.get("/api/v1/keys")`, `rest.del("/api/v1/keys/{key_id}", { path: { key_id } })`) | | `keys`, `tenants`, `governance`, `providers`, `notifications`, `support`, `handoffs`, `evaluations`, `billing`, `marketplace`, `insights`, `memory`, `knowledge` | convenience groups over `rest` for the workspace surface (`keys.create / rotate / revoke`, `tenants.upsert`, `governance.setPolicy`, `providers.check / map`, `notifications.inbox`, `support.open`, `evaluations.run / replay / sample / get`, `billing.invoices`, `marketplace.search`, `insights.usage / savings / audit / sla`, `memory.get / add / forget / conversations`, `knowledge.facts / add / forget / context / suggest / export / train` ...) | Options: `tenant` (`X-OSR-Tenant`), `app` (`X-OSR-App`), `headers`, `timeoutMs`, a custom `fetch`; per call `signal`, `requestId` (`X-Request-Id`) and `tenant`. Non-2xx answers raise `OpenSmartRouteError` with `status`, `detail`, `requestId`, `retryAfter` and a stable `code`; `AuthenticationError` (401), `PermissionDeniedError` (403, `upgradeRequired`), `NotFoundError` (404) and `RateLimitError` (429, `budget` from the `X-Budget-*` headers) are subclasses. Request types (`RouteIn`, `FeedbackIn`, `ChatCompletionIn`, `EmbeddingsIn`) are generated from this API's OpenAPI document; `RouteDecision`, `ChatCompletion`, `ChatCompletionChunk` and `EmbeddingsResponse` type the answers. The package README documents the development and live-test workflow. ## 5. Feedback `POST /api/v1/feedback` reports how a routed answer turned out. The learners (Bradley-Terry, IRT, LinUCB) update on the workspace's own traffic. Included in every plan of the default catalogue (feature `feedback`). ```bash curl -s "$OSR_URL/api/v1/feedback" -H "Authorization: Bearer $OSR_API_KEY" -H "Content-Type: application/json" \ -d '{"request_id": "7f3c...", "target_id": "llm-frontier", "success": true, "quality": 0.9, "latency_ms": 2100}' ``` Fields: `request_id`, `target_id`, `success` (required); `quality` (0..1), `cost_usd`, `latency_ms`, `domains`, `complexity`, `preferred_over` (the id of a target this one beat in a comparison), `role` (the plan slot the outcome grades - `persona`, `skill`, `tool` or `llm` - when it is about a companion of a composed answer rather than the target that answered). The `request_id` must be one the workspace itself routed (`/route`, `/v1/chat/completions`, `/estimate`, MCP): an unknown id, or another workspace's, is a 404, as is an unknown `target_id` - a catalogue target, one of the workspace's installed listings, or a published skill / persona a plan discovered for the request (section 11) are known. Every report is kept against its request: `GET /api/v1/activity` summarises it per row (`outcome`), `GET /api/v1/trace/{request_id}` lists the reports (`outcomes`) next to the spans, and in the enterprise edition the audit chain records an `outcome` entry after the `decision` entry (section 10). The dashboard has the same form without curl: expand a row on `/platform/dashboard/activity` (or open its trace) and report good / poor, a quality score and, optionally, the target that would have done better - the latter is recorded as a pairwise preference for the Bradley-Terry learner. **The platform reports what it can see itself.** Almost no client calls `/feedback`, so an execution's own assessment is kept the same way and needs nobody: when the answer was judged (a handler that rates its own output, or the implicit reading of the answer - a refusal, a truncation, an empty reply), and when a **fallback** settled a comparison (the target that answered a request another one had just failed beat it on that request), the outcome is written against the request and published on the `feedback` topic the SLM service trains from - for the answering target and, with their `role`, for every persona, skill and tool the plan composed the answer with, so the companions share the answer's judgement. An answer nothing was found wrong with is not an assessment and is not recorded. ### The human queue Routing to a person is a real execution, not an error. When a decision lands on a `human` target (escalation intent, high risk, low automated confidence - the defer strategy scores this), `execute: true` and the chat proxy **queue a ticket** instead of failing: the answer names the ticket, costs zero tokens, and the ticket keeps the request text (with PII restored - human targets declare `pii_allowed`) and the router's reason for deferring in a few plain sentences: what the request reads as (domain, difficulty, personal data, jailbreak risk), which targets policy ruled out and why, how the human target ranked against the runner-up, which strategies carried it, and which targets failed before the router re-routed there. The full decision trace stays one link away. | Endpoint | What it does | |---|---| | `GET /api/v1/handoffs` | The workspace's tickets, newest first, with per-status counts (`?status=pending`). | | `GET /api/v1/handoffs/{handoff_id}` | One ticket: text, reason, status, answer. | | `POST /api/v1/handoffs/{handoff_id}/resolve` | A person answers (`{answer, quality?}`); the resolution feeds an outcome to the learners, so deferral itself is learned. | | `POST /api/v1/handoffs/{handoff_id}/cancel` | Withdraw a pending ticket; recorded as a failed outcome. | The dashboard's Human queue (`/platform/dashboard/handoffs`) shows how many tickets wait and for how long, lists each ticket with the request and the reasons side by side and a *Full trace* link into the activity log, and lets a teammate answer it - the answer goes back as the response and is recorded as the outcome with a quality in words (*Solved* ... *Could not help much*) - or cancel it, without curl. ### Evaluation Outcomes say how the router did on the traffic it got; an **evaluation** says how it would do on the prompts you choose, before you send them. `POST /api/v1/evaluations` takes a labelled dataset - the JSON-lines file `osr eval` reads, one object per row - and routes every row on the workspace's own pool: its policy, tenants (`tenant` in the body), allow / deny lists, API-key limits, installed listings and providers all apply, exactly as on `/route`. **Nothing is executed and nothing is metered**: a run calls no model, costs nothing, counts against no quota, and leaves no trace or event in the workspace's history (the rows are not traffic). The dataset is routed and discarded; only the report is kept - the newest 50 runs per workspace, readable at `/platform/dashboard/evaluation`. ```bash curl -s "$OSR_URL/api/v1/evaluations" -H "Authorization: Bearer $OSR_API_KEY" -H "Content-Type: application/json" \ -d '{"name": "support prompts, week 32", "rows": [ {"text": "What is 12 * 37?", "expected": "tool-calculator"}, {"text": "Summarise this clause in plain words: ...", "acceptable": ["llm-mid", "llm-frontier"]}, {"text": "List customers with no orders this year.", "scores": {"skill-sql": 1.0, "llm-mid": 0.7}, "task_type": "sql"}, {"text": "Translate good morning into French."}]}' ``` A row is `text` plus how it is judged: `expected` (the target id that should win), `acceptable` (several ids that all count), or per-target `scores` in `0..1` (RouterBench style; the best score is the label and the chosen target's score is its *realised quality*). `context`, `history` and `task_type` travel with the row. Rows without a label are fine - they still give the mix, the price and the confidence. `objective` (`{"cost": 0.6}`) and `constraints` are those of `/route`. The report carries: | Field | Meaning | |---|---| | `router.accuracy`, `accuracy_ci` | Share of labelled rows whose chosen target is a hit, with the Wilson 95 % interval for that many rows; `hits` and `labelled` are the counts behind it. | | `router.mean_quality` | Mean score of the chosen target over the rows that score targets. | | `router.mean_cost_per_1k`, `mean_latency_ms`, `mean_confidence`, `abstain_rate` | Declared list price and latency of the chosen targets, averaged over the routed rows; the router's confidence in its choice; how often it abstained. | | `router.ece`, `brier`, `reliability` | Calibration on labelled rows - expected calibration error (0 = the router is right exactly as often as it says), Brier score and the ten confidence bins - reported from 20 labelled rows. | | `savings` | The mix's mean price against always sending every row to the dearest target the policy admitted (`pool`), as a share. | | `baselines` | The same rows and the same admissible pool routed by the policies every router must beat - random, always the cheapest, always the strongest `quality_prior`, always the dearest - each with accuracy, interval, quality, price and latency (`baselines: false` skips them). | | `route_mix`, `by_domain`, `confusion`, `misroutes` | Where the rows went (with per-target accuracy and price), accuracy per first domain, the `expected -> got` pairs and the labelled rows the router got wrong with their confidence. | | `frontier` | With `frontier: true` (up to 500 rows): accuracy and mean price at cost weights 0, 0.3, 0.6, 1 and 2 - the accuracy-against-spend curve of your own prompts. | | `decisions` | Every row's decision: the prompt's first words, expected, chosen target and kind, hit / miss / unlabelled, confidence, quality, price, latency, domains and complexity (the first 1,000; the aggregates cover all). | A run routes at most `OSR_PLATFORM_EVAL_MAX_ROWS` rows (default 2000, `413` beyond it) and for at most two minutes (`truncated: true` with `evaluated` when it stopped early). `GET /api/v1/evaluations/sample` returns labelled rows made from the catalogue itself (ontology seed prompts scored by capability fit), so a workspace can see a report before it has a dataset; the page's *Template* button writes the row format. From the command line, `osr eval rows.jsonl --remote` (after `osr login`, or with `--url` / `--token`) sends the same file to the signed-in workspace instead of a local catalogue, prints the headline, the misroutes and the baselines, keeps the run on the Evaluation page and honours `--min-accuracy` for a CI gate; `--baselines`, `--frontier`, `--limit`, `--preset` and `--json` mean what they mean locally. That gate is a few lines in any CI. For GitHub Actions the repository ships a composite action (`.github/actions/osr-eval/action.yml` - copy it into your own repository, or paste the step below): it installs the package, runs `osr eval` locally against a catalogue or remotely against the signed-in workspace, fails the job below `min-accuracy` and writes the accuracy to the job summary and to an `accuracy` output. ```yaml # .github/workflows/routing.yml - fail the build when a catalogue or rule change lowers routing accuracy on: [pull_request] jobs: routing: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - uses: actions/setup-python@v5 with: { python-version: "3.12" } - name: Routing accuracy gate run: | pip install "opensmartroute[yaml]" osr -t targets.yaml -r rules.yaml eval evals/routing.jsonl --baselines --min-accuracy 0.85 - name: The same rows on the hosted workspace # optional: what production would decide env: { OSR_API_KEY: ${{ secrets.OSR_API_KEY }} } run: osr eval evals/routing.jsonl --remote --min-accuracy 0.85 ``` Azure Pipelines, GitLab, Jenkins and CircleCI run the same two commands; exit status `2` is the failed gate. `POST /api/v1/evaluations/replay` is the other direction: it re-routes the prompts the workspace kept under its `log_payloads` policy (section 9; `409` naming the policy when nothing is kept) through today's router - with everything the learners took from your outcomes, every catalogue and rule change since - and compares each decision with the target that answered then: `changed` / `kept` and the `moves` between targets, the kept token counts priced at the new target's list price against the provider cost metered then (`cost`; decision-only requests carry no tokens and are left out), and `agreement` with the outcomes people reported - `good_kept` (accepted answers the router still sends the same way), `good_changed`, `bad_changed` (rejected answers it now sends elsewhere) and `bad_kept`. `days` (1-90) and `limit` bound the window; a prompt masked by the `redacted` mode is replayed as kept, placeholders included. | Endpoint | What it does | |---|---| | `GET /api/v1/evaluations` | The workspace's kept runs, newest first, each with its headline `summary`; `max_rows`, `keep_runs` and `frontier_max_rows` say what the deployment allows. | | `POST /api/v1/evaluations` | Route a labelled dataset (`rows`, `name`, `objective`, `constraints`, `tenant`, `baselines`, `frontier`) and keep the report; `route` scope, `201`. | | `POST /api/v1/evaluations/replay` | Replay the kept prompts of the last `days` through today's router (`limit` newest first) and keep the comparison; `route` scope, `201`, `409` when nothing is kept. | | `GET /api/v1/evaluations/sample?n=40` | A labelled sample dataset built from the catalogue, with the row format explained. | | `GET /api/v1/evaluations/{evaluation_id}` | One run with its full `report`. | | `DELETE /api/v1/evaluations/{evaluation_id}` | Remove a kept run. | **Datasets from production.** The prompts your users actually send are the dataset that matters, and the outcomes they reported are its labels. `POST /api/v1/evaluations/datasets` saves logged requests as a kept dataset - the *Add to dataset* action of the Activity page: name `request_ids`, or take a window over the prompts kept under `log_payloads` (`days`, `limit`, `outcome` `good` / `bad` / `rated` / `any`, `target_id`), or send explicit `rows`, or mix the three. A logged request becomes `{text, expected, scores, context, task_type}`: the kept prompt, the target that answered as `expected` when its outcome was good (with the reported `quality` as its score), the request id under `context.source` so the same request is never added twice. `GET /api/v1/evaluations/datasets` lists them (rows and labelled counts), `GET .../datasets/{id}` returns the rows (`?format=jsonl` downloads the file `osr eval` reads), `POST .../datasets/{id}/rows` adds to one, `POST .../datasets/{id}/run` evaluates it on today's pool and keeps the report (the body carries the run options - `objective`, `constraints`, `tenant`, `baselines`, `frontier`; the report names the `dataset_id`), `DELETE` removes it. A workspace keeps up to 50 datasets of up to 5 000 rows; all of it takes the `route` scope. **Scores from other frameworks.** A dataset downloads in the shape Ragas, DeepEval and Langfuse read (`?format=ragas|deepeval|langfuse`; every row carries its `row` index and, for logged requests, the `request_id` in the metadata), and the scores those tools compute come back with `POST /api/v1/evaluations/datasets/{id}/results`: `format` names the shape - `ragas` (the result table, `to_pandas().to_dict("records")`: numeric columns are the metrics), `deepeval` (test results with `metrics_data`: name, score, success), `langfuse` (run items with their `scores`) or `scores` (plain `{row | request_id | input, scores: {name: value}, passed}` rows from TruLens, Arize or a script) - `tool` the framework for the record, `threshold` passes a row whose every metric reaches it when the tool gave no verdict. Rows join by index, request id or prompt text; the run is kept as an `external` evaluation (per-metric mean / min / max, the pass rate, the per-row scores; 409 when nothing joins), and a row that came from a logged request receives the mean score as an outcome with source `eval:` (`feedback: false` leaves that out), so the activity page, the trace and the learners see what the other tool thought. **The answer judge.** The runs above grade *which target was chosen*; the judge grades *the answer the customer received*. With a `judge` block in the workspace policy (section 9: `sample`, `target`, `threshold`, `endpoints`, `targets`, `max_per_day`) a reproducible share of the executed answers is graded by the judge model after the answer has left, the grade becomes the request's outcome (`feedback` rows with `source: judge:`, read by the learners, the rankings and the replay comparison like a hand-reported one), and every verdict keeps its reason. `GET /api/v1/evaluations/judge` (`days`, `limit`) returns the block in force, what was judged per answering target over the window (judged, graded, good, mean score, cost) and the latest verdicts; `POST /api/v1/evaluations/judge/run` `{request_id}` grades one kept request now (the Activity page's *Judge this answer*; `execute` scope, `409` naming `log_payloads` when the prompt and answer were not kept). Judging is metered as its own endpoint (`judge`) on the workspace's usage, never charged to the request it grades. Every run is recorded in the workspace's audit trail as `evaluation.run`, travels with the data export (`GET /api/v1/me/export` -> `workspace.evaluations`) and is deleted with the workspace. A tenant-pinned API key cannot run evaluations (`403`) - they are the workspace's view of its whole pool. The TypeScript client has them as `client.evaluations.run / replay / sample / list / get / delete`. **Quality gates.** An evaluation says how good the router is on your prompts *today*; a gate says whether it is *still* that good after the catalogue moved, a provider was swapped, a model was retired or somebody edited the workspace policy - before your own customers notice. Keep the dataset you signed off on as a gate (*Keep as a quality gate* on the Evaluation page, or `POST /api/v1/evaluations/gates` with the same `rows`, `objective`, `constraints` and `tenant` as a run) with the thresholds it must hold: `min_accuracy` (an absolute floor, 0..1) and / or `max_drop` (how many accuracy points a run may fall below the run that set the baseline - the first pass; default 0.05). The gate is routed at once - that first pass is the baseline - and again every `OSR_PLATFORM_EVAL_GATES_INTERVAL_S` (six hours; `0` = only on request and after a policy change) and whenever the workspace policy is replaced or removed. Each pass is a kept evaluation named *Gate · …*, so the full report is one click away. A pass under the floor is `fail`, one too far under the baseline `regressed`; the first failing pass files a warning in the workspace inbox (`workspace.evaluation_gate_failed`, linking the report) and the audit trail (`evaluation.gate_failed`), a scheduled pass that keeps failing stays quiet, and the pass that clears both thresholds again files `evaluation_gate_recovered`. A run on request or after a policy change always announces a failure. Nothing is executed or metered. A workspace keeps up to ten gates of up to 500 rows each, at least five of them labelled; creating and deleting a gate takes the admin role, running one the `route` scope. | Endpoint | What it does | |---|---| | `GET /api/v1/evaluations/gates` | The gates with their last verdict (`last_state`, `last_accuracy`, `last_detail`, `last_reason`), the baseline and the counters; `interval_s`, `max_gates`, `max_rows`, `min_labelled` say what the deployment allows. | | `POST /api/v1/evaluations/gates` | Keep a dataset as a gate (`name`, `rows`, `objective`, `constraints`, `tenant`, `min_accuracy`, `max_drop`) and route it now; `201` with the judged gate. | | `POST /api/v1/evaluations/gates/{gate_id}/run` | Route the gate now and judge it. | | `DELETE /api/v1/evaluations/gates/{gate_id}` | Forget the gate (its past runs stay in the list of evaluations). | **Shadow routing.** A dataset and a gate replay prompts; a shadow candidate rides the *live* traffic. Start one (*Shadow routing* on the Evaluation page, or `POST /api/v1/shadow`) with the policy you are considering - objective weights (`quality`, `cost`, `latency`, `quality_floor`) and / or the request-shaping constraint keys of the workspace policy (`deny_targets`, `allow_targets`, `max_cost_per_1k`, `max_latency_ms`, `preferred_max_latency_ms`, `region`, `data_boundary`, `contains_pii`) - and from then on every request the workspace routes is routed a second time under the candidate, inside the same pool and with the same tenant, key limits and installed listings. The second decision is never served, executed, metered or charged; what is kept is two target ids and the cost, quality estimate and latency each would have had (no text). `GET /api/v1/shadow` sums it up: agreement between the two, the spend delta, the quality and latency deltas, where the candidate moves traffic from and to, the requests it would have refused, and a plain-words `verdict` once a few dozen requests are in. When the comparison convinces you, `POST /api/v1/shadow/promote` writes the candidate into the workspace policy - its constraints are merged, its weights become the policy's `objective`, the default for every request that sets none (an explicit objective on a request still wins) - which is a policy change, so every quality gate the workspace keeps runs again on it: a promotion that regresses a kept dataset shows up in the inbox like any other bad edit. One candidate runs at a time; it ends when promoted, stopped (`DELETE /api/v1/shadow`) or after `max_decisions` requests (20,000 by default), and earlier candidates stay readable (`GET /api/v1/shadow/{candidate_id}`). Starting, stopping and promoting take the admin role; a tenant-pinned key is refused. | Endpoint | What it does | |---|---| | `GET /api/v1/shadow` | The running candidate, its `summary` (agreement, cost / quality / latency deltas, `moves`, `refused`, `verdict`, `progress`), the most recent comparisons and the earlier candidates. | | `POST /api/v1/shadow` | Start a candidate (`name`, `objective`, `constraints`, `sample` = share of requests shadowed, `max_decisions`); a running one is stopped. | | `GET /api/v1/shadow/{candidate_id}` | One candidate, running or ended, with its summary and comparisons. | | `POST /api/v1/shadow/promote` | Make the candidate (`candidate_id`, default the running or latest one) the workspace policy. | | `DELETE /api/v1/shadow` | Stop the running candidate. | ### Observed usage: the browser extension's ledger The browser extension (`platform/extension`, product page `/extension`) sees calls a team makes *outside* the router - on ChatGPT, Claude, Gemini, Copilot, Perplexity and the OpenAI, Anthropic, Google and OpenRouter consoles. With "Share observed usage" turned on it reports the **facts** of each call and never the text: ```bash curl -s "$OSR_URL/api/v1/usage/observed" -H "Authorization: Bearer $OSR_API_KEY" -H "X-OSR-App: browser-extension" \ -H "Content-Type: application/json" \ -d '{"items": [{"ts": 1760000000, "source": "chat", "site": "chatgpt.com", "vendor": "openai", "model": "gpt-5", "prompt_tokens": 1284, "completion_tokens": 310, "task_type": "summarise", "complexity": 0.3}]}' ``` | Endpoint | What it does | |---|---| | `POST /api/v1/usage/observed` | Records up to 500 observations (`ts`, `source` = `chat` / `console` / `app`, `model`, optional `vendor`, `site`, token counts, `task_type`, `complexity`). Any other field - a prompt, messages - is refused with `422`. Each row is priced at the observed model's catalogue list price (`estimated_cost_usd`, `null` when the model is unknown) and re-quoted through the router for the same token shape (`recommended_target`, `recommended_cost_usd`). Needs the `feedback` scope. | | `GET /api/v1/usage/observed` | The stored observations, newest first (`?days=`, `?limit=`). | | `GET /api/v1/usage/opportunity` | The report: calls, tokens, observed spend, the routed quote for the priced calls, `gap_usd` / `gap_pct`, breakdowns by model, source, task and day, and the ten models with the widest gap with the target the router would have chosen. | Observations are kept for 90 days and deleted with the workspace. The report compares the same request shapes on both sides - it is a price difference, not a quality claim - and the dashboard shows it on `/platform/dashboard/spend`. ### Insights: the figures read back as findings The dashboard shows numbers; the insights endpoint says what they mean. It reads the workspace's own metered rows - the window against the one before it, per target and per day - and returns findings, each with the evidence behind the sentence and the page that acts on it: cost against the previous window with the targets that moved it, a day whose cost sits far above the average, failures and their rate, speed against the previous window, one target carrying almost all the traffic, decisions without execution, savings against the baseline and spend the extension saw outside the router. No model is involved: every sentence is computed, so asking twice gives the same answer. The dashboard overview carries it as *Ask about your usage*. | Endpoint | Purpose | |---|---| | `GET /api/v1/insights` | `?days=` (1-90, default 7): a `headline`, the `totals` of the window and the one before, the `findings` (`kind`, `severity` = `warning` / `good` / `info`, `title`, `detail`, `evidence`, `action.href`), the `suggestions` people ask and the `method`. | | `POST /api/v1/insights/ask` | `{question, days?}` - a question in plain words (*why did my cost go up*, *is anything slower*, *where could I save*); the subject is read from the question (`intent`: cost, savings, speed, failures, models, outside, volume) and the matching findings come back with one `answer` sentence. A tenant-pinned key gets its tenant's figures only. | ### Bill analysis: the invoice or usage export you already have A team that has not routed yet still has a bill. Every provider, gateway and tracing tool exports usage as a table - a date, a model, request and token counts, often the amount charged - and every provider issues a PDF invoice. Post either and the platform recognises the columns - or, for an invoice, the vendor, the billing period and every line that names a model with an amount - aggregates it per day, provider and model and prices every line twice: what the bill says (or, when the export carries no amounts, the model's catalogue list price) and what the router's recommended target would have charged for requests of the same token shape. Nothing in an export is text - dates, names, quantities and amounts are read, every other column or word (addresses, tax ids, people) is ignored and never stored. The layouts recognised (`source_format`) - the tables from their header row, an invoice from its text: | Format | Where it comes from | What the parser reads | |---|---|---| | `openai-usage` | platform.openai.com → Usage → Export | hourly buckets: `n_requests`, `n_context_tokens_total`, `n_generated_tokens_total` per model | | `openai-costs` | platform.openai.com → Billing → Costs → Export | `line_item` (`gpt-4o, input`) and `amount`; the tokens are backed out of the amount at list price | | `anthropic-usage` | console.anthropic.com → Usage → Export CSV | `usage_input_tokens_no_cache`, cache reads and writes, `usage_output_tokens` per `model_version` | | `azure-openai` | Azure Cost Management → Cost analysis → Download | `MeterName` (`gpt-4o-0806 Inp glbl Tokens`), `Quantity` in thousands, `Cost` | | `google-billing` | Google Cloud Billing → Reports → Download, or the BigQuery billing export | `SKU description` (`Gemini 2.5 Pro Input Tokens`), `Usage amount`, `Cost`, `Service description` | | `aws-cur` | AWS Cost Explorer CSV or a Cost and Usage Report (Bedrock) | `lineItem/UsageType` (`USE1-Claude3Haiku-input-tokens`), `lineItem/UsageAmount`, `lineItem/UnblendedCost` | | `openrouter` | openrouter.ai → Activity → Export | one row per generation: `created_at`, `model`, `tokens_prompt`, `tokens_completion`, `cost` | | `request-log` | LiteLLM spend logs, Langfuse, LangSmith and Helicone exports, this platform's own Activity CSV | one row per request (a timestamp with a clock time, no request count): each row counts as one request | | `generic` | anything else: CSV, TSV, JSON, JSON lines or an `.xlsx` workbook (first sheet, Excel dates understood) | a date, a model and token or amount columns under any of some sixty header spellings; a file without a model column (this platform's Usage CSV, a finance summary) becomes one line per day | | `invoice` | the PDF invoice from Billing → Invoices (OpenAI, Anthropic, OpenRouter and Mistral bill through Stripe; Azure, Google Cloud and AWS issue their own), or its text pasted | the issuer, the billing period (`Aug 1 – Aug 31, 2026`, `Billing period: ...`, else the invoice date), and every line that names a model with an amount - `GPT-4o input tokens 12,345,678 $2.50 / 1M $30.86`, `gpt-4o-0806 Inp glbl Tokens 12,340 1K Tokens 0.0025 30.85`, `Vertex AI Gemini 2.5 Pro Input Tokens ... $10.00`; quantities become tokens (Azure's thousands understood), a line without one has its tokens backed out of the amount at list price; totals, tax, credits and payments are not usage, and the notes say when the model lines do not add up to the invoice total | Cloud bills are roll-ups, so their hourly rows are never counted as requests; a line without a request count is quoted as requests of about 2 000 input tokens. An invoice dates every line at the start of its billing period and the projection per month uses the whole period. A scanned invoice (a picture with no text layer) and a password-protected PDF are refused with what to do instead, and an old `.xls` should be saved as `.xlsx` or CSV. ```bash curl -s "$OSR_URL/api/v1/usage/imports/upload" -H "Authorization: Bearer $OSR_API_KEY" \ -F "file=@openai-invoice-2026-08.pdf" -F "name=August, OpenAI" curl -s "$OSR_URL/api/v1/usage/imports/upload" -H "Authorization: Bearer $OSR_API_KEY" \ -F "file=@openai-usage-2026-08.csv" -F "name=August, OpenAI (usage export)" curl -s "$OSR_URL/api/v1/usage/imports?dry_run=true" -H "Authorization: Bearer $OSR_API_KEY" \ -H "Content-Type: application/json" \ -d '{"name": "one line", "provider": "anthropic", "text": "date,model,input_tokens,output_tokens,cost\n2026-08-01,claude-sonnet-4,120000,30000,0.81"}' ``` | Endpoint | What it does | |---|---| | `POST /api/v1/usage/imports` | Analyses `{name, text, filename?, provider?}` - `text` is the export as the provider gave it (CSV, TSV, JSON or JSON lines, up to 8 MB) or the text of an invoice (its line items with amounts), `provider` names the provider when the rows carry none (`auto` infers it from the model names) - and keeps the analysis unless `?dry_run=true`. The answer is the analysis: `headline` and `story` (the figures in plain sentences), `source_format` (the table above), the columns recognised, totals, `baseline_cost_usd` against `routed_cost_usd`, `savings_usd` / `savings_pct`, the projected monthly figures, `by_provider`, `by_model`, `daily`, the lines (up to 5 000 inline) and `notes` on what was assumed. A file without a date or a token / amount column is refused with `400`, a larger one with `413`. | | `POST /api/v1/usage/imports/upload` | The same analysis for a multipart upload (`file` plus optional `name` and `provider` fields), for curl, the dashboard and integrations that hold the file rather than its text; the way in for a PDF invoice (read from its text layer with `pypdf`) and for an `.xlsx` workbook. | | `GET /api/v1/usage/imports` | The analyses kept for the workspace, newest first, with their headline figures and the `formats` and `providers` the parser accepts. | | `GET /api/v1/usage/imports/samples` | Sample exports in each provider's own layout with invented figures - OpenAI usage and costs, Anthropic, Azure OpenAI, Google Cloud billing, AWS Bedrock, OpenRouter, Mistral, Cohere, xAI, DeepSeek, Groq, Together, LiteLLM, Langfuse, this platform's activity export and the text of an OpenAI invoice (`id`, `provider`, `name`, `source`, `format`, `text`) - to try the analysis before touching a real bill: post one as `text` with `?dry_run=true`. The dashboard's *Try a sample export* menu does exactly that. Every sample is what the parser is tested against. | | `GET /api/v1/usage/imports/{import_id}` | One analysis with its lines; `?format=csv` downloads the lines, `xlsx` a workbook (summary, by provider, by model, daily, lines), `pdf` the report. | | `DELETE /api/v1/usage/imports/{import_id}` | Deletes an analysis and its lines (recorded in the audit trail). | | `GET /api/v1/usage/report` | The workspace's own routed usage as the same report: a `headline` and `story` in sentences, then requests, tokens in / out / cached, provider cost and the workspace's charge per day, model, provider and API key over `?days=` (up to 366, `?tenant=`, `?key_id=`), and the saving against the savings baseline. `?format=csv` downloads the day-model-provider lines, `xlsx` a workbook with every breakdown, `pdf` the report. | The routed side is the estimator's quote under the catalogue and rules of today, so every figure is labelled an estimate; lines whose model the router cannot quote are carried over at what they cost today and the notes say how much of the bill was quoted. Analyses are the customer's own data: kept until they delete them, and deleted with the workspace. ### Client activity: what the extension and the desktop app did Most of what the browser extension and the desktop app do happens on the person's own machine and is never a routed request - the pre-send card, a redaction, a compaction, a writing fix, the command bar opening, an answer pasted back. Each app keeps a small queue of those moments and posts it to the workspace a few minutes later, so the dashboard's Apps page (`/platform/dashboard/apps`) shows how the apps are used, what they caught and what they saved. Every event is **a kind, a hostname, counts and figures - never text**: ```bash curl -s "$OSR_URL/api/v1/usage/activity" -H "Authorization: Bearer $OSR_API_KEY" -H "X-OSR-App: desktop" \ -H "Content-Type: application/json" \ -d '{"items": [{"ts": 1760000000, "app": "desktop", "kind": "answer.received", "target_id": "gpt-4.1-mini", "request_id": "6f1c...", "latency_ms": 2300, "version": "1.5.0"}, {"ts": 1760000010, "app": "browser-extension", "kind": "pii.redacted", "site": "chatgpt.com", "n": 2}]}' ``` | Endpoint | What it does | |---|---| | `POST /api/v1/usage/activity` | Records up to 500 events (`ts`, `app` = `browser-extension` / `desktop`, `kind` as `area.action`, optional bare-hostname `site`, `target_id`, `request_id`, `n`, `tokens`, `saved_usd`, `latency_ms`, `tier`, `task_type`, `version`). A field of any other name is refused with `422`, a `site` with a path or query with `400`. Needs the `feedback` scope. The plan caps events per UTC day: a batch that would cross the cap answers `429` with `Retry-After` and records nothing. | | `GET /api/v1/usage/activity` | The events, newest first (`?days=`, `?limit=`, `?before=` to page, `?app=`, `?kind=pii.*`). | | `GET /api/v1/usage/activity/summary` | Totals per app, kind and site, the per-day series, and what the plan keeps (`retention_days`) and allows (`limit_per_day`, `today`). | What the plan decides: **Free** keeps events 7 days and records up to 5 000 a day, **Pay as you go** and **Pro** 30 days and 50 000 a day, **Team** 90 days and 500 000 a day, **Enterprise** 365 days with no daily cap; a workspace's `retention_days` override (custom agreements) wins over the tier. The retention sweep purges each workspace at its own cutoff. The kinds the first-party clients send are listed in [EXTENSION.md](https://osr-web.kindrock-caaab1a2.centralindia.azurecontainerapps.io/docs/EXTENSION.md) and [DESKTOP.md](https://osr-web.kindrock-caaab1a2.centralindia.azurecontainerapps.io/docs/DESKTOP.md); both apps have a **Log my activity to the workspace** switch (on by default, since there is no text in the events) and administrators can pin it through the extension's managed storage. Every request the extension routes through the platform also carries `X-OSR-App: browser-extension`. Any client may name itself that way (`osr-cli`, `langchain`, your own service name; free text is reduced to a short slug): the value is stored on each metered row and `GET /api/v1/usage` returns `by_app` - requests, tokens and cost per app, `null` for calls that sent no header - which the *Usage* page shows as **By app**. ## 6. Plans and quotas | Plan | Requests / month | Overage | Requests / min | Keys · seats · tenants | Features | Support | |---|---|---|---|---|---|---| | **free** | 15,000 | refused (`429`) | 30 | 2 · 5 · 0 | route + trace, OpenAI-compatible proxy, feedback, your own model providers (`byok`) | community | | **payg** - $0 / month, prepaid credits | unlimited | n/a - paid by the token, `402` when the credits run out | 300 | 10 · 10 · 0 | + plans; every model on offer by the token (*Models by the token*) | community | | **pro** - $49 / month or $490 / year, 90 days free without a card | 300,000 | $10 per 100k | 300 | 10 · 25 · 0 | + plans (`plan`/`execute`) | e-mail, first answer within 48 h | | **team** - $199 / month or $1,990 / year, 90 days free without a card | 2,000,000 | $8 per 100k | 1,000 | 25 · 100 · 10 | + tenants, `/stats`, hash-chained `/audit` | priority, first answer within 8 h | | **enterprise** - from $999 / month, through sales, 90 days free without a card | 20,000,000 | $5 per 100k | 3,000 | 100 · 1,000 · 50 | + organization SSO, region pinning, custom quotas and retention | dedicated, first answer within 4 h, 99.9 % monthly availability target with service credits | Quotas count metered requests (a decision, a chat completion or embedding through the proxy, an estimate, a feedback event) per **calendar month (UTC)**; the counter resets on the first. `GET /api/v1/info` (`plans`) returns the values for the deployment you are calling (`OSR_PLATFORM_PLAN_OVERRIDES` patches them per deployment), `GET /api/v1/me` returns your plan and `GET /api/v1/quota` where you stand: ```json {"plan": "pro", "period": "month", "used": 241300, "limit": 300000, "remaining": 58700, "percent": 80.4, "state": "warning", "warn_at": 0.8, "resets_at": 1767225600, "today": 9120, "requests_per_minute": 300, "overage": {"allowed": true, "usd_per_100k": 10.0, "requests": 0, "estimated_usd": 0.0}} ``` `state` is `ok`, `warning` (80 % and above), `overage` (beyond the quota on a plan with an overage rate and a live subscription - requests keep flowing and are invoiced) or `exhausted` (beyond the quota on any other plan - metered calls answer `429`). The dashboard shows the same bar on the overview, the governance page and *Plan and billing*; the `quota` alert rule fires at 80 % and escalates at 100 % (workspace channels, or the owners by e-mail when the workspace has no e-mail channel - outbox kind `quota_notice`; the `budget` and `wallet` rules reach the owners the same way, as `budget_notice` and `credits_notice`). Upgrades are made in the dashboard (`/platform/dashboard/billing`). * `429 rate limit exceeded` - per-minute window; wait `Retry-After` seconds. * `429 monthly quota exhausted` - `X-Quota-Limit`, `X-Quota-Used`, `X-Quota-Reset` and `X-Upgrade: true`. * `403` with `X-Upgrade: true` - the feature is not in your plan. * `403 ... requires the enterprise edition` with `X-Edition: enterprise` - the deployment runs the community image, which does not expose tenants, statistics or audit regardless of plan. * `403 this API key lacks the '' scope` with `X-Key-Scope` - the key was created with scopes. Every feature a plan sells is live on the workspace the moment the plan applies - there is no separate activation. *Plan and billing* (`/platform/dashboard/billing`) lists them under **What the plan includes**, each one a link to the page that uses it (providers, activity, integrations, tenants, health, audit trail, workspace SSO); a feature the plan lacks names the cheapest plan that has it with an *Upgrade* button. The sidebar marks pages the plan does not include with a lock, and the page itself explains the gap. **Support tiers.** `Plan.support` is `community` (free: documentation, the assistant, the issue tracker), `email` (pro), `priority` (team) or `dedicated` (enterprise, with an SLA). Tickets opened from a workspace on priority or dedicated support start at `high` priority at least - a `normal` or `low` request is raised, a higher one is kept - and the operator notice and inbox line name the plan and the tier (`Acme (team plan, priority support)`). **Service levels.** Each tier states what it promises and the platform measures against it, so the pricing page, the portal and the API read one source (`plans.py`): `support_response_hours` is the first-response target of the support tier (community none, e-mail 48 h, priority 8 h, dedicated 4 h) and `sla_uptime_pct` the monthly availability target the plan carries (99.9 % on enterprise; 0 elsewhere). The target is a contractual figure: the hosted platform is one stack per region, the service credits that apply when a month misses it are defined in the enterprise agreement, and the number the platform reports is its own measurement - not an external monitor. `GET /api/v1/sla` reports the calendar month so far from the workspace's own records: `availability` - the share of its metered requests the platform answered without a 5xx (a refused request - quota, policy, bad input - or a provider error is not a platform failure) against `target_pct`; `support` - the tickets opened this month, how many were answered, the median and slowest first response in hours, how many were answered within `target_hours` and how many are past it (`breached`, answered late or still waiting). `met` is `true`, `false` or `null` (nothing to measure yet); `committed` says whether the plan promises anything. *Plan and billing* shows the same figures under **Service level** on every plan, labelled *committed* or *no commitment on this plan*. **Talk to sales.** Plans sold through sales (`contact_sales`, the enterprise plan) show *Talk to sales* on the pricing page and in the portal instead of a checkout; the button opens the form on `/contact` (`?plan=` preselects the plan). `POST /api/v1/sales/inquiries` (`email`, `name`, `company`, `plan`, `interest` hosted / self-hosted / both / other, `volume`, `message`; no key needed, five per address and day, a filled `website` field marks a bot) records the inquiry with a reference (`OSR-S-00007`), e-mails the requester a confirmation (outbox kind `sales_received`), the support recipients a notice (`sales_operator`) and the console inbox a line; a signed-in caller's workspace is attached. A second request from the same address within a day returns the open inquiry. Operators work the queue under `/platform/admin/sales` (`GET /api/v1/admin/sales/inquiries?status=&q=`, `GET|PATCH /api/v1/admin/sales/inquiries/{key}` - status `new` / `contacted` / `qualified` / `won` / `lost`, `assigned_to`, internal `note`; audit events `sales.inquiry_opened`, `sales.inquiry_updated`) and close the loop on the workspace page: plan, custom quotas, retention and region. **Custom quotas and retention.** Enterprise agreements override a workspace's numbers without a new plan: an operator sends `limits` with `PATCH /api/v1/admin/accounts/{id}` - any of `requests_per_month`, `requests_per_minute`, `max_keys`, `max_tenants`, `max_members` and `retention_days` (whole numbers; the object replaces the previous overrides, `{}` returns the workspace to the plan as sold; audit event `account.limits_changed`). `GET /api/v1/me` then reports the effective plan with `"custom": true` and the overridden values, `GET /api/v1/quota` enforces the custom quota, the key, seat and tenant limits apply as the plan's own would, and *Plan and billing* shows a **custom quotas** badge. `retention_days` gives the workspace its own request-row retention: the retention sweep purges its rows at that age and leaves them out of the deployment-wide purge (`OSR_PLATFORM_RETENTION_DAYS`, which must be on for the sweep to run); `GET /api/v1/governance` reports the effective `retention_days`. The operator console edits the same fields under **Custom quotas and retention** on the workspace page (`/platform/admin/workspaces/`); `GET /api/v1/admin/retention` lists the workspaces with their own retention under `usage.overrides`. ### Billing: subscriptions, trials, overage, invoices and cancellation Paid plans are flat tiers sold as one Stripe subscription per workspace, monthly or yearly. Self-serve billing is on when the deployment sets all three of `OSR_PLATFORM_STRIPE_SECRET` (the `sk_live_...` key), `OSR_PLATFORM_STRIPE_WEBHOOK_SECRET` (the `whsec_...` of the endpoint below) and `OSR_PLATFORM_STRIPE_PRICES` (JSON plan -> price id); `billing: true` in `GET /api/v1/info` tells the web app. Without them every billing endpoint answers `501` and operators set plans by hand (`PATCH /api/v1/admin/accounts/{account_id}`). The Stripe calls are plain HTTPS from the API, no SDK. `OSR_PLATFORM_STRIPE_PRICES` keys: `"pro"` is the monthly recurring price, `"pro:year"` the annual one (sold when present - the pricing page shows the switch), `"pro:overage"` a **metered** price on a Stripe Billing Meter whose event name is `OSR_PLATFORM_STRIPE_METER_EVENT` (default `osr_overage_requests`, unit = requests), and `"pro:overage:year"` the same metered price with a yearly interval - Stripe only accepts items of one interval per subscription, so a plan with overage is sold per year only when both `"pro:year"` and `"pro:overage:year"` exist (a yearly subscriber's overage is then invoiced with the annual renewal). Each metered line lives on its own Stripe product - *OpenSmartRoute Pro - extra requests* with the unit label `request`, *... - metered usage* - because Checkout and the invoice list every line by product name: a buyer must read *extra requests, $0.0001 per request*, not *OpenSmartRoute Pro* twice. A setup run on a deployment created before this moves an overage price left on the plan product to its own (lookup key transferred; set the printed price ids again). `osr-platform-stripe-setup --webhook ` (ships with the platform API; run it inside the API container) creates all of them from the plan catalogue (products, monthly / yearly / metered prices, the meter, the default Billing Portal configuration the *Manage subscription* button opens - payment method, billing details, invoices, cancel at period end, no plan changes - and the webhook endpoint with the seven events below), idempotently - run it again after changing a price and the lookup keys move to the new prices - and prints the JSON, its base64 form for the deployment configuration and the webhook signing secret; the key comes from `STRIPE_SECRET_KEY` in the environment (start with a test-mode key and `--dry-run`): ```json {"pro": "price_...", "pro:year": "price_...", "pro:overage": "price_...", "pro:overage:year": "price_...", "team": "price_...", "team:year": "price_...", "team:overage": "price_...", "team:overage:year": "price_...", "enterprise": "price_...", "enterprise:overage": "price_..."} ``` To also invoice what a workspace spent by the token on the same invoice as its plan, run the setup with `--usage-meter-event` (bare, or with your own event name): it creates a second Billing Meter whose unit is one cent and a metered price per plan (the `"pro:usage"` / `"pro:usage:year"` keys), and prints `OSR_PLATFORM_STRIPE_USAGE_METER_EVENT`. With that variable set, a checkout and a plan change carry the usage item next to the overage one, and the same sync loop reports the month's `charge_usd` in cents (`usage_charge_reports`, one `billing.usage_reported` audit entry per increment). Without it the charges are still priced, stored and shown on *Usage and charges* - they simply never reach a Stripe invoice. **Tax.** Checkouts ask Stripe Tax to compute VAT / GST / sales tax from the buyer's billing address and to collect a tax id, which is what the pricing page and the terms promise. Stripe only does that once the account's **head office address and tax registrations** are set (Stripe -> Settings -> Tax); until then Stripe refuses a session that asks for tax. The platform does not let that stop a purchase: it creates the session again without the tax fields, remembers the refusal (`Billing.tax_problem`) and raises the deployment alert `stripe_tax_unconfigured` (console *Health* and *Finance*, `GET /api/v1/admin/health` -> `alerts`) reading Stripe's tax settings (`Billing.tax_status`, cached an hour) - critical once a checkout had to drop tax, a warning before. Buyers then pay the list price with no tax line; fix the Stripe account, and the next checkout computes tax again. **Activation.** A Stripe account that has not finished activation (business details, bank account, card payments turned on) refuses every paid Checkout Session - yet it *accepts* a free trial ($0 today), so a buyer could type a card into a trial that would be cancelled at its end without a word. The API reads the account from Stripe (`Billing.account_status`, Stripe's account object, cached an hour) and raises the critical deployment alert `stripe_account_inactive` while charges are disabled or card payments are not active, naming what is missing; while it stands, no Checkout Session is created at all - a plan with or without a trial, credits or a paid listing - and the buyer gets `503 Payments are not open on this deployment yet - the operators have been alerted. Contact sales to buy in the meantime.` with `Retry-After` on both `POST /billing/checkout` and `POST /billing/credits/checkout`, never Stripe's developer sentence. Finish the activation under Stripe -> Settings -> Business and Settings -> Payment methods; checkouts go through and the alert clears within the hour. A checkout adds the plan's overage item next to the recurring one; a plan change swaps both. Every `OSR_PLATFORM_OVERAGE_SYNC_S` seconds (default hourly, `0` = never) the API reports each billed workspace's requests beyond its quota this month as meter events (one increment per sync, de-duplicated by identifier and a compare-and-set row in `overage_reports`, so replicas never double-report; each report is a `billing.overage_reported` audit entry). The first checkout of a workspace starts the plan's `trial_days` (90 on every subscription plan) **without a payment method**: the plan applies at once and the workspace returns to free when the trial ends unless it subscribes (see *Free trials, coupons and the launch offer*); `"coupon": ""` pays now instead. A plan whose `trial_needs_card` is true carries its trial into Stripe Checkout instead (card collected, the subscription cancels if none is on file when the trial ends). A workspace that had a subscription or a trial before gets no second trial. | Endpoint | Who | What | |---|---|---| | `POST /api/v1/billing/checkout` `{plan, interval?}` | owner, admin | `interval` is `month` (default) or `year`. A workspace without a live subscription gets a Stripe Checkout URL (`?upgraded=` on return). A workspace that already has one has the subscription's prices swapped in place with prorations - never a second subscription - the plan applies at once and the answer's `url` points back at `/platform/dashboard/billing?changed=`. `400` when the plan is not sold for that interval. | | `GET /api/v1/billing/subscription` | any member | `plan`, `billing_state`, `period_end`, `subscription`, `has_portal`, `invoices_available`, live `cancel_at_period_end` / `stripe_status` / `trial_end` / `interval` (null when Stripe is unreachable), `can_manage`, `purchasable` (per paid plan: intervals sold here and the trial a first checkout starts) and `overage` (`billed`, `usd_per_100k`, `reported_requests` this month). | | `POST /api/v1/billing/portal` | owner, admin | A Stripe Billing Portal session (`{url}`): payment method, billing address, receipts, resume a scheduled cancellation. `404` before the first checkout. | | `GET /api/v1/billing/invoices` | any member | The last 24 invoices: `number`, `status`, `amount_due` (minor units), `currency`, `created`, `hosted_invoice_url`, `invoice_pdf`. `404` before the first checkout. | | `POST /api/v1/billing/cancel` | owner, admin | Cancel at the end of the paid period; the plan stays until Stripe sends `customer.subscription.deleted`. `404` without an active subscription. | | `POST /api/v1/billing/webhook` | Stripe | Signed events (`Stripe-Signature`, HMAC-SHA256 over `t.payload`, 5-minute tolerance); every event id is processed once. | Point the Stripe webhook endpoint at `POST /api/v1/billing/webhook` with these events enabled: `checkout.session.completed`, `customer.subscription.created`, `customer.subscription.updated`, `customer.subscription.deleted`, `invoice.paid`, `invoice.payment_failed`, `charge.refunded` (the last one credits a refund back to the wallet; `osr-platform-stripe-setup` registers all seven). The workspace's `billing_state` (`GET /api/v1/me` -> `account.billing_state`, also `stripe_subscription` and `billing_period_end`) follows the Stripe subscription status: | Stripe status | `billing_state` | Plan | |---|---|---| | `active`, `trialing` | `active` | the paid plan (`metadata.plan`, or the plan whose price the subscription carries) | | `past_due` | `past_due` | kept - Stripe is retrying the charge | | `incomplete` | `incomplete` | unchanged - the first payment has not completed | | `unpaid`, `paused` | `unpaid` | `free` | | `canceled`, `incomplete_expired`, subscription deleted | `canceled` | `free` | `invoice.payment_failed` moves the workspace to `past_due`, writes a `billing.payment_failed` entry to the workspace's audit trail (`GET /api/v1/workspace/audit`) and sends the owners a dunning message (outbox kind `billing_payment_failed`, "Update payment method" linking to the hosted invoice); `invoice.paid` moves it back to `active` and, when the workspace was past due, tells the same owners the invoice is settled (outbox kind `payment_received`; inbox notice `workspace.payment_received`). The dashboard (`/platform/dashboard/billing`) shows the state badge, a warning banner while past due, the renewal or end date, *Manage subscription* (portal), *Cancel at period end* and the invoice table; members see the state and invoices but only owners and admins get the buttons. ### Usage pricing, credits and provider finance Every metered request carries **two prices**: the provider cost the router optimised (`cost_usd`, what the executed model cost upstream) and the customer charge (`charge_usd`, what the workspace pays), plus the token split (`prompt_tokens`, `completion_tokens`, `cached_tokens`), the `provider_id` and `upstream_model` that executed and the `price_version` the charge was computed under - so a margin report months later reproduces it. Successful `/api/v1/route` executions and `/v1/chat/completions` answers return the charge in the body (`result.charge_usd`, and `charge_usd` inside the `opensmartroute` object of a chat answer) and in the `X-OSR-Charge-USD` header. The charge comes from a **pricing engine** with two books per model. The *provider price book* holds rates per unit (`token_in`, `token_out`, `token_cached` in USD per million; `call`, `image`, `second` per unit): `catalogue` rows are the list prices of the model catalogue (*Seed from catalogue* in the console, and rewritten by every catalogue refresh of the rankings loop so a vendor's price change reaches the books on its own - never overwriting the others), `manual` and `contract` rows of a provider beat the list price. *Pricing rules* turn cost into charge: `markup_pct` (`{"pct": 20}`), `markup_fixed` (`{"usd_per_million": 0.5}`), `per_million` (`{"usd_per_million": 3.0, "unit": "token_out"}` sets the customer rate outright), `tier` (`{"tiers": [{"upto_tokens": n, "markup_pct": p}, ...]}` against the month's token volume), `promo` (`{"pct_off": 10}`), `min_margin` (`{"pct": 5}` - the reseller's floor: after every markup and promotion, no unit is charged under the provider rate plus that percent, so a tariff typed below a contract rate or a 100 % giveaway can never sell below cost), `fx` (`{"rate": 0.92}`) and `min_charge` (`{"cents": 1}`). Rules have a scope - `account` (one workspace), `plan` or `deployment` - and the most specific wins, then `priority`; without any markup rule `OSR_PLATFORM_DEFAULT_MARKUP_PCT` applies (default `0`: customers pay provider cost). Every change to a book or rule freezes a new price version (`GET /api/v1/admin/finance/versions`) and writes an audit entry with `before`, `after` and the `reason` the console requires. Three semantics the engine guarantees on the metered row. A charge that prices to **zero on purpose** (a 100 % `promo`, a zero `per_million` tariff) stays zero - only a request that neither the books nor the executing handler could price falls back to the reported provider cost. **Tiers** read the workspace's token volume of the month *before* the request being priced (`tier` boundaries are crossed by the first request past them). The pipeline is **USD-only**: an `fx` rule may scale the charge but `POST /api/v1/admin/finance/rules` refuses one that names another currency (`400`), and a foreign-currency quote that reaches the engine anyway leaves the row *unpriced* (`charge_usd` null) rather than mislabelled. Unpriced rows - also the ones a pricing failure left behind - are retried by the **finance pass** (every `OSR_PLATFORM_FINANCE_SYNC_S` seconds, default 5 minutes, and on `POST /api/v1/admin/finance/wallets/sync`), each written exactly once; the `finance` block of the analytics reports how many remain (`unpriced`), the console shows them as a warning, and revenue and margin exclude them until they are priced. A workspace pays for usage in one of three **billing modes** (`billing_mode` on the account, set by an operator with `PUT /api/v1/admin/finance/accounts/{account_id}/billing-mode`): `subscription` (default - the plan's flat fee plus overage; usage charges reach the same Stripe invoice through a dollar Billing Meter when `OSR_PLATFORM_STRIPE_USAGE_METER_EVENT` names it, one increment per sync with compare-and-set bookkeeping in `usage_charge_reports`), `wallet` (prepaid credits, below) or `invoice` (net terms, recorded for the statement). Credits are bought through a Stripe Checkout in payment mode (`POST /api/v1/billing/credits/checkout` `{amount_cents}`, whole dollars between `OSR_PLATFORM_CREDITS_MIN_CENTS` and `OSR_PLATFORM_CREDITS_MAX_CENTS`, tax collected by Stripe Tax); the webhook's `checkout.session.completed` with `metadata[credits_cents]` writes a `purchase` entry keyed by the event id, `charge.refunded` a `refund` entry keyed by the refund id. The ledger is append-only - corrections are `adjustment` / `promo` entries with a reason (`POST /api/v1/admin/finance/accounts/{account_id}/ledger`) - and the balance is its sum. **Admission and settlement.** Before a call that can incur provider cost runs, an **upper bound** of what it can cost is held on both sides of the ledger: every target the router may pick is priced by the finance engine at the prompt's tokens plus the output allowance (`max_tokens` when the caller sets one, else `OSR_PLATFORM_SPEND_BOUND_OUTPUT_TOKENS`, 4096) under the workspace's tariff, and the dearest quote times `OSR_PLATFORM_SPEND_BOUND_ATTEMPTS` (fallback attempts that may each bill, default 1) is the bound - provider cost for the budgets, customer charge for the wallet. A surface that cannot describe its request falls back to the workspace's mean charge over its last hundred spending requests (at least `OSR_PLATFORM_SPEND_RESERVE_MIN_USD`), which is an estimate, not a cap, and is labelled as such. Budgets (`daily_budget_usd` / `monthly_budget_usd` of the workspace, a tenant or a key) are a **strict cap on provider cost**: a request is refused with `429` when recorded cost + the provider-cost holds of every request still in flight + its own bound reach the limit, `X-Budget-Used` includes the holds and `GET /api/v1/policy` shows them as `held_usd`. A wallet workspace is admitted only while balance - charges not yet settled - customer-charge holds in flight - bound does not go below zero (a bound that exactly meets the remaining credits is admitted; `402` otherwise, `X-Wallet-Balance` and `X-Wallet-Available` in cents) - so a one-cent wallet never admits a two-dollar first call. The hold is written, the checks are repeated against every hold now committed (two replicas admitting at the same instant both see each other and the later one steps back), it is renewed every half TTL while the request is still running, and it is released when the row is recorded; a hold left by a dead process stops being renewed and expires after `OSR_PLATFORM_SPEND_RESERVE_TTL_S` seconds. Every metered row also persists the plan it was sold under and the month's token volume before it, and a row the database refused when the request finished (an outage after the provider was paid) is written to `usage-journal.jsonl` on the data volume and replayed by the next finance pass with its original timestamp - a paid-for request is never invisible for longer than the outage. Repricing (`reprice`) prices such rows under the tariff in force at their own timestamp with those persisted inputs; only a row that tariff still cannot price (a broken rule closed after the fact) is priced under the current tariff, and the pass reports it as `approximate` rather than changing a charge silently. Settlement is cumulative and idempotent: each finance pass debits every wallet workspace *everything charged through today minus everything ever debited* as one `usage` entry whose idempotency key carries the running total - a sub-cent day accumulates, a month-end remainder is taken on the first pass of the next month, a row priced late is owed on the next pass, and two replicas settling the same state write one entry. The `wallet` alert warns at `OSR_PLATFORM_WALLET_LOW_PCT` of the last top-up. | Endpoint | Who | What | |---|---|---| | `GET /api/v1/billing/usage` | any member | Usage with both prices, `group` = `day` (default) / `model` / `provider` / `key` / `tenant`, filters `key_id`, `tenant`, `model`, `provider_id`, `days`; `format=csv` downloads it. | | `GET /api/v1/billing/wallet` | any member | `billing_mode`, `balance_cents`, `last_topup_cents`, `pending_cents` (charged, not yet settled), `reserved_cents` (held for requests in flight), `available_cents` (what one more request is admitted against), purchase bounds and the active wallet alerts. | | `GET /api/v1/billing/wallet/ledger` | any member | The append-only ledger behind the balance, newest first. | | `POST /api/v1/billing/credits/checkout` `{amount_cents, save_payment_method?, auto_topup?}` | owner, admin | A Stripe Checkout URL (payment mode); returns to `/platform/dashboard/billing?credited=`. `501` without Stripe. | | `PUT` / `DELETE /api/v1/billing/wallet/auto-topup` | owner, admin | Auto top-up rule (*Models by the token*, below). | | `GET /api/v1/billing/statement?month=YYYY-MM&format=json\|markdown` | any member | The monthly credits statement: opening and closing balance, every entry, usage by model. | The operator side lives under `/api/v1/admin/finance`: `prices` (list, add, end, `prices/seed`), `rules` (list, add, end), `versions`, `quote` (price a hypothetical request), `alerts` (the money alerts below, as the console's *Margin watch* shows them), `wallets` and `wallets/sync` (run the finance pass now: expire dead holds, reprice, settle), per-workspace `ledger` and `billing-mode`, `providers` (prepaid balance, credit limit, low-balance line, contract terms), `providers/{id}/reconcile?period_start&period_end` (expected spend from the metered rows against the invoices of the period, with the models behind it) and `provider-invoices` (record, then move to `disputed` / `paid` - settlement is recording `paid`). A prepaid provider whose projected balance crosses its low-balance line raises a deployment alert (`provider_balance`). Two more alerts guard the reseller's margin on the console health page and in the alert stream: `margin` - a model whose priced requests of the last 24 hours were charged below provider cost (`critical`) or under `OSR_PLATFORM_MARGIN_ALERT_PCT` gross margin (`warning`, default 5 %) - and `above_list_price` - one per provider, counting the enabled offers whose storefront rate per million tokens exceeds the vendor's catalogue list price (the worst one named), so the platform no longer beats buying direct there. Reads are open to every operator; writes need the `finance` role (price books, rules, ledger, billing mode, invoices, settlement) or `provider_manager` (provider accounts and pricing, the provider endpoints themselves - create, edit, delete, check, target mappings - and the offers listed on them) - `superadmin` passes everything. `GET /api/v1/admin/analytics` carries a `finance` block: usage revenue, provider cost, gross margin, the wallets and the breakdowns by day, provider, model, workspace and plan. ### Models by the token: offers, prices, credits and provider preferences Besides routing with `auto`, the platform sells named models by the token, the way OpenRouter does. You pick a model from the storefront, call it under its catalogue id, and pay the price shown per million tokens; the platform chooses which provider endpoint serves the call unless you pin one. Three ids appear throughout this section: | Id | Example | Meaning | |---|---|---| | Catalogue id | `openai/gpt-4.1` | A model card in the reference catalogue (`GET /api/v1/llms`): capabilities and the vendor's list price. | | Provider | `azure` | An endpoint the deployment can call, configured by the operator under *Providers*. | | Offer | `openai/gpt-4.1@azure` | That model, served by that provider. The `@provider` suffix pins the endpoint. | An offer is an ordinary routing target: `auto` competes it with the rest of the catalogue, plans may use it, the learners grade it. Nothing is added to `targets.yaml` to sell a model. #### Find a model and its price The storefront at [`/models?view=offers`](/models?view=offers) lists every model on offer with two prices: the vendor's list price and the price your workspace pays, per million input and output tokens (cached input when the model supports it). Signed in, you see your own plan's prices; signed out, the deployment's public tariff. Each row opens the endpoints behind it with their health and last week's traffic. The same data over the API: | Endpoint | Returns | |---|---| | `GET /api/v1/offers` | One row per model: `prices.input_usd_per_1m` / `output_usd_per_1m` / `cached_usd_per_1m`, `priced_by` (`rules` when a pricing rule shaped the price, `list` when the vendor's price stands), the endpoints. Filters `q`, `vendor`, `provider`. | | `GET /api/v1/offers/{vendor}/{model}` | One model with its endpoints and its `GET /v1/models` entry. | | `GET /api/v1/llms/{vendor}/{model}` | The catalogue card, with the same `offers` block. | | `GET /v1/models` | The OpenAI-compatible list, in OpenRouter's shape (below). | `GET /v1/models` returns each model on offer with `pricing.prompt` / `completion` / `input_cache_read` in USD **per token** as strings, `context_length`, `architecture`, `top_provider` and `supported_parameters` - the fields OpenRouter clients read - plus an `osr` block with the `endpoints` (provider, upstream id, `executable`, `healthy`, region, data policy) and the vendor `list_price`. Tools written against OpenRouter's listing work unchanged. The price you see before the call is the price recorded after it: usage is metered under the catalogue id even when the provider reports its own name (`gpt-4.1` on Azure is `openai/gpt-4.1`), and the customer price comes from the pricing engine in the section above - a contract row of the provider in the price book beats the list price, then rules per deployment, plan or account apply. #### Call a model by name `POST /v1/chat/completions` (and the Responses and Messages dialects over it) accepts a catalogue id or an offer id in `model`, alongside `auto` and target ids. A `models` list may mix all three. ```bash curl -s https://api.opensmartroute.ai/v1/chat/completions \ -H "Authorization: Bearer $OSR_API_KEY" -H "Content-Type: application/json" \ -d '{"model": "openai/gpt-4.1", "messages": [{"role": "user", "content": "Summarise this thread."}]}' ``` | `model` | What happens | |---|---| | `openai/gpt-4.1` | The router picks among the enabled endpoints serving that model - by price, health and learned quality - and falls back down them when one fails. | | `openai/gpt-4.1@azure` | Only that endpoint. `503` with `Retry-After` when it cannot be used right now. | | `auto` | The whole catalogue, offers included. | The response says which endpoint answered: `model` and the `X-OSR-Target` header carry the offer id (`openai/gpt-4.1@azure`), `X-OSR-Charge-USD` and `charge_usd` in the response's `opensmartroute` object carry what the call cost you, and the usage row records `provider_id` and `upstream_model`. Key, workspace and tenant `allow_targets` / `deny_targets` may name a catalogue id; it stands for every endpoint of that model. #### Steer the choice: provider preferences Preferences travel in the request body under `osr.provider`. OpenRouter's top-level `provider` object is accepted as an alias and is never forwarded upstream. ```json { "model": "openai/gpt-4.1", "messages": [{"role": "user", "content": "..."}], "provider": {"order": ["azure", "openai"], "data_collection": "deny", "sort": "price"} } ``` | Key | Effect | |---|---| | `order: ["azure", "openai"]` | Try the first named provider's endpoint first; the rest are the fallback pool in that order. | | `only: [...]` | Use only these providers' endpoints. | | `ignore: [...]` | Never use these providers' endpoints. | | `data_collection: "deny"` | Skip every endpoint whose provider states that it trains on, logs or retains prompts. | | `zdr: true` | Zero data retention: use only endpoints whose provider has *declared* that it keeps nothing (`zero_retention: true`, or `retains_days: 0` with neither training nor prompt logging). An endpoint with no data policy at all is skipped - the provider has to say so. | | `sort: "price"` \| `"latency"` \| `"throughput"` \| `"quality"` | Re-weight the objective: the cheapest, the fastest or the best endpoint that can answer. | | `max_price: {"prompt": 2, "completion": 8}` | USD per million tokens; dearer endpoints are excluded. | | `allow_fallbacks: false` | Do not fall back to another endpoint (same as `osr.fallbacks: false`). | Preferences that exclude every endpoint of the named model are refused with `400` rather than silently routed elsewhere. An unknown key is a `400` too. A workspace sets defaults for all of its requests with the policy (`PUT /api/v1/policy`, key `provider_preferences`, the same object, or the *Providers* block of the policy dialog); a request's own preferences are layered on top. #### Pay as you go The `payg` plan has no monthly fee and no request quota. Usage is paid from prepaid credits (`billing_mode: wallet`); when they run out, requests stop with `402`. The abuse controls are the plan's `requests_per_minute` and the per-key spend limits below. - **Switch to it:** `POST /api/v1/billing/checkout` `{"plan": "payg"}` applies the plan at once - no Stripe Checkout, no payment method - and puts the workspace on the wallet. It answers `409` while a subscription is live; cancel that first. Moving to a subscription plan later takes the workspace off the wallet again. - **Buy credits:** the billing page offers fixed packs and a custom amount; `POST /api/v1/billing/credits/checkout` is the API. `GET /api/v1/billing/wallet` reports the packs, the purchase fee and the balance. - **Watch the balance from a client:** every priced chat answer of a wallet workspace carries `X-Wallet-Balance`, the credits (in cents) one more request would be admitted against, so an integration can top up before the `402` arrives. | Setting | Default | Effect | |---|---|---| | `OSR_PLATFORM_DEFAULT_PLAN` | `free` | The plan a self-serve signup starts on; `payg` makes new workspaces pay by the token from day one. | | `OSR_PLATFORM_CREDITS_PACKS_CENTS` | `1000,5000,20000` | The credit packs offered on the billing page (within the purchase bounds). | | `OSR_PLATFORM_CREDITS_FEE_PCT` | `0` | Purchase fee, percent of the credits bought. | | `OSR_PLATFORM_CREDITS_FEE_CENTS` | `0` | Purchase fee, fixed part in cents. | | `OSR_PLATFORM_AUTO_TOPUP_COOLDOWN_S` | `3600` | After one auto top-up attempt, charged or declined, the wallet is left alone for this long. | The fee is a second line item in Stripe Checkout and is booked on the ledger as a zero-sum pair of `fee` entries - collected, not spendable - so the balance carries only the credits and revenue analytics separate the fee from the spread. **Auto top-up.** A wallet workspace can have credits bought for it before the balance reaches zero: 1. Buy credits once with `save_payment_method: true` (the billing page: *Keep this card for auto top-up*). The Checkout keeps the card on the workspace's Stripe customer for off-session charges; the webhook stores its label (`visa 4242`), never the number. Passing `auto_topup: {amount_cents, threshold_cents}` on the same purchase switches auto top-up on as soon as it completes. 2. Set or change the rule with `PUT /api/v1/billing/wallet/auto-topup` `{amount_cents, threshold_cents}` - buy `amount_cents` whenever the available balance falls to `threshold_cents` or below (whole dollars within the purchase bounds, the threshold below the amount; `409` without a saved card). `DELETE` turns it off; `?forget_card=true` also detaches the card at Stripe. Owners and admins only. 3. The finance pass (the same one that settles usage, several times a day; `POST /api/v1/admin/finance/wallets/sync` runs it now) charges the saved card off-session and credits the wallet under the PaymentIntent id, so the `payment_intent.succeeded` webhook for the same intent is a no-op. One attempt per cooldown, whatever the outcome: a declined card is recorded as `auto_topup.error`, raises the `wallet:auto_topup` alert on the billing page and in the workspace alerts, and is not retried until the cooldown passes or the rule is saved again. `GET /api/v1/billing/wallet` reports the rule under `auto_topup` (`enabled`, `amount_cents`, `threshold_cents`, `payment_method`, `last_attempt_at`, `error`, `available` - the deployment has Stripe). Every step is in the audit log: `billing.payment_method_saved`, `billing.auto_topup_changed`, `ledger.auto_topup`, `billing.auto_topup_failed`. **Referrals: give credits, get credits.** Every workspace has a referral link, `/r/`, on *Refer a team* in the dashboard (`GET /api/v1/referrals`: the code, the link, the terms, the referrals so far and what they earned; `POST /api/v1/referrals/invite` e-mails the link to one address with a line from the sender, twenty a day). The site remembers the code the way it remembers the first touch and hands it to the signup (`attribution.ref` on `POST /api/v1/signup` and on a sign-in with a provider that creates the account); the latest referral link a browser opened is the one that counts. A referral is *signed up* when the new workspace exists, *qualified* when its owner has confirmed the address and the workspace has made its first successful request, and *rewarded* when the finance pass has credited both wallets - `OSR_PLATFORM_REFERRAL_REWARD_CENTS` to the referrer and `OSR_PLATFORM_REFERRAL_WELCOME_CENTS` to the new workspace (both `500` by default), as `promo` ledger entries that name the referral. A referrer earns at most `OSR_PLATFORM_REFERRAL_MAX_REWARDS` rewards (`20`; `0` = no cap) - a referral past the cap is *capped* and the invited side still gets its welcome credit. Self-referrals (the same person or address, a member of the inviting workspace) are not recorded and a workspace is referred once. `GET /api/v1/referrals/lookup?code=` tells a visitor on a link whether it is live and what they are offered, naming the inviter by initials only; the inviter sees the invited workspaces by initials. `OSR_PLATFORM_REFERRAL_ENABLED=false` switches the programme off (links keep working as plain sign-up links); all four are run-time settings. Operators read the programme on `GET /api/v1/admin/referrals` and run the qualification pass with `POST /api/v1/admin/referrals/sweep`. Events: `referral.signed_up`, `referral.rewarded`, `referral.capped`, `referral.invited`. **Money that moves is announced.** Whenever credits land on the wallet - a purchase at checkout or an automatic top-up - the owners receive *Credits added* (outbox kind `credits_added`: the amount, the balance now, the source and any purchase fee) and the workspace inbox gets `workspace.credits_added`; a Stripe refund sends *Payment refunded* (`credits_refunded`, the balance after it) and files `workspace.credits_refunded`. Each goes out once per ledger entry, whichever of the finance pass or the webhook wrote it. Both are `account` mail and always go out. #### Per-key spend limits `limits.spend_limit_cents` (per calendar month) and `limits.spend_limit_daily_cents` (per UTC day) on `POST /api/v1/keys` cap what one key may **charge** the workspace: charge recorded so far, plus the charge side of its requests in flight, plus the bound of the request being admitted. A request that would pass the cap is refused with `429`, `X-Key-Limit: spend_limit_cents` and the `X-Budget-*` headers in dollars. Budgets (`daily_budget_usd`) remain the cap on provider cost. #### Operators: list, price and pause offers The console page **Offers** (`/platform/admin/offers`) does everything in this section without curl. 1. **Connect a provider** under *Providers* and press **Check**. The check calls the provider's `/models` and reports, under `offers`, which upstream ids the reference catalogue recognises. 2. **List models.** One click creates an offer for every recognised id. Upstream ids that resolve to the same card are one offer (`gpt-4.1` and `gpt-4.1-2025-04-14` both mean `openai/gpt-4.1`). Azure and Bedrock do not return deployments from `/models`: name them in the `models` field. 3. **State the data policy** the provider publishes - `trains`, `retains_days`, `logs_prompts`, `statement_url`, `stated_at`. This is what `data_collection: deny` reads; an empty policy never excludes the endpoint. 4. **Set the price** on the *Finance* page: a contract row for the provider in the price book and the mark-up rules. Without either, the vendor's list price stands. 5. **Pause or remove** an offer from the same page when a provider is unreliable or a contract ends. Paused offers leave the storefront and `GET /v1/models` at once. The same over the API: | Endpoint | Purpose | |---|---| | `POST /api/v1/admin/providers/{provider_id}/check` | Probe the provider; `offers.recognised` lists the upstream ids the catalogue knows. | | `POST /api/v1/admin/providers/{provider_id}/offers` `{models?, region?, data_policy?}` | List models: one offer per recognised id (`models` names them explicitly, else the probe's list is used). | | `DELETE /api/v1/admin/providers/{provider_id}/offers` | Remove every offer of the provider. | | `GET /api/v1/admin/offers` | Every offer with its provider, prices and state. | | `GET /api/v1/admin/offers/desk` | The reseller's desk: every offer with what the deployment pays the provider per million tokens and what a workspace on the public tariff is charged (`rates`, with `priced_from` `contract` when a provider price-book row sets the cost, else `list`), the margin per unit, whether the offer declares zero data retention, and the last seven days - requests, tokens, success rate, realised provider cost, customer charge and margin. `totals` sums the week; `losing` lists the offers that lose money on paper or in practice. | | `POST /api/v1/admin/offers` `{model, provider_id, upstream_model?, region?, data_policy?, enabled?, notes?}` | Add one offer by hand. | | `PATCH` / `DELETE /api/v1/admin/offers/{vendor}/{model@provider}` | Pause (`enabled: false`), re-point (`upstream_model`), edit the data policy, or remove. | `OSR_PLATFORM_OFFERS_AUTO_LIST=true` lists the recognised models on every provider check without the button. **An Ollama server offers only the models it is mapped to run.** Several Ollama servers often share one model store, so each one's `/api/tags` lists every model on the disk while only the mapped one is resident and kept warm; an offer for any other model would send a chat into a cold load of tens of gigabytes and time out. List models therefore creates offers only for the upstream ids the server's target mappings name (the rest are returned under `unserved`), and every sync pauses an enabled offer whose server is not mapped to run the model, with the reason in the offer's notes. A provider is recognised as an Ollama server by its kind, by port `11434` or by a name starting with `ollama-`. Providers without mappings are not restricted. **A provider you cannot pay is unhealthy, not expensive.** When a prepaid provider's projected balance is exhausted (see *Usage pricing, credits and provider finance*), every target it executes - offers and hand-written mappings alike - leaves the pool `auto` routes over, a request pinning one of them answers `503` with `Retry-After`, the storefront and `GET /v1/models` mark the endpoint as not executable, and the `provider_balance` alert names the provider on the console health page and in the alert stream. A workspace's own provider (BYOK) is never affected: it pays for its own calls. ### Free trials, coupons and the launch offer A **coupon** grants a paid plan for a number of days without payment - no Stripe, no payment method. Redeeming one puts the workspace on the plan at once and records `account.trial_end` (and `coupon_code`) on `GET /api/v1/me`; when the trial ends without a subscription having taken over, the workspace returns to `free` (on its next request and by a sweep every `OSR_PLATFORM_TRIAL_SWEEP_S` seconds, default 15 minutes; audit entry `billing.trial_ended`). The owners hear at each step, whether or not the deployment has Stripe: *Your* (plan) *trial has started* the moment it begins (outbox kind `trial_started`: until when, that no card was taken, what happens at the end), *Your* (plan) *trial ends on* (date) three days before (`trial_ending`, the lifecycle campaign) and *Your* (plan) *trial has ended* when the sweep moves the workspace back (`trial_ended`); the workspace inbox carries the same (`workspace.trial_started`, `workspace.trial_moved` when the trial moves to another plan, `workspace.trial_ended`) and each is a `plan.changed` event (`reason` `trial`, `trial_moved`, `trial_expired`) for the subscribed channels. While a coupon trial runs the quota is the plan's, overage is not billed (there is no subscription), and the workspace may move to any other plan the coupon covers with the same end date. A Stripe subscription that completes during the trial clears the deadline. Coupons work on deployments without Stripe. The **plans' own free trials** ride the same machinery. Every subscription plan whose `trial_days` is set and whose `trial_needs_card` is false (the shipped catalogue: Pro, Team and Enterprise, 90 days each) is carried by the built-in promotion `FREE-TRIAL`, which the API keeps in step with the catalogue at start and whenever an operator changes a plan (`PATCH /api/v1/admin/plans/{name}`): it covers exactly those plans, advertises the longest of their trials and grants each plan its own number of days. The first `POST /api/v1/billing/checkout` of a workspace therefore starts the trial at once, without Stripe and without a card - on a deployment that has no Stripe at all, and on one whose Stripe account cannot charge yet. One trial per workspace: a workspace that had any trial or subscription before is refused (`409`) and pays instead. `GET /api/v1/info` shows the promotion with `builtin: true`; the running trial reports `coupon: null`; the operator console lists the row as created by `plans` and cannot type over its terms for long. Set every plan's `trial_days` to `0` and the promotion switches itself off. The **launch offer** is an operator's coupon on top, created at start-up from the environment and applied automatically (the newest live promotion wins a checkout): | Variable | Default | Meaning | |---|---|---| | `OSR_PLATFORM_LAUNCH_COUPON` | unset | The code, e.g. `LAUNCH100` (3-32 letters, digits, dashes). Unset = no launch coupon. | | `OSR_PLATFORM_LAUNCH_COUPON_DAYS` | `100` | Days of the plan the coupon grants. | | `OSR_PLATFORM_LAUNCH_COUPON_UNTIL` | unset | `YYYY-MM-DD` (UTC) after which it can no longer be redeemed; unset = until an operator disables it. | | `OSR_PLATFORM_TRIAL_SWEEP_S` | `900` | How often expired coupon trials are moved back to free (`0` = only on the workspace's next request). | An **auto-apply** coupon is the promotion: `GET /api/v1/info` advertises it as `promotion` (`code`, `trial_days`, `plans`, `valid_until`, `description`, `builtin`), the pricing page and the plan cards say "*N*-day free trial", and `POST /api/v1/billing/checkout` applies it to every eligible workspace **instead of** starting a Stripe Checkout - nobody has to type the code. Eligible = the coupon covers the plan, is enabled, within its deadline and cap, the workspace has no live subscription and has not redeemed this code before (for the built-in trial: has never had a trial). The answer then carries `coupon`, `builtin`, `trial_days`, `trial_end` and a `url` back to `/platform/dashboard/billing?redeemed=`. Send `"coupon": ""` to skip the trial and the promotion and pay (the *Subscribe now* button on the trial card), or `"coupon": ""` to use a specific one. | Endpoint | Who | What | |---|---|---| | `GET /api/v1/billing/coupons` | any member | `promotion` on offer, the workspace's running `trial` (`plan`, `coupon`, `trial_end`, `days_left`), `eligible` per paid plan and the workspace's `redemptions`. | | `POST /api/v1/billing/coupons/validate` `{code, plan?}` | any member | The coupon's public terms plus `eligible` and `reason` for this workspace. `404` unknown code. 20 lookups per workspace per minute. | | `POST /api/v1/billing/coupons/redeem` `{code, plan}` | owner, admin | Apply the coupon: `400` when it does not cover the plan or ran out, `409` when the workspace redeemed it before or already pays for a subscription. | | `GET|POST /api/v1/admin/coupons`, `GET|PATCH /api/v1/admin/coupons/{code}`, `POST /api/v1/admin/coupons/sweep` | operator | Create (`code`, `trial_days`, `plans` - empty = every paid plan, `auto_apply`, `max_redemptions`, `valid_until`, `description`), list with counters, see who redeemed a code, change its terms or disable it, run the expiry sweep now. | The dashboard's *Plan and billing* page shows the running trial (days left, end date, *Subscribe now* when Stripe is configured), marks the plans the promotion covers, and has a *Have a coupon code?* form; the operator console manages codes at `/platform/admin/coupons`. ## 7. Organizations Signup creates a personal workspace. An **organization** is a second kind of workspace with its own members, roles, API keys, tenants, plan and usage; one user can belong to several and switches between them in the dashboard (`/platform/dashboard/workspace`, `POST /api/v1/workspaces/{account_id}/switch`). Creating one (`POST /api/v1/workspaces` with `name` and an optional `slug`) makes you its owner and issues its first API key. A personal workspace that already carries a paid plan, keys and history can instead become an organization in place: `POST /api/v1/workspace/convert` with `name` (and an optional `slug`) changes only its kind - plan, subscription, keys, tenants and usage stay on the same row, the browser session stays on it, and the user receives a fresh free personal workspace alongside (audit `workspace.converted`). The *Members* page of a personal workspace offers exactly this, so the seats a team or enterprise plan sells become usable without moving anything. Roles are `member` < `admin` < `owner`: | Action | Role | |---|---| | Route, read usage, savings and activity; read the roster and the teams | `member` | | Manage the roster and the keys of a team they lead (below): mint, rename, rotate, revoke | `member` with the team `lead` role | | Create and revoke API keys, change tenants, invite and remove members, create teams, rename the workspace, checkout, billing portal and cancellation | `admin` | | Grant or remove `owner`, transfer ownership, configure organization SSO, delete the connection or the organization | `owner` | API keys belong to the workspace and act with its full rights; the role model applies to signed-in browser sessions. Members are invited by email (`POST /api/v1/workspace/invites`): the invitee gets a message with the link when the deployment has a mail server, and the platform returns the same link once to the inviter so it can be shared directly. `POST /api/v1/workspace/invites/{id}/resend` sends a pending invitation again with a fresh link and a full validity period (the previous link stops working; audit `member.invite_resent`); invitations that expired without being accepted are purged by the retention sweep. Seats and tenants are limited per plan (`max_members`, `max_tenants` in `GET /api/v1/info`). An organization always keeps at least one owner. **Transferring ownership** is one call: `POST /api/v1/workspace/transfer` with the new owner's `user_id` and the role the current owner keeps (`keep_role`: `admin` - the default -, `member`, `owner` to share ownership, or `leave` to hand over and go), owner only, audited as `workspace.transferred`, both people e-mailed; the crown button on **Members** does the same. There is no per-workspace "suspend" for a person: **remove the member** (their sessions on the workspace end at once, the invitation can be sent again later), or an operator disables the whole user account. **Organization SSO** (enterprise) lets people sign in with the company identity provider: `PUT /api/v1/workspace/sso` with `kind` (`github`, `google`, `microsoft`, `gitlab`, `linkedin` or `oidc` with an `issuer`), `client_id`, `client_secret`, the allowed email `domains` and the `default_role` given to anyone who signs in from one of those domains. The workspace needs a slug first; the sign-in page shows the provider as `org-`. A `role_map` (`{"claim": "groups", "map": {"osr-admins": "admin"}, "require_match": false}`) derives the workspace role from the provider's group claim instead: the mapping is re-applied at every sign-in (a user added to or removed from a mapped group changes role on the next login, recorded as `member.role_sso` in the workspace audit; the last owner is never demoted), unmatched users get `default_role`, and `require_match: true` refuses anyone outside the mapped groups. Entra `groups` / `roles` / `wids`, Okta `groups`, Keycloak `realm_roles` and Cognito `cognito:groups` are read from the id token and userinfo; ask the provider for the claim with the connection's `scopes` (`groups`). **SCIM provisioning** (organizations) is the push side of the same identity provider: single sign-on lets a person *in* when they sign in, SCIM 2.0 (RFC 7643 / 7644) lets Entra ID, Okta, OneLogin, JumpCloud and the like create the account before the first sign-in and **remove it the moment HR closes it**. An owner mints the provisioning token on **Workspace › Single sign-on** (*Provisioning (SCIM)*; `POST /api/v1/workspace/scim/token`, shown once, valid a year, one live token per organization - minting again replaces it, `DELETE` revokes it) and pastes it with the base URL `/scim/v2` into the provider's SCIM connector. The connector then calls: | Call | What happens | |---|---| | `GET /scim/v2/ServiceProviderConfig`, `/ResourceTypes`, `/Schemas` | What is supported: users, PATCH, `filter`, no bulk, no passwords (sign-in stays with the provider). | | `GET /scim/v2/Users` | The members as SCIM users; `filter=userName eq "..."` / `externalId eq "..."`, `startIndex`, `count` (up to 200). | | `POST /scim/v2/Users` | Provision: the person's account is created if it does not exist (with a personal workspace like any signup; the address counts as confirmed because the provider vouches for it) and joins the organization with the SSO connection's `default_role` (`member` without one) - a `roles` entry of `admin` or `member` sets the role, `owner` is refused; the plan's seat limit applies as for an invitation; `409` when already a member. | | `PATCH` / `PUT /scim/v2/Users/{id}` | `name`, `displayName`, `roles`, `externalId` and `active`: `active: false` **removes the membership and ends the person's sessions on the workspace** - the user and their other workspaces stay - and `active: true` restores it. Paths such as `name.givenName` and `emails[type eq "work"].value` are understood. | | `DELETE /scim/v2/Users/{id}` | Deprovision, the same as `active: false` (`204`). The last owner is never removed (`409`). | | `GET /scim/v2/Groups` | Empty: roles come from the SSO `role_map` at sign-in or from `roles` here; teams are not provisioned. | Errors come back in SCIM's own shape (`urn:ietf:params:scim:api:messages:2.0:Error` with `status`, `detail`, `scimType`). Every change is in the workspace audit trail with *identity provider (SCIM)* as the actor - `member.provisioned`, `member.deprovisioned`, `member.role_scim`, `user.scim_updated` - and a deprovisioning files the same *member removed* notice in the workspace inbox as a removal by hand. `GET /api/v1/workspace/scim` (admin role) shows whether provisioning is on, the token's age and use and how many people came through it. Not offered, by decision rather than omission: **SAML** (every supported identity provider - Entra, Okta, Google, Keycloak, Cognito - also speaks OIDC, which is what the `oidc` connection kind uses; SAML-only providers can front the platform through an OIDC bridge) and the OpenAI **Batch API** (asynchronous batch files - the router prices and routes each request live; send them concurrently). ### Teams A large customer does not want one admin minting every key. A **team** groups some of an organization's members - a product squad, an agency's client desk, the data-science group - and gives that group its own place in the workspace: * **Tenant binding.** A team may be bound to one of the workspace's [tenants](#8-tenants). Keys minted for the team are pinned to that tenant, so its constraints, budgets and usage roll-up apply, and the team's spend shows on the tenant. `GET /api/v1/workspace/teams` reports each bound team's requests, tokens and cost this month and today. A tenant bound to a team cannot be deleted until the team is unbound (`409`). * **Leads.** A team member with the team role `lead` manages the roster (adds and removes members, not other leads) and the team's keys - mints them (`POST /api/v1/keys` with `team: ""`; the key carries `team` and is pinned to the team's tenant), renames, rotates and revokes them - without holding the workspace `admin` role. Only admins make leads. This is the delegation that lets a customer run its own people: the admin creates the team, binds it and names a lead; the lead does the rest. A team with keys cannot be re-bound to another tenant (`409`; revoke its keys first); deleting a team detaches its active keys (they keep working as the tenant, managed by admins from then on - `keys_detached` in the answer). * **Attribution.** `GET /api/v1/workspace/members` lists every person's teams, `GET /api/v1/me` the caller's own (`teams`), and the audit trail records `team.created`, `team.updated`, `team.deleted`, `team.member_added` and `team.member_removed` (also notification events). | Endpoint | Role | Notes | |---|---|---| | `GET /api/v1/workspace/teams` | `member` | Every team with `members`, `leads`, `keys` pinned to its tenant and `usage`; `mine` names the caller's own teams. | | `POST /api/v1/workspace/teams` `{name, slug?, description?, tenant?, members?}` | `admin` | `slug` defaults to the name; `members` seeds the roster (`[{user_id, role}]`). `409` when the slug exists. | | `GET /api/v1/workspace/teams/{slug}` | `member` | The team with its roster (team role and workspace role per person), its active `keys` (prefixes) and `can_manage`. | | `PATCH /api/v1/workspace/teams/{slug}` `{name?, description?, tenant?}` | `admin` | `tenant: ""` unbinds. `409` while the team has keys and the tenant would change. | | `DELETE /api/v1/workspace/teams/{slug}` | `admin` | Members stay in the workspace; the team's active keys are detached and keep working as the tenant. | | `POST /api/v1/workspace/teams/{slug}/members` `{user_id, role?}` | `admin` or team `lead` | The person must already be a member of the workspace. Only admins set `role: "lead"`. | | `DELETE /api/v1/workspace/teams/{slug}/members/{user_id}` | `admin` or team `lead` | A lead cannot remove another lead (but may leave). | Leaving or being removed from the workspace drops every team membership; deleting the organization drops its teams. Personal workspaces have no teams (`400`). The **Members** page carries the *Teams* card (create, bind, roster, leads) and shows each member's teams; the **Tenants** page names the teams that run under each tenant; the key dialog on **API keys** offers the teams the caller may mint for. Operators see the teams and their rosters on the workspace's console page (`teams` on `GET /api/v1/admin/accounts/{account_id}`), and the data export (`GET /api/v1/me/export`) carries them. ### Regions The hosted platform runs as independent stacks per region - an EU stack and a US stack - each with its own database, so a workspace's keys, usage and traces never leave the region it was created in. A stack declares its region with `OSR_PLATFORM_REGION` (`eu`, `us`; unset = a single-region deployment that enforces nothing). Every workspace created on a stack - at signup, `POST /api/v1/workspaces` or by an operator - is pinned to that region (`region` on the account in `GET /api/v1/me` and `GET /api/v1/info` reports the stack's `region` and `regions_enforced`), and API keys are minted where the workspace lives. Using a key on the other stack answers `403` with `code: "region_mismatch"`, `region: ""` and `detail: "this workspace is pinned to region ''; use the endpoint"`; browser sessions get the same answer, which is how the dashboard tells you your workspace lives in the other region. Every authenticated response carries `X-OSR-Region` with the answering stack's region, so a client can check it is talking to the right one. Operators are not pinned, and public endpoints work on every stack. Deploying two stacks = two deployments of the same template with `OSR_PLATFORM_REGION` set to `eu` / `us` and DNS such as `eu.api.` / `us.api.` in front of each. Moving a workspace is deliberate: an operator changes its `region` (`PATCH /api/v1/admin/accounts/{account_id}` with `{"region": "us"}`, audited as `account.region_changed`; `""` unpins it), and the data itself is exported from one stack and imported on the other by hand (`GET /api/v1/me/export`). This is not the routing constraint of the same name: `constraints.region` on a request (or a tenant's `region`) chooses *targets* deployed in a region; the workspace region chooses the *stack* that accepts the key. ## 8. Tenants Tenants (enterprise) apply a fixed set of hard constraints per customer, team or region without the caller repeating them. Create a tenant, then send its slug in `X-OSR-Tenant` (or `osr.tenant`). ```bash curl -s -X PUT "$OSR_URL/api/v1/tenants" -H "Authorization: Bearer $OSR_API_KEY" -H "Content-Type: application/json" \ -d '{"slug": "eu-health", "config": {"region": "eu", "data_boundary": "private", "max_cost_per_1k": 0.01, "deny_targets": ["llm-frontier"]}}' curl -s "$OSR_URL/api/v1/route" -H "Authorization: Bearer $OSR_API_KEY" -H "X-OSR-Tenant: eu-health" \ -H "Content-Type: application/json" -d '{"text": "Anonymise this discharge summary ..."}' ``` Tenant config keys: `region`, `data_boundary`, `max_cost_per_1k`, `max_latency_ms`, `preferred_max_latency_ms`, `deny_targets`, `allow_targets`, `contains_pii`, `daily_budget_usd`, `monthly_budget_usd`, `log_payloads`, `marketplace_discovery`, `data_protection`, `output_guard`, `allowed_ips`. Explicit request constraints win over the tenant defaults; the workspace policy (next section) applies on top of both. Executed spend is attributed to the tenant, so `GET /api/v1/governance` shows spend and budgets per tenant, and `GET /api/v1/tenants` returns each tenant with `usage` (`month` and `today`: requests, tokens, `cost_usd`), its `budgets` with what is used and remaining, and `keys` - the number of API keys pinned to it. `GET /api/v1/usage` adds `by_tenant` (requests, tokens and cost per tenant slug; `null` is the workspace's own traffic) and `?tenant=` narrows the daily series and the target and endpoint breakdowns to one tenant; `GET /api/v1/audit?tenant=` keeps only that tenant's hash-chained records, so a per-customer audit extract is one export away. Audit reads are bounded: each chain file is read from its tail and the scan stops once `limit` matching records are in hand, and `next_before` in the answer is the cursor for the following page (`GET /api/v1/audit?limit=100&before=`). The *Activity*, *Usage* and *Audit trail* pages offer the tenant filter. **Tenant-pinned keys** are the isolation boundary for a customer or team you hand a key to: `POST /api/v1/keys` with `{"name": "acme", "tenant": "acme"}` mints a key whose every request runs as that tenant - the header is not needed, naming another tenant answers `403`, and the key can never act as the whole workspace. Its requests count against the tenant's budgets and appear under the tenant in usage and audit. The boundary holds on every surface: the key routes, executes and reports outcomes as its tenant; it reads only its tenant's rows on `/activity`, `/trace/{id}`, `/events`, `/usage`, `/usage/report`, `/savings`, `/audit`, `/handoffs`, `/keys`, `/tenants`, `/me` and `/governance` (another tenant's request id answers `404`, `?tenant=` answers `403`); every workspace-management call - keys, policy, tenants, providers, members, channels, billing, listings and the other reads of the workspace - answers `403` with `X-Key-Tenant: `. A tenant with active pinned keys cannot be deleted (`409`); revoke them first. The *API keys* page offers the tenant on creation and shows the scope of every key. **Fair share** (enterprise, `OSR_PLATFORM_FAIR_SHARE=true`) keeps one workspace or tenant from crowding out the others on a shared deployment: the SDK's dominant-resource-fairness middleware watches requests, tokens and cost per workspace and per tenant over a sliding window (`OSR_PLATFORM_FAIR_SHARE_WINDOW_S`, 300 s), steers whoever takes more than `OSR_PLATFORM_FAIR_SHARE_SOFT` (1.5) x its share to cheaper targets and caps premium targets above `OSR_PLATFORM_FAIR_SHARE_HARD` (3) x - the trace shows `fairshare.throttled` / `fairshare.capped` events. `GET /api/v1/governance` -> `fair_share` reports the window, the counters and the dominant shares of the caller's own workspace and tenants only; `controls.fair_share` says whether it is on. Operators see every tenant across workspaces on `/platform/admin/tenants` and can edit a tenant's configuration or delete it there (`PUT|DELETE /api/v1/admin/accounts/{account_id}/tenants/{slug}`). ## 9. Governance: workspace policy, budgets and controls A **workspace policy** applies to every request of the workspace, whichever key or tenant sent it. It takes the same keys as a tenant plus budgets, and needs the `admin` role: ```bash curl -s -X PUT "$OSR_URL/api/v1/policy" -H "Authorization: Bearer $OSR_API_KEY" -H "Content-Type: application/json" \ -d '{"config": {"data_boundary": "private", "deny_targets": ["llm-frontier"], "monthly_budget_usd": 250}}' ``` `GET /api/v1/policy` returns the policy, the accepted keys and the current budget lines; `DELETE` removes it. Constraints merge deterministically: a boundary or region fills in when the request has none, cost and latency caps take the stricter value, deny lists are merged and allow lists intersected. Unknown keys, unknown target ids and an id present in both `allow_targets` and `deny_targets` are rejected with `400`. **Budgets** count executed spend (`execute: true`, `/v1/chat/completions`, MCP `ask`) - quotes and route-only calls are free. `daily_budget_usd` resets at midnight UTC, `monthly_budget_usd` on the first of the month. Once a workspace or tenant budget is exhausted, routed calls return `429 ... budget ... exhausted` with `Retry-After` and the `X-Budget-*` headers until the period rolls over or the limit is raised. **Request logging** is off by default: the platform keeps the decision facts of every request (signals, ranking, policy rejections, execution steps, tokens, cost) but no prompt or answer text. A workspace that wants to read them next to each trace opts in with `log_payloads` in the policy (or per tenant): `redacted` stores the prompt and the answer with PII masked first (`opensmartroute.security.Redactor`: e-mails, phone numbers, card and ID numbers become ``-style placeholders) and credential shapes masked as well (API keys, cloud and repository tokens, private keys become ``), `full` stores them as sent. Each text is cut to `OSR_PLATFORM_REQUEST_LOG_MAX_CHARS` characters (`truncated` marks the row). The stored payload appears as `payload` on `GET /api/v1/trace/{request_id}` and on the trace drawer of the *Activity* page, `GET /api/v1/activity` marks the rows that have one (`logged`) and reports the mode in force (`logging`), the data export carries every row (`request_logs`), the rows follow the usage retention, and `DELETE /api/v1/activity/logs` (admin) erases them all while the activity rows, traces and outcomes stay. The governance posture flags `full` logging as a warning and `redacted` as an informational finding. **Data protection** decides what happens to personal and health data *before* the router or any provider sees the text - the policy key `data_protection` (per workspace, or per tenant in the tenant's config, which wins): ```json {"config": {"data_protection": {"mode": "mask", "entities": ["pii", "phi"], "restore": true}}} ``` | `mode` | What happens to a request that carries personal data | |---|---| | `detect` (default) | Nothing is changed; the request is marked as carrying personal data, so it may only route to targets whose catalogue entry allows it (`pii_allowed`) - a public model pinned by name answers `422`. | | `mask` | Every detected value becomes a labelled placeholder (``, ``, ``) in the current turn **and** in the conversation history before anything is routed, so the request may run on any target; the same value gets the same placeholder throughout. Targets that are allowed to see personal data still receive the originals. With `restore: true` (default) the placeholders in the answer - streamed too - are turned back into the values for the client; `false` hands the client the placeholders. | | `block` | A request that carries any of the chosen entities is refused with `422` before any spend, naming the kinds found (`mrn (1), email (2)`), never the values. | | `off` | The detection is switched off entirely (not even the routing mark). | `entities` chooses what counts: the groups `pii` (e-mail, phone, card, national ids, passport), `phi` (medical record number, NPI, DEA, Medicare id, insurance member id, ICD-10 code, date of birth) and `financial` (card, IBAN), or single names; the default is everything. Every response carries `X-OSR-Data-Protection: ` when something was found, and the usage row's `facts.pii` records the mode, the entity counts and whether the text was masked (`GET /api/v1/activity`, the trace drawer) - the values themselves are never stored. Under `mask` the stored request log (`log_payloads`) keeps the masked text. The same `facts` object carries `cache: hit`, `fallback_from` (the targets that failed before the answer came from another one) and `upstream_request_id` - the provider's own id of the call (`x-request-id` at OpenAI, `request-id` at Anthropic, `apim-request-id` at Azure), what a support ticket with the provider needs. **Output guard** decides what happens to an answer *before* the customer sees it - the policy key `output_guard` (per workspace, or per tenant, which wins): `{"mode": "redact", "pii": true, "secrets": true, "injection_threshold": 0.7}`. Every executed answer, streamed or not, is screened for leaked secrets (cloud and API keys, tokens, private keys), personal data and instruction injection. `flag` reports the finding and leaves the text; `redact` replaces secrets and personal data by placeholders (``, ``) - a streamed answer is held until it was screened, then sent; `block` refuses an answer that carries a secret or an injection with `400` and redacts personal data otherwise; `off` (default) leaves answers alone. `pii: false` keeps personal data in place (a support answer may quote an address) while secrets still go. The finding - kinds and counts, never the values - is on the answer (the `output_guard` field of the response's `opensmartroute` block on chat completions, `result.output_guard` on `/api/v1/route`), in the `X-OSR-Output-Guard` header (`redact:secrets=1,pii=2`) and on the usage row as `facts.output_guard`. Dashboard: *Governance › Controls*, next to the data protection. **Allowed addresses** fence every routed request of the workspace: `allowed_ips` in the policy (addresses or CIDR blocks, at most 32) refuses a request from anywhere else with `403` and `X-Policy-Limit: allowed_ips` before anything is routed, judged on the client address past the trusted proxies - never a header the caller wrote. A tenant's own `allowed_ips` replaces the workspace's for its requests. Management calls (the policy itself, keys, billing) are not fenced, so a locked-out address can always be fixed; `allowed_ips` on one key (above) fences that key everywhere. **Marketplace discovery** decides which published skills and personas a plan may use *without* an install (section 11): `marketplace_discovery` is `verified` (the deployment default unless the operator changed it - only listings an operator marked verified), `all` (every published listing) or `off` (the workspace's installs only). Discovered companions fill plan slots and never answer on their own; every decision names the listings it used under `marketplace`, each with `installed: true|false`. **Default objective.** `objective` in the policy (`{"quality": 1, "cost": 0.6, "latency": 0.1}`, any of `quality`, `cost`, `latency`, `quality_floor`) is the objective of every request that sets none; a request's own `objective` still wins. It is what promoting a shadow candidate writes (the *Evaluation* page, section above). **Response cache.** `response_cache_ttl_s` (seconds, up to seven days) answers an identical chat request the workspace already paid for from the store instead of a provider: the same model name, messages, options and tenant hash to one key, the stored answer comes back with zero tokens and cost on its usage row, a `cache` entry (`{"hit": true, "stored_at": ...}`) in the answer's routing block and an `X-OSR-Cache: hit | miss | bypass` header. A caller shortens the life of one answer with `X-OSR-Cache-TTL: 60` (never lengthens it past the policy), skips the lookup with `X-OSR-Cache: bypass`, keeps an answer out of the store with `X-OSR-Cache: no-store`, or sets `osr.cache: false` in the body. `GET /api/v1/cache` reports hits, misses and what the hits saved; `DELETE /api/v1/cache` (admin) empties the workspace's entries. The store is Redis when the deployment has one, else the database. `response_cache_mode: "similar"` adds near-duplicate matching: every entry also keeps a *context key* (the request with its last user message blanked - model, earlier turns, tools, sampling) and a signature of that message (hashed character shingles of its normalised text, never the text), and a request that misses the exact key is compared with the entries of the same context and served from the closest one whose similarity reaches `response_cache_similarity` (0.5-1, default 0.85). That catches the rewordings that make up most repeats - case, punctuation, a typo, a word - without an embedding call per request; the answer's `cache` entry says `"match": "similar"` with the score, which `X-OSR-Cache-Similarity` carries too. It is surface similarity, not meaning: two questions that differ in one decisive word can score high, so keep the threshold where your traffic makes it safe and leave `exact` for workspaces where that cannot be tolerated. Both keys are in the policy dialog. **Session affinity.** `session_affinity: true` keeps a conversation on the target that served it: a request that carries a session (`context.session_id`, or the hosted `osr.conversation_id`) routes to the target of the previous turn while that target stays executable, allowed by the policy and not behind an open breaker; an explicit `model` in the request still wins. Agent loops stop flipping models mid-task and prompt caches on the provider side keep their prefix. **Aliases.** `aliases` maps the model names your clients already send to what the router understands - a target id, a catalogue model, an offer (`vendor/model@provider`) or a preset (`osr/auto:cheap`): `{"gpt-4o": "auto", "claude-3-5-sonnet": "llm-frontier"}`. Resolution is one step and happens before everything else on `/v1/chat/completions` (`model` and `models`), so a code base migrates to the router without a search-and-replace of model names. **Answer judge.** `judge` turns on the LLM judge for a share of the executed answers: `{"sample": 0.1, "target": "llm-frontier", "threshold": 6, "endpoints": ["chat"], "targets": ["llm-small"], "max_per_day": 500}`. The request id decides the sample (reproducibly), the judge model grades the answer against the prompt (0-10 with a reason), the grade lands in the `feedback` table as `quality` with `source: judge:` - exactly where a hand-reported outcome lands, so the learners, the rankings and the replay comparison use it - and the verdict with its reason stays on `GET /api/v1/evaluations/judge`. Judging is a metered call of its own (`endpoint: judge`) and runs on a thread after the answer has left; `max_per_day` bounds it. Section 5 has the endpoints. `GET /api/v1/governance` is the compliance view of the workspace in one payload: | Block | Contents | |---|---| | `controls` | What the deployment enforces: `input_guard`, `pii_redaction`, `steering_strip`, `output_guard` (`flag` / `redact` / `block` or `false`), `metrics`, `tracing`, `audit`, `health_breakers`, `decision_cache`, `learning` (`persisted` or `in-process`). | | `audit` | Whether the hash-chained audit trail is on, its length and the result of the last verification. | | `retention_days` | How long usage rows are kept before the background sweep purges them. | | `policy`, `policy_keys`, `budgets`, `spend` | The workspace policy, its accepted keys, budget lines with used and remaining amounts, spend today and this month. | | `quota` | The same payload as `GET /api/v1/quota`: requests used this month against the plan's quota, `state`, reset time and the overage so far. | | `tenants` | Each tenant with its monthly spend and budget lines. | | `catalogue` | Targets per `data_boundary`, targets that refuse PII, executable targets, providers, and which targets remain reachable under the policy. | | `keys` | Hygiene of the active API keys: `active`, `unscoped`, `no_expiry`, `expired`, `limited` (budgets or allow lists), `stale` (unused for 90+ days), `tenant_pinned`. | | `guard_activity` | The guard verdicts of the last 24 hours counted from the workspace's trace events - `guard.blocked` / `guard.flagged` / `guard.redacted` (input guard), `output_guard.flagged` / `output_guard.blocked`, `route.no_route`, `tenant.rejected`; `null` when tracing is off. | | `posture` | The compliance findings a reviewer would raise, most severe first, each with a stable `code`, a `severity` (`critical` / `warning` / `info`), the reason, the dashboard page that fixes it and a `scope`: `workspace` for what the customer can change (no budget, no reach policy, unscoped / non-expiring / expired / stale keys, an organization without SSO on a plan that includes it or without a role map) and `deployment` for what only the platform operator holds (guards or audit off, output guard only flagging, chain broken, tracing off, unbounded retention). `score` (0-100; a critical costs 25, a warning 10, an info 3) counts the **workspace** findings only; `deployment` carries the platform's own findings with their own `score`, so a customer is never judged by a setting they cannot touch. | `GET /api/v1/governance/report` (admin role) is the evidence bundle behind that view: the governance payload plus the members and their roles, the SSO connection (client id and role map, never the secret), every active key with its scopes, limits, expiry and last use, the platform's identity (brand, versions, region) and who generated the report when; `?format=markdown` downloads it as a Markdown document for an audit file. Every generation is recorded in the workspace's audit log (`governance.report`). The dashboard renders the same data at `/platform/dashboard/governance` - the posture card links each finding to the page that fixes it, the guard-activity table links into the events page filtered on the event, the key-hygiene card into the keys page, and "Compliance report" downloads the Markdown evidence - with a policy editor for admins. ## 10. Observability and caching The platform observes itself: every span, event, counter and alert below is produced, stored and served by the platform - there is no external metrics system, tracing backend or alerting tool to run. Every deployment exposes liveness and readiness probes and the counters without a key: | Endpoint | Meaning | |---|---| | `GET /healthz` | Process is up: edition, version, target count, uptime. | | `GET /readyz` | `200` when the database, catalogue, router and (enterprise) audit file all answer; `503` with the failing check otherwise. Also reports `database_dialect` (`postgres`), `redis` and `events` (the event bus) - those two are informational and never fail readiness. | | `GET /metrics` | The counters in text exposition format: `osr_platform_http_requests_total{method,route,status}`, `osr_platform_http_request_duration_ms_*`, `osr_platform_http_in_flight`, plus the router's `osr_route_decisions_total`, `osr_route_latency_ms_*`, `osr_outcome_*` and `osr_decision_cache_*`. | Responses carry `X-Request-Id` (send your own to correlate with your logs) and `Server-Timing`; the API writes one JSON access-log line per request (`osr.platform.access`) with the route template, status, latency and account - never the prompt. `GET /api/v1/stats` (enterprise) adds `http`, `controls`, `cache` and `observability` snapshots; `/platform/dashboard/health` shows them live. ### Time series and alerts | Endpoint | Returns | |---|---| | `GET /api/v1/telemetry/series?window=24h` | The workspace's metered traffic bucketed over `1h` (per minute), `6h` (5 min), `24h` (15 min), `7d` (3 h) or `30d` (per day): requests, failures, `error_rate`, p50 / p95 latency, cost and tokens per bucket, totals per target and per endpoint, and the window total. Computed from the platform's own usage records, so it agrees with the activity log to the row. Plans with `stats` also get `http`: the deployment's requests, 4xx, 5xx and p50 / p95 / p99 latency per bucket from the per-minute counters the process keeps for 24 hours. | | `GET /api/v1/alerts` | Active conditions, most severe first, each with the dashboard page that shows or fixes it. Workspace rules on every plan: the monthly request quota at 80 % (`warning`), exhausted (`critical`) or in overage (`warning`, with the estimated bill); a daily or monthly budget of the workspace or a tenant at 80 % (`warning`) or exhausted (`critical`); 20 % or more of the last 15 minutes' metered requests failing (10+ requests); prompts or answers blocked by the guards in the last 24 hours (`warning`, `guard_blocked`) and answers the output guard flagged for secrets, PII or injection (`info`, `output_guard_flagged`) - both link into the events page filtered on the guard events. Deployment rules on plans with `stats`: a failing readiness check, 5 % or more 5xx over the last five minutes (20+ requests), p95 above 2 000 ms, an open or half-open circuit breaker per target, a routing-drift alarm in the last hour, a failed autopilot cycle, a routing SLM last fitted more than 30 days ago, and tracing being off. The money rules - a prepaid provider's balance (`provider_balance`), a model sold below cost or under the margin floor (`margin`), an offer priced above the vendor's list price (`above_list_price`), Stripe Tax and the Stripe account (`stripe_tax_unconfigured`, `stripe_account_inactive`) - are the operator's alone: they appear on the console (`GET /api/v1/admin/health`, `/api/v1/admin/finance/alerts`), never on a customer's dashboard - see *Usage pricing, credits and provider finance*. `status_monitor` (`critical`): the status page's sampler has recorded nothing for three intervals, so `/status` says unknown. `backup_stale` (`critical`, operator only): the nightly `osr-db-backup` job records every finished archive in the `backups` table; when the newest is older than 36 hours the console's Health page and the operator notifier say so - a deployment where the job never ran shows nothing. | The overview shows the last 24 hours (traffic, latency) and the alerts; `/platform/dashboard/health` adds the deployment series and the same alerts, refreshed with the page. ### Notifications: alert delivery Alerts are evaluated by the API itself - no external scheduler. Every `OSR_PLATFORM_ALERTS_INTERVAL_S` seconds (default 60; `0` turns the loop off and leaves *Evaluate now* to operators) one replica - the one holding the `osr:alerts:leader` lease in Redis, or the only one without Redis - re-evaluates the workspace rules for every workspace that was active in the last 15 minutes or has a channel or an open alert, plus the deployment rules. Each condition becomes a **notification episode**: opened when the rule starts firing, refreshed while it stays on (an escalation from `warning` to `critical` is delivered again), resolved when it clears. Episodes land in an inbox with unread counts per severity, are published to the `alerts` Kafka topic in cluster mode, and are delivered to the workspace's **channels**: | Kind | Target | Delivery | |---|---|---| | `email` | An address | Through the platform mailer (Microsoft 365, ACS or SMTP; otherwise the outbox at `/platform/admin/mail`). | | `webhook` | An `https://` URL | `POST` of `{event, alert, workspace, url}` with `X-OSR-Event` (`alert.firing`, `alert.escalated`, `alert.resolved`, `alert.test`), `X-OSR-Delivery` and, when the channel has a secret, `X-OSR-Signature: t=,v1=` = HMAC-SHA256 of `.`; verify like Stripe signatures. | | `slack` | An incoming-webhook URL | Block Kit message with severity, rule, value against threshold and a link to the page that fixes it. | | `teams` | An incoming-webhook URL | MessageCard with the same facts. | | `pagerduty` | An Events API v2 routing key (32 characters) | `trigger` on firing and escalation, `resolve` when the alert clears, one `dedup_key` per episode so PagerDuty keeps a single incident; severity, rule, value and the dashboard link travel as custom details. | A channel filters by minimum severity (`info`, `warning`, `critical`) and optionally by rule names, can be disabled, records its last delivery and last error, and has *Send test*. Failed webhook deliveries are retried on the following evaluation rounds up to three attempts. Up to 20 channels per workspace; creating or changing them takes the `admin` role. Operators have the same inbox and channels for the **deployment** scope (readiness, 5xx rate, p95, circuit breakers, drift, autopilot, SLM age, tracing) at `/platform/admin/notifications`. **Account notices.** The same inbox also carries what happened to your own account, so a security message does not depend on a mailbox that may be filtered, forwarded or shared. Each one arrives as an entry with `state: "notice"` (it never fires or resolves - it is simply read), a `rule` of `security.*`, and a link to the page that acts on it; the *Account* filter on `/platform/dashboard/notifications` shows only these. Every one of them is sent by e-mail as well, under the `security` category that cannot be switched off. | Notice | When | |---|---| | `security.new_sign_in` | The account was opened on a browser and system it has not been seen on for 30 days. Carries the device, the method, the address and the time on your own clock. | | `security.password_changed` / `security.password_reset` | The password was changed from the account page, or a reset link was used. Every other browser is signed out. | | `security.two_factor_on` / `security.two_factor_off` | The second factor was switched on or off by you. | | `security.two_factor_reset` | An **operator** reset the second factor (the usual reason is a lost phone) and signed every browser out. | | `security.sessions_revoked` | An **operator** signed the account out of every browser. | | `security.sign_in_attempts` | The per-address password limit tripped - wrong passwords in a row. Nobody got in. | | `security.sign_in_method_added` / `security.sign_in_method_removed` | An identity provider was added to, or removed from, the ways into the account. | | `security.account_disabled` / `security.account_enabled` | An **operator** closed or re-opened the account. | A notice belongs to one person: other members of the organization never see it, it follows its owner from one workspace to the next, and it is not in the operator console's deployment inbox. Notices are not delivered to channels either - a workspace Slack room is not the place for one person's sign-in. **Workspace notices.** The same mechanism records what happened to the *workspace*, under `workspace.*` rules and `scope: "workspace"`, so the people who live in the dashboard see it even though only the owners are mailed. Every member of the workspace sees these; they stay in the workspace they belong to. | Notice | When | |---|---| | `workspace.key_created` / `workspace.key_revoked` | An API key was minted or revoked, with who did it and the key's prefix. Revocation is a `warning` - anything still signing with it now gets `401`. | | `workspace.member_joined` | An invitation was accepted, with the role taken. | | `workspace.member_role` | A member's role changed, from what to what and by whom. | | `workspace.member_removed` | A member was taken off the workspace and their sessions here ended (`warning`). | | `workspace.plan_changed` | The plan moved - upgrade, downgrade, cancellation or trial end - with the reason. | | Endpoint | Purpose | |---|---| | `GET /api/v1/notifications?state=firing&unread=true&limit=100` | The workspace inbox (`items`) and unread counts per severity (`unread`). `state` is `firing`, `resolved` or `notice`; the reader's own account notices are included whichever workspace they are in. | | `GET /api/v1/notifications/unread` | Unread counts only - the dashboard bell polls this. | | `POST /api/v1/notifications/read` | `{"ids": [...]}` marks those read, `{"ids": null}` marks everything read. | | `GET|POST /api/v1/notifications/channels` | List channels (with the rule names you can filter on) or create one: `kind`, `name`, `target`, optional `secret`, `min_severity`, `rules`, `enabled`. | | `PATCH|DELETE /api/v1/notifications/channels/{id}` | Change or remove a channel. | | `POST /api/v1/notifications/channels/{id}/test` | Deliver a test notification and return `ok`, `status`, `error`. | | `GET /api/v1/notifications/deliveries` | The last delivery attempts (channel, event, attempt, status, error). | | `GET|POST /api/v1/admin/notifications` and `.../channels`, `.../deliveries` | The operator (deployment scope) equivalents; `POST /api/v1/admin/notifications/evaluate` runs one round now and returns what it opened, resolved, delivered and failed. | **Platform events.** Besides alerts, a `webhook`, `slack` or `teams` channel can subscribe to what happens in the workspace: `events` on the channel is a list of event names or patterns (`"*"`, `"key.*"`), validated against `event_types` in the channel listing. The event types are `request.completed` / `request.failed` (one per metered request - high volume), `outcome.reported`, `handoff.pending` / `handoff.resolved` / `handoff.cancelled`, `key.created` / `key.revoked` / `key.rotated` / `key.scoped` / `key.limited`, `member.invited` / `member.joined` / `member.left` / `member.removed` / `member.role`, `team.created` / `team.updated` / `team.deleted` / `team.member_added` / `team.member_removed`, `plan.changed`, `billing.invoice_paid` / `billing.payment_failed` (as Stripe confirms the subscription invoice), `account.deleted`, `support.opened` / `support.answered` / `support.replied` / `support.closed`, `governance.report` and `workspace.reports`. Every event is something the platform recorded - it is queued in the outbox on the request path (one insert) and delivered by the next evaluation round with the same envelope and `X-OSR-Event` / `X-OSR-Delivery` / `X-OSR-Signature` headers as alerts; `GET /api/v1/notifications/deliveries` shows the attempts. The same publications reach the Kafka topics below when a broker is configured, so an external consumer sees the deployment-wide stream and a workspace only its own events. ### Shared state and replicas The platform keeps its state in PostgreSQL: `OSR_PLATFORM_DATABASE_URL` is required, and every API replica and domain service connects to the same database. The data volume only holds what is local to a process (learner state, the audit chain, the SLM bundle, the autopilot cache). The hash-chained audit trail has one chain **per process**: the API appends to `audit.jsonl`, each domain service to `audit..jsonl` (`audit.routing.jsonl`, `audit.openai.jsonl`, `audit.mcp.jsonl`), because a hash chain has exactly one writer; `/api/v1/audit`, the trace view and the governance status verify every chain on the volume and merge the records (`chains` counts them, `problem` names the broken file). On a clustered deployment (Redis set, several replicas of one role) each replica writes its own file, `audit...jsonl`, because a file lock on a network share (Azure Files mounted `nobrl`, NFS) is not honoured across hosts and two replicas appending to one file interleave and break the chain; the readers merge all of them and the go-live checklist (*Audit trail*) shows whether every chain verifies. On a single-host deployment two processes of one role share a chain file safely: every append takes an exclusive file lock and continues from the last line on disk, so a second writer extends the chain instead of forking it. A chain that no longer verifies (a line removed, a record edited, a fork written before 1.8.1) is a critical deployment alert (`audit_chain_broken`) and every workspace's compliance posture shows it; a superadmin resolves it from the console's *Health* page (`POST /api/v1/admin/audit/rotate` with a reason): the broken file is kept beside the new one as `..archived`, never edited, and the new chain opens with a `chain.rotated` record naming the archive, the problem, the reason and the operator - the rotation is itself on the record and in the operator audit log. Spans and events are persisted by every process that routes or executes (routing, openai, mcp) and read by the API, so a chat completion served by the OpenAI service is traceable at `/api/v1/trace/{request_id}` like a `/route` decision. A deployment that runs several replicas moves the remaining shared parts to services that are themselves containers - no managed cloud database, cache or broker is required, and the same images run on a laptop, a Kubernetes cluster or Azure Container Apps: | State | Service | Setting | |---|---|---| | Accounts, users, keys, tenants, policies, usage, feedback, providers, marketplace, notifications | PostgreSQL | \OSR_PLATFORM_DATABASE_URL\ (required) | | Spans and events (logs) - kept out of the main database so log volume never contends with it | PostgreSQL (own database, created on first start) | \OSR_PLATFORM_TELEMETRY_DATABASE_URL\ | | Rate-limit windows, learner state (enterprise: every replica learns read-modify-write into one shared state and adopts what the others learned every `OSR_PLATFORM_LEARNER_REFRESH_S` seconds, default 30), `/v1/responses` conversation state | Redis | `OSR_PLATFORM_REDIS_URL` | | Platform events as JSON topics `osr.`: `usage` (one per metered request), `feedback`, `training` (prompt text for the SLM service, only with `OSR_PLATFORM_TRAINING_EVENTS`), `admin` (account lifecycle and operator actions), `alerts`, `billing` (plan changes, invoices), `handoffs`, `support`, `sales` | Kafka (any Kafka-protocol broker) | `OSR_PLATFORM_KAFKA_BOOTSTRAP`, `OSR_PLATFORM_KAFKA_TOPIC_PREFIX` | | Photographs operators upload for the public site (`/platform/admin/photos`): the bytes, under a content-addressed name with an immutable cache header, in a container with public read; the database keeps only the address. Without a container the bytes stay in the database and the API serves them. | Azure Blob Storage (REST, no SDK) | `OSR_PLATFORM_SITE_PHOTOS_URL` (the container URL; only an `https://` address on `*.blob.core.windows.net` or a sovereign cloud's blob endpoint is accepted, anything else is ignored - the start-up check and the console's Photographs page both say so, and the start-up check also warns when a container has neither a SAS nor a managed identity to sign with), signed with either `OSR_PLATFORM_SITE_PHOTOS_SAS` (a container SAS with read, write and delete) or, on Azure, the container app's managed identity - `OSR_PLATFORM_AZURE_CLIENT_ID` names the user-assigned identity, which needs *Storage Blob Data Contributor* on the account; the Azure template sets all of this up | `GET /api/v1/admin/overview` -> `storage` says which mode a deployment runs (`{database, clustered, redis, events, topics}`; `clustered` is true once Redis shares the rate limits, `topics` is the full list above as the broker sees it) and the operator console shows it on `/platform/admin`. The public `GET /api/v1/info` never names where state lives: that is an operator fact. Event payloads carry the same fields as the activity log and never the prompt text. The topic list is one constant (`cluster.TOPICS`): the Azure template creates exactly these event hubs for a managed broker (Event Hubs does not auto-create topics), a self-hosted broker creates them on first use, and a test fails when code publishes on a topic that is not in the list. An authenticated broker - Event Hubs, Confluent Cloud, any SASL or TLS endpoint - is reached with `OSR_PLATFORM_KAFKA_SECURITY_PROTOCOL` (`SASL_SSL`, `SASL_PLAINTEXT`, `SSL`), `OSR_PLATFORM_KAFKA_SASL_MECHANISM` (`PLAIN`, `SCRAM-SHA-256`, `SCRAM-SHA-512`) and the `OSR_PLATFORM_KAFKA_SASL_USERNAME` / `OSR_PLATFORM_KAFKA_SASL_PASSWORD` pair; for Event Hubs that is the literal user `$ConnectionString` and the namespace connection string. Such a gateway hangs up on a connection left idle for a few minutes - and so does the network in front of a self-hosted broker (the internal ingress of Azure Container Apps after four) - while the driver would hold one for nine, so the clients, whatever the protocol, release theirs first after `OSR_PLATFORM_KAFKA_CONNECTIONS_MAX_IDLE_MS` (default 180000, three minutes; `0` leaves the question to the driver). Without it a quiet topic finds the socket already gone on its next read: no event is lost - the producer retries and the consumer rebuilds - but the log fills with `Connection lost: socket disconnected` and `ConnectionResetError: [Errno 104] Connection reset by peer` under *Error receiving network data*, once per idle connection, every few minutes in every service. Both shared services degrade rather than stop the API. When Redis is unreachable the rate limiter answers from its per-replica window and stays off Redis for a five-second cool-down before trying again (`/readyz` `redis` shows `ok`, `latency_ms` or the error); the shared learners keep learning in memory - a refresh that cannot read Redis leaves the replica's state untouched instead of blanking it, and the next successful save catches up. A request never waits for Kafka: `emit` only enqueues, one sender thread per process hands the events to the producer (queue bounded at 10 000 events; beyond that the newest are dropped and counted as `events.overflow`). Events are acknowledged by every in-sync replica (`acks=all`), retried with back-off and dropped - counted and reported as `events.dropped` / `queued` / `last_error` - only when the broker stays unreachable. The SLM service consumes `training` and `feedback` with manual offset commits after each ingested batch (at-least-once: a record is never committed before it was processed) and reports the consumer group, commits, errors and per-topic lag under `consumer` in `GET /model`. The self-hosted Compose stack starts PostgreSQL with the platform; the Azure template provisions it as a container app with internal TCP ingress next to the API and web app. A deployment that predates 1.1 kept its state in a `platform.sqlite3` file on the data volume. Import it once with `osr-platform-import-sqlite /data/platform.sqlite3 --database-url postgresql://...` (ships with the platform API; run it inside the API container - safe to re-run; rows already in PostgreSQL are left alone), then remove the file. In that stack the routing SLM is its own service too: the API publishes each routed prompt to the `training` topic (opt-in, `OSR_PLATFORM_TRAINING_EVENTS`) together with the decision it took and the ranked propensities of the top candidates (`target_id`, `ranked`); the SLM service remembers prompts, decisions and workspace in `autopilot/requests.jsonl` on the shared volume - a restart or a deploy loses nothing - pairs prompts with the outcomes reported through `POST /api/v1/feedback` (feedback rows) and distils the ensemble's ranking on every routed prompt that has no outcome yet (decision rows, newest `OSR_SLM_DECISION_ROWS_MAX`), trains a challenger on a schedule, keeps it only when it beats the champion on a held-out split, and writes the promotion to the shared volume from which every API replica reloads it (a timer every `OSR_PLATFORM_SLM_RELOAD_S` seconds in every process that routes, idle replicas included). Synthetic seed rows are added only while real rows are scarce (`OSR_SLM_SYNTHETIC_PER_TEMPLATE` rows per prompt template - 8 in the shipped stacks, about 500 rows and a 100-row holdout, so the gate can tell two architectures apart before real traffic arrives); `GET /model` reports `prompts_remembered`, `decisions_remembered`, `outcomes` and the cycle's `sources` (feedback, decisions, datasets, synthetic). The trainer builds every challenger with the query encoder `OSR_SLM_ENCODER` names (`attention` in the shipped stacks - a transformer block over the token sequence on top of the hashed features, about five milliseconds per prediction; `hashed` for the bag-of-features encoder alone): when it differs from the champion's the challenger is a fresh model with the configured architecture and the gate decides, so the served model changes encoder only where it wins the holdout; `GET /model` shows `encoder`. `OSR_SLM_EMBEDDER` adds a real embedding model to every challenger as frozen dense features (`openai:@`, any OpenAI-compatible `/embeddings`): the Azure stack points it at the embedding model the deployment already serves for `embed-small` (its model server's `/v1`, nothing leaves the environment; `azd env set OSR_SLM_EMBEDDER none` keeps the hashed and attention features only, any other value names an endpoint of your own), Compose takes it from the same variable (`openai:qwen3-embedding:4b@http://ollama:11434/v1` with the `llm` profile). The SLM then learns its head on what a sentence transformer knows a prompt means, while the hashed features keep the surface cues; the promoted file names the embedder, so every API replica rebuilds the same client on reload (`OSR_SLM_EMBEDDER_TIMEOUT_S` 5 s in the API, 120 s for the trainer's batches) and a request the endpoint cannot embed in time is scored by the other strategies alone. Both start from the warm-start model the API image ships at `/app/models/slm.json` (`OSR_PLATFORM_SLM`) - the attention model bundled inside the SDK package, trained on the example catalogue, its labelled dataset and follow-up conversations and measured on a held-out seed - so a fresh deployment routes with a learned strategy from the first request and the first promotion has a champion to beat. Operators see the model in service, the evidence ingested, every cycle and a prompt probe on `/platform/admin/slm` (`GET /api/v1/admin/slm`, `/slm/reports`, `POST /slm/cycle`, `/slm/predict`). **The labelled floor.** The holdout a promotion is judged on is self-agreement: the router's own decisions on the deployment's traffic. On a deployment whose traffic is one kind of job (production's was its own blog and SEO writers) the served model promoted itself twenty times on holdouts of 0.92-0.97 and routed 23 of the 70 hand-labelled reference rows acceptably - and, heard in the ensemble, pulled it from 66 to 39 of 70. The SDK package now ships that labelled set next to the bundled model, and every promotion is held to it (`OSR_SLM_LABELLED_FLOOR`, on): a challenger that routes fewer of the applicable labelled rows acceptably than the champion, by more than `OSR_SLM_LABELLED_TOLERANCE` (5 %) of them, is refused whatever its holdout says, and the report carries both scores under `labelled`. At start the service holds the champion it loads from the volume to the same floor against the packaged model and adopts the packaged model when the champion fell below it (the replaced model stays as `slm.prev.json`; `GET /model` -> `floor` says what happened). Rows count only when the deployment's catalogue holds their targets; with fewer than `OSR_SLM_LABELLED_MIN_ROWS` (20) such rows the floor is off, so a catalogue of your own is judged by its own evidence alone. **Rehearsal.** A floor alone would leave the loop stuck: a challenger fitted on one workspace's traffic learns that traffic perfectly (31 holdout wins, 0 losses) and forgets the reference set (44 of 70), so the floor refuses it - every cycle, for ever, and nothing is ever promoted. The applicable labelled rows therefore also *train* every challenger, replicated until they are about `OSR_SLM_LABELLED_REHEARSAL` (0.25) of its training rows (0 = off); the holdout stays the deployment's own rows. The same challenger then keeps 69-70 of the 70 reference rows while it learns the traffic, and is promoted; the report's `sources` count the rows under `labelled`. The same cycle trains the model's **slot models**: the outcomes credited to a plan's persona, skill and tool slots (`role` on the `feedback` topic - the platform's own assessment of every composed answer carries one per companion, `POST /api/v1/feedback` takes one) become one small encoder per role, saved in the same file and promoted with the primary, so the ensemble's SLM also votes on which persona or skill a request wants - a marketplace listing that gathered a record included. A role trains from `OSR_SLM_SLOT_MIN_ROWS` (20) rows on; each cycle report lists the roles under `slots` (rows, accuracy, skill, or why the role was skipped) and `GET /api/v1/learning` shows the served slots under `slm.slots`. Workspaces do not all route alike, so the service also trains **per-workspace models**: once a workspace has `OSR_PLATFORM_SLM_WORKSPACE_MIN_ROWS` (100; `0` = off) rows of its own - outcomes joined to prompts through the `training` topic plus its routed prompts' decision rows, attributed by `account_id` - each shared cycle also trains a challenger for it, starting from the shared champion and seeing only that workspace's rows. It is promoted to `autopilot/slm..json` under the same gates (a ten-row holdout at least, a significant win over the model it replaces - the shared champion the first time), and the API replicas pick the file up with the shared one and serve it for that workspace's requests only; everyone else keeps the shared model. Each of these models is a multi-megabyte bundle every replica holds in memory and every cycle trains in series, so a deployment keeps at most `OSR_PLATFORM_SLM_WORKSPACE_MAX_MODELS` (20; `0` = no cap) of them - the workspaces with the most evidence. `GET /api/v1/learning` lists them under `workspace_slms`, the service's `/model` under `workspaces`, and `POST /predict` takes an `account_id` to rank with a workspace's model. ### Services The API is one image that can run as one process or as several: the documentation JSON (`docs`), the public model catalogue and rankings (`rankings`), the marketplace (`marketplace`), provider management (`providers`), signup and account links (`onboarding`), identity and workspaces (`accounts`: sign-in, sessions, organizations, keys, tenants, policy, governance, usage), Stripe billing (`billing`), the operator console API (`admin`), the MCP endpoint (`mcp`), the OpenAI-compatible proxy (`openai`), the routing core with its trace, event and learning reads (`routing`) and the routing SLM (`slm`) each have their own entry point and port. `/api/v1/info`, `/api/v1/status` and `/api/v1/estimate` always stay with the API. The API remains the single public origin and forwards a domain's paths to its service when `OSR_PLATFORM__URL` is set (streamed completions are relayed as they arrive; a service that stays silent longer than `OSR_PLATFORM_PROXY_TIMEOUT_S`, default 300 s - a cold self-hosted model loading its weights - answers `503`); the answer carries `X-OSR-Service` naming the process that served it. The topology is an operator fact: the public `GET /api/v1/info` does not list it; operators see the map and live health per service on `/platform/admin/services` (`GET /api/v1/admin/services`). The Compose stack and the Azure template deploy every service as its own container next to the gateway. ### Traces and events The router records every request as a tree of **spans** (`request`, `route`, `plan`, `execute`) with **events** inside them (`route.signals`, `route.policy`, `route.rank`, `route.fallback`, `guard.*`, `execute.step`, `learn.outcome`, `cache.hit`, ...). The newest ones stay in an in-memory buffer (1000 by default, `OSR_OBSERVABILITY_MEMORY_EVENTS`) and every one of them is also written to the platform database by the platform's own telemetry store (`OSR_PLATFORM_TELEMETRY_STORE`, on by default) - off the request path, from a background writer, so tracing never slows routing. Events and traces therefore survive restarts, can be read by time range and are purged after `OSR_PLATFORM_TELEMETRY_RETENTION_DAYS` (14). They are readable per workspace: | Endpoint | Returns | |---|---| | `GET /api/v1/trace/{request_id}` | The activity row plus every span and event of one request, in time order, with `trace_id`, `spans` and the root `duration_ms`; `outcomes` (the feedback reported for the request), `audit` (its hash-chained records, plans with `audit`) and `payload` (the stored prompt and answer when the workspace's `log_payloads` policy kept them, else `null`). `source` says whether the rows came from the buffer or the store. `404` when the request is not the workspace's; `events: []` once the request is older than the telemetry retention. | | `DELETE /api/v1/activity/logs` | Erases every prompt and answer stored under the `log_payloads` policy (admin role); activity rows, traces and outcomes stay. Returns `deleted` and the `logging` mode still in force. | | `GET /api/v1/events?name=route.*&kind=&level=&request_id=&since=&until=&limit=200` | Events of the workspace's requests (oldest first, `limit` up to 2000; `name` may end with `*`; `since` / `until` are epoch seconds, default the last 6 hours). Deployment-wide events without a request id (breaker transitions, autopilot) are included on plans with `stats`. `source`, `persisted`, `retention_start` and `retention_days` describe the store. `404` when tracing is off entirely. | | `GET /api/v1/events/stream?name=&kind=&level=&request_id=&max_s=3600` | The same events pushed as they are recorded, as server-sent events (`text/event-stream`): one `event: ` / `data: {…}` frame per event, a `: ping` comment every 15 seconds while quiet, and the stream ends after `max_s` seconds (default and ceiling one hour) - reconnect to continue. Every replica reads the shared store, so the stream sees every replica's events. Same filters and `404` as the list endpoint. | | `GET /api/v1/status` | Public readiness: the `/readyz` checks plus edition, versions, uptime, controls and tracing state; `503` while a check fails. `Cache-Control: no-store`. The public status page of the site (`/status`) polls it every 15 seconds underneath the board below, so a browser that cannot reach the API says so at once. | | `GET /api/v1/status/board` | The public status page (`/status`): `overall` (`operational`, `degraded`, `partial_outage`, `major_outage`, or `unknown` when the platform's own checks have not run for three intervals - stale checks are never shown as operational), `monitor` (`enabled`, `last_sample_at`, `stale`, the last pass's `error`), every **component** the platform samples itself - the API's readiness, the database, the cache, the event bus, mail delivery, each remote domain service, the web app and every enabled model provider - with its last check (`status`, `latency_ms`, `checked_at`, `detail`), `uptime_30d_pct` / `uptime_90d_pct` and the per-day `history` (checks, failures, latency), the `incidents` (`open`, `recent` = the last 30 days) with their updates, and `served` / `served_90d`: metered requests per UTC day and the share the platform failed with a 5xx. One replica samples every `OSR_PLATFORM_STATUS_SAMPLE_S` seconds (default 60; 0 = off) and keeps `OSR_PLATFORM_STATUS_HISTORY_DAYS` (default 90) of roll-up; raw samples live a week. The web app is checked at `OSR_PLATFORM_WEB_URL` - or at `OSR_PLATFORM_WEB_PROBE_URL` when the public address does not resolve from the API's own network (compose sets `http://web:3000`). A core component (`api`, `database`) or a domain service that fails three checks in a row **opens an incident on its own** (`automatic: true`, severity `critical` for a core component) and resolves it after three healthy checks; a provider down degrades the banner but never opens one - the router fails over. Cached 10 s (`ETag`). | | `GET /api/v1/status/incidents?days=90&limit=50`, `GET /api/v1/status/incidents/{id}` | Incidents opened or active in the window, newest first, each with `severity` (`minor` / `major` / `critical`), `status` (`investigating` / `identified` / `monitoring` / `resolved`), `components`, `started_at`, `resolved_at` and every update. | | `GET /api/v1/status/feed` | RSS 2.0 of incident updates (one item per update) - what "subscribe to status updates" points at. | | `GET /api/v1/services` | What this deployment offers as a service, computed from its state at the time of the call (no key): every first-party capability - routing, the OpenAI-compatible proxy, quotes, plans, learning, the routing SLM, your own providers, human hand-off, the MCP server, traces, savings, governance, tenants and audit, alerts, usage digests, support, the marketplace, the model catalogue, the blog, the newsletter, single sign-on, billing, the CLI, self-hosting, the browser extension and desktop app - with `status` (`live` works now, `limited` runs with a stated restriction such as no model provider or no mail transport, `off` not on this deployment or edition), `detail` (the reason), `endpoint` (`METHOD /path`, always a real route), `access` (`public` / `key` / `operator`), `plan` (the cheapest plan that includes it), `docs`, `page` and `facts` (small live figures from the same stores the dashboards read). The site renders it at `/services`; the welcome mail and every newsletter issue link to it. Cached for a minute (`ETag`). Nothing about the platform's own editorial use of the router is included. | | `GET /api/v1/facts` | The public numbers of this deployment, each section carrying its **class** so no page can put a reference-catalogue count, an imported index and a live counter in one sentence by accident (no key): `version` (platform, SDK and the build label), `catalogue` (class `catalogue`: the reference model index - `models`, `vendors` - indexed, not necessarily executable here), `storefront` (class `live`: `models` sold by the token, `endpoints`, `executable_models`), `deployment` (class `live`: `targets`, `executable_targets`, `kinds`, `providers`, `vendors`, whether the routing `slm` is loaded and the `autopilot` runs), `activity` (class `live`: lifetime `requests`, `requests_24h`, `workspaces`, `workspaces_routing`, `executed_requests`, the measured `saved_usd` / `saved_pct` against `baseline`) and `marketplace` (class `imported`: `indexed` listings and `publishers_indexed` in total, `installs`, `reviews`, and `native` / `imported` / `sources` once the manifest scan has run - `null` before). `classes` spells each class out in the words the site prints next to the number. The site renders it at `/whats-live` together with the services catalogue and the provider table. Cached for a minute (`ETag`). | `POST /api/v1/route` responses include `trace_id` and the `X-OSR-Trace-Id` header when the request was traced, and `GET /api/v1/activity` marks rows whose spans are kept (in memory or in the store) with `traced: true` and summarises the feedback reported for each request in `outcome` (`POST /api/v1/feedback` is stored per request, so the loop from decision to result is visible per row). A caller's **W3C `traceparent`** header is honoured: the request's spans join the caller's trace (`X-OSR-Trace-Id` returns that trace id), and a well-formed client `X-Request-Id` (8-64 characters of `A-Za-z0-9._:-`, not yet used by the workspace) becomes the decision's `request_id` - what `/feedback` and `/trace/{request_id}` take. The TypeScript SDK, the browser extension and the desktop app send both on every call by default (`trace: false` on the SDK client turns it off), so every request of every client is one traceable row; the extension's side panel and the desktop's answer line link *answered by* to the request's trace on the dashboard. Attributes hold ids, numbers and short labels; the request text is represented only by its digest and length. The dashboard shows the history at `/platform/dashboard/events` (window of 1 hour to 7 days, search by request or trace id), opens a trace from any activity row (with a *How it was routed* story: what the request looked like, who was allowed to answer, how the candidates ranked with their expected quality, price and latency, *why* the winner beat the runner-up strategy by strategy with each vote's rationale, the plan slots filled or skipped, the decision and the execution steps with tokens and cost) and adds a *Trace* tab to the playground. `GET /api/v1/trace/{request_id}` also returns `usage`: tokens (prompt / completion), the metered cost, the target's list price and what the same tokens would have cost on the savings baseline (the dearest model the workspace itself ran in the last 30 days, or the catalogue's dearest when it ran none - the same rule as `GET /api/v1/savings`; `usage.baseline.source` says which), so every request shows what it saved. The deployment-wide figures the site shows (`GET /api/v1/stats/public`: `requests`, `tokens`, `cost_usd`, `success_rate`, `top_targets` and the `savings` block, plus `internal` for the deployment's own jobs) are read from a lifetime ledger (`platform_counters`, one row per target and scope) that every metered request adds to and nothing subtracts from: the retention sweep, a workspace's own `retention_days` and a deleted workspace remove request rows, never what has already been counted. `since` is the oldest counted request. The ledger seeds itself from the request rows a deployment still has the first time it is read, so an upgrade starts from today's totals rather than from zero. Only the windows (`requests_24h`, `tokens_24h`, the latency figures) and `accounts_routing` follow the rows that exist. Public catalogue reads (`/api/v1/info`, `/api/v1/models*`, `/api/v1/rankings`, `/api/v1/llms*`, `/api/v1/catalogue`, `/api/v1/stats/public`) return an `ETag` and `Cache-Control: public, max-age=60, stale-while-revalidate=60`; send `If-None-Match` to get `304` when nothing changed. Operators tune this and the decision cache with: | Setting | Default | Effect | |---|---|---| | `OSR_PLATFORM_METRICS_PUBLIC` | `true` | `false` restricts `/metrics` to requests carrying `X-Admin-Token`. | | `OSR_PLATFORM_TRUSTED_PROXIES` | `private` | Which `X-Forwarded-For` hops are the deployment's own proxies (ingress, web proxy, gateway) when working out a caller's address for per-IP limits and audit records: `private` skips every RFC 1918, loopback and link-local hop, `none` uses the TCP peer only, or list CIDRs. Entries a client wrote itself are never trusted. | | `OSR_PLATFORM_GEOIP_HEADER`, `OSR_PLATFORM_GEOIP_SERVICE`, `OSR_PLATFORM_GEOIP_TOKEN` | unset | Where sign-ins come from: a request header the ingress sets with the country code, and / or an IP geolocation service (a preset or a URL template with `{ip}`) with its key. See [Sign-in locations and alerts](#sign-in-locations-and-alerts). | | `OSR_PLATFORM_NETWORK_DATA_RETENTION_DAYS` | `365` | Days the address and the place of sessions and account events are kept before the retention sweep removes them (the events stay); `0` keeps them as long as the row. | | `OSR_PLATFORM_EVAL_GATES_INTERVAL_S` | `21600` | How often every workspace's quality gates are routed again and judged (six hours); `0` = only on request and after a policy change. See [Evaluation](#evaluation). | | `OSR_PLATFORM_HTTP_CACHE_MAX_AGE_S` | `60` | `Cache-Control` lifetime of public reads; `0` sends `no-store`. | | `OSR_PLATFORM_STATS_ROLLUP_S` | `120` | How often the public aggregates that scan the request table (lifetime latency, workspaces routing, per-target traffic of the week) are recomputed; every replica and service reads the one stored copy, a stale copy is served while one process refreshes it. `0` computes on every read. | | `OSR_PLATFORM_ROUTE_CACHE_TTL_S` | `0` | Reuse a routing decision for identical requests within this window (`0` = off). | | `OSR_PLATFORM_CHAT_TOOL_MIN_CONFIDENCE` | `0.5` | A built-in tool (calculator, web search) answers `/v1/chat/completions` only when the router is at least this confident it fits the request; below it the chat is routed again without tools, and a tool is never the fallback for a failed model. When every runnable model's breaker is open the chat answers `503` with `Retry-After`. `0` lets tools compete like any target. | | `OSR_PLATFORM_FAIR_SHARE` | `false` | Enterprise: dominant-resource fair share across workspaces and tenants (`_WINDOW_S` 300, `_SOFT` 1.5, `_HARD` 3 tune it; section 8). | | `OSR_PLATFORM_OUTPUT_GUARD` | `flag` | Response-side guard on every model / agent answer (`opensmartroute.security.OutputGuard`): `flag` records leaked secrets (API keys, private keys, JWTs), personal data and instruction injection in the trace and under the `output_guard` entry of the answer's metadata block; `redact` also replaces secrets and PII with placeholders; `block` refuses the answer (`400`); `off` disables it. Under `redact` / `block` a streamed answer is buffered and screened before its first token leaves. | | `OSR_PLATFORM_RETENTION_DAYS` | `365` | Usage rows older than this are purged every six hours; `0` keeps everything. | | `OSR_PLATFORM_REQUIRE_EMAIL_VERIFICATION` | `true` | Hold every account at the confirmation step until the signed-in user - or, for an API key, an owner of the workspace - has confirmed the e-mail address (`403`, `X-Email-Verification: required`, everywhere but `/me`, `/onboarding`, `/auth/*`, `/legal`). `false` only on a stack that is thrown away afterwards. | | `OSR_PLATFORM_SIGNUP_TEST_DOMAINS` | `false` | Accept signups from domains reserved for testing (`example.com`, `*.test`, `*.local`, ...). The smoke, load and e2e suites need it on their throw-away stacks; production keeps it off so test users never land in the customer database - the operator's own doors (`POST /api/v1/admin/users`, `POST /api/v1/admin/accounts`, members added by e-mail) refuse those domains too, and the retention sweep removes anything on such a domain that still got in (audited as `test_data.purged`, actor `system`). Operators list what a run left behind with `GET /api/v1/admin/test-data` and remove it at once with `DELETE /api/v1/admin/test-data` (superadmin, audited). | | `OSR_PLATFORM_REFERRAL_ENABLED` | `true` | Run the referral programme: a workspace's link earns both sides credits (section 6, *Referrals*). | | `OSR_PLATFORM_REFERRAL_REWARD_CENTS` | `500` | Credits (cents of the wallet) the referrer earns per qualified referral. | | `OSR_PLATFORM_REFERRAL_WELCOME_CENTS` | `500` | Credits the invited workspace starts with once it qualifies. | | `OSR_PLATFORM_REFERRAL_MAX_REWARDS` | `20` | Rewards one referrer may earn; `0` = no cap. | | `OSR_PLATFORM_CAPTCHA_PROVIDER` | `off` | A bot challenge on the sign-up form: `turnstile` (Cloudflare) or `hcaptcha`. The form renders the provider's widget and the API verifies its token with the provider before anything else runs; a missing or failed token is refused (`400`, `X-Signup-Refused: captcha`). Needs the two keys below. | | `OSR_PLATFORM_CAPTCHA_SITE_KEY` | - | The provider's public site key (published on `GET /api/v1/info` as `captcha`). | | `OSR_PLATFORM_CAPTCHA_SECRET` | - | The provider's secret the API verifies tokens with; never leaves the API. | | `OSR_PLATFORM_UNVERIFIED_ACCOUNT_TTL_DAYS` | `14` | A user who never confirmed the address is removed after this many days, with the personal workspace only they belonged to (the retention sweep); `0` keeps them. The platform's own `*.local` service workspaces are exempt. | | `OSR_PLATFORM_REQUEST_LOG_MAX_CHARS` | `20000` | Longest prompt / answer kept per request under a `log_payloads` policy (longer texts are cut and marked `truncated`). | | `OSR_PLATFORM_TELEMETRY_STORE` | `true` | Persist every span and event to the platform database (the events page, traces and `since` / `until` queries outlive the in-memory buffer). | | `OSR_PLATFORM_TELEMETRY_DATABASE_URL` | unset | A separate PostgreSQL database for the telemetry store (e.g. `postgresql://osr:pw@host:5432/osr_telemetry`), so span/event volume never contends with the main database; created on first start when missing. Unset keeps them in the main database. | | `OSR_PLATFORM_TELEMETRY_RETENTION_DAYS` | `14` | Persisted spans and events older than this are purged with the same sweep; `0` keeps them. | | `OSR_PLATFORM_LOG_EXPORT_TICK_S` | `30` | How often the accounts owner looks for request-log export destinations whose interval has passed and ships their next batch (one replica per tick); `0` = only on *Export now*. | ### Request-log export to your own storage The usage rows and the kept payloads stay queryable through the dashboard and the API; a team that runs its own data lake or SIEM wants the same rows where its other logs are, continuously, without polling `GET /api/v1/activity`. A **log export** does that: `POST /api/v1/workspace/log-export` names a destination and the platform writes every new row there in batches of up to 2 000, on the destination's own `interval_s` (60 s and up), with a cursor so nothing is sent twice and nothing is skipped when a batch fails (the error is recorded, the cursor stays, the next pass retries). Destinations: `azure_blob` - a container URL (`https://.blob.core.windows.net/`) with a container SAS as `credential` (or the API's managed identity when it is empty), one JSONL blob per batch under `/YYYY/MM/DD/