# Install the platform The **hosted platform** at [opensmartroute.ai](https://opensmartroute.ai) needs no installation - sign up, create a key, follow the [platform quickstart](https://osr-web.kindrock-caaab1a2.centralindia.azurecontainerapps.io/docs/QUICKSTART_PLATFORM.md). This page is for running the **whole platform yourself**: the routing API with its OpenAI-compatible endpoint, the web app (public site, dashboard, operator console), the routing SLM service and the state behind them. The same images and the same `OSR_PLATFORM_*` settings run everywhere; only the way they are started differs. | Where | How | For | |---|---|---| | Azure, from the portal | [Azure Marketplace](#azure-marketplace) - the solution template with a wizard, one resource group, no tooling (the same template deploys from the published package until the listing is live) | Teams that want the platform in their own subscription | | Your own servers or any cloud VM | [Docker Compose](#your-own-servers-docker-compose) - PostgreSQL, Redis, Kafka and the platform as containers, from the published Compose bundle | On-premises, air-gapped, other clouds | | Kubernetes | [`osr serve` Helm chart](https://osr-web.kindrock-caaab1a2.centralindia.azurecontainerapps.io/docs/deploy.md) for the router alone; the platform on Kubernetes runs the same containers as Compose | Sidecar / control-plane routing without the hosted-platform surfaces | Every file named on this page - the Compose bundle, the Helm chart, the Marketplace package and its two files, the checksums of the PyPI release - is on the **[downloads page](https://opensmartroute.ai/downloads)** with its version and SHA-256; the stable links `https://opensmartroute.ai/download/` always resolve to the current release and `https://opensmartroute.ai/download/latest.json` lists them for scripts. The source repository is private; nothing here needs it. Just the Python library or the `osr` command line: [README - Install](https://osr-web.kindrock-caaab1a2.centralindia.azurecontainerapps.io/docs/readme.md#install). Calling the platform from TypeScript or JavaScript: `npm install @opensmartroute/sdk` ([TypeScript SDK](https://osr-web.kindrock-caaab1a2.centralindia.azurecontainerapps.io/docs/sdk-typescript.md)). ## What you get Every install path deploys the same product: - `osr-platform-api` - the FastAPI platform: `POST /api/v1/route`, the OpenAI-compatible `/v1` proxy, MCP server, accounts, keys, workspaces, tenants, governance, marketplace, billing, notifications, support tickets and the operator API. In cluster mode it also runs once per domain (docs, rankings, marketplace, providers, onboarding, accounts, billing, admin, mcp, openai, routing) behind the API gateway, plus the routing SLM service. - `osr-platform-web` - the Next.js web app: the public site and documentation, `/platform` (sign-in, dashboard, playground, publish flow) and `/platform/admin` (the operator console). - State: PostgreSQL (always), Redis and Kafka (cluster mode), a data volume for the audit chain and the routing SLM. - Model providers: open-weight models through Ollama by default - on Azure the template runs the curated set as its own container apps (CPU) or one serverless-GPU server and maps every tier and specialist target to it, with Ollama Cloud for the frontier open weights when you give it a key; Azure OpenAI deployments only when you switch them on (`azureOpenAiEnabled`). Anything OpenAI-compatible everywhere (connected in the console under *Providers*, or by workspaces with their own keys under *Model providers*). Release images: `crosrua2c6onyidguw.azurecr.io/osr-platform-api:2.1.0` and `crosrua2c6onyidguw.azurecr.io/osr-platform-web:2.1.0` (anonymous pull; `:latest` follows the newest release, `:` the newest patch). The web image is built without a baked hostname: it reads `OSR_PUBLIC_WEB_URL` at run time and the browser uses the origin it is talking to, so one image serves any domain. ## Azure Marketplace The offer **OpenSmartRoute platform** is an Azure Application (solution template): it deploys into a resource group of your subscription, you own every resource, nothing is shared with the publisher and there is no software charge on the Azure bill - you pay the Azure resources it creates. **Listing status.** The offer package is the Partner Center submission; until the listing is live the offer does not appear in a Marketplace search. The same template deploys today from the published package - `mainTemplate.json` and `createUiDefinition.json` from the [downloads page](https://opensmartroute.ai/downloads) - with the same wizard when you use the portal's *Deploy a custom template*, or from the command line: ```bash curl -fsSLO https://opensmartroute.ai/download/marketplace-template # opensmartroute-mainTemplate-.json az group create -n rg-osr -l centralindia az deployment group create -g rg-osr -f opensmartroute-mainTemplate-*.json \ -p adminUsername=admin adminPassword='' platformEdition=enterprise clusterMode=false ``` Every wizard field below is a parameter of that template (`az deployment group what-if` lists them with their descriptions); the outputs, first sign-in, custom-domain, upgrade and backup steps are identical. ### Deploy 1. In the Azure portal open *Marketplace*, search for **OpenSmartRoute**, choose the plan and *Create* (or *Deploy a custom template* -> *Build your own template in the editor* -> *Load file* with the downloaded `mainTemplate.json`, then *Edit UI definition* with `createUiDefinition.json`). 2. **Basics** - subscription, a new or existing resource group, the region (Container Apps must be offered there - and Azure OpenAI with the default `gpt-4.1` family when you switch those deployments on), a deployment name, the first **operator username and password** (the console sign-in), the edition (Enterprise: tenants, audit, statistics, persisted learners; Community: the lightest footprint). 3. **Size and state** - *Cluster mode* (Redis + Kafka, the API split into domain services scaled on demand; off = one API replica with PostgreSQL only, right for a trial) and *Managed state* (Azure Database for PostgreSQL with point-in-time restore, Azure Cache for Redis, Event Hubs - recommended for production; off = state containers on Azure Files, cheapest). 4. **Hostnames** - leave empty to use the Container Apps hostnames, or enter your web and API hostnames (and let the template create the Azure DNS zone). TLS certificates are issued in a second run once DNS resolves (below). 5. **E-mail** - Microsoft 365 (an Entra app registration with the application permission `Mail.Send`, sending as the From mailbox), Azure Communication Services or SMTP for verification, reset, invitation, alert, usage-report and support-ticket mail; the first transport set wins in that order. Without a transport every message waits in the console's *Mail* page; nothing is lost. 6. **Models and learning** - the self-hosted model server (Ollama, CPU or serverless GPU; on by default, it answers every tier), an Ollama Cloud key for the frontier open weights, Azure OpenAI deployments (off by default), the autopilot (retrain the routing model from your own feedback) and the marketplace catalogue refresh. 7. **Sign-in and access** - password sign-in, self-serve signups per IP (0 = operators create every workspace), SSO providers, a static admin token for scripts, public `/metrics`. 8. **Images and billing** - the release to deploy and the registry to pull from (yours, after mirroring), Stripe for self-serve paid plans. 9. *Review + create*. A cluster-mode deployment takes about ten minutes; the first Ollama start pulls about 16 GB of weights in the background. The deployment's **Outputs** tab lists `webUrl`, `platformUrl`, `adminLoginUrl`, `apiUrl`, the hostnames and static IP for your DNS records, the `asuid` verification id and, when enabled, the Azure OpenAI endpoint. ### First sign-in 1. Open `adminLoginUrl` (`/platform/admin/login`) and sign in with the operator username and password from Basics. The console shows the deployment's health, users, workspaces, plans, providers and the support desk. 2. *Providers*: the Ollama model server the template started is connected and every tier (`llm-small`, `llm-mid`, `llm-frontier`, `llm-onprem`) and specialist target already executes on it - Qwen3.5 4B on a CPU profile, the dedicated-node models (GLM-4.7-Flash, gpt-oss-20b) on a dedicated profile, gpt-oss-120b / DeepSeek-V4-Flash through Ollama Cloud when a key was given. Add OpenAI, Azure OpenAI, Anthropic, Bedrock, Vertex, vLLM or any OpenAI-compatible endpoint here, or let workspaces bring their own keys. 3. Open `webUrl`, *Sign up* (or create the workspace from the console when self-serve signup is off), create an API key and route a request - [platform quickstart](https://osr-web.kindrock-caaab1a2.centralindia.azurecontainerapps.io/docs/QUICKSTART_PLATFORM.md) with `OSR_API_URL=`. 4. `osr login --url ` signs the command line in to your deployment. ### Custom domain (two runs) 1. First run with your hostnames entered and *Issue TLS certificates* off: the apps bind the hostnames without TLS. 2. Create the DNS records from the outputs: `CNAME -> webHostname`, `CNAME -> apiHostname` (an apex domain gets `A -> staticIp`), and `TXT asuid. -> domainVerificationId` for each hostname. If the template created the zone, delegate the domain to `dnsNameServers` instead - the records are already there. 3. Run the deployment again (*Redeploy* on the deployment, same values) with *Issue TLS certificates* on. Azure issues free managed certificates and the hostnames answer over HTTPS. ### Upgrade, backup, remove - **Upgrade**: redeploy with a newer *OpenSmartRoute release* (image tag). The database schema migrates itself at start-up - the same numbered ledger, pre-upgrade backup on the data share (`backups/pre-upgrade-*.jsonl.gz`) and downgrade rules as under *Operate* below; state lives in PostgreSQL and on the data share, never in the containers. When an upgrade replaces the database container itself, the new replica waits for the previous one to shut down before it opens the database (Container Apps runs both for a moment); expect the API to pause for up to a minute. - **Backup**: a nightly job writes a logical database backup to the deployment's backup share (14 days kept). Managed state adds point-in-time restore on the server. Restore: load the newest archive into a fresh database with `psql`, then restart the API. - **Remove**: delete the resource group. Nothing is left outside it. ### Sizing and cost (Azure list prices, rough) | Configuration | Monthly Azure cost | Suits | |---|---|---| | Community, cluster mode off, Azure Files state | ~40-70 USD (Container Apps consumption, storage, Log Analytics) + model usage | Trial, a team | | Enterprise, cluster mode, Azure Files state | ~150-250 USD | Production without SLA requirements | | Enterprise, cluster mode, managed state (B1ms PostgreSQL, C0 Redis, 1 TU Event Hubs) | ~250-400 USD | Production with backups, SLAs, HA options | | + Ollama on serverless GPU (T4) | pay per second of GPU while replicas run | Private / specialist models without cloud keys | Model usage runs on the self-hosted Ollama apps (part of the Container Apps figure; a serverless GPU is billed per second while replicas run) and, when you connect them, on Ollama Cloud or a closed-model provider at their own prices; the dashboard's *Savings* page shows what routing saved against always calling the frontier model. ## Your own servers (Docker Compose) Runs on any host with Docker Engine 24+ (Linux, or Docker Desktop for a workstation): on-premises, air-gapped, or a VM in any cloud. Minimum 4 vCPU / 8 GiB / 40 GB disk for the platform; add what your models need if you run them on the same host. ### 1. Get the stack Download the Compose bundle of the current release - `docker-compose.yml`, `.env.example` and the Ollama overlays in the layout this page uses - and unpack it: ```bash curl -fsSLO https://opensmartroute.ai/download/compose # opensmartroute-compose-.zip unzip opensmartroute-compose-*.zip && cd opensmartroute-*/ cp platform/.env.example platform/.env ``` The bundle's `.env.example` already names the images of its own release; the stack runs from the published images and needs no source. ### 2. Configure Edit `platform/.env`: the passwords and the session secret (all of them - the defaults are placeholders), the public URLs of the web app and the API as your users will reach them, the release images, and the e-mail transport. Everything is documented in the file; the same variables are explained in platform/README.md (`platform/README.md`). ### 3. Start ```bash docker compose -f platform/docker-compose.yml --env-file platform/.env pull docker compose -f platform/docker-compose.yml --env-file platform/.env run --rm api python -m osr_platform.app --check docker compose -f platform/docker-compose.yml --env-file platform/.env up -d curl -fsS http://localhost:8080/readyz # every check ok once PostgreSQL is up and the schema is created ``` The `--check` line is the configuration preflight: it reads `.env` the way the API will and lists what is still a placeholder or inconsistent - the shipped passwords, a session secret missing on a multi-replica stack, public URLs still on `localhost`, e-mail confirmation required without a mail transport, a providers file that is not in the container - each with the variable that fixes it, and exits non-zero on an error. The same findings are logged at start-up and shown on the console's *Health* page. Web `http://localhost:3000`, operator console `http://localhost:3000/platform/admin/login` (the operator username and password from `.env`), API `http://localhost:8080/api/v1/info`. `OSR_API_IMAGE` / `OSR_WEB_IMAGE` in `.env` name the release images; a checkout of the source can leave them empty to build locally (`up -d --build`). **Just the router with a dashboard?** `platform/docker-compose.minimal.yml` starts four containers - PostgreSQL, Redis, one API process serving every domain, the web app - with `OSR_PLATFORM_PROFILE=minimal`: accounts, keys, routing, the OpenAI-compatible `/v1` proxy, MCP, billing and the console, and none of the jobs a hosted deployment runs for itself (blog writer, newsletter, growth engine, marketplace harvest). No Kafka, no separate SLM trainer. The same `.env` applies; switch to `docker-compose.yml` later without migrating anything - both stacks use the same database schema. ### 4. Connect models The stack has no cloud account. Either connect providers in the console under *Providers* (any OpenAI-compatible endpoint: OpenAI, Azure OpenAI, Anthropic through a gateway, vLLM, Ollama, ...) or start the on-prem model server next to it - the `llm` profile of the same Compose file - and point the router at its providers file: ```bash # in platform/.env: # COMPOSE_PROFILES=llm # OSR_PLATFORM_PROVIDERS=/config/providers.ollama.yaml docker compose -f platform/docker-compose.yml --env-file platform/.env up -d # on an NVIDIA host add the GPU overlay: docker compose -f platform/docker-compose.yml -f deploy/compose/ollama.gpu.yaml --env-file platform/.env up -d ``` The `ollama` service pulls the curated open weights of deploy/compose/providers.ollama.yaml (`deploy/compose/providers.ollama.yaml`) (Qwen 3.5, Qwen2.5-Coder, GLM-OCR, TranslateGemma, DeepSeek-R1, a Qwen3 embedding model; `OSR_OLLAMA_MODELS` trims the list on a small machine) and that file maps the private / specialist targets to it - `deploy/compose` is mounted at `/config` in every platform container, which is why the path above works. `OSR_PLATFORM_PROVIDERS` points the API at a providers file for anything else. ### 5. Put it behind TLS Terminate TLS in your reverse proxy (nginx, Caddy, Traefik, an ingress) in front of ports 3000 (web) and 8080 (API), set `OSR_PLATFORM_WEB_URL` / `OSR_PLATFORM_BASE_URL` to the public `https://` origins, and remove the host port mappings you do not want exposed. Several API replicas share PostgreSQL / Redis / Kafka: `docker compose up -d --scale api=3` behind the proxy, exactly what the Azure template does. ### Operate - **Backups**: `docker compose -f platform/docker-compose.yml exec postgres pg_dump -U osr osr | gzip > osr-$(date +%F).sql.gz` (and `osr_telemetry`); restore with `psql` into a fresh volume. Without `pg_dump` at hand, the API container does it from its own connection: `docker compose ... run --rm api python -m osr_platform.maintenance backup /data/backups/osr-$(date +%F).jsonl.gz` writes every row of every table as one gzip file, `... restore --yes` (API stopped) empties the tables and loads it back in one transaction. - **Upgrade**: set the new tag in `.env`, `pull`, `run --rm api python -m osr_platform.app --check`, `up -d`. The schema is a numbered ledger (`schema_migrations`): the new release applies the migrations it knows and the database has not run yet, in order, and records each with the release that wrote it - `python -m osr_platform.maintenance schema` shows the ledger, applied and pending. Before a pending migration that is not additive runs on a database that already holds data, the API writes a backup to `/backups/pre-upgrade--.jsonl.gz` on its own (`OSR_PLATFORM_PRE_UPGRADE_BACKUP=false` turns that off, `OSR_PLATFORM_BACKUPS_KEEP` is how many it keeps); an additive change needs none - the way back from it is the older release itself - and a fresh install never writes one. The backup never holds the schema lock, so replicas already serving keep answering while it is written. Skipping releases is fine: the migrations between them run in sequence at the next start. - **Downgrade**: set the previous tag and `up -d`. Every migration is marked *additive* (it only added columns, indexes or tables) or not. An older release starts against a database a newer one migrated when every newer change is additive - it logs a warning (also shown on the console's *Health* page under *Configuration*) and ignores the columns it does not know. A non-additive newer change stops start-up with the release that wrote it named: run that release, or put the database back with `python -m osr_platform.maintenance restore` of the `pre-upgrade-*` backup the upgrade took (a restore refuses a backup written by a newer schema than the running release knows; an older one loads and the columns added since take their defaults). - **Edition switch**: `OSR_PLATFORM_EDITION` is read at start-up and every table exists in both editions, so moving between `community` and `enterprise` is a restart with the new value and back - nothing to migrate. The database remembers which edition opened it: the first start after a change logs what the switch means (community writes no audit log and does not read the persisted learner state, so an enterprise stack that ran as community for a while has that gap in its audit chain) and the console's *Health* page shows the same sentence under *Configuration* until the next restart. - **Air-gapped**: on a connected machine `docker pull` both images (and `postgres:16-alpine`, `redis:7-alpine`, `apache/kafka:3.9.0`), `docker save | gzip`, move, `docker load`; or mirror them into your registry (`az acr import --source crosrua2c6onyidguw.azurecr.io/osr-platform-api:2.1.0 ...`) and set `OSR_API_IMAGE` / `OSR_WEB_IMAGE` (or the Marketplace wizard's *Image registry*). - **Health**: `GET /readyz` on the API (database, Redis, Kafka, mail transport, routing probe), `/platform/admin/health` in the console (readiness, traffic, breakers, alerts, the configuration preflight), `GET /metrics` for Prometheus. ## Monitoring Every install monitors itself: the platform's own telemetry store keeps every span and event in PostgreSQL, `GET /metrics` serves Prometheus text, `GET /readyz` and `GET /api/v1/status` answer probes and status pages, the console's *Health* page shows readiness, traffic, latency, circuit breakers, alerts and the configuration preflight. Nothing else is required. **On Azure** the template and the Marketplace offer also create a workspace-based **Application Insights** resource (`appi-osr-`, next to the Log Analytics workspace that already receives the container console logs; the *Application Insights* checkbox of the wizard, `OSR_APPLICATION_INSIGHTS=false` in the azd environment turns it off) and hand every platform container its connection string. Every process then exports requests, routing spans (`dependencies`), routing events (`customEvents`), warnings and exceptions under the request's operation id, so the end-to-end transaction view, the application map, KQL, workbooks and Azure Monitor alerts work out of the box. The export is `OSR_PLATFORM_APPINSIGHTS=auto`: it only runs where a connection string is present *and* the host is Azure. **On your own servers** nothing is exported anywhere. To report a self-hosted stack into an Azure subscription anyway, create an Application Insights resource there, put its connection string in `OSR_PLATFORM_APPINSIGHTS_CONNECTION_STRING` and set `OSR_PLATFORM_APPINSIGHTS=on` in `.env`; `OSR_PLATFORM_APPINSIGHTS_SAMPLE_RATE` bounds the ingestion volume. The settings are described in [docs/PLATFORM.md](https://osr-web.kindrock-caaab1a2.centralindia.azurecontainerapps.io/docs/PLATFORM.md#monitoring-what-is-always-on-and-application-insights-on-azure). ## Where the downloads come from Every release publishes the Compose bundle, the Helm chart, the Marketplace package and the checksums of the PyPI build to the [downloads page](https://opensmartroute.ai/downloads), together with a `latest.json` manifest (version, build time, per-file name, size and SHA-256) for scripts. Versioned files are immutable; `latest.json` always points at the current release. The Azure Marketplace offer is the same solution template, published through Partner Center; a plan's image tag always names release images that already exist, so a deployment never pulls a moving target.