# External security review pack Everything an independent reviewer needs to assess the OpenSmartRoute control plane against the threat model in [SECURITY.md](https://osr-web.kindrock-caaab1a2.centralindia.azurecontainerapps.io/docs/SECURITY.md): scope, trust boundaries, the evidence the project already produces, how to reproduce it, the questions we want answered and how a report is published. It exists so that an independent review can be run by a third party without a briefing call. ## 1. Engagement | | | |---|---| | Subject | `opensmartroute` 2.1.0 (`pip install opensmartroute`), the `osr serve` image, the hosted platform at `https://api.opensmartroute.ai` / `https://opensmartroute.ai` | | Type | White-box: full source, tests, CI and deployment code are shared with the reviewer for the engagement (the source repository is private) | | Rules of engagement | Test against your own `osr serve` / `python -m osr_platform.app` instance or a free-plan workspace you created; no load testing of the hosted platform; no access to other workspaces' data | | Contact | `security@opensmartroute.ai` ([../SECURITY.md](https://osr-web.kindrock-caaab1a2.centralindia.azurecontainerapps.io/docs/security-policy.md)); acknowledgement within 72 h | | Publication | Findings fixed or accepted first; then the report is linked in section 7 and in the roadmap | ## 2. System under review ``` caller ── HTTPS ──> platform/web (Next.js, proxies /api,/v1,/mcp) ──> platform/api (FastAPI, osr_platform) │ API keys, sessions, plans, metering, │ workspaces, audit, admin └──> opensmartroute (SDK: signals, policy, strategies, learners) └──> targets (Azure OpenAI, MCP tools, agents, humans) self-hosted: caller ──> osr serve (src/opensmartroute/server.py) ──> opensmartroute ──> targets library: your code ──> Router / RouterBuilder ──> targets ``` Trust boundaries, from the outside in: 1. **Internet -> web / API.** Anonymous: landing, docs, `/api/v1/models`, `/rankings`, `/stats/public`, `/status`, `/estimate` (rate limited), `/signup` (5 per IP per day). Everything else needs an API key (`osr_live_`, hashed at rest) or a browser session (`osr_sess_`, 30 days, SSO via PKCE with a signed state). Admin routes need `X-Admin-Token`. Cross-workspace reads are 404 by construction (`db.usage_by_request(account, request_id)` scopes every trace / event lookup). 2. **`osr serve`.** Optional bearer / `X-API-Key` middleware (`serve --token`, `--require-auth`, `OSR_SERVER_AUTH_TOKENS`); `OPEN_PATHS` are the probes, `/metrics`, `/whoami` and the OpenAPI UI. The Helm chart and the Docker image are documented as "front it with your gateway". 3. **Router -> request text.** `InputGuard` (size, gadget and steering scrub, optional PII redaction) runs before signals; `Policy` runs before scoring and cannot be outweighed; the LLM judge sees a fenced, sanitised prompt and its output is parsed item by item. 4. **Router -> catalogue.** Imported MCP tool descriptions, A2A cards and `SKILL.md` files pass the injection lexicon and the learned gadget score; MCP manifests can be signed (HMAC-SHA256 / Ed25519). 5. **Router -> execution.** `OriginPolicy` (parameter provenance), `ResourceLimiter` (steps, calls, depth, tokens, cost, wall-clock), breakers and budgets sit between a decision and a handler call; `OutputGuard` (secrets, PII, relayed injection - flag / redact / block) sits between the handler's answer and the caller. 6. **Router -> state.** JSON only, atomic writes, hashed keys, versioned schema, quarantine of corrupt state, optional AES-256-GCM at rest, hash-chained audit with `verify()`. 7. **Platform -> workspace.** API keys carry scopes (`route`, `execute`, `teach`, `manage`), an expiry and per-key limits (daily / monthly budget, allowed targets, requests per minute); the workspace policy and tenant budgets apply on top; `GET /api/v1/governance/report` renders the controls in force, the key posture and the guard verdicts of the last 24 h as the evidence pack a reviewer would otherwise assemble. ## 3. Threat model [SECURITY.md, "Threat model & controls"](https://osr-web.kindrock-caaab1a2.centralindia.azurecontainerapps.io/docs/SECURITY.md#threat-model--controls) lists T1-T19 with the OWASP Top 10 / OWASP LLM Top 10 mapping and the implementing code; ["Threat-model delta: 0.3 -> 0.4"](https://osr-web.kindrock-caaab1a2.centralindia.azurecontainerapps.io/docs/SECURITY.md#threat-model-delta-03---04) lists the residual risk we already know about per threat. The reviewer is asked to confirm, refute or extend that table rather than start from a blank page. ## 4. Evidence the project produces | Evidence | Where | How to reproduce | |---|---|---| | Static analysis on every push: `ruff` (incl. the `S` bandit rules), `bandit`, `mypy` | CI, every push | `python -m ruff check src tests scripts; python -m bandit -q -r src -c pyproject.toml; python -m mypy src` (in the engagement checkout) | | Weekly security scan: `bandit` over SDK and platform, `pip-audit`, `npm audit`, safety suite | the weekly security run | shared with the engagement | | Unit tests for the controls (guard, output guard, moderation, gadget detector, sanitiser, provenance, limits, signed manifests, audit chain, encrypted store) | the security test suites (49 dedicated tests plus hardening and store suites) | `python -m pytest tests/test_security.py tests/test_hardening.py -q` (in the engagement checkout) | | Safety-routing red-team suite (paraphrase, gadget and steering attacks against safety routing) gates CI and every release | `security.safety.run_safety_suite` | `osr -t examples/targets.yaml safety --learned-guard` | | Platform authorisation tests (API keys, scopes and per-key limits, sessions, workspaces, SSO state, device flow, operator MFA, admin token, trace scoping, governance report) | the platform test suite (419 tests) | `python -m pytest platform/api/tests -q` (in the engagement checkout) | | Zero runtime dependencies in the SDK core; optional extras imported lazily | `pyproject.toml` | `pip install opensmartroute && pip show opensmartroute` (Requires: none) | | Reproducible releases: the sdist and wheel are built once by the release process, published with `SHA256SUMS` and uploaded to PyPI from that artifact | the release verification stage | `sha256sum -c SHA256SUMS` (from the [downloads page](https://opensmartroute.ai/downloads)) against the PyPI files | | Container hardening: non-root image, read-only root filesystem, dropped capabilities, NetworkPolicy, PDB in the Helm chart | the published image and Helm chart | `helm template` over the published chart | | Content-free logs and traces (ids, digests and lengths only) | `observability.text_digest`, `LoggingTelemetry` | run `osr serve` with a `FileSink` and inspect | | Hash-chained audit log with tamper detection | `enterprise.FileAuditSink.verify` | edit one line of an audit file and run `verify()` | ## 5. Questions for the reviewer Ordered by the impact a positive finding would have. 1. **Rerouting (T1, T2).** Can a request author steer the decision to a chosen target through the text alone - gadgets the learned detector misses, steering the scrub does not remove, or judge-prompt escapes past the sanitiser? Success = a reproducible prompt that moves a target's propensity by more than the ensemble's stated confidence margin on `examples/targets.yaml`. 2. **Policy bypass (T3).** Does any strategy, plan slot, fallback, cascade or `exclude`/`candidates` path reach a target that `Policy.check` rejected (PII to a `pii_allowed: false` target, region or tenant constraint)? The platform's `/api/v1/policy` overlay is in scope. 3. **Cross-workspace access (platform).** Any read or write that crosses `account_id`: traces, events, activity, savings, API keys, invites, SSO connections, marketplace listings, admin routes without the admin token, IDOR on `request_id`. 4. **Authentication (platform).** API-key hashing and comparison, session fixation / expiry, SSO PKCE and state signature, the RFC 8628 device flow (`slow_down`, code entropy, approval binding), signup rate limiting, the OpenAI-compatible `/v1` proxy's key handling. 5. **Catalogue poisoning (T13, T14).** An MCP tool description, A2A card or `SKILL.md` that passes the import filter and changes routing or smuggles a parameter past `OriginPolicy`. 6. **Learner poisoning (T5).** Feedback (`/feedback`, `Outcome`) sequences that move a target's learned quality faster than the documented prior / confidence ramp allows, or state files that load and are trusted when they should be quarantined. 7. **Resource amplification (T6, T15).** Inputs or plans that escape `InputGuard` limits, `ResourceLimiter` caps or the knapsack meters; unbounded growth in any in-memory map or state store. 8. **Data at rest and in logs (T4, T16).** Any code path that writes request text to logs, traces, cache keys, feedback, audit or metrics; weaknesses in `EncryptedStateStore` (nonce reuse, key rotation). 9. **Supply chain (T11).** Release workflow, attestation coverage, Docker base image, the `install.sh` / `install.ps1` bootstrap scripts served from the web app, the npm package `@opensmartroute/sdk`. 10. **Answer-side leakage (T19).** A model answer that carries a credential, personal data or an instruction past `OutputGuard` in `redact` / `block` mode - credential shapes outside `SECRET_PATTERNS`, PII entities the `Redactor` misses, injection phrasing under `injection_threshold`, or the streamed path where the guard can only flag after the fact. ## 6. Known gaps (do not re-report) - The SDK does not authenticate callers; `osr serve` is unauthenticated unless `--require-auth` or tokens are configured, and the Helm chart says so. - The gadget detector ships trained on synthetic gadgets; the injection lexicon is English-centric. - A compromised LLM judge can still bias ranks among legitimate candidate ids. - A valid-looking poisoned learner snapshot is indistinguishable from real learning; write access to the state store is the boundary. - Audit verification is only meaningful if the log file is write-once at the OS level. - Encryption at rest is opt-in; key management is the operator's (`load_secret`). - The output guard under `redact` / `block` never lets an unscreened token out of a stream: by default it buffers the streamed answer (`buffer_streams`, the caller sees the screened text chunked after the fact); with `buffer_streams=False` a `StreamScreen` holds back the newest `OSR_SERVER_OUTPUT_GUARD_HOLD_CHARS` characters and releases only cleared text, so a secret split across chunks is caught - the trade-off is that an injection phrase longer than the hold may have partly left before `block` stops the stream. `flag` inspects a stream after the fact. The hosted platform defaults to `flag` (`OSR_PLATFORM_OUTPUT_GUARD`), `osr serve` to `off`. ## 7. Review log | Date | Reviewer | Scope | Report | Findings (open / fixed) | |---|---|---|---|---| | 2026-10-09 | internal (privacy data-flow and retention) | section 8 of this page: every store of personal data on the hosted platform, its processor and its retention, checked against the code | this page | 2 open / 2 fixed (listed under 8.4) | | 2026-10-14 | internal (registration abuse, black-box against production) | the self-serve door after a day of scripted registrations (nine accounts, one random `*.cc.cd` subdomain each, `Hermes ` names, some confirmed their address, IPs rotated under the 5-a-day cap); security headers of the API and the web app; enumeration on sign-up, password reset and login | this page | 1 open / 3 fixed (listed under 9) | A completed review adds a row here, a "Threat-model delta" row in [SECURITY.md](https://osr-web.kindrock-caaab1a2.centralindia.azurecontainerapps.io/docs/SECURITY.md) for anything that changed, and a CHANGELOG entry for every fix. Until a row exists, the independent report is an open adoption item, not a shipped control. ## 8. Privacy: data flow and retention (hosted platform) Reviewed 2026-10-09 against the code; the retention figures below are the defaults a fresh deployment runs with and a test holds this table to the constants, so a change in either shows up as a red build. The customer-facing summary is the [privacy policy](https://opensmartroute.ai/privacy); [PLATFORM.md, "Privacy and data handling"](https://osr-web.kindrock-caaab1a2.centralindia.azurecontainerapps.io/docs/PLATFORM.md#14-privacy-and-data-handling) has the request-path diagram. ### 8.1 Where personal data goes | Data | Written by | Leaves the platform to | Purpose | Store | |---|---|---|---|---| | Request text (prompt, messages, tool arguments) | `/api/v1/route`, `/v1/*`, `/mcp` | the model provider the router chose (the workspace's own key, or the deployment's); PII masked first unless the target is marked `pii_allowed` | answer the request | **not stored** unless the workspace sets `log_payloads` (`redacted` masks PII *and* credential shapes; `full` keeps the text) -> `request_logs` | | Decision rows: target, cost, tokens, latency, signals, outcome | every request | nowhere | usage, billing, learning, savings | `usage`, `feedback` | | Conversations and remembered facts (`osr.conversation`, memory) | the chat proxy when asked | the provider, as the context of the next turn | continue a thread, personalise | `conversations`, `conversation_turns`, `memories` | | Spans and events (ids, digests, lengths - no text) | every request | nowhere (an optional OTLP exporter when the operator configures one) | traces, health | `telemetry_events` | | Account: e-mail, name, password hash, identities, sessions, MFA secret | signup, sign-in | the mail provider (Microsoft Graph) for verification, notices and newsletters; the identity provider on SSO; Stripe (e-mail, name, workspace id) when a plan is bought | identity, billing | `users`, `sessions`, `identities` | | Sign-in place (IP, resolved country / city) | sign-in, key use, operator sign-in | a GeoIP service the operator configures (IP only) | security alerts ("new country") | `sessions`, `audit_log`, `geo_cache` | | Browser and client events (extension / desktop / IDE: counts, hosts, no text) | the apps, when the person opted in | nowhere | the person's own usage page | `client_events`, `observed_usage` | | Visitor journey (page, referrer, UTM, anonymous id) | the public site | Google Analytics 4 and the advertising conversion endpoints only when the operator configured them and the visitor consented | the activation funnel | `journey_events`, `journey_answers` | | Uploaded bills and usage exports | bill analysis | nowhere (parsed in process) | the AI-spend report | `spend_imports` | | Support tickets, sales inquiries, product feedback (incl. screenshots) | the forms | the mail provider (acknowledgements), the operators' inbox | support, sales | their own tables | | Audit records (actor, action, subject - no text) | every privileged action | nowhere | accountability | `audit_log` (SQL) and the hash-chained JSONL files | ### 8.2 Retention Deletion runs in the retention sweep (every 6 hours, once at start); a workspace on Team or Enterprise can set its own shorter request-row retention, which wins over the deployment default. | Store | Kept for | Set by | |---|---|---| | `usage`, `feedback`, `request_logs` (request rows) | 365 days | `OSR_PLATFORM_RETENTION_DAYS` (0 keeps forever) | | `telemetry_events` (spans, events) | 14 days | `OSR_PLATFORM_TELEMETRY_RETENTION_DAYS` | | `observed_usage` (chat-site usage the extension observed) | 90 days | fixed | | `client_events` (extension / desktop activity) | 7 / 30 / 90 / 365 days by plan (Free / Pro and Pay-as-you-go / Team / Enterprise) | the plan's `activity_days` | | `journey_events`, `journey_answers` | 400 days | fixed | | `status_samples` (raw uptime samples) | 7 days | fixed; daily roll-ups keep `OSR_PLATFORM_STATUS_HISTORY_DAYS` + 30 | | mail outbox | 30 days | fixed | | IP address and place on `audit_log`, `sessions`, `operator_sessions` | 365 days, then scrubbed to null (the event row stays) | `OSR_PLATFORM_NETWORK_DATA_RETENTION_DAYS` | | `geo_cache` (IP -> place) | 30 days valid, purged after 60 days | fixed | | newsletter engagement | 180 days | fixed | | unverified accounts | 14 days | `OSR_PLATFORM_UNVERIFIED_ACCOUNT_TTL_DAYS` | | conversations, memories | until the person or the workspace deletes them (per-thread cap of 400 turns, per-subject memory cap) | - | | `audit_log` (SQL) | at least 365 days | the retention default, floored at a year | ### 8.3 Rights `GET /api/v1/me/export` returns the user's profile, identities, sessions and history and the workspace's plan, members, key prefixes, tenants, policy, SSO, usage, requests, outcomes, request logs, evaluation reports and - since this review - conversations with their turns and remembered facts. `POST /api/v1/me/delete` removes the user (sessions, identities, memberships; workspaces they alone owned are disabled with their keys); `POST /api/v1/workspace/delete` removes an organization and everything scoped to it, memory included; `DELETE /api/v1/activity/logs` erases kept payloads on their own. ### 8.4 Findings of this review - **Fixed.** `log_payloads: redacted` masked PII but not credentials: a pasted API key reached the log row verbatim. Redacted mode now runs the output guard's secret patterns as well. - **Fixed.** The export did not carry conversations or memories although the privacy policy promises the dashboard removes them; it does now (`workspace.memory`). - **Open.** Conversations have no age-based retention; a workspace that never deletes keeps them until it is deleted. The per-thread and per-subject caps bound the volume, not the age. - **Open.** Journey events are unlinked, not deleted, when a workspace is deleted (the anonymous visitor rows stay for their 400 days); the privacy policy says so, a reviewer may want it tightened. ## 9. Findings of the 2026-10-14 registration review - **Fixed.** A domain rule cannot keep up with a bot that mints a subdomain per account. The signup guard now scores the shape of the registration (digits woven into the local part, a letters-and-digits subdomain label, a word-and-hex display name, a long consonant run) and refuses what adds up to a machine (`block_generated`, on by default); the per-domain daily cap counts by the registered domain; `cc.cd` and the tempmail.plus family join the temporary-inbox list. Evidence: `platform/api/tests/test_signup_guard.py` pins the nine observed registrations as refused and a list of ordinary addresses as accepted. - **Fixed.** Clearing the accounts already made meant one rule and one disable per row. The Users page's bulk actions *Block their e-mail domains* / *Block their addresses* write the blocklist rule and disable the user in one move, audited per user. - **Fixed.** The bot challenge (Turnstile / hCaptcha) was configurable on the API but the web app's Content Security Policy would have blocked its script and iframe in production; both providers are now in `script-src`, `frame-src` and `connect-src`. - **Open (accepted).** `POST /api/v1/signup` answers `409` for an address that already has an account, which lets a caller test whether an address is registered. Password reset (`202` either way) and sign-in do not. The `409` is kept because the form shows the API key on the spot and sends the person to sign in otherwise; the per-address sign-up limit (`OSR_PLATFORM_SIGNUP_PER_IP_PER_DAY`, 5 by default) and the IP blocklist bound the rate at which a list can be tested. A deployment that must not disclose membership runs with `OSR_PLATFORM_SIGNUP=false` and invitations. - **Operator action, not code.** The hosted deployment runs without a bot challenge; set `OSR_PLATFORM_CAPTCHA_PROVIDER=turnstile` with the site key and secret, a per-domain cap and `auto_block_after` on the Access page.