Loading
90 days of the enterprise, pro and team plans free, no card needed. See the plans
API gateway with AI plugins
Kong adds AI plugins to the API gateway many platform teams already run: an OpenAI-compatible proxy across providers, token-based rate limiting, prompt guards and load balancing between models. OpenSmartRoute is the routing brain those plugins call: a cost/quality objective per request, hard policy before scoring, a trace for every decision and learning from your outcomes.
“Kong authenticates, limits and meters API traffic at the edge; OpenSmartRoute selects the target behind it by cost, quality, latency and policy.”
Our reading of public documentation; corrections welcome through the support page.
| Capability | OpenSmartRoute | Kong AI Gateway |
|---|---|---|
| Routes to LLMs | Yes | Yes |
| Routes to agents, skills, tools, workflows and humans | Yes | No |
| Hard policy before scoring (region, data boundary, PII, tenant, cost)Static routing rules per plugin config; no data-boundary or PII gating of the decision | Yes | Partial |
| Per-request trace with ranked candidates and rejections | Yes | No |
| Learns from your outcomes (per target, per tenant) | Yes | No |
| Multi-step plans (persona -> skill -> model) | Yes | No |
| Self-hosted, air-gapped, zero runtime dependenciesSelf-hosted with the gateway's own database and footprint | Yes | Partial |
| Prompt-injection and steering-gadget defences on the routerPattern-based prompt guard; no learned gadget detector on the decision | Yes | Partial |
| Output guard on responses (PII, secrets, injection: flag, redact or block)Prompt guard and PII sanitiser plugins on the request; response screening is limited | Yes | Partial |
| Savings ledger, bill analysis and routing auditEvery request records baseline and routed cost; upload last month's provider export to see what routing would have saved before you switch | Yes | No |
| OpenAI-compatible proxy (chat, embeddings, Responses, moderations, audio, images)Chat and embeddings across providers through the ai-proxy plugins | Yes | Partial |
| Anthropic Messages API (/v1/messages)Anthropic as an upstream provider behind the OpenAI-shaped route | Yes | Partial |
| Virtual keys: per-key budgets, rate limits, allowed targets, scopes and expiryConsumers and token-based rate limits; no per-key budget or target allow-list | Yes | Partial |
| Per-target rate limits, budgets and circuit breakersLoad balancing and health checks per upstream; no spend budget per model | Yes | Partial |
| Python and TypeScript SDKsAny OpenAI client pointed at the gateway route | Yes | Partial |
| MCP server for IDEs and agentsMCP traffic can be proxied; the gateway is not an MCP server for routing | Yes | Partial |
| Browser extension and desktop app for the people on the teamPrice, personal-data and writing checks on the device before a prompt is sent, on ChatGPT, Claude, Gemini and in any desktop app | Yes | No |
| Provider marketplace with creditsModels by the token: every catalogue model on offer per provider with the price you pay, one key, a prepaid credits wallet and provider preferences (order, only, ignore, max_price) | Yes | No |
| Request/response observability UI for every callGateway analytics and logs; no per-request decision trace | Yes | Partial |
Declare one upstream service - the OpenSmartRoute proxy - behind the ai-proxy route. Kong keeps authentication, rate limiting and plugins; the router keeps the decision, the policy and the trace.
# kong.yaml (declarative): one upstream, the router decides the model
services:
- name: osr
url: https://osr.internal/v1
routes:
- name: chat
paths: [/v1/chat/completions]
# clients send model: "osr/auto:cheap" (or a target id) and the router picks the providerRun the Routing Audit on a week of anonymised logs: savings versus your current mix, policy violations your routing allowed, and where agents or humans would have been the better answer.
Other comparisons: OpenRouter, LiteLLM, Portkey, Cloudflare AI Gateway, Bifrost (Maxim AI), Helicone, Not Diamond, Martian, Unify, Requesty, Building it yourself.
Photo: Mario Gogh
