neon-ai-gateway

Imported from sickn33/agentic-awesome-skills (skills/neon-ai-gateway/SKILL.md). Install upstream with npx skills add sickn33/agentic-awesome-skills --skill neon-ai-gateway. Copyright stays with the author (Apache-2.0).

Neon AI Gateway

Confirm current project/region availability in the official quickstart. The Neon AI Gateway is the LLM inference layer built into your Neon branch: one API and one Neon credential give you access to frontier and open-source models from Anthropic, OpenAI, Google, Meta, Alibaba, DeepSeek, and Databricks — powered by Databricks. SDK integrations require the matching dialect, explicit gateway credential and a model supporting the requested features.

Use this skill to help the user send model calls through the gateway, wire it into the AI SDK or Mastra, and switch providers without rewiring code. Deliver a working inference request, a configured agent, or a precise answer from the official Neon docs.

When to Use

Reach for the AI Gateway whenever an app or agent needs to call an LLM and the user would rather not manage model providers themselves:

• One credential instead of many provider accounts. A single Neon credential reaches the entire model catalog across seven providers. No separate OpenAI / Anthropic / Google billing, keys, or signups to provision and rotate.
• Switch models without rewiring. The unified endpoint is OpenAI-compatible and works with every model in the catalog — change one model field to move between Claude, GPT, and Gemini. Standard SDKs require matching routes, credentials and feature validation.
• AI follows your branches. Each branch has its own gateway endpoint, scoped with the same lineage as your database. AI requests from a preview/feature branch are isolated to that branch — the same isolation your data already gets — which makes preview, CI, and agent environments self-contained.
• No extra infrastructure, and it's already next to your data. The gateway lives inside your Neon project (and is injected into Neon Functions automatically), runs on the same Databricks infrastructure that serves trillions of tokens a month, and supports streaming (SSE) out of the box.

If the user already has a deep, single-provider integration and no interest in Neon branching or multi-model routing, a direct provider SDK is fine — but the moment they want one credential, model portability, or branch-scoped AI, this is the reason to use it.

What It Does

• One API for all models — Frontier and open-source models behind a single endpoint, addressed by their catalog ID (e.g. claude-sonnet-4-6, gpt-5-mini, gemini-2-5-flash).
• Standard SDKs, one URL change — OpenAI SDK and AI SDK (OpenAI-compatible MLflow/Responses routes), Anthropic SDK (native Messages), google-genai (native Gemini).
• Branch-scoped — Each branch gets its own gateway host; the Neon credential authorizes requests for that branch and its descendants.
• Streaming — Verify streaming and event handling for the selected endpoint/model.

Setup

The gateway is part of neon.ts (see the neon skill for the branch-first workflow and neon.ts basics). Enable it under preview.aiGateway:

// neon.ts
import { defineConfig } from "@neon/config/v1";

export default defineConfig({
  preview: {
    aiGateway: true,
  },
});

neon deploy   # provisions the gateway on the linked branch

Neon Infrastructure as Code (neon.ts)

The preview.aiGateway toggle above is part of neon.ts, Neon's infrastructure-as-code file — one TypeScript file declares the gateway alongside every other branch service, in version control (see the neon skill for the full reference). Reconcile it against a branch the Terraform way:

neon config status   # print the branch's live config (is the gateway on?)
neon config plan     # dry-run diff of what apply would change
neon config apply    # enable the gateway on the branch  (neon deploy is an alias)

The gateway is branch-scoped: each branch gets its own gateway host. When a neon.ts is present, neon checkout applies the policy as it creates a branch, so a fresh preview/CI branch comes up with the gateway already enabled. Checking out an existing branch doesn't reconcile it — run neon deploy to apply changes. Provisioning (config apply / deploy), link, and checkout also pull the branch's gateway credentials into your local .env.local, so local runs hit the same branch gateway as the deployed function (no manual env pull needed).

For typed, validated access to the injected credentials, pass the same config object to parseEnv from @neon/env — it returns an env.aiGateway namespace (apiKey, baseUrl) derived from your neon.ts.

Environment variables

Current documented variables are NEON_AI_GATEWAY_TOKEN (a server-side bearer with ai_gateway:invoke scope) and NEON_AI_GATEWAY_BASE_URL (bare HTTPS branch host). Do not assume OPENAI_* aliases are injected; configure clients explicitly. Credential files must be private and gitignored. Credentials authorize descendants as documented; a branch endpoint is not an application-user authorization boundary. Revocation must be explicit where expiry is not enforced.

Append /v1 for all-provider chat completions, /openai/v1 for Responses, /anthropic for the Anthropic SDK, or /gemini for google-genai. Native SDKs append their own API segments. The older longer /ai-gateway routes remain aliases, but model listing uses GET /v1/models. Confirm model/feature compatibility before switching dialects.

Build agents with the Vercel AI SDK (recommended)

The Vercel AI SDK is the recommended way to call the gateway and build agents from TypeScript: one set of primitives (generateText, streamText, tool calling, structured output) over every catalog model, with first-class streaming for the long agent responses Neon Functions are built to host.

On a Neon Function that streams text and generates images, configure the provider explicitly rather than assuming OPENAI_* environment aliases:

import { createOpenAI } from "@ai-sdk/openai";
const openai = createOpenAI({apiKey: process.env.NEON_AI_GATEWAY_TOKEN, baseURL: `${process.env.NEON_AI_GATEWAY_BASE_URL}/openai/v1`});
import { streamText } from "ai";

const result = streamText({
  model: openai("gpt-5-mini"),
  messages,
  tools: {
    image_generation: openai.tools.imageGeneration({
      outputFormat: "jpeg",
      size: "1024x1024",
    }),
  },
});
return result.toUIMessageStreamResponse();

For multi-provider routing from a single call, the dedicated @neon/ai-sdk-provider reads NEON_AI_GATEWAY_BASE_URL + NEON_AI_GATEWAY_TOKEN and routes each model to the best endpoint (Anthropic → Messages, OpenAI/Codex → Responses, everything else → MLflow):

import { neon } from "@neon/ai-sdk-provider";
import { generateText } from "ai";

const { text } = await generateText({
  model: neon("claude-haiku-4-5"), // or gpt-5-3-codex, gemini-2-5-flash, ...
  prompt: "Summarize Postgres for me.",
});

To build an agent — a model that calls tools in a loop and then answers — add tools and a stopWhen budget. The loop runs in-process, but still needs request deadlines, cancellation, concurrency and cost limits:

import { neon } from "@neon/ai-sdk-provider";
import { generateText, tool, stepCountIs } from "ai";
import { z } from "zod";

const { text } = await generateText({
  model: neon("claude-sonnet-4-6"),
  prompt: "How many open todos do I have, and what's the oldest one?",
  tools: {
    listTodos: tool({
      description: "List the user's open todos.",
      inputSchema: z.object({}), // AI SDK v5+: `inputSchema`, not `parameters`
      execute: async () => listOwnedOpenTodos(authenticatedUser.id), // Server-bound identity and allowlisted fields; implement this adapter.
    }),
  },
  stopWhen: stepCountIs(5), // let the model call tools, then summarize
});

For a full AI SDK agent deployed as a Neon Function (streaming, tool calling, image generation, persistence), see the neon-functions skill's references/ai-sdk.md.

Build agents with Mastra (recommended)

Mastra is the recommended framework when you want batteries-included agents — built-in memory, tools, workflows, and tracing — with the model still pointed at the gateway. A memory-backed agent (threads/messages in Postgres via @mastra/pg) running as a Neon Function reads env.aiGateway from parseEnv and uses the chat-completions (MLflow) dialect:

import { Agent } from "@mastra/core/agent";
import { parseEnv } from "@neon/env";
import config from "../neon";

const env = parseEnv(config);
const gatewayUrl = `${process.env.NEON_AI_GATEWAY_BASE_URL}/v1`;

export const personalAssistant = new Agent({
  id: "personal-assistant",
  name: "personal-assistant",
  instructions:
    "You are a warm, concise personal assistant with long-term memory.",
  model: {
    id: `neon/claude-haiku-4-5`,
    url: gatewayUrl,
    apiKey: env.aiGateway.apiKey,
  },
  memory,
});

Use with plain SDKs (lower-level)

When you don't need an agent framework — a single completion, an existing provider-SDK integration, or native provider features — call the gateway with the plain SDKs. Use an explicit gateway token and dialect; do not accidentally fall back to a direct provider credential:

import OpenAI from "openai";

const client = new OpenAI({apiKey: process.env.NEON_AI_GATEWAY_TOKEN, baseURL: `${process.env.NEON_AI_GATEWAY_BASE_URL}/openai/v1`});

const res = await client.responses.create({
  model: "gpt-5-mini", // swap to claude-sonnet-4-6, gemini-2-5-flash, ...
  input: "What is Neon?",
});

For the unified chat-completions dialect (/mlflow/v1) instead, point the client at it. The ergonomic way is to swap the dialect on the injected base URL rather than rebuild it (same move the Mastra example makes):

const client = new OpenAI({
  apiKey: process.env.NEON_AI_GATEWAY_TOKEN,
  baseURL: `${process.env.NEON_AI_GATEWAY_BASE_URL}/v1`,
});

const res = await client.chat.completions.create({
  model: "claude-sonnet-4-6",
  messages: [{ role: "user", content: "What is Neon?" }],
});

The Anthropic SDK and google-genai work the same way for native provider features — point them at the /anthropic and /gemini routes on the bare gateway host (${NEON_AI_GATEWAY_BASE_URL}/ai-gateway/anthropic, ${NEON_AI_GATEWAY_BASE_URL}/ai-gateway/gemini).

Model identifiers

Use a model's catalog ID directly in the model field — e.g. claude-sonnet-4-6, gpt-5-mini, gemini-2-5-flash. No provider prefix is needed. To look up the exact identifiers the gateway serves, which underlying model each maps to, and their context windows, pricing, and capabilities, use any of:

• models.dev Neon provider page: https://models.dev/providers/neon — the external catalog; cross-check against the gateway model list and official docs of the Neon provider's model IDs and their underlying models. The machine-readable catalog is at https://models.dev/api.json (the neon key).
• Models doc: see Further reading.

Availability

AI Gateway requires a paid Neon plan and prepaid credits. Confirm current region/project eligibility, model catalog, pricing and retention before provisioning or sending user data. The examples are fragments with missing application adapters; no inference, deployment or billing result is established by configuration alone.

Neon Documentation

The Neon documentation is the source of truth and the AI Gateway is evolving rapidly, so always verify against the official docs. Any doc page can be fetched as markdown by appending .md to the URL or by requesting Accept: text/markdown. Find the right page from the docs index (https://neon.com/docs/llms.txt) and the changelog announcements.

Further reading

• https://neon.com/docs/ai-gateway/overview.md
• https://neon.com/docs/ai-gateway/get-started.md
• https://neon.com/docs/ai-gateway/models.md
• https://neon.com/docs/ai-gateway/chat-completions.md
• https://neon.com/docs/ai-gateway/anthropic-messages.md
• https://neon.com/docs/ai-gateway/openai-responses.md
• https://neon.com/docs/ai-gateway/gemini.md
• https://neon.com/docs/ai-gateway/authentication.md
• https://neon.com/docs/ai-gateway/troubleshooting.md

Limitations

• Use this skill only when the task clearly matches its upstream product or API scope.
• Verify commands, API behavior, pricing, quotas, credentials, and deployment effects against current official documentation before making changes.
• Do not treat generated examples as a substitute for environment-specific tests, security review, or user approval for destructive or costly actions.