LLM gateways

Agents behind Vercel AI Gateway, OpenRouter, LiteLLM, Portkey, Helicone and others: tagged sessions, billed cost per call, and A/B runs across models.

An LLM gateway gives you one key for many models. coding.sh tags sessions that went through one, adds what each call cost and which provider served it, and runs A/B experiments across models with that one key.

Supported gateways

Gateway	`gateway enrich`	`compare --gateway`	Key
Vercel AI Gateway	`/v1/generation`	`vercel`	`AI_GATEWAY_API_KEY`
OpenRouter	`/api/v1/generation`	`openrouter`	`OPENROUTER_API_KEY`
LiteLLM proxy	`/spend/logs`	`litellm`	`LITELLM_PROXY_API_KEY`
Cloudflare AI Gateway	logs API	no	`CLOUDFLARE_API_TOKEN`
Portkey	`/v1/logs`	no	`PORTKEY_API_KEY`
Helicone	`/v1/request`	no	`HELICONE_API_KEY`
Ramp Router (router.com)	cost from the response	`ramp`	`RAMP_ROUTER_API_KEY`
Requesty	cost from the response	no	`REQUESTY_API_KEY`
TrueFoundry, Kong, Bifrost, Martian	detect only	no
CLIProxyAPI	usage queue (--gateway cliproxyapi)	no	`CLIPROXYAPI_MANAGEMENT_KEY`

Every gateway in the table is detected: sessions get a gateway:<name> tag. Keys come from the standard variable, or from a [gateway.<name>] table:

toml

# ~/.coding.sh/config.toml (only needed to change the defaults)
[gateway]
default = "openrouter"            # what a bare compare --gateway uses

[gateway.litellm]
baseUrl = "http://localhost:4000"  # or LITELLM_PROXY_API_BASE

[gateway.cloudflare]
accountId = "<32 hex>"             # or CLOUDFLARE_ACCOUNT_ID
gatewayId = "prod"

Route your agent

Point the agent's base URL at the gateway. The examples below use Vercel AI Gateway with AI_GATEWAY_API_KEY. For OpenRouter, use https://openrouter.ai/api as Claude Code's base URL and your OpenRouter key.

Claude Code

json

// ~/.claude/settings.json
{ "env": {
  "ANTHROPIC_BASE_URL": "https://ai-gateway.vercel.sh/claude-code",
  "ANTHROPIC_AUTH_TOKEN": "<your AI Gateway key>",
  "ANTHROPIC_API_KEY": ""
} }

Codex

toml

# ~/.codex/config.toml
model_provider = "vercel"
model = "openai/gpt-5"

[model_providers.vercel]
name = "Vercel AI Gateway"
base_url = "https://ai-gateway.vercel.sh/codex/v1"
env_key = "AI_GATEWAY_API_KEY"
wire_api = "responses"

AI SDK

Send its telemetry to coding.sh collector, as on the OpenTelemetry page.

const { text } = await generateText({
  model: "openai/gpt-5", // any creator/model, billed through the gateway
  prompt,
  experimental_telemetry: { isEnabled: true },
});

OpenCode uses its built-in vercel or openrouter provider, and Pi uses vercel-ai-gateway or openrouter.

Tag sessions and add billed cost (built)

shell

coding.sh gateway status          # which agents route through a gateway, keys set or not
coding.sh gateway enrich          # billed cost + serving provider per call (last 30 days)
coding.sh sessions --flag gateway:openrouter

Detection is read-only. coding.sh never prints or stores your keys, and re-running enrich makes no new API calls. Cloudflare, Portkey and Helicone need the log id from their response header: the SDKs record it, agents don't keep it. See coding.sh gateway for every option.

Compare models through a gateway (built)

shell

coding.sh compare sonnet gpt5 --gateway --k 3
coding.sh compare sonnet gpt5 --gateway openrouter

A bare --gateway uses [gateway] default, else the gateway with a key, else Vercel. See coding.sh compare.

Trace an app that calls a gateway (built)

Apps traced with the TypeScript or Python SDK detect every gateway in the table on each model call. The call records the gateway and its request id, so the session is tagged and gateway enrich can look it up.

shell

// TypeScript
const openai = observe(new OpenAI({ baseURL: "https://openrouter.ai/api/v1", apiKey: process.env.OPENROUTER_API_KEY }));

# Python
client = coding_sh.observe(OpenAI(base_url="https://gateway.ai.cloudflare.com/v1/<account>/<gateway>/compat"))

How calls are matched to a gateway

The SDKs detect a gateway by base URL (a self-hosted one by its URL env var, like LITELLM_PROXY_API_BASE), by a response header only that gateway sets, or by the response id. They read headers only, never a body twice.

Request ids come from cf-aig-log-id (Cloudflare), helicone-id, x-litellm-call-id, x-portkey-trace-id and x-kong-request-id, or the response id (gen_… Vercel, gen-… OpenRouter, rqsty-… Requesty).

For enrich, OpenRouter and Requesty calls link by their response id, and LiteLLM calls through its OpenTelemetry export. Calls through a Vercel vmc/ virtual model record the model it resolved to.

Trace drains (partial)

Vercel Pro and Enterprise plans can drain one OpenTelemetry trace per gateway request. Tunnel to coding.sh collector and use <tunnel>/v1/traces as the drain URL. Each request becomes one model call with its billed cost, and failovers are not counted as extra calls. enrich then links each request to the agent session that made it.

OpenTelemetry exports from LiteLLM, Cloudflare, Portkey, OpenRouter (Broadcast), TrueFoundry and Bifrost are tagged with their gateway and cost.