---
title: "LLM gateways: OpenRouter, LiteLLM, Vercel AI Gateway"
description: "Agents behind Vercel AI Gateway, OpenRouter, LiteLLM, Portkey, Helicone and others: tagged sessions, billed cost per call, and A/B runs across models."
canonical: https://coding.sh/docs/ai-gateway
human_url: https://coding.sh/docs/ai-gateway
markdown_url: https://coding.sh/docs/ai-gateway.md
agents_url: https://coding.sh/agents.md
llms_url: https://coding.sh/llms.txt
updated: 2026-11-11
---

# LLM gateways

Agents behind Vercel AI Gateway, OpenRouter, LiteLLM, Portkey, Helicone and others: tagged sessions, billed cost per call, and A/B runs across models.

An LLM gateway gives you one key for many models. coding.sh tags sessions that went through one, adds what each call cost and which provider served it, and runs A/B experiments across models with that one key.

## Supported gateways

| Gateway | `gateway enrich` | `compare --gateway` | Key |
| --- | --- | --- | --- |
| Vercel AI Gateway | `/v1/generation` | `vercel` | `AI_GATEWAY_API_KEY` |
| OpenRouter | `/api/v1/generation` | `openrouter` | `OPENROUTER_API_KEY` |
| LiteLLM proxy | `/spend/logs` | `litellm` | `LITELLM_PROXY_API_KEY` |
| Cloudflare AI Gateway | logs API | no | `CLOUDFLARE_API_TOKEN` |
| Portkey | `/v1/logs` | no | `PORTKEY_API_KEY` |
| Helicone | `/v1/request` | no | `HELICONE_API_KEY` |
| Ramp Router (router.com) | cost from the response | `ramp` | `RAMP_ROUTER_API_KEY` |
| Requesty | cost from the response | no | `REQUESTY_API_KEY` |
| TrueFoundry, Kong, Bifrost, Martian | detect only | no |  |
| CLIProxyAPI | usage queue (--gateway cliproxyapi) | no | `CLIPROXYAPI_MANAGEMENT_KEY` |

Every gateway in the table is detected: sessions get a `gateway:<name>` tag. Keys come from the standard variable, or from a `[gateway.<name>]` table:

toml

```
# ~/.coding.sh/config.toml (only needed to change the defaults)
[gateway]
default = "openrouter"            # what a bare compare --gateway uses

[gateway.litellm]
baseUrl = "http://localhost:4000"  # or LITELLM_PROXY_API_BASE

[gateway.cloudflare]
accountId = "<32 hex>"             # or CLOUDFLARE_ACCOUNT_ID
gatewayId = "prod"
```

## Route your agent

Point the agent's base URL at the gateway. The examples below use Vercel AI Gateway with `AI_GATEWAY_API_KEY`. For OpenRouter, use `https://openrouter.ai/api` as Claude Code's base URL and your OpenRouter key.

### Claude Code

json

```
// ~/.claude/settings.json
{ "env": {
  "ANTHROPIC_BASE_URL": "https://ai-gateway.vercel.sh/claude-code",
  "ANTHROPIC_AUTH_TOKEN": "<your AI Gateway key>",
  "ANTHROPIC_API_KEY": ""
} }
```

### Codex

toml

```
# ~/.codex/config.toml
model_provider = "vercel"
model = "openai/gpt-5"

[model_providers.vercel]
name = "Vercel AI Gateway"
base_url = "https://ai-gateway.vercel.sh/codex/v1"
env_key = "AI_GATEWAY_API_KEY"
wire_api = "responses"
```

### AI SDK

Send its telemetry to `coding.sh collector`, as on the [OpenTelemetry](https://coding.sh/docs/otel) page.

ts

```
const { text } = await generateText({
  model: "openai/gpt-5", // any creator/model, billed through the gateway
  prompt,
  experimental_telemetry: { isEnabled: true },
});
```

OpenCode uses its built-in `vercel` or `openrouter` provider, and Pi uses `vercel-ai-gateway` or `openrouter`.

## Tag sessions and add billed cost (built)

shell

```
coding.sh gateway status          # which agents route through a gateway, keys set or not
coding.sh gateway enrich          # billed cost + serving provider per call (last 30 days)
coding.sh sessions --flag gateway:openrouter
```

Detection is read-only. coding.sh never prints or stores your keys, and re-running `enrich` makes no new API calls. Cloudflare, Portkey and Helicone need the log id from their response header: the SDKs record it, agents don't keep it. See [coding.sh gateway](https://coding.sh/docs/cli/gateway) for every option.

## Compare models through a gateway (built)

shell

```
coding.sh compare sonnet gpt5 --gateway --k 3
coding.sh compare sonnet gpt5 --gateway openrouter
```

A bare `--gateway` uses `[gateway] default`, else the gateway with a key, else Vercel. See [coding.sh compare](https://coding.sh/docs/cli/compare).

## Trace an app that calls a gateway (built)

Apps traced with the [TypeScript or Python SDK](https://coding.sh/docs/sdk) detect every gateway in the table on each model call. The call records the gateway and its request id, so the session is tagged and `gateway enrich` can look it up.

shell

```
// TypeScript
const openai = observe(new OpenAI({ baseURL: "https://openrouter.ai/api/v1", apiKey: process.env.OPENROUTER_API_KEY }));

# Python
client = coding_sh.observe(OpenAI(base_url="https://gateway.ai.cloudflare.com/v1/<account>/<gateway>/compat"))
```

How calls are matched to a gateway

The SDKs detect a gateway by base URL (a self-hosted one by its URL env var, like `LITELLM_PROXY_API_BASE`), by a response header only that gateway sets, or by the response id. They read headers only, never a body twice.

Request ids come from `cf-aig-log-id` (Cloudflare), `helicone-id`, `x-litellm-call-id`, `x-portkey-trace-id` and `x-kong-request-id`, or the response id (`gen_…` Vercel, `gen-…` OpenRouter, `rqsty-…` Requesty).

For `enrich`, OpenRouter and Requesty calls link by their response id, and LiteLLM calls through its OpenTelemetry export. Calls through a Vercel `vmc/` virtual model record the model it resolved to.

## Trace drains (partial)

Vercel Pro and Enterprise plans can drain one OpenTelemetry trace per gateway request. Tunnel to `coding.sh collector` and use `<tunnel>/v1/traces` as the drain URL. Each request becomes one model call with its billed cost, and failovers are not counted as extra calls. `enrich` then links each request to the agent session that made it.

OpenTelemetry exports from LiteLLM, Cloudflare, Portkey, OpenRouter (Broadcast), TrueFoundry and Bifrost are tagged with their gateway and cost.

To set coding.sh up for a user, follow the agent workflow at [/agents.md](https://coding.sh/agents.md).