LESSON 4 OF 5 · DEPLOYMENT ARCHITECTURE
A proxy passes traffic. A gateway understands it.
By the end of this lesson, you can explain when a client needs an LLM gateway over a proxy, configure one for any deployment path, and show how session headers turn gateway logs into per-team chargeback.
Proxy vs. gateway
The distinction that matters for enterprise clients
Enterprise clients with multiple development teams making model calls will eventually ask: who is spending what? A proxy cannot answer that question. A gateway can.
Proxy
Forwards traffic
Routes requests from inside the network to an external endpoint. Applies firewall rules, TLS inspection, and egress controls. Sees packets. Does not understand what is inside them.
Gateway
Understands model traffic
Parses the request, applies auth policy, attributes cost to a team, enforces rate limits, logs the interaction, and routes to the right provider. The proxy gets traffic out of the building. The gateway governs what happens to it.
What a gateway provides
Five capabilities a proxy cannot replicate
A proxy has no concept of a team, a budget, or a provider. A gateway has all three. These are the five things enterprise clients with more than one team will eventually ask for.
01
Centralised authentication
Provider credentials live in one place the platform team controls. Developers hold short-lived gateway tokens. Credential rotation happens once, centrally, not across every developer machine.
02
Per-team cost attribution
Session headers map each request to a team token. The gateway answers who spent what without parsing request bodies or estimating from cloud invoices.
03
Budget enforcement
Rate limits and spend caps applied per team at the request boundary, before the model call is made. This is an enforcement feature, not a reporting feature.
04
Model routing
Switch providers or model versions behind a stable endpoint. A client running Bedrock and Vertex side by side presents both through one base URL. No developer machine changes required on a provider swap.
05
Compliance audit logging
Every model interaction logged in the client's own stack. On cloud paths where OTel telemetry to Anthropic defaults off, as covered in 3.2, the gateway audit log is where observability comes from.
Configuration
A base URL, a token, and a skip flag
Developers and architects: this section is for you. The pattern is identical across deployment paths. Only the variable names change.
# Gateway exposing the Anthropic API format export ANTHROPIC_BASE_URL=https://llm-gateway.client.com export ANTHROPIC_AUTH_TOKEN=team-token-from-gateway # Bedrock behind the gateway export CLAUDE_CODE_USE_BEDROCK=1 export ANTHROPIC_BEDROCK_BASE_URL=https://llm-gateway.client.com/bedrock export CLAUDE_CODE_SKIP_BEDROCK_AUTH=1 # Vertex behind the gateway export CLAUDE_CODE_USE_VERTEX=1 export ANTHROPIC_VERTEX_BASE_URL=https://llm-gateway.client.com/vertex export CLAUDE_CODE_SKIP_VERTEX_AUTH=1
The skip-auth flags are the centralised-auth payoff in code: Claude Code stops signing requests itself because the gateway injects provider credentials server-side. For rotating per-user tokens, configure apiKeyHelper in settings to run a script that fetches from the client's vault. One requirement for the gateway team's acceptance criteria: the gateway must forward anthropic-beta and anthropic-version headers intact. A gateway that strips them silently degrades Claude Code features, and that failure looks like a product bug until someone inspects the traffic.
Session headers
How Claude Code labels its own traffic
Every Claude Code request carries attribution headers the gateway can read without parsing the request body. These three headers are the foundation of per-team chargeback.
Session level
X-Claude-Code-Session-Id
A unique ID for the current Claude Code session. The gateway aggregates all requests in a session under this ID. This gives you session-level cost and duration reporting without touching request bodies.
Subagent level
X-Claude-Code-Agent-Id
Present on requests from subagents. When Claude Code fans out into parallel subagents, each carries its own agent ID so the gateway can attribute cost to individual agent threads within the session.
Parent relationship
X-Claude-Code-Parent-Agent-Id
Links a subagent to its parent. Enables full nested agent attribution: a session that spawns five subagents still rolls up correctly into a single team's cost report at the CoE review.
What this enables
Per-team chargeback
Map gateway tokens to teams and the three headers give you cost per team per week, sessions per developer, and budget alerts before the next invoice. The CoE metrics review and the CFO conversation, served by the same data pipe.
Non-technical leads: these headers are what enable per-team cost reporting in the Day 30 readout. You do not configure them. Claude Code sends them automatically. The gateway reads them.
Reference implementation
LiteLLM as the starting point
Anthropic's gateway documentation uses LiteLLM as its reference implementation. It is not the only option, but it has the most tested configuration patterns for Claude Code deployments.
LiteLLM proxies support both Anthropic Messages format and OpenAI-compatible format pass-through. For clients who already run OpenAI-format tooling, this means a single gateway can serve both workflows without a separate integration layer. The base-URL override and skip-auth flags documented in Section 3 work identically whether the path is the Anthropic API, Bedrock, or Vertex. Configuration details and currently tested versions are in the Anthropic LLM gateway documentation at code.claude.com/docs/en/llm-gateway.
Security notice
LiteLLM PyPI versions 1.82.7 and 1.82.8 were compromised with credential-stealing malware. Do not install those versions. Before any client deployment using LiteLLM, verify the current safe version against the Anthropic LLM gateway documentation. This is a supply chain risk, not a product limitation.
Positioning principle
Every enterprise client with more than one team will eventually ask who is spending what. Recommend a gateway before they ask, not after the first invoice surprises someone.
Section: Gateway recommendation and configuration
Add the gateway section to your Deployment Decision Document. This section records whether the engagement needs a gateway and the key configuration decisions the platform team needs to action.
1.
State whether the client requires a gateway. Use the positioning principle from this lesson as the decision criterion: more than one team, chargeback requirement, or multi-cloud routing need.
2.
If yes, record which deployment path the gateway will front (Anthropic API, Bedrock, Vertex, or a combination) and confirm which skip-auth flags apply for each path.
3.
Note the gateway acceptance criterion for the platform team handoff: the gateway must forward anthropic-beta and anthropic-version headers intact. Flag this as a pilot go/no-go requirement.
4.
If recommending LiteLLM, confirm the installed version against the Anthropic LLM gateway documentation before recording it in the document.
Lesson 4 out of 5 complete
The network layer is designed. One question left: where does the data live?
You can now recommend a gateway with a reason tied to the client's actual needs, configure it for any deployment path, and explain how session headers and per-team attribution turn gateway logs into a chargeback model the CFO can read.
UP NEXT · LESSON 5
Security questionnaire preparation
Turn deployment architecture decisions into audit-ready answers. Encryption by path, ZDR eligibility, Compliance API scope, and the data questions every enterprise security team asks.