Feat(aigw): Route Claude CLI traffic through {{site.ai_gateway}} and Vertex AI

[jbaross]

Description

Fixes #issue

Preview Links

/ai-gateway/use-claude-code-with-ai-gateway-vertex/

Checklist

• Tested how-to docs. If not, note why here.
• All pages contain metadata.
• Any new docs link to existing docs.
• All autogenerated instructions render correctly (API, decK, Konnect, Kong Manager).
• Style guide (capitalized gateway entities, placeholder URLs) implemented correctly.
• Every page has a description entry in frontmatter.
• Add new pages to the product documentation index (if applicable).

[netlify]

✅ Deploy Preview for kongdeveloper ready!

Name	Link
🔨 Latest commit	7d41f8d
🔍 Latest deploy log	https://app.netlify.com/projects/kongdeveloper/deploys/6a577c048706f5000867ad9a
😎 Deploy Preview	https://deploy-preview-5952--kongdeveloper.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.
🤖 Make changes	Run an agent on this branch

---

To edit notification comments on pull requests, go to your Netlify project configuration.

[Copilot]

Pull request overview

Adds a new AI Gateway v2 how-to that walks users through routing Claude Code/Claude CLI traffic through {{site.ai_gateway}} to Google Vertex AI using kongctl, aligning with the existing “Claude CLI through AI Gateway” doc set.

Changes:

• Introduces a new how-to page for configuring an AI Model Provider (Vertex) and an AI Model for Anthropic-compatible Claude CLI traffic.
• Adds Vertex + Claude Code CLI prerequisites and a verification walkthrough.

[felderi]

Replicated this how-to end-to-end against Vertex AI Claude — the config as written doesn't work; fixes below, all validated live (Claude Code → Kong → Vertex Claude returning 200). Confirmed on the latest AI Gateway 2.0 dev image against a real Vertex project (claude-sonnet-4-5@20250929 in us-east5).

🔴 Blockers

1. No beta-field strip → the generic 400 (this is the error you hit). Claude Code sends beta body fields (output_config, context_management, mcp_servers, container, service_tier) that Vertex's Anthropic schema rejects with 400 …Extra inputs are not permitted. A/B proven live: identical Claude-Code payload through Kong → strip policy on = 200, strip off = 400 output_config.effort: Extra inputs are not permitted. Add a request-transformer-advanced remove.body for those five fields. Do not strip model (it drives model selection).

2. The model id claude-sonnet-4-6 doesn't exist on Vertex (404). Vertex publisher model ids are name@YYYYMMDD — use e.g. claude-sonnet-4-5@20250929. (Enable the model in Vertex AI Model Garden first, and use a region where Claude is offered, e.g. us-east5.)

3. VERTEX_UPSTREAM_URL format is never documented (step 8 = <your_upstream_url>). For the Vertex driver this must be the full :rawPredict URL — project and region are encoded here, there are no separate provider/target fields for them:

https://<region>-aiplatform.googleapis.com/v1/projects/<project>/locations/<region>/publishers/anthropic/models/<model>:rawPredict

✅ Correct as-is / good to know

• Provider type: vertex + auth.type: gcp + service_account_json: !env GCP_SERVICE_ACCOUNT_JSON is correct — keep it. (Note: kongctl explain … model_providers incorrectly lists only auth.type: basic; the Konnect API accepts gcp.)
• No anthropic_version handling needed — the vertex driver auto-injects anthropic_version: vertex-2023-10-16 into the body itself. (This differs from the Azure Foundry how-to, where anthropic-version must be added as a header.)
• Avoid the target's gcp_environment{project_id, location_id, endpoint_id} fields — those are for custom-deployed Vertex endpoints (they require endpoint_id and fail config-load without it), not publisher Claude. Use upstream_url.

🟡 Minor

Launch note: claude --model <X> must pass the AI Model name (claude-code-vertex-sonnet), not the target model id (claude-sonnet-4-5@20250929) — passing the target trips the model selector → 503.

✅ Validated drop-in

Env vars (document the VERTEX_UPSTREAM_URL format):

export GCP_SERVICE_ACCOUNT_JSON="$(cat /path/to/service-account.json)"
# FULL :rawPredict URL — project + region are encoded here:
export VERTEX_UPSTREAM_URL="https://us-east5-aiplatform.googleapis.com/v1/projects/<project>/locations/us-east5/publishers/anthropic/models/claude-sonnet-4-5@20250929:rawPredict"

Add the missing request-compat policy:

ai_gateway_policies:
  - ref: claude-code-compat
    name: claude-code-compat
    ai_gateway: ai-quickstart
    type: request-transformer-advanced
    enabled: true
    global: false
    config:
      remove:
        headers: [anthropic-beta]
        querystring: [beta]
        body: [output_config, context_management, mcp_servers, container, service_tier]

Fix the model (real id + upstream_url + attach the policy):

ai_gateway_models:
  - ref: claude-code-vertex-sonnet
    name: claude-code-vertex-sonnet          # the value passed to `claude --model`
    ai_gateway: ai-quickstart
    type: model
    enabled: true
    formats: [{ type: anthropic }]
    config:
      route: { paths: [/] }
      model: { name_header: true }
    capabilities: [generate]
    policies: [ !ref claude-code-compat#name ]
    targets:
      - name: claude-sonnet-4-5@20250929     # NOT claude-sonnet-4-6 (404s on Vertex)
        provider: vertex-prod
        config:
          type: vertex
          upstream_url: !env VERTEX_UPSTREAM_URL   # full :rawPredict URL

Verify (unchanged):

ANTHROPIC_BASE_URL=http://localhost:8000/ claude --model 'claude-code-vertex-sonnet'

Happy to pair on the rewrite. (Companion note: PR #5948 (Azure) has the analogous fixes — the two docs share the beta-field-strip and model-selector items; Vertex differs in that the driver auto-injects the version and needs the full :rawPredict upstream.)

[felderi]

Validation feedback — I stood this up end-to-end (Claude Code → Kong → Vertex, returning 200 with msg_vrtx_… responses for both Sonnet 4.5 and 4.6), so the approach is sound. Notes, roughly in severity order:

1. (Breaking) The three separate kongctl apply blocks break the model's policy !ref. Same structural issue as the Azure how-to (#5948): the AI Model apply references policies: [ !ref claude-code-compat#name ], but claude-code-compat is created in a separate apply and isn't re-declared there → resource not found: claude-code-compat. !ref only resolves within a single apply. Fix: combine provider + policy + model into one apply, or re-declare the policy via _external in the model apply.

2. (Breaking) VERTEX_UPSTREAM_URL format is under-documented — the prereq value will 404. The prereq describes the API Endpoint as "typically https://{location}-aiplatform.googleapis.com", but upstream_url must be the full :rawPredict path:

https://<region>-aiplatform.googleapis.com/v1/projects/<project>/locations/<region>/publishers/anthropic/models/<model>:rawPredict

e.g. …/publishers/anthropic/models/claude-sonnet-4-5@20250929:rawPredict (verified 200). Just the host → 404. Worth a sentence noting publisher Claude uses upstream_url; the derive path (gcp_environment) is for Gemini on Vertex Model Garden only (its endpoint_id is required and scoped to that).

3. Target model name vs label mismatch. The target is name: claude-sonnet-4-5@20250929 (Sonnet 4.5) but display_name/title say 4.6. Both work — just make the target match the label (claude-sonnet-4-6 if you want 4.6, or update the labels to 4.5).

4. Gateway ownership: use _external, don't fully declare it. The doc declares ai_gateways: - ref: ai-quickstart, name: ai-quickstart, …, but that gateway's CP/DP is created and owned by the AI quickstart/script — kongctl should only manage the children hanging off it. Reference it link-only so kongctl doesn't try to own it:

ai_gateways:
  - ref: ai-quickstart
    _external:
      selector:
        matchFields:
          name: ai-quickstart

5. Copy-paste from the Azure doc: the AI Policy section says "remove extra headers that Azure does not support" — should say Vertex.

6. Stale prose around the model block: it mentions config.model.alias: claude-code-vertex-sonnet, but the config uses model: { name_header: true } (no alias). There's also a stray display_name: "Claude Code - Vertex - Sonnet 4.6" line sitting outside the code fence (renders as body text).

One heads-up for testers: binding the Vertex upstream requires a data-plane build newer than 2.0.1-rc.1 — that rc leaves the target upstream as the internal placeholder (ai-gateway.upstream.local) and 503s, while a current kong-ai-gateway-dev build serves it correctly. Bedrock/Azure are unaffected on both.

[cloudjumpercat]

LGTM!