diff --git a/README.md b/README.md index 5ebc7bc..4ce2969 100644 --- a/README.md +++ b/README.md @@ -84,6 +84,16 @@ ready-to-run configs: [`examples/remote/`](examples/remote/). Architecture, auth planes, pipeline internals, operations, and embedding guide: [`docs/architectural/`](docs/architectural/). +## Deployment + +Running it outside hoop, on GCP or AWS: [`docs/deployment/`](docs/deployment/). +Start with [the constraints page](docs/deployment/README.md) — the daemon +keeps sessions, approvals, and auth-server state in memory, so it is +single-replica by construction and a round-robin load balancer in front of +two instances answers `404 unknown session`. That page also covers the +container image, TLS termination, state volumes, and the exposure +checklist; + ## HTTP surface | Endpoint | Purpose | @@ -126,10 +136,10 @@ cat sessions/.jsonl | jq -r '[.time,.event,.tool // "",.reason // ""] | @ts ## State -mcpproxy stores OAuth tokens and DCR client registrations under -`$MCPPROXY_STATE_DIR` (default `~/.mcpproxy`), encrypted with AES-256-GCM -under a generated keyfile. The embedded auth server keeps its sessions in -memory. +mcpproxy stores outbound OAuth tokens under `$MCPPROXY_STATE_DIR` (default +`~/.mcpproxy`), encrypted with AES-256-GCM under a generated keyfile. DCR +client registrations are not persisted, so a restart re-registers. The +embedded auth server keeps its sessions in memory. ## Embedding diff --git a/docs/architectural/README.md b/docs/architectural/README.md index 737220c..576eedb 100644 --- a/docs/architectural/README.md +++ b/docs/architectural/README.md @@ -10,3 +10,6 @@ - [running.md](running.md): build, run, configure, operate, debug - [embedding.md](embedding.md): the gateway as a library, plus the hoop integration seams + +Deployment guides (GCP, AWS, containers, hoop comparison) live one level +up in [../deployment/](../deployment/). diff --git a/docs/architectural/auth.md b/docs/architectural/auth.md index a981800..2421392 100644 --- a/docs/architectural/auth.md +++ b/docs/architectural/auth.md @@ -72,8 +72,9 @@ Design points: at the IdP. That removes the browser step and adds a trust configuration on the IdP side. -Tokens and DCR registrations persist under `$MCPPROXY_STATE_DIR` (default -`~/.mcpproxy`), encrypted with a generated keyfile. +Tokens persist under `$MCPPROXY_STATE_DIR` (default `~/.mcpproxy`), +encrypted with a generated keyfile. DCR registrations do not: the daemon +leaves the client store unset, so each restart re-registers. ## Plane 3: the embedded authorization server diff --git a/docs/architectural/running.md b/docs/architectural/running.md index feb4588..4ff7341 100644 --- a/docs/architectural/running.md +++ b/docs/architectural/running.md @@ -77,9 +77,13 @@ jq -r '[.time, .event, .tool // "", .reason // ""] | @tsv' sessions/.jsonl ### State -- `$MCPPROXY_STATE_DIR` (default `~/.mcpproxy`): OAuth tokens plus DCR - client registrations, AES-256-GCM encrypted. `tokens.key` is the - keyfile, so protect it and back it up alongside `tokens.enc`. +- `$MCPPROXY_STATE_DIR` (default `~/.mcpproxy`): outbound OAuth tokens, + AES-256-GCM encrypted. `tokens.key` is the keyfile, so protect it and + back it up alongside `tokens.enc`. Losing the key while keeping the + ciphertext makes the daemon fail to start. +- DCR client registrations are **not** persisted: the daemon leaves the + client store unset, so each restart re-registers with the upstream + authorization server. - The embedded auth server keeps sessions and codes in memory, so clients re-authenticate after a restart. diff --git a/docs/deployment/README.md b/docs/deployment/README.md new file mode 100644 index 0000000..cbf24fe --- /dev/null +++ b/docs/deployment/README.md @@ -0,0 +1,182 @@ +# Deploying mcpproxy + +Guides for running the standalone daemon outside hoop: + +- [gcp.md](gcp.md): Cloud Run and GKE +- [aws.md](aws.md): ECS Fargate and EKS +- [container.md](container.md): the image every cloud guide builds on + +Read this page first. Every constraint below applies to both clouds, and +each one has bitten a plausible deployment plan. + +## The daemon is single-replica by construction + +All mutable state lives in Go maps in one process. Nothing is shared, +coordinated, or externalized: + +| State | Where | Consequence | +|---|---|---| +| MCP sessions | `sess map[string]*conn` ([gateway/gateway.go](../../gateway/gateway.go)) | a session is served by exactly one process | +| Held approvals | in-memory, holds live channels and timers ([approval/store.go](../../approval/store.go)) | a restart drops pending approvals | +| Embedded auth server | five maps plus the signing key ([auth/authserver/store.go](../../auth/authserver/store.go)) | a restart logs everyone out | +| stdio children | one process group per session ([backend/stdio.go](../../backend/stdio.go)) | pinned to the process that spawned them | + +Start with **one replica** and scale vertically. That is not a placeholder +recommendation; the sections below are what you must solve before a second +replica is correct. + +### Sessions require affinity, and the failure is a 404 + +`initialize` mints a random session id and stores the connection in a +process-local map. Every later request presents `Mcp-Session-Id`, and the +lookup is a bare map read: a miss returns `404 unknown session` +([gateway/http.go](../../gateway/http.go)). + +Two daemons, a session opened on the first, one request sent to the second: + +``` +initialize on A -> 200 session: 456561d29fc6354bb20940410b314428 +A's session -> A : 200 OK +A's session -> B : 404 Not Found <-- round-robin LB + body: unknown session +``` + +Behind a round-robin load balancer with N replicas, roughly (N-1)/N of +post-initialize requests fail this way. Clients see intermittent 404s, not +a clean outage, which is the worst kind of failure to debug. + +If you must run more than one replica, the load balancer has to hash on the +`Mcp-Session-Id` **request header**. Cookie-based stickiness does not help: +MCP clients are not browsers and will not return a cookie. This rules out +AWS ALB's built-in stickiness, which is cookie-based in both its +duration-based and application-based forms. GCP can do it with +`HEADER_FIELD` affinity; see [gcp.md](gcp.md). + +### Approvals cannot be load balanced at all + +Session affinity does not rescue the review API. A reviewer is a different +client, on a different path, with no session header, and the approval id +lives in the URL. No routing rule can send `POST /approvals/{id}/approve` +to the replica holding that call, and the wrong replica returns 404. + +Run one replica if you use approvals. + +### The embedded auth server is worse + +Its OAuth legs arrive from three different user agents (the MCP client +registers, a browser authorizes and hits the callback, the client redeems +the code) with no shared routing key. Register on A and authorize on B and +you get `400 invalid_client`. Each replica also publishes a different JWKS, +cached for an hour. + +Do not run `auth_server` with more than one replica. Set +`auth_server.signing_key_file` to a mounted path, or the key is regenerated +on every restart and every previously issued token breaks. + +## Storage + +Two directories, with opposite sharing rules. + +**`$MCPPROXY_STATE_DIR`** (default `~/.mcpproxy`, created `0700`) holds +exactly two files: `tokens.enc`, the AES-256-GCM sealed outbound OAuth +grants, and `tokens.key`, the raw 32-byte key at `0600`. + +- Give each replica its **own** state dir. The write path is atomic + (temp + rename) but the mutex is process-local with no file lock, so + concurrent replicas clobber each other's grants last-writer-wins, and + each runs its own refresh loop against the same rotating refresh token. +- Persist it if any backend uses `auth: oauth`. If the key is regenerated + against a surviving `tokens.enc`, the daemon **fails to boot** with a + wrong-key error rather than starting fresh. +- Re-authorizing an OAuth backend needs the loopback browser flow, which is + impossible in a headless container. Seed the grant locally and mount the + resulting files, or use `auth: static` / token exchange instead. +- The default depends on `$HOME`. A container without `HOME` set resolves + the state dir to `/.mcpproxy` at the filesystem root. Always set + `MCPPROXY_STATE_DIR` explicitly. + +**`wal_dir`** is the one thing that is safe to share. Session ids are +128-bit random and each session owns its own `.jsonl`, so filenames +never collide. Records are buffered (32 KiB) and fsynced when the session +ends, so a `SIGKILL` loses the tail of every live session. Give the daemon +at least 60s to shut down. + +## Base image + +The daemon spawns `command:` directly, so a `scratch` or `distroless` image +fails for stdio backends with `exec: not found`, surfacing as +`502 failed to start session`. All ten stdio examples in this repo use +`npx`, so the image needs Node (and Python if you use `uvx`). Remote-only +configs (`transport: streamable-http`) have no such requirement and can use +a minimal image. + +`npx -y` also downloads from the npm registry on the session hot path. Bake +your MCP servers into the image instead; see [container.md](container.md). + +## Sizing + +Live child processes = sessions × stdio backends. They are not pooled. Size +`pids` limits and memory accordingly, and budget about 3 fds per child. + +## Shutdown + +`SIGTERM` drains HTTP for up to 10s, then closes every session, which +process-group kills stdio children (`SIGTERM`, then `SIGKILL` after a 3s +grace). Worst case is roughly 6.25s per backend after the drain. Allow +**at least 60s** of termination grace; a shorter window truncates WAL +records and can orphan children. + +Verified locally: three live sessions, `SIGTERM`, zero orphaned children. + +## Exposure checklist + +The daemon serves **plaintext HTTP only**. There is no TLS field in the +config. Terminate TLS at the load balancer. + +Before exposing it beyond localhost: + +- [ ] `listen: 0.0.0.0:` — the default `127.0.0.1:8000` accepts no + external traffic. `${PORT}` expands, which Cloud Run needs. +- [ ] `inbound.mode` set to `oidc`, `github`, `awssts`, or `static`. The + default is `anonymous`, and `local` resolves the OS user once at + startup, which is meaningless in a container. +- [ ] `approvals.api_token` set. Empty means the review API is + unauthenticated: `GET /approvals/` returns `200 []` to anyone. +- [ ] `MCPPROXY_STATE_DIR` set explicitly. +- [ ] `auth_server.signing_key_file` set if `auth_server` is enabled. + +### Known rough edge + +With `inbound.mode: oidc` and no `auth_server`, the daemon advertises its +RFC 9728 resource identifier as `"http://" + listen` +([cmd/mcpproxyd/main.go](../../cmd/mcpproxyd/main.go)). Behind a +TLS-terminating load balancer that publishes something like +`http://0.0.0.0:8080` to clients, and there is no config override. Clients +that rely on protected-resource discovery will not find the gateway; point +them at the issuer directly. + +## Health checks + +`GET /healthz` returns `200 ok` and is always registered. It reports +process liveness only — not backend health — which is what you want for a +liveness probe, since a dead backend should end one session, not restart +the daemon. + +## Metrics + +Set `telemetry.prometheus_path` to expose `/metrics`. Useful series: + +- `mcpproxy_messages_total{direction,method,tool,decision,backend}` +- `mcpproxy_message_duration_seconds{direction,backend,method}` + +Alert on `decision="kill"` (rug pull) and on `mcp.guardrail_block` in the +logs. + +`mcpproxy_sessions_active` is exported but **always reads 0**. The +increment and decrement calls exist in `telemetry` but nothing in the +daemon or gateway invokes them. Do not build an alert on it. +`examples/12-observability.yaml` advertises the gauge; that comment is +misleading. + +The `tool` label is unbounded by design. A backend exposing many tools +gives you one series per tool. diff --git a/docs/deployment/aws.md b/docs/deployment/aws.md new file mode 100644 index 0000000..66857b0 --- /dev/null +++ b/docs/deployment/aws.md @@ -0,0 +1,206 @@ +# Deploying mcpproxy on AWS + +Read [README.md](README.md) first: the daemon is single-replica by +construction, and that shapes every choice below. Build the image with +[container.md](container.md). + +Two options: + +| | ECS Fargate | EKS | +|---|---|---| +| Ops burden | low | high | +| Persistent state | EFS access point | PVC (EBS) | +| Session affinity | not achievable via ALB | not achievable via ALB | +| Best for | most deployments | you already run EKS | + +**Neither can safely run more than one task.** The reason is specific to +AWS and worth stating plainly. + +## ALB stickiness does not work for MCP + +ALB target-group stickiness is cookie-based in both forms: load-balancer +generated (`AWSALB`) and application-based. Both require the client to +store and return a `Set-Cookie` value. + +MCP clients are not browsers. They will not return the cookie, so every +request hashes fresh and lands on an arbitrary target. Post-`initialize` +requests then hit a task that has never heard of the session and get +`404 unknown session`. + +ALB offers no request-header-based routing to a *specific* target — only +listener rules that select a target group, which does not identify a +task. There is no configuration that makes multi-task mcpproxy correct +behind an ALB. + +**Run exactly one task.** Scale vertically. + +## ECS Fargate + +### Task definition + +```json +{ + "family": "mcpproxy", + "networkMode": "awsvpc", + "requiresCompatibilities": ["FARGATE"], + "cpu": "1024", + "memory": "2048", + "runtimePlatform": { "cpuArchitecture": "ARM64", "operatingSystemFamily": "LINUX" }, + "executionRoleArn": "arn:aws:iam::ACCOUNT:role/ecsTaskExecutionRole", + "taskRoleArn": "arn:aws:iam::ACCOUNT:role/mcpproxyTaskRole", + "volumes": [ + { + "name": "state", + "efsVolumeConfiguration": { + "fileSystemId": "fs-XXXX", + "transitEncryption": "ENABLED", + "authorizationConfig": { "accessPointId": "fsap-XXXX", "iam": "ENABLED" } + } + } + ], + "containerDefinitions": [ + { + "name": "mcpproxy", + "image": "ACCOUNT.dkr.ecr.REGION.amazonaws.com/mcpproxy:TAG", + "portMappings": [{ "containerPort": 8080, "protocol": "tcp" }], + "environment": [ + { "name": "PORT", "value": "8080" }, + { "name": "MCPPROXY_STATE_DIR", "value": "/var/lib/mcpproxy" } + ], + "secrets": [ + { + "name": "MCPPROXY_APPROVAL_TOKEN", + "valueFrom": "arn:aws:secretsmanager:REGION:ACCOUNT:secret:mcpproxy-approval-token" + } + ], + "mountPoints": [ + { "sourceVolume": "state", "containerPath": "/var/lib/mcpproxy", "readOnly": false } + ], + "healthCheck": { + "command": ["CMD-SHELL", "node -e 'require(\"http\").get(\"http://localhost:8080/healthz\",r=>process.exit(r.statusCode===200?0:1)).on(\"error\",()=>process.exit(1))'"], + "interval": 30, + "timeout": 5, + "retries": 3, + "startPeriod": 10 + }, + "stopTimeout": 90, + "linuxParameters": { "initProcessEnabled": true }, + "logConfiguration": { + "logDriver": "awslogs", + "options": { + "awslogs-group": "/ecs/mcpproxy", + "awslogs-region": "REGION", + "awslogs-stream-prefix": "mcpproxy" + } + } + } + ] +} +``` + +Three fields deserve explanation: + +- **`stopTimeout: 90`** (max 120 on Fargate). ECS sends `SIGTERM`, waits, + then `SIGKILL`s. The daemon drains HTTP for 10s, then tears down each + session's backends with a 3s grace before `SIGKILL`, serialised per + session. The default 30s can cut that short, truncating WAL records. +- **`initProcessEnabled: true`**. Each session spawns a process tree + (`npm exec` → `sh` → `node` for an `npx` backend). The daemon + process-group kills them, but an init process reaps any stragglers + instead of leaving zombies to accumulate against the task's pid budget. +- **`healthCheck` uses `node`, not `curl`.** `node:22-slim` ships no curl, + so a `curl -fsS` probe fails every time and ECS kills a healthy task. + Node is guaranteed present on that base. On a distroless image there is + no shell at all — drop the container health check and rely on the ALB + target-group check instead. + +### Service + +```sh +aws ecs create-service \ + --cluster mcpproxy \ + --service-name mcpproxy \ + --task-definition mcpproxy \ + --desired-count 1 \ + --launch-type FARGATE \ + --deployment-configuration 'maximumPercent=100,minimumHealthyPercent=0' \ + --health-check-grace-period-seconds 30 \ + --network-configuration 'awsvpcConfiguration={subnets=[subnet-XXXX],securityGroups=[sg-XXXX]}' +``` + +`maximumPercent=100` with `minimumHealthyPercent=0` forces stop-then-start. +The rolling default would briefly run two tasks, and during that overlap +the ALB splits traffic between a task holding live sessions and a new one +that 404s them. A brief outage is the honest tradeoff for a daemon whose +sessions cannot migrate. + +### EFS for state + +Only needed for `auth: oauth` backends. Use an access point with the +container's uid so the daemon can create `0700` directories: + +```sh +aws efs create-access-point --file-system-id fs-XXXX \ + --posix-user 'Uid=10001,Gid=10001' \ + --root-directory 'Path=/mcpproxy,CreationInfo={OwnerUid=10001,OwnerGid=10001,Permissions=0700}' +``` + +Never mount one access point into two tasks. The grant store has no file +lock; concurrent writers clobber each other last-writer-wins, and both run +refresh loops against the same rotating refresh token. + +Consider skipping EFS entirely: `auth: static` from Secrets Manager, or +token exchange, avoids durable state. The interactive OAuth flow needs a +loopback browser and cannot complete in Fargate regardless. + +### ALB + +```sh +# SSE streams send no keepalive; the 60s default closes idle streams. +aws elbv2 modify-load-balancer-attributes --load-balancer-arn ARN \ + --attributes Key=idle_timeout.timeout_seconds,Value=3600 + +aws elbv2 modify-target-group-attributes --target-group-arn ARN \ + --attributes Key=deregistration_delay.timeout_seconds,Value=90 +``` + +Target-group health check: path `/healthz`, matcher `200`, interval 30s. + +Terminate TLS on the ALB with an ACM certificate; the daemon is plaintext +only. Restrict the task security group to the ALB security group. + +### IAM + +`inbound.mode: awssts` validates a caller's signed STS identity against +`aws_role_mappings`, needing no AWS credentials on the task. The task role +is only for pulling secrets and mounting EFS. Grant nothing else. + +## EKS + +Use the GKE Deployment manifest in [gcp.md](gcp.md) — it is portable — with +these substitutions: + +- `storageClassName: gp3` on the PVC, `ReadWriteOnce`. +- AWS Load Balancer Controller ingress annotations: + +```yaml +metadata: + annotations: + alb.ingress.kubernetes.io/scheme: internal + alb.ingress.kubernetes.io/target-type: ip + alb.ingress.kubernetes.io/healthcheck-path: /healthz + alb.ingress.kubernetes.io/load-balancer-attributes: idle_timeout.timeout_seconds=3600 + alb.ingress.kubernetes.io/certificate-arn: arn:aws:acm:REGION:ACCOUNT:certificate/XXXX +``` + +Keep `replicas: 1` and `strategy: Recreate`. The ALB stickiness limitation +above applies identically here; `target-type: ip` routes to pod IPs but +still cannot pin a session to a pod without a cookie. + +Use IRSA for Secrets Manager access rather than node instance roles. + +## Cost note + +`--min-instances`-style always-on is mandatory here too: there is no +scale-to-zero option that preserves sessions. A single 1 vCPU / 2 GB +Fargate task running continuously is the floor. diff --git a/docs/deployment/container.md b/docs/deployment/container.md new file mode 100644 index 0000000..a510a54 --- /dev/null +++ b/docs/deployment/container.md @@ -0,0 +1,163 @@ +# Container image + +This repo ships no Dockerfile. Both cloud guides build on the image below. + +Read [README.md](README.md) first for the constraints that shape it. + +## Choosing a base + +The daemon `exec`s each stdio backend's `command:` directly, so the image +must contain that command's runtime. This is the single most common +deployment failure. + +| Backends | Base | Notes | +|---|---|---| +| remote only (`transport: streamable-http`) | `gcr.io/distroless/static` | nothing is spawned | +| stdio via `npx` | `node:22-slim` | all ten stdio examples here | +| stdio via `uvx` | `python:3.13-slim` + `uv` | | + +Get this wrong and the daemon starts, passes health checks, and fails every +`initialize` with `502 failed to start session`. + +## Dockerfile + +```dockerfile +# syntax=docker/dockerfile:1 +FROM golang:1.26 AS build +WORKDIR /src +COPY go.mod go.sum ./ +RUN go mod download +COPY . . +# CGO_ENABLED=0: no cgo in the daemon, and a static binary runs on any base. +RUN CGO_ENABLED=0 go build -trimpath -o /out/mcpproxyd ./cmd/mcpproxyd +RUN CGO_ENABLED=0 go build -trimpath -o /out/configcheck ./cmd/configcheck + +# node:22-slim because the stdio backends below run under npx. Use +# gcr.io/distroless/static for a remote-only config. +FROM node:22-slim +RUN useradd --create-home --uid 10001 mcpproxy +COPY --from=build /out/mcpproxyd /usr/local/bin/ +COPY --from=build /out/configcheck /usr/local/bin/ + +# Pre-install the MCP servers you actually use. Without this, `npx -y` +# reaches registry.npmjs.org on the session hot path: every initialize +# pays the download, and an npm outage becomes your outage. +RUN npm install -g @modelcontextprotocol/server-everything@2026.7.4 + +# HOME matters: the default state dir is $HOME/.mcpproxy, and an unset HOME +# resolves it to /.mcpproxy at the filesystem root. Set it explicitly anyway. +# The VOLUME below is created root-owned, so a non-root USER cannot write +# to it. Create and chown it first, or the daemon dies at startup with +# "wal: create dir ...: permission denied". +RUN mkdir -p /var/lib/mcpproxy && chown 10001:10001 /var/lib/mcpproxy +USER 10001 +ENV HOME=/home/mcpproxy \ + MCPPROXY_STATE_DIR=/var/lib/mcpproxy +VOLUME /var/lib/mcpproxy + +COPY config.yaml /etc/mcpproxy/config.yaml +EXPOSE 8080 +ENTRYPOINT ["mcpproxyd", "-config", "/etc/mcpproxy/config.yaml"] +``` + +Pin the MCP server version. `npx -y some-server` resolves to whatever is +latest at session start, which is a live dependency on third-party code +inside your security gateway. Rug-pull detection catches a catalog that +changes mid-session, not one that changed between deploys. + +## Config + +`listen` must bind `0.0.0.0`; the `127.0.0.1:8000` default accepts no +traffic from outside the container. `${VAR}` expands anywhere in the file, +so `${PORT}` works where the platform assigns a port. + +```yaml +listen: 0.0.0.0:${PORT} + +inbound: + mode: oidc + issuer: https://your-idp.example.com + audience: mcpproxy + +backends: + everything: + transport: stdio + command: ["npx", "-y", "@modelcontextprotocol/server-everything"] + +approvals: + api_token: ${MCPPROXY_APPROVAL_TOKEN} + +telemetry: + prometheus_path: /metrics + +wal_dir: /var/lib/mcpproxy/sessions +``` + +Validate before shipping — `configcheck` is in the image for exactly this: + +```sh +docker run --rm --entrypoint configcheck -e PORT=8080 \ + -v "$PWD/config.yaml:/c.yaml" your-image /c.yaml +``` + +`--entrypoint` is required: the image's ENTRYPOINT is `mcpproxyd`, so a +bare `docker run your-image configcheck /c.yaml` passes `configcheck` to +the daemon as an argument and starts the server instead. + +## Verify the image + +Do not trust a container that merely starts. Exercise a real MCP session, +which is what proves the stdio runtime is present: + +```sh +docker run -d --name mcpproxy -p 8080:8080 -e PORT=8080 your-image + +curl -fsS localhost:8080/healthz # -> ok + +SID=$(curl -sD- -o/dev/null -X POST localhost:8080/mcp \ + -H 'Content-Type: application/json' \ + -H 'Accept: application/json, text/event-stream' \ + -d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{ + "protocolVersion":"2025-11-25","capabilities":{}, + "clientInfo":{"name":"probe","version":"1"}}}' \ + | awk -F': ' '/[Mm]cp-[Ss]ession-[Ii]d/{print $2}' | tr -d '\r') + +curl -sS -X POST localhost:8080/mcp \ + -H 'Content-Type: application/json' \ + -H 'Accept: application/json, text/event-stream' \ + -H "Mcp-Session-Id: $SID" \ + -d '{"jsonrpc":"2.0","id":2,"method":"tools/list"}' +``` + +A `tools/list` that returns tools means the backend spawned. A `502` on +`initialize` means the runtime is missing from the base image. + +Check that shutdown reaps children, since an orphaned MCP server outliving +its session is a credential-holding process nobody is watching. `node:22-slim` +ships no `ps`, so read `/proc` directly: + +```sh +# one leaf MCP server per live session +docker exec mcpproxy sh -c \ + 'c=0; for d in /proc/[0-9]*; do + tr "\0" " " < $d/cmdline 2>/dev/null \ + | grep -q "^node /usr/local/bin/mcp-server-everything" && c=$((c+1)) + done; echo $c' + +docker stop -t 60 mcpproxy +``` + +Each session actually spawns a three-process tree (`npm exec` → `sh` → +`node`), which is why teardown signals the whole process group rather than +the direct child. + +## Image hygiene + +- Run as non-root. The daemon needs no privileges; it binds an unprivileged + port and spawns children as the same user. +- Do not bake secrets. `${VAR}` expansion at config load is the intended + path; inject through the platform's secret manager. +- `mcpproxyd -version` prints the build tag, `dev` unless you set + `-ldflags "-X main.version=vX.Y.Z"`. +- A read-only root filesystem works if `MCPPROXY_STATE_DIR` and `wal_dir` + are writable mounts. diff --git a/docs/deployment/gcp.md b/docs/deployment/gcp.md new file mode 100644 index 0000000..068d73e --- /dev/null +++ b/docs/deployment/gcp.md @@ -0,0 +1,236 @@ +# Deploying mcpproxy on GCP + +Read [README.md](README.md) first: the daemon is single-replica by +construction, and that shapes every choice below. Build the image with +[container.md](container.md). + +Two options: + +| | Cloud Run | GKE | +|---|---|---| +| TLS + managed certs | built in | needs Ingress/Gateway + ManagedCertificate | +| Scale to zero | yes, but see below | no | +| Session affinity | on/off switch, cookie-based | `HEADER_FIELD` on the backend service | +| Persistent state | GCS FUSE or Filestore | PersistentVolumeClaim | +| Best for | remote backends, static/OIDC inbound | stdio backends, approvals, auth server | + +Pick Cloud Run for a remote-backend gateway. Pick GKE if you need stdio +backends at volume, approvals, or the embedded auth server. + +## Cloud Run + +### What works + +`listen: 0.0.0.0:${PORT}` — Cloud Run injects `PORT`, and the config +expands `${VAR}` at load, so no wrapper script is needed. Verified: the +daemon logs `addr=0.0.0.0:8080`. + +### Pin it to one instance + +```sh +gcloud run deploy mcpproxy \ + --image=REGION-docker.pkg.dev/PROJECT/repo/mcpproxy:TAG \ + --region=REGION \ + --min-instances=1 \ + --max-instances=1 \ + --no-cpu-throttling \ + --timeout=3600 \ + --port=8080 \ + --set-env-vars=MCPPROXY_STATE_DIR=/var/lib/mcpproxy \ + --set-secrets=MCPPROXY_APPROVAL_TOKEN=mcpproxy-approval-token:latest +``` + +Every flag above is load-bearing: + +- `--max-instances=1`. Cloud Run's session affinity is best-effort and + cookie-based; MCP clients send no cookies. Without this cap you get + intermittent `404 unknown session` as requests spread across instances. +- `--min-instances=1`. Scaling to zero destroys every live session, all + pending approvals, and the auth server's in-memory state. +- `--no-cpu-throttling`. Cloud Run throttles CPU outside a request by + default. The daemon runs background work between requests — the session + reaper, backend pumps that deliver server-initiated SSE traffic, and + OAuth refresh. Throttled, an SSE notification stalls until the next + inbound request. +- `--timeout=3600`. The default 300s kills the `GET /mcp` SSE stream. + +Even so, Cloud Run may replace the instance for maintenance, which drops +sessions. Clients re-`initialize` and recover; approvals in flight do not. + +### State + +Cloud Run's filesystem is ephemeral. For `auth: oauth` backends, mount +durable storage or the daemon regenerates `tokens.key` and then **fails to +boot** against a surviving `tokens.enc`: + +```sh +gcloud run services update mcpproxy \ + --add-volume=name=state,type=cloud-storage,bucket=my-mcpproxy-state \ + --add-volume-mount=volume=state,mount-path=/var/lib/mcpproxy +``` + +GCS FUSE is not POSIX-complete. It handles this workload because the daemon +writes with temp-file + rename and only two small files are involved, but +**never point two services at the same bucket path** — there is no file +lock, and concurrent writers clobber grants. + +Simpler: avoid `auth: oauth` on Cloud Run. Use `auth: static` with a +Secret Manager secret, or token exchange. The loopback OAuth flow needs a +browser on the daemon's host and cannot complete headless anyway. + +### Health checks + +```sh +gcloud run services update mcpproxy \ + --liveness-probe=httpGet.path=/healthz,initialDelaySeconds=5,periodSeconds=30 +``` + +Do not add a startup probe that expects backends to be reachable. +`/healthz` reports process liveness only, which is what you want: a dead +backend should end one session, not restart the daemon. + +### Inbound auth + +Cloud Run IAM (`--no-allow-unauthenticated`) authenticates the *caller +service account*, not the MCP user. That is a useful outer perimeter, but +identity in audit events and per-user token grants still comes from +`inbound.mode`. Set both. + +## GKE + +### Single-replica Deployment + +```yaml +apiVersion: apps/v1 +kind: Deployment +metadata: + name: mcpproxy +spec: + # Not a placeholder. Sessions, approvals and auth-server state are all + # in-process; a second replica serves 404s for the first one's sessions. + replicas: 1 + strategy: + # Never run two pods at once: the old pod owns live sessions and the + # new one cannot see them. + type: Recreate + selector: + matchLabels: { app: mcpproxy } + template: + metadata: + labels: { app: mcpproxy } + spec: + # Backends get SIGTERM, then SIGKILL after a 3s grace, serialised per + # session, after a 10s HTTP drain. Too short a window truncates WAL + # records and can orphan stdio children. + terminationGracePeriodSeconds: 90 + securityContext: + runAsNonRoot: true + runAsUser: 10001 + fsGroup: 10001 + containers: + - name: mcpproxy + image: REGION-docker.pkg.dev/PROJECT/repo/mcpproxy:TAG + ports: [{ containerPort: 8080 }] + env: + - name: PORT + value: "8080" + - name: MCPPROXY_STATE_DIR + value: /var/lib/mcpproxy + - name: MCPPROXY_APPROVAL_TOKEN + valueFrom: + secretKeyRef: { name: mcpproxy, key: approval-token } + volumeMounts: + - { name: state, mountPath: /var/lib/mcpproxy } + - { name: config, mountPath: /etc/mcpproxy } + livenessProbe: + httpGet: { path: /healthz, port: 8080 } + periodSeconds: 30 + readinessProbe: + httpGet: { path: /healthz, port: 8080 } + periodSeconds: 10 + resources: + requests: { cpu: 500m, memory: 512Mi } + # Live child processes = sessions x stdio backends, not pooled. + limits: { memory: 2Gi } + volumes: + - name: state + persistentVolumeClaim: { claimName: mcpproxy-state } + - name: config + configMap: { name: mcpproxy-config } +``` + +The PVC must be `ReadWriteOnce`. `ReadWriteMany` invites a second writer, +which corrupts the grant store. + +Do not add a HorizontalPodAutoscaler. Scale the pod, not the replica count. + +### If you genuinely need more than one replica + +Only viable when **all** of the following hold: no approvals, no embedded +auth server, and each replica gets its own state volume (a StatefulSet with +`volumeClaimTemplates`). Then hash on the session header: + +```yaml +apiVersion: cloud.google.com/v1 +kind: BackendConfig +metadata: + name: mcpproxy +spec: + sessionAffinity: + affinityType: "HEADER_FIELD" + timeoutSec: 3600 # SSE streams outlive the 30s default + connectionDraining: + drainingTimeoutSec: 90 +``` + +`HEADER_FIELD` affinity hashes on `consistentHash.httpHeaderName` and +requires `localityLbPolicy: RING_HASH` or `MAGLEV`; consistent hashing is +supported on `INTERNAL_MANAGED` and `INTERNAL_SELF_MANAGED` backend +services, so check that your load balancer flavour supports it before +relying on this. Set the header name to `Mcp-Session-Id`. + +This still leaves a hole: `initialize` carries no session header, so the +first request hashes on an absent value. Sessions land somewhere and stay +there, which is what matters, but distribution is uneven. + +### SSE and idle timeouts + +The `GET /mcp` stream sends bytes only when the server has something to +say, with no keepalive frames. An idle stream can be closed by any +intermediary. Set `timeoutSec` above your expected idle gap; the session +reaper's 30-minute timeout is hardcoded and not configurable, so 3600 is a +safe ceiling. + +### TLS and the OIDC discovery caveat + +Terminate TLS at the Ingress or Gateway; the daemon is plaintext only. + +If you use `inbound.mode: oidc` without `auth_server`, the daemon +advertises its RFC 9728 resource identifier as `"http://" + listen`, so +clients discover `http://0.0.0.0:8080`. There is no override. Configure +clients with the issuer directly. + +## Observability + +Scrape `/metrics` when `telemetry.prometheus_path` is set: + +```yaml +apiVersion: monitoring.googleapis.com/v1 +kind: PodMonitoring +metadata: + name: mcpproxy +spec: + selector: + matchLabels: { app: mcpproxy } + endpoints: + - port: 8080 + path: /metrics + interval: 30s +``` + +Alert on `mcpproxy_messages_total{decision="kill"}`. Do not alert on +`mcpproxy_sessions_active` — it always reads 0 (see +[README.md](README.md)). + +For traces, set `telemetry.otlp_endpoint` to an OpenTelemetry Collector +exporting to Cloud Trace.