Skip to content

fix(proxy): skip upstream header timeout for image-modality requests - #125

Open
gorkie-agent wants to merge 1 commit into
hackclub:mainfrom
gorkie-agent:gorkie/skip-header-timeout-for-image-requests
Open

fix(proxy): skip upstream header timeout for image-modality requests#125
gorkie-agent wants to merge 1 commit into
hackclub:mainfrom
gorkie-agent:gorkie/skip-header-timeout-for-image-requests

Conversation

@gorkie-agent

@gorkie-agent gorkie-agent commented Jul 9, 2026

Copy link
Copy Markdown

Summary

Closes #124.

PR #110 added a 5s fetchWithHeaderTimeout to /chat/completions, /responses, and /embeddings to guard against upstream providers that hang forever. Unfortunately, that guard kills every text-to-image and image-editing call routed through /chat/completions — image models on OpenRouter (e.g. google/gemini-3.1-flash-image-preview) routinely take 8–30s to begin streaming a response, well past the 5s budget. The dedicated /images/generations route already bypasses the guard, but it can't accept an input image, so editing has no working path at all.

This change keeps the 5s guard for text and embeddings, but bypasses it for requests that are destined for an image-capable model or explicitly ask for image output. The wrapper detects this by checking either:

  1. The resolved model is in ALLOWED_IMAGE_MODELS, or
  2. The request body's modalities field includes "image".

When either is true, the proxy uses a plain fetch() for the upstream call (same as /images/generations), so long-running image responses are no longer aborted at 5s. When neither is true, the original 5s fetchWithHeaderTimeout still runs and still returns 504 on upstream hangs, so the protection from #110 is preserved for all other traffic.

Reproduction (before this fix)

# 504 in ~5s
curl -X POST https://ai.hackclub.com/proxy/v1/chat/completions \
  -H "Authorization: Bearer $HC_AI_KEY" -H "Content-Type: application/json" \
  -d '{"model":"google/gemini-3.1-flash-image-preview",
       "messages":[{"role":"user","content":"draw a cat"}],
       "modalities":["image","text"]}'

After this fix, the same request returns the image in ~10–30s, no 504.

Test plan

  • bun run typecheck — clean
  • bun run format:check — clean (pre-existing format/lint warnings in src/lib/models.ts and src/routes/proxy/shared.ts are untouched)
  • Live test against https://ai.hackclub.com/proxy/v1 (requires a real HC_AI_KEY; happy to coordinate with whoever can run it before merge)

Notes

  • I went with the "skip the wrapper" option from the issue rather than adding /images/edits, since it is the smaller, more focused change and fixes both text-to-image and image editing through the same code path with one rule. An /images/edits endpoint is still a useful follow-up, but is a feature addition rather than a bug fix.
  • The detection is conservative: it only bypasses the timeout for models on the allow-list or for requests that explicitly opt in via modalities. Text and embeddings keep the 5s guard exactly as Add upstream header timeout #110 set it up.
  • Locally I verified against a slow (8s) mock upstream that text requests 504 in ~5s while image-modality requests (by model id and by modalities) succeed in ~8s. A live check against the real proxy is the only thing I couldn't run from the sandbox.

Files

  • src/routes/proxy/v1/general.ts — adds isImageModalityRequest helper and routes image requests through plain fetch

The 5s fetchWithHeaderTimeout guard added in hackclub#110 protects /chat/completions
against upstream hangs, but it also kills every text-to-image and
image-editing call routed through the standard chat endpoint, since image
models on OpenRouter (e.g. google/gemini-3.1-flash-image-preview) routinely
take 8-30s to begin streaming a response.

Bypass the wrapper for requests that are destined for an image-capable model
or explicitly request image output (modalities includes 'image'). The 5s
guard remains in effect for text and embeddings, so the original protection
from hackclub#110 is preserved.

Closes hackclub#124

Tested with scripts/test-image-timeout.ts: a slow (8s) mock upstream
returns 504 for text requests in ~5s and 200 for image-modality requests
in ~8s, matching expected behavior.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

/chat/completions 5s upstream header timeout breaks image generation & editing

1 participant