diff --git a/src/content/changelog/workers-ai/2026-08-28-reject-if-busy.mdx b/src/content/changelog/workers-ai/2026-08-28-reject-if-busy.mdx new file mode 100644 index 00000000000..2550376a5b6 --- /dev/null +++ b/src/content/changelog/workers-ai/2026-08-28-reject-if-busy.mdx @@ -0,0 +1,40 @@ +--- +title: Reject busy synchronous inference requests +description: Fail synchronous inference requests instead of waiting for capacity. +date: 2026-08-28 +--- + +import { TypeScriptExample } from "~/components"; + +The `rejectIfBusy` option lets synchronous Workers AI inference requests fail when capacity is unavailable. Use it when your application should not wait in a capacity queue. + +Pass the option as the third argument to the Workers AI binding: + + + +```ts +const response = await env.AI.run( + "@cf/google/gemma-4-26b-a4b-it", + { + messages: [{ role: "user", content: "Explain capacity queues." }], + }, + { rejectIfBusy: true }, +); +``` + + + +For the native REST API, add the option to the request body: + +```bash +curl --request POST \ + --url "https://api.cloudflare.com/client/v4/accounts/$ACCOUNT_ID/ai/run/@cf/google/gemma-4-26b-a4b-it" \ + --header "Authorization: Bearer $CLOUDFLARE_API_TOKEN" \ + --header "Content-Type: application/json" \ + --data '{ + "messages": [{ "role": "user", "content": "Explain capacity queues." }], + "options": { "rejectIfBusy": true } + }' +``` + +Refer to [Reject busy requests](/workers-ai/features/reject-if-busy/) for OpenAI-compatible usage and error behavior. diff --git a/src/content/docs/workers-ai/configuration/open-ai-compatibility.mdx b/src/content/docs/workers-ai/configuration/open-ai-compatibility.mdx index 8634b9ea48d..63a76b5307b 100644 --- a/src/content/docs/workers-ai/configuration/open-ai-compatibility.mdx +++ b/src/content/docs/workers-ai/configuration/open-ai-compatibility.mdx @@ -64,6 +64,14 @@ curl --request POST \ ' ``` +### Fail fast when capacity is unavailable + +For synchronous Chat Completions, set `options.rejectIfBusy` in the top-level request body. This makes the request fail instead of waiting in a capacity queue. + +OpenAI clients that preserve custom fields can send this option. Clients that remove unknown fields do not apply it, so requests proceed normally. + +Refer to [Reject busy requests](/workers-ai/features/reject-if-busy/) for examples and error behavior. + ### AI Gateway These endpoints are also compatible with [AI Gateway](/ai-gateway/usage/providers/workersai/#openai-compatible-endpoints). diff --git a/src/content/docs/workers-ai/features/reject-if-busy.mdx b/src/content/docs/workers-ai/features/reject-if-busy.mdx new file mode 100644 index 00000000000..cc32afdaf40 --- /dev/null +++ b/src/content/docs/workers-ai/features/reject-if-busy.mdx @@ -0,0 +1,91 @@ +--- +pcx_content_type: concept +title: Reject busy requests +description: Fail synchronous inference requests when capacity is unavailable. +sidebar: + order: 5 +products: + - workers-ai +--- + +import { TypeScriptExample } from "~/components"; + +Set `rejectIfBusy` when your application should not wait in a capacity queue. Workers AI rejects the synchronous inference request if capacity is unavailable. + +## Send a REST request + +For the native REST API, add `rejectIfBusy` to the request `options` object: + +```bash +curl --request POST \ + --url "https://api.cloudflare.com/client/v4/accounts/$ACCOUNT_ID/ai/run/@cf/google/gemma-4-26b-a4b-it" \ + --header "Authorization: Bearer $CLOUDFLARE_API_TOKEN" \ + --header "Content-Type: application/json" \ + --data '{ + "messages": [ + { + "role": "user", + "content": "Explain what a capacity queue is." + } + ], + "options": { + "rejectIfBusy": true + } + }' +``` + +## Use the Workers binding + +For the Workers AI binding, pass `rejectIfBusy` in the third argument to `env.AI.run()`: + + + +```ts +const response = await env.AI.run( + "@cf/google/gemma-4-26b-a4b-it", + { + messages: [ + { + role: "user", + content: "Explain what a capacity queue is.", + }, + ], + }, + { rejectIfBusy: true }, +); +``` + + + +Do not add `rejectIfBusy` to the model input object. The binding only applies this option from the third argument. + +## Call Chat Completions + +For OpenAI-compatible Chat Completions, add `options` at the top level of the request body: + +```bash +curl --request POST \ + --url "https://api.cloudflare.com/client/v4/accounts/$ACCOUNT_ID/ai/v1/chat/completions" \ + --header "Authorization: Bearer $CLOUDFLARE_API_TOKEN" \ + --header "Content-Type: application/json" \ + --data '{ + "model": "@cf/google/gemma-4-26b-a4b-it", + "messages": [ + { + "role": "user", + "content": "Explain what a capacity queue is." + } + ], + "options": { + "rejectIfBusy": true + } + }' +``` + +OpenAI clients that preserve custom fields can send this option. Clients that remove unknown fields do not apply it, so requests proceed normally. + +## Handle capacity errors + +Rejected requests return HTTP status `429` and internal error code `3040`. The error message is `Capacity temporarily exceeded, please try again.` + +Refer to [Workers AI errors](/workers-ai/platform/errors/) for error details. diff --git a/src/content/docs/workers-ai/platform/errors.mdx b/src/content/docs/workers-ai/platform/errors.mdx index dabf3b0279b..6154d78efb2 100644 --- a/src/content/docs/workers-ai/platform/errors.mdx +++ b/src/content/docs/workers-ai/platform/errors.mdx @@ -28,4 +28,4 @@ Below is a list of Workers AI errors. | Timeout | `3007` | `408` | Request timeout | | Aborted | `3008` | `408` | Request was aborted | | Account limited | `3036` | `429` | You have used up your daily free allocation of 10,000 neurons. Please upgrade to Cloudflare's Workers Paid plan if you would like to continue usage. | -| Out of capacity | `3040` | `429` | No more data centers to forward the request to | +| Out of capacity | `3040` | `429` | `Capacity temporarily exceeded, please try again.` Also returned when `rejectIfBusy` rejects a request because capacity is unavailable. |