feat(models): unified createAI provider for Workers AI and AI Gateway (ai-sdk, pi-ai) - #2214
feat(models): unified createAI provider for Workers AI and AI Gateway (ai-sdk, pi-ai)#2214mattzcarey wants to merge 3 commits into
Conversation
… (ai-sdk, pi-ai)
Adds experimental agents/models/ai-sdk and agents/models/pi-ai: one createAI
factory per framework over the Workers AI binding and AI Gateway.
Workers AI is the only catalog the package keeps up with: ai("@cf/...") runs
through env.AI.run with a compat layer that maps Workers AI onto strict OpenAI
chat completions. Every other vendor brings its own provider: ai(model) clones
the user's model per call, swaps its fetch for the gateway's universal request
(env.AI.gateway(id).run) and passes the vendor's body through untouched. Binding
only, no vendor package in agents, each subpath tree-shakes to its own framework.
Includes embeddings, image, transcription, speech and reranking on Workers AI,
ProviderV4 compliance (registry and default-provider hook), a framework-neutral
core with a capture-backed Workers AI quirk table, 306 tests, docs, an example
Worker exercising every call form, and a changeset.
🦋 Changeset detectedLatest commit: 9e0af71 The changes in this PR will be included in the next version bump. This PR includes changesets to release 2 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
agents
@cloudflare/ai-chat
@cloudflare/codemode
hono-agents
@cloudflare/shell
@cloudflare/think
@cloudflare/voice
@cloudflare/worker-bundler
commit: |
…ted fetch A streaming leg that fails before its first output part (an error part, an error finish, or a broken body) now hands over to the next fallback leg, with its preamble dropped; a leg that has produced output keeps its failure. The routed fetch reads the body and signal from a Request when a provider passes one instead of (url, init).
⚪ agents import sizesMeasured 279 runtime imports as minified bundles. The primary size is gzip; raw minified size is included for diagnosis. An existing import growing by more than 10% is marked red. This report is informational.
Compared Changed imports (12)
All 279 current runtime imports
Reported by agent-think[bot]. |
There was a problem hiding this comment.
Devin Review found 1 new potential issue.
1 flag not posted on this PR by your GitHub settings — view it in Devin Review. (Configure)
| const logId = binding.aiGatewayLogId; | ||
| if (logId !== null && logId !== undefined) { | ||
| logIds.set(response, logId); |
There was a problem hiding this comment.
🟡 Concurrent requests misreport gateway logs
When concurrent calls settle together, binding.aiGatewayLogId can already identify another call. Provider metadata then links a response to the wrong log.
Prompt for agents
The binding-wide aiGatewayLogId property is mutable and shared across concurrent env.AI.run calls. In packages/agents/src/models/core/transport.ts, bindingTransport snapshots it only after awaiting raw.run, leaving a race where another completion can overwrite the property first. Preserve response-to-log correlation without relying on a shared post-await property. Prefer the response's cf-aig-log-id header when available, or change the transport and metadata flow so a per-call correlation value is captured atomically with that call's completion. Add a concurrency test where two runs complete close together and verify each response receives its own log ID.
Was this helpful? React with 👍 or 👎 to provide feedback.
Summary
Adds two experimental subpath exports,
agents/models/ai-sdkandagents/models/pi-ai: onecreateAIfactory per framework over the Workers AI binding and AI Gateway, following the Workers AI + AI Gateway unification (the model id is the whole API, the gateway is an option, Cloudflare holds the vendor credential).One rule decides everything:
ai("@cf/…")runs throughenv.AI.runwith a compat layer (core/chat-completions) that maps Workers AI onto strict OpenAI chat completions, keyed by model family from live conformance captures. Only@cf/strings are accepted.ai(model)takes any AI SDK v4 model the user built, clones it per call, swaps itsfetchfor AI Gateway's universal request (env.AI.gateway(id).run), and hands the vendor's body and headers through untouched. The vendor's provider builds and parses; we add gateway options, fallback andproviderMetadata.cloudflare.agentstakes no vendor package as a dependency or peer.@ai-sdk/providerand@earendil-works/pi-aiare optional peers; each subpath tree-shakes to its own framework.ai.embedding,ai.image,ai.transcription,ai.speech,ai.reranking(+ theirProviderV4aliases). The provider is a fullProviderV4, so it works withcreateProviderRegistryand asglobalThis.AI_SDK_DEFAULT_PROVIDER.agents/models/pi-aiis the equivalent for pi-ai (not a layer over the AI SDK one):ai("@cf/…"),ai(getBuiltinModel("anthropic", "claude-opus-4-8")), andai("anthropic/claude-opus-4-8")when pi's own generated gateway registry knows the id. Dispatch by the pi model'sapi, pi's converters, pi's metadata for thinking.What is in the change
packages/agents/src/models/core/— framework-neutral: binding transport (run+universal), gateway options and precedence, gateway provider table (host → slug, fromai-gateway-provider), Workers AI id types, SSE decoding, errors, image helpers, and the Workers AI ↔ OpenAI compat layer (quirk table + request/response/stream normalizers, every row cited to a capture).packages/agents/src/models/ai-sdk/andpackages/agents/src/models/pi-ai/— identical skeletons (index,settings,catalog,errors,transport,models/*,wires/*).packages/agents/src/tests/models/— 306 tests, fixtures taken from live captures.docs/agents/models.md,docs/agents/models-pi-ai.md,examples/next/models(a Worker exercising every call form of both modules), changeset (agentsminor).Verification
pnpm vitest --run --project workers src/tests/models: 20 files, 306 tests.pnpm run typecheck(122 projects),pnpm run build,pnpm run check:exports,sherif,oxlint,oxfmt --check .all green.dist/models/ai-sdkreferences no pi-ai;dist/models/pi-aireferences no@ai-sdk; neither imports a vendor package.Known gaps (documented)
reasoning_detailsreplay is not implemented.@cf/black-forest-labs/flux-2-*need a multipart body the JSON run path cannot send; they are off the autocomplete list rather than refused at runtime.reasoning_effortshape; othercompat.thinkingFormatvalues are the vendor package's job (stated in the docs).Follow-ups
workers-ai-providerandai-gateway-providerin cloudflare/ai as thin shims over these modules, with deprecation notices./ai/v1/*endpoints wrap rather than translate; two id spellings between the run path and the universal request) are written up separately for the gateway team.