feat(sdk/go/ai): support the Infron gateway - #874
Open
meridah7 wants to merge 1 commit into
Open
Conversation
6 tasks
Infron is an OpenAI-compatible inference gateway that serves the standard <provider>/<model> ids, so a model moves across by prefix alone: infron/moonshotai/kimi-k2.6 routes the same model the bare id names. Follows the provider shape already in this package rather than inventing a new one: - infron_attribution.go mirrors the existing attribution helper. Infron accepts the same HTTP-Referer / X-Title pair, and the attribution env vars already configured for the existing gateway are honored as fallbacks, so a deployment that already declares itself as 'AgentField AI' keeps that identity after switching gateways. - Config gains IsInfron(); DefaultConfig() reads INFRON_API_KEY and points at https://llm.onerouter.pro/v1. - client.go attaches attribution on both the sync and streaming paths. - marshalRequest opts Infron into native usage accounting and strips the routing-only 'infron/' model prefix before the request goes out (stripInfronPrefix, mirroring the prefix handling on the media path). The gateway serves the bare id, so leaving the prefix on returns 'No available providers for model infron/...'. Only a copy of the Request is rewritten; the caller's Request is untouched. One real difference is handled rather than papered over: Infron returns the native cost at the top level of the body and of the final stream chunk, rather than nested under usage. Parsed naively that leaves Usage.Cost nil, which the cost tracker reads as 'price unknown' -- usage still recorded, but with no cost and an empty cost_source instead of 'provider'. Response/StreamChunk now carry the top-level field and normalizeNativeCost folds it into Usage.Cost, so every existing consumer keeps reading one place. An explicit usage.cost always wins. A gateway key that was already honored before Infron existed keeps precedence, so adding an Infron key never reroutes an existing deployment. llm.onerouter.pro is deliberately NOT added to vouchedRewriteDomains: max_tokens and max_completion_tokens behaved identically in probing and neither could be shown to be enforced, so the conservative legacy max_tokens path stays, per the reasoning already in that comment.
meridah7
force-pushed
the
feat/infron-provider
branch
from
August 4, 2026 22:03
8574fd0 to
6323eaa
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds the Infron gateway to
sdk/go/ai, in the shape this package already uses to describe a gateway, so selecting it is an env-var change rather than a fork.Infron is an OpenAI-compatible gateway serving the standard
<provider>/<model>ids, so nothing about a model's identity changes when it runs there:infron/moonshotai/kimi-k2.6routes the same model the bare id names. I followed the attribution/config helpers already in the package rather than introducing a second way to describe a provider.Disclosure: I work on Infron. Everything below is checkable from the diff and the commands in the test plan.
sdk/go/ai/infron_attribution.goHTTP-Referer/X-Titlepair, and the attribution env vars already configured in a deployment are honored as fallbacks, so a deployment that already declares itself as "AgentField AI" keeps that identity after switching gateways.sdk/go/ai/config.goIsInfron();DefaultConfig()readsINFRON_API_KEYand points athttps://llm.onerouter.pro/v1.sdk/go/ai/client.gosdk/go/ai/model_params.goinfron/prefix before the request goes out.sdk/go/ai/response.gocostfield +normalizeNativeCost()onResponseandStreamChunk. See below.docs/ENVIRONMENT_VARIABLES.md,sdk/go/ai/README.mdsdk/go/ai/infron_attribution_test.goThe one real difference, handled rather than papered over
This package reads native cost from
usage.cost. Infron reports it at the top level of the body, and of the final stream chunk:{ "model": "deepseek/deepseek-v4-flash", "cost": 0.000002, "usage": { "prompt_tokens": 12, "completion_tokens": 9 } } // no usage.costParsed as-is that leaves
Usage.Cost == nil, whichrecordLLMUsagereads as price unknown: the call is still recorded, but with a nil cost and an emptycost_sourceinstead of"provider". Nothing errors, and it only shows up later as a hole in the cost data.ResponseandStreamChunknow carry the top-level field, andnormalizeNativeCost()folds it intoUsage.Coston parse, so every existing consumer keeps reading one place. An explicitusage.costalways wins; the fold only fills a gap.The model prefix is stripped before the wire
infron/is a routing marker for callers that select a gateway by model string, but the gateway serves the bare id, so leaving the prefix on returnsNo available providers for model infron/moonshotai/kimi-k2.6.stripInfronPrefixremoves it inmarshalRequest, mirroring the prefix handling this package already does on the media path. Only a copy of theRequestis rewritten, so the caller'sRequestandconfig.Modelare untouched andIsInfron()still reports the truth.Worth flagging either way: the chat path does not strip the pre-existing gateway prefix today (only the media path does), so a model string carrying that prefix hits the same wall against its own gateway. I left that alone rather than change existing behavior inside a PR about a new provider, but happy to send it separately if you want the two symmetrical.
Backwards compatibility
A gateway key that was already honored before Infron existed keeps precedence. Adding
INFRON_API_KEYto an existing environment never reroutes it;TestDefaultConfigExistingGatewayWinsOverInfronpins that.IsInfron()also does not match a bare shared model id (moonshotai/kimi-k2.6), only the explicitinfron/prefix or the Infron host, so gateways cannot be confused by model alone.Type of change
Usage.Costinstead of being droppedTest plan
Rebased on
mainat4bc8ce7and re-run today.cd sdk/go && go test ./...—agent,ai,client,did,inputs,typesall okcd sdk/go && go test -race ./ai/...— cleangofmt -lclean on every touched file;go vet ./ai/...cleanai.Config, callsai.NewClient, and exercisesComplete()andStreamComplete(). The attribution headers are asserted by putting a capturing reverse proxy in front of the real gateway, so what is checked is what actually went on the wire:One pre-existing failure, unrelated to this PR:
TestOpenCodeConcurrencyLimit_RealSubprocessinsdk/go/harnessfails identically on a clean checkout ofmainon macOS — same test, same parse error, verified side by side before opening this. It shells out todate +%s%N, which BSDdatedoes not support, so the test parses a literalN. It passes in CI on Linux. Happy to send that as a separate fix if useful.Test coverage
sdk-go).coverage-baseline.jsonchange needed, and here is why:mainsdk/go/aistatement coverageMeasured back to back with
-count=1; the number moves about 0.1 pp between runs on its own. That is against amax_surface_dropof 1.0 and amin_surfaceof 84.0, on the smaller of the two numbers feeding thesdk-gosurface. Patch coverage on the non-test lines this PR adds is 92.7% (101/109 coverable added lines), againstmin_patch = 80.0; the new file itself is at 95.7%,IsInfronand the prefix strip at 100%.Notes
llm.onerouter.prois deliberately not added tovouchedRewriteDomains. I probed bothmax_tokensandmax_completion_tokensand they behaved identically, with neither demonstrably enforced, so I left the conservative legacymax_tokenspath in place per the reasoning already in that comment. Easy to add if you have better information.open_codeprovider + theINFRON_API_KEYauto-select path): feat: add Infron as an open_code gateway provider SWE-AF#126. The two are independent; either can land alone.