[Fix] Authenticate model provider API keys before saving them - #1053
Open
daniel-lxs wants to merge 1 commit into
Open
[Fix] Authenticate model provider API keys before saving them#1053daniel-lxs wants to merge 1 commit into
daniel-lxs wants to merge 1 commit into
Conversation
Connecting a hosted provider persisted the key without ever calling the provider: the save path only made a network call for the four providers with dynamic model discovery, and even those committed the key before discovery ran. A typo or a revoked key saved cleanly and first surfaced as a failed task run, attributed to Roomote rather than the credential. Probe the provider with one bounded authenticated GET before anything is written, on both the setup wizard and Models settings save paths. The endpoint and auth header come from the provider's inference-gateway descriptor, so adding a provider is a table entry. Only a rejection from the provider blocks the save; a timeout, rate limit, or outage is reported as unverified and lets the save through.
Contributor
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
Connecting a hosted model provider persisted the API key without ever authenticating it. The only network call in the save path was model discovery, gated on
provider.dynamicModels, which is set on just four providers (github-copilot,litellm,ollama,vllm). Every hosted provider, Anthropic and OpenAI included, contacted nothing at all on save. A typo, a revoked key, or a key pasted into the wrong provider saved cleanly, reported the provider connected, and first surfaced as a task failing at run time, which reads as a Roomote fault rather than a rejected credential. Even for the four dynamic providers the key was committed inside a transaction that closed before discovery ran, and the failure came back as a "couldn't list models" discovery error.Both save paths, the setup wizard and Models settings, now probe the provider with one bounded authenticated GET before anything is written:
anthropic,openai,google,xai,moonshotai,openrouter,togetherai.INFERENCE_GATEWAY_PROVIDERSdescriptor (upstreamBaseUrl+authHeader), so the only new per-provider data is a path. Adding a provider later is a table entry, not a code change.unknownand lets the save through, so a provider having a bad day cannot wedge settings.Anthropic rejected the API key (ANTHROPIC_API_KEY), status 401: "invalid x-api-key" Check the value and save it again.Key resolution is submitted value, then runtime env, then stored value, using
||rather than??because the settings form posts an empty string for fields already satisfied by a runtime env var. A key missing everywhere skips the probe and still gets the existing required-field error.Follows the shape of
validateTeamsBotCredentials(bounded timeout, save fails with the error attributed to the credential field) andvalidateAdoToken(tri-state, provider message quoted).How it was tested
Unit tests for the new module cover the probe URL and auth header per provider, the rejection message, the non-blocking
unknowncases, and the skip for endpoint providers; the save-path test asserts nothing is written when the provider rejects the key. 99 tests pass inapps/web/src/trpc/commands/task-models/.Each of the seven endpoints was also probed live with a bogus key to confirm it exists and rejects:
invalid x-api-keyIncorrect API key providedAPI key not valid.Incorrect API key provided.Invalid AuthenticationMissing Authentication headerUnauthorizedGoogle and xAI reject with 400 rather than 401, which is why the probe table has a per-provider
rejectionStatusesescape hatch, set for those two only. Each probe is a fixed GET with no body, so a 400 there is about the key and not the request. OpenRouter uses/api/v1/keyrather than/api/v1/models, which is public and would call any string valid; it was confirmed to actually read the header (three distinct 401 bodies for malformed, well-formed-unknown, and absent).Not verified: that a valid key returns 200 on each of the seven, which needs real keys.
pnpm lint:fast,pnpm check-types:fast, andpnpm knipare clean.setup-new/launch-lifecycle.test.tsfails 11 tests on a stale local test DB schema, reproduced identically on a stashed tree, unrelated to this change.Checklist
pnpm lintandpnpm check-typespass locallypnpm changeset