Skip to content

Add configurable custom providers - #23

Open
0vp wants to merge 4 commits into
mainfrom
feature/custom-providers
Open

Add configurable custom providers#23
0vp wants to merge 4 commits into
mainfrom
feature/custom-providers

Conversation

@0vp

@0vp 0vp commented Sep 3, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • add configurable OpenAI Chat Completions, OpenAI Responses, and Anthropic Messages providers
  • add the /providers setup and management flow with API-key, environment-variable, and keyless authentication
  • support model discovery and manual models, custom headers and request arguments, thinking controls, image inputs, tools, and conversation continuation
  • add dynamic BYOK registration, routing, catalog integration, encrypted key storage, and provider lifecycle handling
  • harden credential transport, recursive tool schemas, signed tool-call provenance, and Responses continuation metadata isolation
  • document custom-provider configuration and extend live schema verification tooling

Validation

  • bun run validate
    • 1,636 passed
    • 9 skipped
    • 0 failed
  • bun run build
  • live OpenAI-compatible Chat and Responses probes
  • real Ink TUI inference with high thinking
  • live Featherless GPT-OSS thinking and Read tool continuation
  • live custom Gemini high-thinking and signed Read tool continuation

Comment thread src/core/keys/ProviderKeyController.ts Outdated
{
...config,
providers: next,
...((!config.model || !currentModelReachable) && models[0]

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This branch runs before the previousId check, so re-saving a provider that's currently disabled resets config.model to models[0] even when the selected model is still listed. Repro: disable a provider, edit it, save without changes.

memoryProfile: readMemoryProfileConfig(config),
notify: typeof config.notify === "boolean" ? config.notify : undefined,
verbose: typeof config.verbose === "boolean" ? config.verbose : undefined,
providers: parseCustomProviders(config.providers),

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Entries that fail parsing are dropped here, and every writer spreads this result back to disk. A hand-edited provider with a typo gets deleted on the next unrelated save (/model, logout, etc.) with no warning.

Comment thread src/ui/App.tsx
const refreshCredentials = useCallback((): void => {
const wasExpert = config.isExpertModeEnabled;
refreshClientCredentials(config, client);
const persistedModel = readBackboardConfig().model;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This overrides --model. In that case the persisted model differs from config.modelString on purpose, so toggling any provider in /providers switches the session's model.

});
const statuses: ProviderKeyStatus[] = registry.adapters.map((adapter) => {
const entry = saved[adapter.id];
const definition = definitions.find(

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A hand-edited config.json entry with a reserved id like openai shows up here as a keyless custom provider but never routes (the registry skips reserved ids). Probably better to reject reserved ids in the parser.

setDraft((current) => ({
...current,
authType,
credential: "",

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This wipes the saved variable name when editing an env provider, so it has to be retyped every time.

@muhammadbalawal muhammadbalawal left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up review of d99dd30: the five earlier findings are fixed (branch order in saveDefinition, opaque provider preservation, reserved-id rejection, --model guard, env credential prefill) — nice work. Four smaller issues remain, posted inline below.

body.prompt_cache_key = request.cacheKey;
}
if (request.maxOutputTokens ?? modelConfig?.maxOutputTokens) {
body.max_tokens = request.maxOutputTokens ?? modelConfig?.maxOutputTokens;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Chat adapter sends deprecated max_tokens, rejected by OpenAI reasoning models

When a custom openai-chat provider configures maxOutputTokens on a model, this sends it as max_tokens. OpenAI's Chat Completions API (and Azure OpenAI) hard-reject max_tokens for reasoning models (o-series, gpt-5 family) with a 400 "Unsupported parameter: 'max_tokens' ... Use 'max_completion_tokens'", so every request for such a model fails. Azure OpenAI is exactly the endpoint class this protocol targets. Sending max_completion_tokens instead works for reasoning models, and servers that only understand max_tokens can still receive it through extraArgs. The built-in openai adapter is unaffected only because request.maxOutputTokens is never populated and it has no models config.

Comment thread src/config/providers.ts
};
if (typeof value.enabled === "boolean") provider.enabled = value.enabled;
const auth = parseAuth(value.auth);
const headers = stringRecord(value.headers);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P3] One malformed header silently drops all headers — and the loss is persisted

stringRecord returns null if any single header entry is invalid (non-string value, embedded CR/LF), and parseProvider keeps the provider with headers omitted entirely — unlike an invalid baseUrl, which rejects the whole provider. Consequences: (1) all valid headers, including a required Authorization, are silently stripped so requests go out unauthenticated and fail confusingly; (2) usesCredentials(auth, null) no longer sees the credential header, so an auth: none provider with an Authorization header and an http non-localhost baseUrl passes validation it would otherwise fail; (3) because the entry as a whole still parses, the opaque-entry preservation in saveBackboardConfig does not protect it, and the next config rewrite persists the entry without its headers field, permanently deleting the user's headers. Rejecting the whole provider here (making it opaque, like other invalid fields) fixes all three.

Comment thread src/providers/byok/registry.ts Outdated
return saved?.enabled ? saved.key : null;
}

hasCredential(

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P3] Dead exports left behind by the registry migration

ProviderRegistry.hasCredential (here), byokAdapter (L114), and byokAdapterFor (L121) have zero callers across src/, tests/, and scripts/ — every call site now uses ProviderRegistry/BUILTIN_PROVIDER_REGISTRY directly. byokAdapter's semantics also silently changed from a never-throwing record lookup to one that throws Unknown provider, which nothing covers. Remove them, or wire hasCredential into the router's hasKeyFor check where a credential-presence test would arguably be more correct than hasProviderKeyFor.

Comment thread scripts/verify-tool-schemas.ts Outdated
({ provider }) =>
!filter ||
provider.includes(filter) ||
config.modelString.includes(filter),

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P3] config.modelString.includes(filter) makes the -f filter a no-op

This clause doesn't depend on the loop item — config.modelString is constant for the whole run. Whenever the -f value happens to match the configured default model string (e.g. -f claude while the default model is anthropic/claude-...), the OR short-circuits to true for every entry, so all provider catalogs are fetched instead of only the filtered one, and a catalog failure in an unrelated, filtered-out provider aborts the whole run. Dropping this clause (leaving !filter || provider.includes(filter)) restores the documented behavior; the later probes already handle model-name filters via ${provider}/${model}.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants