Dynamic multi-provider AI engine & custom endpoints (BYOM) - #2
Merged
Conversation
Add OpenAI as a first-class provider and support user-registered OpenAI-compatible endpoints (Ollama, LM Studio, vLLM, OpenRouter, Groq). Model catalogs are now discovered live from each provider's model listing API instead of hardcoded lists. Server: - New OpenAICompatibleProvider core (fetch-based, no SDK) shared by the branded OpenAI provider and custom endpoints - Adaptive structured output: json_schema → json_object → prompt-based JSON with per-endpoint/model mode caching; tool calls degrade to plain chat on engines that reject tools - Live model discovery for Gemini/Claude/OpenAI/custom with guardrails: 1-year recency window, non-chat modality stripping (embeddings, audio, TTS, image/video, moderation), and alias/snapshot dedupe - SQLite-backed catalog cache (model_catalog) + custom_endpoints table - /api/endpoints CRUD with pre-flight connectivity checks - /api/config/refresh-models for 1-click catalog re-sync - Keys masked as ••••1234 in all responses; masked values never re-saved; key removal drops the catalog and cached provider instances - Key verification now lists models (no token spend) and triggers immediate discovery on save Frontend: - Settings: OpenAI key card, connection-error badges, masked key hints, custom endpoint manager with quick-fill presets, Refresh Models button, family labels in the grouped model picker - Sidebar: grouped picker includes custom endpoints; empty state links to Settings when nothing is configured - Store: active model safely falls back to the first available model when its provider/endpoint disappears Tests: 59 new tests covering recency/modality filtering, alias dedupe, key masking, adaptive JSON fallbacks, endpoint connectivity, namespaced custom model routing, and the settings/sidebar UX flows. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Bugbot is not enabled for your account, so this pull request was not reviewed. Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs. |
Model filtering (server): - New "latest generation per family" heuristic on top of the existing guardrails: classify models into families (Gemini Pro/Flash/Flash-Lite/ Gemma; Claude Fable/Opus/Sonnet/Haiku; OpenAI Flagship/Balanced/Fast/ Mini/Nano/Reasoning incl. GPT-5.6 sol/terra/luna tier names), extract generation numbers, and keep only the newest generation of each family, preferring stable releases over previews and collapsing suffix variants - Strip specialized non-chat systems: image generation (Nano Banana), music (Lyria), robotics, deep-research agents, computer-use models - OpenAI static fallback updated to the GPT-5.6 family (Sol/Terra/Luna) - Verified live: Gemini 27→4, Claude 10→4, OpenAI shows current lineup Model selection UX (frontend, mobile-first): - New ModelPicker component replaces the sidebar <select> and the long settings radio list: bottom sheet on mobile, command-palette dialog on desktop, with search, provider-grouped rows with brand accents, family chips, catalog refresh, provider-management shortcut, and an empty state that guides setup - Mobile header gains a model chip — model switching now works on mobile - Settings redesign: provider cards with status pills (Connected / Not configured / Error), active-model summary card with a Change-model action, and matching card styling for custom endpoints - Design stays within the app's token system; light/dark themes and reduced-motion are respected; 44px touch targets on mobile Tests: filter heuristics (generation extraction, preview detection, latest-per-family, specialized-model exclusion) plus reworked UI tests for the picker flows; verified visually via headless-browser screenshots in both themes and both breakpoints. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Upgrades the AI layer into an open, auto-updating, multi-provider platform:
gpt-4o-2024-08-06) collapse into their floating alias.custom:<endpointId>:<model>.json_schema→json_object→ prompt-enforced JSON, with the working mode cached per endpoint+model, so flashcard generation and other structured features work on local models with basic JSON support. Tool calls degrade to plain chat on engines that rejecttools.••••1234); masked values sent back by forms are never re-saved; removing a key drops the provider's catalog and cached instances, and the client resets the active model pointer to the first available alternative.UX
Testing
🤖 Generated with Claude Code