feat: support any OpenAI-compatible embeddings endpoint - #184
Open
tianyiswufeng wants to merge 1 commit into
Open
feat: support any OpenAI-compatible embeddings endpoint#184tianyiswufeng wants to merge 1 commit into
tianyiswufeng wants to merge 1 commit into
Conversation
The OpenAI embedder hardcodes https://api.openai.com/v1/embeddings, so --embedding openai only works against OpenAI itself. Anything else that speaks the same wire format - Azure OpenAI, ollama, vLLM, LM Studio, or a gateway - has no way in, and the local provider is the only fallback. Resolve the API root the same way the API key is already resolved: an optional baseUrl option, then OPENAI_BASE_URL, then OpenAI. Nothing changes for existing users; with the variable unset the request URL is byte-identical to before. - openai-embedder.ts: DEFAULT_OPENAI_BASE_URL + resolveOpenAIBaseUrl(), trailing slashes trimmed, fetch targets `${baseUrl}/embeddings` - load-local-env.ts: allowlist OPENAI_BASE_URL so target-root .env.local and .env reach it, matching OPENAI_API_KEY and OPENAI_MODEL - doctor: print the endpoint it will call, so a misconfigured base URL is visible before a run fails - scripts/check-openai-embedder-base-url.js: resolution precedence plus a loopback HTTP server asserting batching, bearer auth, and that requests land on <OPENAI_BASE_URL>/embeddings No config-schema change: the base URL travels with the key, not the stored embedding config.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
libs/knowledge-graph/graph-context/openai-embedder.tshardcodes the endpoint:model,dimensions, andbatchSizeare all configurable, so--embedding openaiis one string away from working with anything that speaks the same wire format - Azure OpenAI, ollama, vLLM, LM Studio, an on-prem deployment, or a gateway. Today none of those are reachable, and the only fallback is the local provider.Change
Resolve the API root the same way the API key is already resolved - option, then environment, then default:
openai-embedder.ts-DEFAULT_OPENAI_BASE_URL,resolveOpenAIBaseUrl(), optionalbaseUrlonOpenAIEmbedderOptions, the request now targetsfetch(`${this.baseUrl}/embeddings`).load-local-env.ts-OPENAI_BASE_URLadded torepoEnvKeys, so target-root.env.local/.envreach it just likeOPENAI_API_KEYandOPENAI_MODEL.doctor- printsEmbeddings endpoint: <url>under the existingOPENAI_API_KEYline, so a wrong base URL is visible before a run fails.scripts/check-openai-embedder-base-url.js- new check wired intonpm test: resolution precedence and trailing-slash trimming, plus a loopback HTTP server asserting batching, bearer auth, and that requests actually land on<OPENAI_BASE_URL>/embeddings.Fully backward compatible. With
OPENAI_BASE_URLunset the request URL is byte-identical to today. No config-schema change and no version bump: the base URL travels with the key, not with storedembeddingconfig, which also keeps it out of committed config files.Worth noting for anyone pointing this at a non-OpenAI endpoint: the existing response-length validation
already makes a
dimensionsmismatch fail loudly rather than writing bad vectors into the store, andgraph-context/config.tskeys the embedding cache onprovider:model:dimensions, so switching endpoints re-embeds instead of mixing vector spaces. Both behaviours are unchanged here; the README bullet points users atembedding.model/embedding.dimensionssince thetext-embedding-3-small/ 1536 default is OpenAI-specific.Verification
npm ci && npm run typecheck && npm test && npm audit --omit=dev --audit-level=highall pass locally on Node 22.22.3 - 18 checks green including the new one. Also exercised end to end outside the test suite:OpenAIEmbedder+cosineSimilarityagainst a hosted OpenAI-compatible endpoint (bge-m3, 1024 dims, batch of 4), where retrieval ranked the expected document first on two different queries.Disclosure: I work on that gateway, which is how I hit the hardcoded URL. The change is provider-neutral - no new default, no vendor reference in code or docs - and the same one unblocks purely local setups like ollama and vLLM. Happy to drop the doctor line or the README bullet if you would rather keep the diff to the two functional files.