Skip to content

feat: support any OpenAI-compatible embeddings endpoint - #184

Open
tianyiswufeng wants to merge 1 commit into
Autoloops:mainfrom
tianyiswufeng:feat/openai-embeddings-base-url
Open

feat: support any OpenAI-compatible embeddings endpoint#184
tianyiswufeng wants to merge 1 commit into
Autoloops:mainfrom
tianyiswufeng:feat/openai-embeddings-base-url

Conversation

@tianyiswufeng

@tianyiswufeng tianyiswufeng commented Aug 3, 2026

Copy link
Copy Markdown

Problem

libs/knowledge-graph/graph-context/openai-embedder.ts hardcodes the endpoint:

const response = await fetch("https://api.openai.com/v1/embeddings", {

model, dimensions, and batchSize are all configurable, so --embedding openai is one string away from working with anything that speaks the same wire format - Azure OpenAI, ollama, vLLM, LM Studio, an on-prem deployment, or a gateway. Today none of those are reachable, and the only fallback is the local provider.

Change

Resolve the API root the same way the API key is already resolved - option, then environment, then default:

export function resolveOpenAIBaseUrl(baseUrl?: string): string {
  const configured = (baseUrl ?? process.env.OPENAI_BASE_URL)?.trim();
  if (!configured) return DEFAULT_OPENAI_BASE_URL;
  return configured.replace(/\/+$/, "");
}
  • openai-embedder.ts - DEFAULT_OPENAI_BASE_URL, resolveOpenAIBaseUrl(), optional baseUrl on OpenAIEmbedderOptions, the request now targets fetch(`${this.baseUrl}/embeddings`).
  • load-local-env.ts - OPENAI_BASE_URL added to repoEnvKeys, so target-root .env.local / .env reach it just like OPENAI_API_KEY and OPENAI_MODEL.
  • doctor - prints Embeddings endpoint: <url> under the existing OPENAI_API_KEY line, so a wrong base URL is visible before a run fails.
  • scripts/check-openai-embedder-base-url.js - new check wired into npm test: resolution precedence and trailing-slash trimming, plus a loopback HTTP server asserting batching, bearer auth, and that requests actually land on <OPENAI_BASE_URL>/embeddings.
  • README - one bullet documenting the variable.

Fully backward compatible. With OPENAI_BASE_URL unset the request URL is byte-identical to today. No config-schema change and no version bump: the base URL travels with the key, not with stored embedding config, which also keeps it out of committed config files.

Worth noting for anyone pointing this at a non-OpenAI endpoint: the existing response-length validation

throw new Error(`OpenAI returned ${item.embedding.length} dimensions; expected ${this.options.dimensions}.`);

already makes a dimensions mismatch fail loudly rather than writing bad vectors into the store, and graph-context/config.ts keys the embedding cache on provider:model:dimensions, so switching endpoints re-embeds instead of mixing vector spaces. Both behaviours are unchanged here; the README bullet points users at embedding.model / embedding.dimensions since the text-embedding-3-small / 1536 default is OpenAI-specific.

Verification

npm ci && npm run typecheck && npm test && npm audit --omit=dev --audit-level=high all pass locally on Node 22.22.3 - 18 checks green including the new one. Also exercised end to end outside the test suite: OpenAIEmbedder + cosineSimilarity against a hosted OpenAI-compatible endpoint (bge-m3, 1024 dims, batch of 4), where retrieval ranked the expected document first on two different queries.

Disclosure: I work on that gateway, which is how I hit the hardcoded URL. The change is provider-neutral - no new default, no vendor reference in code or docs - and the same one unblocks purely local setups like ollama and vLLM. Happy to drop the doctor line or the README bullet if you would rather keep the diff to the two functional files.

The OpenAI embedder hardcodes https://api.openai.com/v1/embeddings, so
--embedding openai only works against OpenAI itself. Anything else that
speaks the same wire format - Azure OpenAI, ollama, vLLM, LM Studio, or
a gateway - has no way in, and the local provider is the only fallback.

Resolve the API root the same way the API key is already resolved:
an optional baseUrl option, then OPENAI_BASE_URL, then OpenAI. Nothing
changes for existing users; with the variable unset the request URL is
byte-identical to before.

- openai-embedder.ts: DEFAULT_OPENAI_BASE_URL + resolveOpenAIBaseUrl(),
  trailing slashes trimmed, fetch targets `${baseUrl}/embeddings`
- load-local-env.ts: allowlist OPENAI_BASE_URL so target-root .env.local
  and .env reach it, matching OPENAI_API_KEY and OPENAI_MODEL
- doctor: print the endpoint it will call, so a misconfigured base URL
  is visible before a run fails
- scripts/check-openai-embedder-base-url.js: resolution precedence plus
  a loopback HTTP server asserting batching, bearer auth, and that
  requests land on <OPENAI_BASE_URL>/embeddings

No config-schema change: the base URL travels with the key, not the
stored embedding config.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant