Skip to content

copilot: support 'extra_body' for BYOK (top_k, chat_template_kwargs, … - #329215

Open
gpotter2 wants to merge 1 commit into
microsoft:mainfrom
gpotter2:main
Open

copilot: support 'extra_body' for BYOK (top_k, chat_template_kwargs, …#329215
gpotter2 wants to merge 1 commit into
microsoft:mainfrom
gpotter2:main

Conversation

@gpotter2

@gpotter2 gpotter2 commented Aug 5, 2026

Copy link
Copy Markdown

Many inference engines have extra 'custom' attributes that are out of the classic responses / chat-completions format. This is commonly referred to as 'extra_body', because that's how it was called in the OpenAI API.

This PR:

  • adds 'extraBody' to mimic this functionality. This is a generalization of modelOptions, which currently has a hardcoded list of two allowed values. Arguably this could replace 'modelOptions', but for now I chose to keep this backwards compatible.
  • fixes a 'TODO' in 'requestLoggerImpl.ts' to include those parameters in the debug log

Some examples:

  • SGLang with GLM 5.2 (I tested this one)
"extraBody": {
	"chat_template_kwargs": {
		"enable_thinking": false
	}
}
  • vLLM top_k
"extraBody": {
	"top_k": 50
}

This comes with a side PR with the doc :D microsoft/vscode-docs#10101

fixes #321372

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds BYOK extraBody support for forwarding provider-specific request parameters to OpenAI-compatible endpoints.

Changes:

  • Propagates extraBody through BYOK model configuration and metadata.
  • Merges extra properties into all supported request-body formats.
  • Expands request logging and adds metadata propagation coverage.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
extensions/copilot/src/platform/endpoint/common/endpointProvider.ts Adds endpoint metadata for extra body properties.
extensions/copilot/src/extension/prompt/vscode-node/requestLoggerImpl.ts Logs additional request-body options.
extensions/copilot/src/extension/byok/vscode-node/customOAIProvider.ts Propagates custom OpenAI configuration.
extensions/copilot/src/extension/byok/vscode-node/customEndpointProvider.ts Propagates custom endpoint configuration.
extensions/copilot/src/extension/byok/vscode-node/azureProvider.ts Propagates Azure BYOK configuration.
extensions/copilot/src/extension/byok/node/openAIEndpoint.ts Applies extra properties to outgoing bodies.
extensions/copilot/src/extension/byok/common/test/byokProvider.spec.ts Tests metadata propagation.
extensions/copilot/src/extension/byok/common/byokProvider.ts Adds shared BYOK capability propagation.

Comment thread extensions/copilot/src/platform/endpoint/common/endpointProvider.ts
Comment thread extensions/copilot/src/extension/byok/node/openAIEndpoint.ts Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add support extra_queryextra_body and extra_headers for the custom endpoint AI model

3 participants