Skip to content

docs: clarify AI request protocol handling - #13731

Open
kayx23 wants to merge 7 commits into
apache:masterfrom
kayx23:agent/document-ai-protocol-detection
Open

docs: clarify AI request protocol handling#13731
kayx23 wants to merge 7 commits into
apache:masterfrom
kayx23:agent/document-ai-protocol-detection

Conversation

@kayx23

@kayx23 kayx23 commented Jul 23, 2026

Copy link
Copy Markdown
Member

Description

Several AI request formats share the same body fields: Bedrock Converse, Anthropic Messages, and OpenAI Chat Completions can contain messages, while OpenAI Responses and Embeddings both use input. APISIX resolves these overlaps by checking URI-specific formats before body-only formats, but the plugin documentation did not explain the detection order, required URI suffixes, or downstream endpoint selection consistently.

The related AI plugins also handle the detected formats differently. Some inspect or modify each protocol's native content structure, while ai-prompt-template replaces the complete request body and ai-request-rewrite creates a separate provider request. The previous prompt-template description incorrectly limited templates to Chat Completions even though the plugin accepts any JSON template supported by the downstream plugin.

This PR updates the English and Chinese plugin references to:

  • document the complete request detection order for Bedrock Converse, Anthropic Messages, OpenAI Responses, Chat Completions, Embeddings, and passthrough JSON
  • explain which URI suffixes must be preserved and that custom URI prefixes remain valid
  • clarify automatic OpenAI endpoint selection and remove redundant Embeddings endpoint overrides
  • document the separate Anthropic Chat Completions and native Messages endpoints
  • add Responses API route and request examples for ai-proxy and ai-proxy-multi
  • document format-specific behavior for prompt decoration, prompt guarding, RAG, caching, Lakera Guard, and AWS and Aliyun content moderation
  • clarify that ai-prompt-template replaces the incoming body with an arbitrary rendered JSON template
  • add a Responses API prompt-template example that performs an OpenAI web search restricted to apisix.apache.org and shows the resolved request and cited response
  • clarify that ai-request-rewrite creates a separate non-streaming provider request instead of classifying and proxying the client's request as an AI protocol

Which issue(s) this PR fixes:

N/A

Validation

  • verified request detection, protocol adapters, provider endpoint selection, cache behavior, moderation behavior, prompt rewriting, and template replacement against the implementation and protocol tests
  • validated the Responses API web-search template end to end with APISIX and OpenAI; the request returned HTTP 200 with a completed web search call and URL citation
  • ran the repository-pinned markdownlint on all changed pages
  • ran utils/fix-zh-doc-segment.py and utils/check-category.py
  • parsed the new Responses request and response JSON examples
  • ran git diff --check

Checklist

  • I have explained the need for this PR and the problem it solves
  • I have explained the changes or the new features added to this PR
  • I have added tests corresponding to this change (documentation-only; existing protocol tests cover this behavior)
  • I have updated the documentation to reflect this change
  • I have verified that this change is backward compatible

@dosubot dosubot Bot added size:L This PR changes 100-499 lines, ignoring generated files. doc Documentation things labels Jul 23, 2026
@kayx23 kayx23 changed the title docs(ai-proxy): document OpenAI request protocols docs: document OpenAI request protocols in AI plugins Jul 23, 2026
@kayx23 kayx23 changed the title docs: document OpenAI request protocols in AI plugins docs: clarify AI request protocol handling Jul 23, 2026
@dosubot dosubot Bot added size:XL This PR changes 500-999 lines, ignoring generated files. and removed size:L This PR changes 100-499 lines, ignoring generated files. labels Jul 24, 2026
juzhiyuan
juzhiyuan previously approved these changes Jul 24, 2026
Yilialinn
Yilialinn previously approved these changes Aug 4, 2026

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the English and Chinese documentation for APISIX AI plugins to more consistently explain how overlapping AI request formats are detected (URI-suffix-first vs body-only), how providers map detected protocols to upstream endpoints (notably OpenAI Responses vs Embeddings), and how related AI plugins behave for each protocol (rewrite/template/guard/decorator/RAG/cache/moderation).

Changes:

  • Document request protocol detection order (Bedrock Converse / Anthropic Messages / OpenAI Responses / Chat Completions / Embeddings / passthrough) and required URI suffix behavior.
  • Clarify provider endpoint selection (OpenAI auto-selects Responses vs Embeddings endpoints; remove redundant Embeddings endpoint overrides) and add Responses API examples for ai-proxy / ai-proxy-multi.
  • Clarify protocol-specific behavior across plugins (RAG, prompt guard/decorator, cache, Lakera Guard, AWS/Aliyun moderation) and that ai-prompt-template replaces the entire request body with an arbitrary rendered JSON template.

Reviewed changes

Copilot reviewed 22 out of 22 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
docs/en/latest/plugins/ai-request-rewrite.md Clarifies rewrite call independence from client protocol; documents non-streaming constraints.
docs/zh/latest/plugins/ai-request-rewrite.md Same as English, localized.
docs/en/latest/plugins/ai-rag.md Documents protocol-aware enrichment behavior and URI-first detection order.
docs/zh/latest/plugins/ai-rag.md Same as English, localized.
docs/en/latest/plugins/ai-proxy.md Adds protocol detection order + Responses API routing/examples; updates provider endpoint explanations.
docs/zh/latest/plugins/ai-proxy.md Same as English, localized.
docs/en/latest/plugins/ai-proxy-multi.md Adds protocol detection order + Responses API load-balancing example; updates provider endpoint explanations.
docs/zh/latest/plugins/ai-proxy-multi.md Same as English, localized.
docs/en/latest/plugins/ai-prompt-template.md Clarifies full-body replacement semantics; adds Responses web-search template example.
docs/zh/latest/plugins/ai-prompt-template.md Same as English, localized.
docs/en/latest/plugins/ai-prompt-guard.md Documents protocol-specific inspection behavior + detection order.
docs/zh/latest/plugins/ai-prompt-guard.md Same as English, localized.
docs/en/latest/plugins/ai-prompt-decorator.md Documents protocol-specific decoration behavior + detection order.
docs/zh/latest/plugins/ai-prompt-decorator.md Same as English, localized.
docs/en/latest/plugins/ai-lakera-guard.md Clarifies protocol coverage and request/response scanning behavior.
docs/zh/latest/plugins/ai-lakera-guard.md Same as English, localized.
docs/en/latest/plugins/ai-cache.md Clarifies exact-vs-semantic cache applicability across protocols.
docs/zh/latest/plugins/ai-cache.md Same as English, localized.
docs/en/latest/plugins/ai-aws-content-moderation.md Documents supported protocols and protocol-native extraction/deny formatting.
docs/zh/latest/plugins/ai-aws-content-moderation.md Same as English, localized.
docs/en/latest/plugins/ai-aliyun-content-moderation.md Documents supported protocols + URI-first detection order and format-specific behavior.
docs/zh/latest/plugins/ai-aliyun-content-moderation.md Same as English, localized.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread docs/en/latest/plugins/ai-proxy.md Outdated
Comment thread docs/zh/latest/plugins/ai-proxy.md Outdated
Comment thread docs/zh/latest/plugins/ai-proxy-multi.md Outdated
Comment thread docs/en/latest/plugins/ai-proxy-multi.md Outdated
@kayx23
kayx23 dismissed stale reviews from Yilialinn and juzhiyuan via 3f78320 August 4, 2026 09:47
@kayx23
kayx23 requested a review from juzhiyuan August 4, 2026 09:49
@kayx23
kayx23 requested a review from Yilialinn August 4, 2026 09:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

doc Documentation things size:XL This PR changes 500-999 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants