Skip to content

feat(ai): real retrieval — Ask AI answers from the docs - #13

Merged
oreofeolurin merged 1 commit into
feat/ai-composer-textarea-attachmentsfrom
feat/ai-retrieval
Jun 11, 2026
Merged

feat(ai): real retrieval — Ask AI answers from the docs#13
oreofeolurin merged 1 commit into
feat/ai-composer-textarea-attachmentsfrom
feat/ai-retrieval

Conversation

@oreofeolurin

Copy link
Copy Markdown
Contributor

Stacked on #11 (feat/ai-composer-textarea-attachments) — merge that first, or I'll rebase onto main once it lands.

Problem

Ask AI couldn't answer even when the answer was on the current page. Root cause: it was handed no document content — only a section-name string ("you are viewing the 'Releases' section"), and pickSources() returned hardcoded fake labels. So it replied "the provided context does not cover this." Mintlify, by contrast, retrieves the relevant page text and grounds on it.

Fix — lib/ai/retrieval.ts (client-side RAG)

  1. Current page always included — reads the page's prose from the DOM (.docs-prose / .api-doc). Fixes "on the Services page it still couldn't answer."
  2. Cross-page retrieval — ranks the build-time llms-full.txt corpus against the question with saturating tf·idf + title boost, plus a light stemmer (so create a service matches the Services page). Attaches the top 3 pages → answers off-page questions, Mintlify-style.
  3. Real "Used N sources" — reflects the pages actually retrieved.

Pure client-side (one cached fetch of /llms-full.txt) → works in BYOK mode and on pure-static hosting; degrades to current-page-only if llms-full.txt is absent. Proxy context cap raised 8k→16k.

Validation

Ranking unit-checked against a rune-like corpus:

Question Top result
how do i create a service Services (1.97)
how do releases roll back on failure Releases (3.81)
authenticate api requests Authentication (1.93)
expose a service on a port Services, Networking

tsc clean · next lint clean · production build prerenders all 25 pages.

Known gap

llms-full.txt is docs-only, so API-reference operation bodies aren't in the corpus yet (the current API page's DOM text is still included). Follow-up: add API operations to the corpus.

Before: the model received only a section-name string ('you are viewing the
"Releases" section') and pickSources() returned hardcoded fake labels — so it
had no actual content and replied 'the provided context does not cover this',
even on the very page that answered the question.

Now lib/ai/retrieval.ts grounds every answer in real docs, client-side:
- Always includes the current page's text (read from the DOM: .docs-prose /
  .api-doc), so on-page questions are answered from the page.
- Ranks the build-time llms-full.txt corpus against the question (saturating
  tf·idf + title boost, with a light stemmer so 'create a service' matches the
  'Services' page) and attaches the top pages — so off-page questions are
  answered too (Mintlify-style).
- 'Used N sources' now reflects the pages actually retrieved, not placeholders.

Pure client-side (one cached fetch of /llms-full.txt) — works in BYOK mode and
on pure-static hosting; degrades to current-page-only if llms-full.txt is
absent. Proxy context cap raised 8k→16k to fit the grounding context.

Known gap: llms-full.txt is docs-only, so API-reference operation bodies aren't
in the corpus yet (the current API page's DOM text is still included).
@oreofeolurin
oreofeolurin merged commit dd3c2ec into feat/ai-composer-textarea-attachments Jun 11, 2026
1 check passed
@oreofeolurin
oreofeolurin deleted the feat/ai-retrieval branch June 11, 2026 05:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant