[Ask AI 01] feat: switch docs assistant to Luna - #228
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
2 Skipped Deployments
|
raviships
changed the title
5fe2474 to
6dfec00
Compare
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 6dfec000d9
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| - If a feature doesn't exist in EdgeStore, say so clearly. | ||
|
|
||
| After providing your answer, use the "provideLinks" tool to share relevant documentation links with the user. Include links to the documentation pages you referenced in your answer.`; | ||
| Before writing your final answer, call the "provideLinks" tool exactly once with the documentation pages you referenced. After the tool returns, write a complete, non-empty answer to the user. Do not call "provideLinks" again, and do not treat the tool call as the answer.`; |
There was a problem hiding this comment.
When a question requires four sequential getDocs retrieval rounds, provideLinks becomes the fifth step; its new executor produces a tool result, after which stepCountIs(5) stops generation before Luna can write the complete answer now required by this prompt. Increase the step allowance or use a completion condition that reserves a step after provideLinks, otherwise sufficiently complex documentation questions can still return links with no answer text.
Useful? React with 👍 / 👎.
1 participant
Change
provideLinksbefore it writes the final answer.provideLinksso the model can continue after the tool call.Without the executor, Luna could stop at the link call and return no answer text.
Benchmark
The benchmark asked five documentation questions twice for each configuration. Every run used the same EdgeStore pages, tool schemas, and prompt. Quality is the percentage of question-specific checks passed. Cost uses estimated OpenAI token pricing.
provideLinkscallLuna with no reasoning cut estimated cost by 77% and median total time by 33%. Its quality score was 2.5 percentage points lower, and its first event arrived 92 ms later. Low reasoning cost more, took longer, and scored lower in this run.
Checks
pnpm buildpasses.pnpm --filter docs exec tsc --noEmit --pretty falsepasses after rebuilding the workspace packages.