Migrate AI harness to Vercel AI SDK - #3
Merged
Conversation
…t fixes Replace the five hand-written provider classes with one engine (server/ai/engine.js) built on the Vercel AI SDK v7. All model calls now go through runText, runStructured, and chatStream. Fixes: - Remove sampling parameters everywhere (temperature broke Claude Opus 4.8) - Stop injecting the full flashcard/guide database into the system prompt; the model now retrieves notes just-in-time via search tools - Drop the end-of-life @google/generative-ai SDK and the per-request Gemini explicit-cache creation (implicit caching covers it) - Add retries with backoff, per-call timeouts, and client-disconnect abort propagation on every model call - Cap tool-calling loops at 8 steps - Stream all providers through one path with SSE heartbeats - Validate structured output with Zod against provider-native JSON modes; raise a clear error on truncation instead of a JSON-parse failure - Fail clearly on unknown model IDs instead of silently routing to Gemini - Log model, token usage, finish reason, and duration for every call - Send real message arrays in non-streaming chat instead of flattened text Net deletion of ~1,300 lines. Adds engine routing tests; suite is 167 passing. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Bugbot is not enabled for your account, so this pull request was not reviewed. Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Replaces the hand-written provider abstraction (5 provider classes, ~2,300 lines) with a single engine built on the Vercel AI SDK v7. Every model call now goes through three functions in
server/ai/engine.js:runText,runStructured, andchatStream. Net deletion of ~1,300 lines.Fixes
temperature, which newer Claude models reject with a 400. No call sends sampling parameters now.search_flashcards/search_guidetools (a test chat turn dropped to ~900 input tokens).@google/generative-aireached end-of-life Nov 2025. Also removes the per-request Gemini explicit-cache creation that paid a cache-write on every call and never read it back.Wire compatibility
The SSE format (
data: {text}/{tool}/[DONE]) and all route contracts are unchanged — no client changes needed.Testing
npm test: 167 passing (12 files), including 11 new tests for model routing and the custom-endpoint namespacenpm run lint: clean🤖 Generated with Claude Code