Skip to content

feat: enhance prompt caching with full provider support#1

Merged
Patel230 merged 2 commits into
mainfrom
dev
May 8, 2026
Merged

feat: enhance prompt caching with full provider support#1
Patel230 merged 2 commits into
mainfrom
dev

Conversation

@Patel230
Copy link
Copy Markdown
Contributor

@Patel230 Patel230 commented May 8, 2026

Summary

  • Add cache_control for tool definitions and complex message types
  • Parse cache metrics from both Anthropic and OpenAI responses
  • Unified CacheCreationTokens/CacheReadTokens in EyrieUsage
  • Bump Go to 1.26.1

Test plan

  • New cache_test.go with 6 tests covering all scenarios
  • All existing tests pass

Patel230 added 2 commits May 7, 2026 19:40
…ting

- StreamChatWithContinuation: auto-continues on max_tokens (3x, 32K cap)
- StreamChatContinue on EyrieClient for easy integration
- SanitizeMessages: detects orphaned tool_use, injects synthetic error results
- Integrated sanitization into Anthropic and OpenAI Chat/StreamChat
- WeightedProvider: weighted random selection with failover on retriable errors
- StopReason now propagated in both Anthropic and OpenAI stream processors
- Add cache_control annotations for tool definitions (last tool cached)
- Handle tool_use/tool_result blocks in cached request builder
- Parse cache_creation_input_tokens and cache_read_input_tokens from Anthropic API
- Parse prompt_tokens_details.cached_tokens from OpenAI API
- Add CacheCreationTokens/CacheReadTokens to unified EyrieUsage
- Bump Go version to 1.26.1 for toolchain consistency
@Patel230 Patel230 merged commit 5537290 into main May 8, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant