Skip to content

feat(ai): modernize the Anthropic path and fix conversation replay - #663

Open
yupanzi wants to merge 1 commit into
kite-org:mainfrom
yupanzi:feature/ai-claude-updates
Open

feat(ai): modernize the Anthropic path and fix conversation replay#663
yupanzi wants to merge 1 commit into
kite-org:mainfrom
yupanzi:feature/ai-claude-updates

Conversation

@yupanzi

@yupanzi yupanzi commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Summary

Modernize the Anthropic path and fix conversation replay:

Anthropic request surface

  • Move to the Beta Messages API with adaptive thinking, context management, and output_config.effort. Feature use is gated per model (anthropicModelSupportsModernFeatures) so models that reject the modern surface keep the classic request shape.
  • New Reasoning Effort setting (low/medium/high/xhigh/max, default xhigh), exposed in general settings next to Max Tokens, with hints explaining both knobs.
  • AIMaxTokens is sent as configured (never clamped) and the default is provider-aware: 64000 for Anthropic, 8192 for OpenAI. Existing installs with the old 4096 default are migrated (covered by general_setting_upgrade_test.go). Default Anthropic model bumped to claude-opus-5.

Conversation replay

  • The frontend now sends tool round-trips structurally (role: tool with id/name/args/result) and the backend rebuilds real tool_use/tool_result blocks. Tool messages without a result (denied/cancelled) are skipped so no dangling tool_use is produced. Covered by conversation_history_test.go.

Robustness

  • Secret data in tool results is masked before reaching the model.
  • SSE streams emit a keepalive comment every 20s; the chart's gateway request timeout is raised to 900s and the matching ingress-nginx annotations are documented in values.yaml — these are two halves of the same fix.
  • Resource tool schemas now state their caps (list items, log tail lines/bytes) so the model narrows queries instead of truncating blindly.
  • AI chat markdown tables no longer overflow on long unbreakable tokens.

Why

The Anthropic integration predates the current Claude API surface: budget_tokens now returns 400 on current models, the 4096-token default truncates answers (thinking and answer share max_tokens), and tool turns were replayed to the model as flattened [Tool: ...] text, which poisons later turns into emitting tool calls as plain text/XML instead of native tool use. Long agent turns also died behind ingress-nginx, which closes a connection after 60s of backend silence while a tool is legitimately still running.

Related issue

Closes #668

Validation

  • go test ./pkg/ai/... ./pkg/model/... ./pkg/settings/... (new: conversation history, handler streaming, settings migration tests)
  • pnpm run type-check && pnpm run lint && pnpm run test
  • helm lint + helm template for the chart change

Checklist

  • I reviewed this PR myself before requesting review.
  • I understand the changes, including AI-generated parts (if any).
  • For new features, a feature request issue is linked.
  • I cleaned up AI noise (unnecessary comments, dead code, and unrelated changes).
  • This PR is reasonably scoped (or split into smaller PRs).

🤖 Generated with Claude Code

Update the Claude integration to the current API surface and fix several
agent-loop problems:

- Move the Anthropic path to the Beta Messages API with adaptive thinking,
  context management, and output_config.effort. Effort is the reasoning-depth
  knob on current Claude models (budget_tokens returns 400 there); it is
  exposed as a new AIEffort setting (low/medium/high/xhigh/max, default xhigh)
  in the general settings. Use is gated per model by
  anthropicModelSupportsModernFeatures, so models that reject the modern
  request surface keep the classic shape.
- Send AIMaxTokens as configured and make the default provider-aware (64000
  for Anthropic, 8192 for OpenAI): on current Claude models thinking and
  answer share max_tokens, so a small default truncates the answer
  mid-sentence. The default Anthropic model becomes claude-opus-5.
- Bound the conversation with per-message and total character budgets, trimming
  oldest-first, so a long session cannot silently overflow the request.
- Strip tool calls the model emits as plain text or XML instead of through the
  native mechanism, and instruct it not to in the system prompt. A textual call
  is never executed, and echoing it back poisons later turns into repeating it.
- Mask Secret data in tool results before they reach the model.
- Emit an SSE keepalive comment every 20s. An agent turn is legitimately silent
  while a tool runs, and ingress-nginx closes a connection after 60s of backend
  silence. Raise the chart's gateway request timeout to 900s and document the
  matching ingress-nginx annotations next to it.
- Tighten resource tool schemas and descriptions with explicit caps, and fix
  long-token overflow in AI chat markdown tables.
- Cover conversation replay, handler streaming, settings migration and the
  agent loop with tests.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@yupanzi
yupanzi force-pushed the feature/ai-claude-updates branch from d92664d to d231670 Compare September 1, 2026 04:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

AI: Anthropic requests fail or truncate on current Claude models

1 participant