release(0.13.10): vendor extractor edge cases#68
Merged
Conversation
Pre-fix the SDK 0.13.9 extractors silently dropped five
classes of vendor responses from billing/observability:
1. Cohere v2 tool_calls path (line 403) read the top-level
payload["tool_calls"] — but Cohere v2 moved
tool_calls under message.tool_calls (the OpenAI
shape, just nested). Every v2 Cohere call shipped with
tool_names=[] and the backend's loop detection could
not see Cohere tool use. Same drift fix adds the
usage.tokens.cached_tokens cache-hit read (always 0
before) and the UPPERCASE finish_reason vocabulary
(COMPLETE | MAX_TOKENS | TOOL_CALL) — the
_FINISH_REASON_MAP already lower-cased both
vocabularies, so the only missing piece was the test
snapshot.
2. Mistral num_cached_tokens (flat field on usage,
not nested under prompt_tokens_details.cached_tokens
like OpenAI's). The OpenAI extractor only read the nested
shape, so Mistral customers always saw cache_read_tokens=0
even when the inference cache hit. Fix reads the Mistral
flat field as a fallback inside the same chain.
3. Gemini 2.5+ thoughtsTokenCount (reasoning tokens
in usageMetadata) — was hard-coded to 0, so thinking-mode
Gemini calls had no visible reasoning column on the
dashboard. Reasoning tokens are part of
candidatesTokenCount upstream so the total stays
correct without adjustment.
4. Anthropic 4.5+ output_tokens_details.thinking_tokens
(extended-thinking mode) — was hard-coded to 0 for the same
reason. The pre-fix comment ("reasoning tokens are part
of output_tokens") was correct for the non-thinking
baseline, but the thinking-mode field was still readable
and was being dropped. Now we read the breakdown while
keeping the total at input+output (Anthropic bills
thinking tokens at the output rate upstream).
5. AWS Bedrock finish_reason for the Mistral-on-Bedrock
/ OpenAI-compat and Llama-on-Bedrock adapter shapes. The
pre-fix extractor only read top-level stopReason /
stop_reason (Anthropic + Llama top-level). Mistral's
OpenAI-compat shape puts the field under
choices[0].finish_reason and was always None. The
matched_shape discriminator (already tracked in the
tool-detection block) tells us which body to read from
and the new branch picks choices[0].finish_reason when
matched_shape == 'openai_choices'.
The same audit also identified the following as should-fix but
deferred to a follow-up PR (none of them is a billing gap — all
are visibility / observability gaps):
- Anthropic cache_creation.ephemeral_{1h,5m}_input_tokens
TTL breakdown (different billing rates; Bedrock does not
yet expose the breakdown as of 2026-Q3)
- Anthropic server_tool_use.{web_search_requests,
web_fetch_requests} — server-side tool invocations not
visible to loop detection
- Gemini multimodal *TokensDetails[] (TEXT vs IMAGE vs
AUDIO) — image-heavy calls mask the real cost driver
- Cohere billed_units.{search_units, classifications} for
RAG / classify workloads
- Cohere reasoning models (command-a-reasoning-*)
- Bedrock Converse API (separate envelope from InvokeModel)
Wire format: unchanged. Backends on 1.0.0 keep working
unchanged. Recommended upgrade path: 0.13.9 -> 0.13.10.
Tests (in tests/test_extractors.py, 8 new test snapshots):
- test_cohere_v2_message_tool_calls_path — v2 nested
message.tool_calls returns the right tool_names
- test_cohere_v2_cached_tokens — tokens.cached_tokens
surfaces as cache_read_tokens
- test_cohere_v1_top_level_tool_calls_fallback — v1
callers (legacy top-level tool_calls) keep working
- test_openai_mistral_num_cached_tokens — Mistral
usage.num_cached_tokens fallback in the OpenAI extractor
- test_gemini_2_5_thinking_tokens — thoughtsTokenCount
surfaces as reasoning_tokens while the total stays at
totalTokenCount
- test_anthropic_extended_thinking_tokens —
output_tokens_details.thinking_tokens surfaces
alongside cache_read_input_tokens /
cache_creation_input_tokens already extracted
- test_bedrock_mistral_finish_reason_via_choices —
Mistral-on-Bedrock OpenAI-compat finish_reason is now
captured
- test_bedrock_llama_finish_reason_via_top_level —
Llama-on-Bedrock stop_reason snake_case is captured
(this one already worked, but it had no test snapshot
before — left in for regression coverage)
Local verification:
* pytest tests/test_extractors.py
tests/test_crewai_patch.py tests/test_runtime.py
tests/test_runtime_branches.py
tests/test_track_batch_retry.py
tests/test_track_span_context.py
tests/test_v3_wire_contract.py
tests/test_release_polish.py — 185 passed, 1 skipped
(8 new tests net-new from this commit; no regression
on the 177 tests that were green on master).
* ruff check src/ — "All checks passed!".
* mypy src/ — 11 pre-existing errors (langgraph overload
mismatches at lines 1818, 1821, 1827; same count as
origin/master). No new mypy findings from this commit.
No public API change. No SDK_MIN_VERSION bump. The
_openai_extractor host map (line 567) already covers Mistral
and Azure so this commit is purely an extractor-body
improvement.
Bump version 0.13.9 -> 0.13.10. Prepends the v3.24 / 0.13.10 changelog entry to __version__.py and backfills the inline pyproject.toml comment block for 0.13.6..0.13.10 (the previous comment trail stopped at 0.13.5, which made the pyproject side of the release notes drift behind the docstring). The 5 vendor edge cases themselves ship in the preceding commit 3ac7231. No on-wire change. No SDK_MIN_VERSION bump. Backends on 1.0.0 keep working unchanged. Recommended upgrade path: 0.13.9 -> 0.13.10.
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Release 0.13.10 — vendor extractor edge cases
Bump version 0.13.9 → 0.13.10. Two commits on top of
origin/master:3ac7231— fix(extractors): close 5 vendor edge cases missed in 0.13.9 auditPre-fix the SDK silently dropped five classes of vendor responses from billing/observability:
payload["tool_calls"]; v2 nests undermessage.tool_calls. Now walks both v1 and v2 shapes. Same patch addsusage.tokens.cached_tokens(cache-hit read was always 0) and the UPPERCASE finish_reason vocabulary (COMPLETE | MAX_TOKENS | TOOL_CALL).num_cached_tokens— flat field onusage, not nested underprompt_tokens_details.cached_tokenslike OpenAI's. Now read as fallback in the same chain. Host map already covers Mistral.thoughtsTokenCount— was hard-coded to 0; thinking-mode Gemini calls had no visible reasoning column. Surfaced asreasoning_tokenswhile total stays attotalTokenCount.output_tokens_details.thinking_tokens— extended-thinking mode was dropped for the same reason. Now read while total stays atinput+output(Anthropic bills thinking tokens at the output rate upstream).choices[0].finish_reason; was alwaysNone.matched_shape == 'openai_choices'discriminator picks the right branch.Deferred to a follow-up PR (visibility gaps, not billing): Anthropic cache-creation TTL breakdown, server-tool-use counters, Gemini multimodal
*TokensDetails[], Cohere RAG units, Cohere reasoning models, Bedrock Converse API.32876a1— chore(release): 0.13.10__version__to0.13.10insrc/nullrun/__version__.py.versionto0.13.10inpyproject.toml.__version__.py.pyproject.tomlinline comment block for 0.13.6..0.13.10 (the previous trail stopped at 0.13.5).Verification
pytest tests/test_extractors.py tests/test_crewai_patch.py tests/test_runtime.py tests/test_runtime_branches.py tests/test_track_batch_retry.py tests/test_track_span_context.py tests/test_v3_wire_contract.py tests/test_release_polish.py— 185 passed, 1 skipped (8 net-new tests; no regression on the 177 green on master).ruff check src/nullrun/__version__.py pyproject.toml— All checks passed!mypy src/nullrun— Success: no issues found in 34 source files.tests/test_drift_fixes_2026_07_04.pyare unchanged fromorigin/master— not a regression of this release.Wire format / API
SDK_MIN_VERSIONbump.