fix(adapter): tolerate missing role in TraceToMessages prompt entries#523
Open
itsuzef wants to merge 1 commit intomicrosoft:mainfrom
Open
fix(adapter): tolerate missing role in TraceToMessages prompt entries#523itsuzef wants to merge 1 commit intomicrosoft:mainfrom
itsuzef wants to merge 1 commit intomicrosoft:mainfrom
Conversation
Some tracers (notably AgentOps when re-emitting prior turns inside ``gen_ai.prompt.N.*``) drop the ``role`` key on assistant tool-call entries and tool-response entries. ``convert_to_openai_messages`` then crashed with ``KeyError: 'role'`` at the first such entry, taking down the entire rollout's adapter pass and silently dropping the trajectory's gradient signal during APO. Recover when the role is unambiguous (``tool_calls`` -> ``assistant``, ``tool_call_id`` -> ``tool``) and otherwise log a warning and skip just the offending message, preserving the rest of the trajectory's signal. Adds regression coverage in tests/adapter/test_messages_adapter.py for both the inferred-role path and the unidentifiable-skip path. Fixes microsoft#425 Fixes microsoft#311
Author
|
@microsoft-github-policy-service agree |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Some tracers (notably AgentOps when re-emitting prior turns inside
gen_ai.prompt.N.*) drop therolekey on assistant tool-call entries and tool-response entries.convert_to_openai_messages(agentlightning/adapter/messages.py:134) then crashed withKeyError: 'role'at the first such entry, taking down the entire rollout's adapter pass and silently dropping the trajectory's gradient signal during APO.This change makes the adapter recover when the role is unambiguous, and otherwise log a warning and skip just the offending message:
tool_callspresent, norole→ inferassistanttool_call_idpresent, norole→ infertoollogger.warning(...)once and skip that single entry, preserving the rest of the trajectory's signalThe adapter is the contract between any OTel-emitting tracer and the algorithm layer, so the right place to absorb this kind of upstream variance is here — even if AgentOps tightens its serialization, third-party tracers (Phoenix, hand-rolled
agl.emit_*, etc.) will keep producing the same shape.No API change. Zero new pyright errors against
messages.py.Fixes #425
Fixes #311
Verification
Validated end-to-end inside a
linux/amd64container running the officialghcr.io/astral-sh/uv:python3.12-bookwormimage with the project's own lockfile, mirroring theubuntu-latestCI runners:I also confirmed both new regression tests genuinely fail when the patch is removed, with the exact
KeyError: 'role'atmessages.py:134reported in #425 — so the test coverage is meaningful and pinned to the user-reported repro shape.Test plan
tests/adapter/test_messages_adapter.py::test_trace_messages_adapter_recovers_assistant_role_from_tool_calls— covers the AgentOps-style re-emitted assistant tool-call entry (norole, hastool_calls) and the matching tool response (norole, hastool_call_id)tests/adapter/test_messages_adapter.py::test_trace_messages_adapter_skips_unidentifiable_prompt_entry— covers the unrecoverable case: garbage entry is skipped with a warning, the rest of the trajectory passes throughtests/adapter/test_messages_adapter.pystill pass (no behavior change for well-formed traces)