fix(multi_format_api): avoid zero-column DataFrame on empty JSON arrays#93
Merged
gabrielbressan-tfy merged 1 commit intoJul 10, 2026
Merged
Conversation
When a JSON response contained a structurally valid but empty record array
(e.g. {"results": [], "next_url": "..."}), parse_json returned a bare
pd.DataFrame() — shape (0, 0). DuckDB rejects a zero-column frame when the
planner runs `SELECT * FROM df`, raising "Need a DataFrame with at least one
column". That opaque error surfaced to the agent as a query problem rather
than an empty result, triggering wasteful query-rewrite retries.
The empty branch now returns a 0-row frame with at least one column:
the envelope's top-level scalar siblings (next_url, count, status, ...)
when present, else an `_empty` sentinel. A valid-but-empty response now
returns a clean empty result set instead of a DuckDB traceback.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The
multi_format_apihandler (display name URL Reader) crashes when an API returns a structurally valid but empty record array — e.g.{"results": [], "next_url": "..."}.parse_jsonreturned a barepd.DataFrame(), which is shape(0, 0)— zero rows and zero columns (pandas can't infer column names from an empty list of dicts). When the API-handler planner then runsSELECT * FROM df, DuckDB rejects the zero-column frame:This is a DuckDB schema-inference limitation, not a data problem — the query is valid and the data is legitimately empty. But the error that bubbles up to the agent is a generic DuckDB traceback with no hint that "your filters matched zero rows," so the LLM misreads a data condition as a syntax problem and starts mutating the query (guessing
record_path, column names, etc.) — a wasteful retry-storm.Fix
The empty-record branch in
parse_jsonnow always returns a 0-row frame with at least one column:next_url,count,status, …) as columns — mirroring what the non-empty path already does when it reattaches envelope metadata._emptysentinel column when there's no envelope metadata to borrow (e.g. a bare top-level[]).A valid-but-empty response now returns a clean empty result set, giving the agent an unambiguous "valid query, zero rows" signal instead of a DuckDB traceback.
XML and CSV paths always emit ≥1 column, so JSON was the only affected format.
Tests
df.shape[1] >= 1.{"results": [], "next_url": "..."}envelope case (columns preserved,resultskey excluded).[]sentinel fallback.All 15 tests in the handler module pass (
/venv/bin/python -m pytestin thedev-mindsdbcontainer).🤖 Generated with Claude Code