SCRUM-1160-adapters(bigquery): retry once on the exact bytes-billed shortfall - #331
Open
emnarahmouni-exmergo wants to merge 1 commit into
Open
SCRUM-1160-adapters(bigquery): retry once on the exact bytes-billed shortfall#331emnarahmouni-exmergo wants to merge 1 commit into
emnarahmouni-exmergo wants to merge 1 commit into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes : #320
explore queryfailed inside its own confirmed budget onBigQuery, because the server-side
maximum_bytes_billedcap was pinned tothis command's own reservation against the cumulative
session_ceiling(sized to the dry-run estimate), not the wider per-command budget the
operator actually confirmed.
Root cause
CostGate.remaining_for_statement()bounds the per-statement cap bymin(effective_ceiling(), self._reserved). That clamp to_reservedisdeliberate (issue #159): without it, two commands sharing one
session_ceilingcould each claim the full remaining session headroom andjointly overspend it. The reservation is sized to the dry-run estimate,
and
charge()already widens it when a fresh dry run drifts higher.The gap #320 hits is different: BigQuery's own execution-time rounding of
bytes billed can exceed any dry-run estimate, however accurate that
estimate was at dry-run time. No dry run predicts it, so the existing
drift-widening never catches it, and the cap stays pinned to the original
(too-tight) reservation even though the confirmed budget has ample room.
The fix
BigQuery's refusal already states the exact byte count it needed:
"Query exceeded limit for bytes billed: 163595928. 164626432 or higher required."On that refusal,BigQueryAdapter._runnow parses therequired count, widens the charge by the gap between it and what was
already charged (going through the same locked
charge()/admission pathan ordinary estimate drift already uses), and retries the statement once
with the new cap.
succeeds and the retry completes within budget.
OverCeilingError(the exactbehavior today, just now attributing the refusal to the true byte count
rather than a too-tight reservation), and no second server call is made.
session_ceilingis not enforced under concurrency: the cost gate snapshots spend at construction #159 concurrency guarantee is untouched: the retry widens through theidentical shared, locked reservation mechanism, so two commands sharing a
session ceiling still can't jointly overspend it.