Skip to content

fix: Hessian auto-quant crash and qconfig_dict mutation bugs - #29

Merged
Adithya-Thonse merged 2 commits into
TexasInstruments:mainfrom
musicalplatypus:pr/auto-quantization-bugs
Aug 5, 2026
Merged

fix: Hessian auto-quant crash and qconfig_dict mutation bugs#29
Adithya-Thonse merged 2 commits into
TexasInstruments:mainfrom
musicalplatypus:pr/auto-quantization-bugs

Conversation

@musicalplatypus

Copy link
Copy Markdown
Contributor

Summary

Two bugs in Hessian-based auto-quantization (auto_quantization.py/qconfig_types.py), both real, reproducible on the code as it stands.

Bare TypeError instead of graceful degradation

run_auto_quantization()'s optimal_bitwidth stays None whenever no calibration_dataloader is supplied, or compute_hessian_sensitivity() can't compute sensitivities (it returns ({}, {}) when sample inputs/targets/criterion aren't available) — a path callers like audio_classification/train.py already anticipate and try to degrade gracefully from, logging "Could not obtain sample data... Proceeding without it". That graceful-degradation intent was defeated a few lines later:

total_bit_budget = optimal_bitwidth * total_params

TypeError: unsupported operand type(s) for *: 'NoneType' and 'int'.

Fixed by returning the caller's already-built qconfig_mapping unchanged when optimal_bitwidth is None — this is the mapping the caller constructed from the manually-specified uniform bitwidth before auto-quantization was attempted, making it the semantically correct fallback (not a forced all-float32 fallback, and not a crash).

apply_mixed_precision() mutates the caller's live config dict

qconfig_dict['weight']['bitwidth'] = bit_width
qconfig_dict['activation']['bitwidth'] = bit_width

Two of its four call sites — qconfig_types.py's manual mixed-precision path, and auto_quantization.py's default Hessian auto-quantization path (i.e. essentially every successful quantized run) — pass the original, un-copied qconfig_dict, which is frequently the same dict object retained as self.qconfig_type on the live model wrapper (quant_base.py). Any downstream logging, checkpoint metadata, or reuse of that config silently observed whichever bitwidth tier this function processed last, not the model's actual/intended default. Two other call sites already build a throwaway copy before calling in — confirming the mutation was an oversight, not intended behavior.

Fixed in the function itself, not just the two unsafe call sites: builds a local copy per bit-width tier instead of mutating the caller's dict, so every caller is protected regardless of whether it already defended itself.

Verification

3 of 4 added tests fail on the unmodified code — both crash tests reproduce the exact TypeError, and the mutation test observes the caller's dict actually change from 8 to 4 — and pass after the fix. The 4th test confirms the mutation fix doesn't break the actual mixed-precision assignment. Also verified this merges cleanly alongside pr/dataloader-pinning-fix (#20), which touches a sibling file (quant_base.py) in the same directory.

🤖 Generated with Claude Code

t5fkg8d44d-beep and others added 2 commits August 4, 2026 18:30
Two bugs in Hessian-based auto-quantization:

1. run_auto_quantization() crashed with a bare TypeError instead of failing
   gracefully. optimal_bitwidth stays None whenever no calibration_dataloader
   is supplied, or compute_hessian_sensitivity can't compute sensitivities
   (returns ({}, {}) when inputs/targets/criterion aren't available) -- a
   path callers like audio_classification/train.py already anticipate and
   try to degrade gracefully from, logging "Could not obtain sample data...
   Proceeding without it". That graceful-degradation intent was defeated a
   few lines later by `total_bit_budget = optimal_bitwidth * total_params`,
   raising TypeError: unsupported operand type(s) for *: 'NoneType' and 'int'.

   Fixed by returning the caller's already-built qconfig_mapping unchanged
   when optimal_bitwidth is None -- this is the qconfig_mapping the caller
   constructed from the manually-specified uniform bitwidth before auto-
   quantization was attempted, making it the correct, non-crashing fallback
   (not a forced all-float32 fallback, and not a crash).

2. apply_mixed_precision() wrote qconfig_dict['weight']['bitwidth'] /
   ['activation']['bitwidth'] directly onto whatever dict it was handed.
   Two of its four call sites (qconfig_types.py's manual mixed-precision
   path, and auto_quantization.py's default Hessian auto-quantization path
   -- i.e. essentially every successful quantized run) passed the original,
   un-copied qconfig_dict, which is frequently the same dict object retained
   as self.qconfig_type on the live model wrapper (quant_base.py). Any
   downstream logging, checkpoint metadata, or reuse of that config
   silently observed whichever bitwidth tier this function processed last,
   not the model's actual/intended default. Two other call sites already
   worked around this by building a throwaway copy before calling in --
   confirming the mutation was an oversight, not intended behavior.

   Fixed in the function itself (not just the two unsafe call sites): builds
   a local copy per bit-width tier instead of mutating the caller's dict,
   so every caller is protected regardless of whether it already built its
   own defensive copy.

Verified: 3 of 4 added tests fail on the unmodified code (both crash tests
reproduce the exact TypeError; the mutation test observes the caller's dict
change from 8 to 4) and pass after the fix. The 4th test confirms the
mutation fix doesn't break the actual mixed-precision assignment.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…ewrote

Both CodeRabbit and the earlier Opus review flagged that
test_apply_mixed_precision_still_applies_the_requested_bitwidth_per_layer
used {32: ["fc"]}, which takes the bit_width == 32 "disable quantization"
path -- a branch the mutation fix never touched -- so nothing verified
the rewritten bw_qconfig_dict construction actually produces a qconfig at
the requested bitwidth. Now uses {4: ["fc"]} and asserts the produced
qconfig's real quantization ranges (signed 4-bit weights: quant_max 7;
unsigned 4-bit activations: quant_max 15, values confirmed empirically
against get_default_qconfig before asserting). The 32-bit disable path
keeps its own separate test.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@Adithya-Thonse
Adithya-Thonse merged commit 640183a into TexasInstruments:main Aug 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants