Time the stages of the Hessian-inverse pipeline - #3
Open
a-rahimi wants to merge 1 commit into
Open
Conversation
Add a small stage-timing registry (timing.py) and instrument the pipeline: the functorch derivatives pass, K assembly, and both solver paths (splu: CSC conversion / factorization / substitution; block: pivot / LDU factorization / substitution), plus per-step gradient computation and the trust-region subproblem in train_newton, which prints a per-stage summary table at the end of training. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Stacks on #2 (now merged). Adds lightweight wall-clock instrumentation so each stage of the Hessian-inverse pipeline is visible:
src/timing.py(new, stdlib-only): a module-level registry with arecord(name)context manager (time.perf_counter),reset(), andreport()which renders an aligned table sorted by total time.src/hessian.py: stagessetup/derivatives(the functorch pass),solve/assemble-K, and per-solver stages — splu:to-csc/factorize/substitute/pack; block:pivot/factorize/substitute(Tridiagonal.solvespelled out verbatim at the call site so the two phases time separately;block_partitioned_matrices.pyuntouched).src/train_newton.py:train/gradientandtrain/trs-subproblemstages; prints the summary table to stderr at the end of training.No numerical behavior changes — instrumentation is context managers around existing statements.
Example (trust-region,
--tr-solver efficient --batch-size 32 --num-steps 5)Already actionable: the secular loop runs ~11 solves per step, and
to-csc+assemble-K(~118 ms/solve) are recomputed per trial λ even though only the diagonal changes — caching the CSC structure and patching the diagonal would cut ~30% off the subproblem.Tests
137 passed(133 + 4 new for the timing registry).🤖 Generated with Claude Code