Add accuracy tracking to PRs. - #224
Merged
Merged
Conversation
Accuracy reportNo |
jenskeiner
force-pushed
the
feature/accuracy-checks
branch
from
July 1, 2026 08:13
a8a3ffe to
77e3a1e
Compare
jenskeiner
force-pushed
the
feature/accuracy-checks
branch
from
July 1, 2026 08:39
77e3a1e to
cfdc22b
Compare
jenskeiner
marked this pull request as ready for review
July 1, 2026 10:03
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
CodSpeed benchmarks that run on PRs and compare to the baseline on
developalready give us an idea if code changes had any impact on performance. The same is not true for accuracy.The existing automated tests compute various transformations and compare the results to a reference. If the error exceeds a defined bound, the case fails, and thus ultimately the whole CI run. This way, some inadvertent breaks can be captured.
However, it would be more useful to track changes to accuracy figures more precisely. This could flag degrading or improving accuracy that does not cross a threshold. This is particularly interesting in light of upcoming performance and accuracy work, as in #222.
This PR adds some custom code to add the following:
develop.At this stage, this new way of presenting accuracy figures seems to be working ok, although I would imagine there's still a lot of room for improvement. The goal of getting this in now is to gather more evidence that the changes in #222 not only improve performance of the direct 1D NDFT transforms, but also the accuracy by avoiding the accumulation of round-off errors.