Compute and output normalized error above tolerance - #112
Open
stelliom wants to merge 1 commit into
Open
Conversation
Collaborator
Author
|
@huppd Follow up on our Slack conversation. What do you think? |
huppd
reviewed
Aug 3, 2026
huppd
left a comment
Collaborator
There was a problem hiding this comment.
Thanks for this proposition. I like it :) also the clipping is a good idea.
| err_above_tol = (err_rel_tol - 1.).clip(lower=0.) | ||
| log_dataframe( | ||
| logger, | ||
| "\nError relative to tolerance", |
Collaborator
There was a problem hiding this comment.
Suggested change
| "\nError relative to tolerance", | |
| "\nErrors above tolerance (normalized): max((Differences - Tolerances) / Tolerances, 0)", |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adds the computation of a fourth metric for the output of check.py. Instead of only showing the "Error relative to tolerance":
We also show the "Normalized error above tolerance":
This let's user see how much above the tolerance the checked variables are, in a normalized fashion, such that errors for different variables can be compared. This is especially useful when
err_rel_tolis very close to1, since the precision of the output only shows 2 decimal digits. With this new output you can see how much above the tolerance the values are even if they are only slightly above (i.e.err_rel_tol ~ 1).Note that values that do not exceed the tolerance are clipped to
0, this allows for spotting errors quicker.