Fix/false positive calibration#6
Merged
Merged
Conversation
Real-world RAGTruth benchmarking exposed that the default "flag if any
sentence is unsupported" over-flags long answers, and that ~17% of clean
RAGTruth-QA responses contain a meta/refusal sentence ("unable to answer",
"the passages do not mention X") that NLI scores as unsupported even though
it is not a hallucination.
- athena_verify/claims.py: conservative check-worthiness filter that skips
questions and meta/refusal statements. core.py marks them NOT_A_CLAIM so
they are never flagged, and compute_overall_trust excludes them.
- nli.py: optional MiniCheck-DeBERTa-v3-Large backend (nli_model="minicheck"),
a purpose-built grounding checker, with no new dependencies. Benchmarking
showed it does not beat the default NLI on RAGTruth-QA response-level, so it
stays opt-in rather than default.
- assets/benchmarks.png: measured-improvements figure (synthetic FP reduction;
real-world zero-shot accuracy on RAGTruth-QA and HaluEval-QA).
Synthetic regression check: unchanged at 4.6% FP / 95.0% F1. 162 tests pass.
- README/RESULTS: lead with real-world zero-shot results — RAGTruth QA balanced accuracy 0.71, HaluEval QA accuracy 0.69 (GPT-3.5-to-GPT-4 prompted range, fully local, ~25 ms). Honest framing: LettuceDetect's higher response-level F1 comes from fine-tuning on RAGTruth, so it is domain-specific; athena is zero-shot. Note the imbalance-suppressed F1 transparently. Embed the measured-improvements plot. - PLAN_NEXT.md: handoff for the next iteration — optional RAGTruth-trained backend to close the in-domain F1 gap (kept opt-in so zero-shot stays the default), embedding backoff, premise retrieval, and launch execution.
There was a problem hiding this comment.
Pull request overview
This PR reduces false positives in athena-verify by introducing a “check-worthiness” filter that excludes non-claim sentences (questions, refusals, “not mentioned” meta statements) from hallucination scoring and overall aggregation, and it updates benchmark documentation to reflect the new calibration/real-world results.
Changes:
- Add check-worthiness heuristics (
is_question,is_checkworthy) and apply them during per-sentence status classification (NOT_A_CLAIM). - Update overall trust aggregation to exclude
NOT_A_CLAIMsentences (and pass when there are no verifiable claims). - Introduce opt-in MiniCheck model aliases/adapter, plus new tests and documentation/benchmark updates.
Reviewed changes
Copilot reviewed 8 out of 9 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/test_claims.py | Adds unit tests for the new check-worthiness heuristics. |
| README.md | Updates benchmark section with real-world results and methodology notes. |
| PLAN_NEXT.md | Adds a next-iteration planning/handoff document for the calibration + benchmarks work. |
| benchmarks/RESULTS.md | Adds real-world benchmark results and reproduction notes. |
| athena_verify/nli.py | Adds MiniCheck model aliases and a MiniCheck adapter behind the existing .predict() interface. |
| athena_verify/core.py | Marks non-claim sentences as NOT_A_CLAIM during per-sentence classification. |
| athena_verify/claims.py | Implements check-worthiness filtering logic and meta-statement detection. |
| athena_verify/calibration.py | Excludes NOT_A_CLAIM from overall score/unsupported-ratio aggregation and defines pass behavior when no claims exist. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+70
to
+78
| for start in range(0, len(pairs), batch_size): | ||
| batch = pairs[start : start + batch_size] | ||
| texts = [f"{premise}{self._sep}{hypothesis}" for premise, hypothesis in batch] | ||
| enc = self.tokenizer( | ||
| texts, max_length=2048, truncation=True, padding=True, return_tensors="pt" | ||
| ) | ||
| with torch.no_grad(): | ||
| logits = self.model(**enc).logits | ||
| out.extend(torch.softmax(logits, dim=-1)[:, 1].tolist()) |
Comment on lines
+23
to
+26
| # Purpose-built grounding checkers — far better calibrated on abstractive | ||
| # RAG answers than generic NLI, at the cost of a larger model. Opt in with | ||
| # nli_model="minicheck". See docs/models.md. | ||
| "minicheck": "lytang/MiniCheck-DeBERTa-v3-Large", |
Comment on lines
+232
to
+235
| # Questions and meta/refusal statements aren't verifiable claims — never | ||
| # flag them as hallucinations (they're usually the honest, correct response). | ||
| if not is_checkworthy(sentence): | ||
| return trust, "NOT_A_CLAIM" |
Comment on lines
+142
to
+146
| # Only verifiable claims count toward the overall score; questions and | ||
| # meta/refusal sentences are marked NOT_A_CLAIM and excluded. | ||
| claims = [s for s in sentences if s.support_status != "NOT_A_CLAIM"] | ||
| if not claims: | ||
| return 1.0, True |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.