Compute tail p-values with sf, not 1 - cdf

[stefan-jansen]

The defect

A tail probability written as 1 - dist.cdf(x) returns exactly 0.0 once x passes
roughly 8.35, and is already wrong by 60% at 8.3: cdf rounds to 1.0 long before the
tail mass underflows, so the subtraction cancels every remaining significant digit.
dist.sf evaluates the tail directly.

Measured on stats.t with df=4231:

|t|	2 * (1 - cdf)	2 * sf
6.00	2.138e-09	2.138e-09
8.30	2.220e-16	1.385e-16
8.94	0	5.702e-19
9.50	0	3.409e-21

compute_ic_hac_stats is the one that surfaced it: a downstream notebook rendered
HAC t(3 lags): -8.94 (p=0) to a reader. The true value is 5.7e-19.

It appears in two spellings. The inline one is grep-visible. The other is not:

probability = float(norm.cdf(z_score))
p_value = float(1 - probability)

That cancels exactly as hard and reads as arithmetic rather than as a tail, which is why
the first pass over this package missed it.

What changed

18 sites.

Inline form (14):

• metrics/ic_inference.py - compute_ic_summary_stats, compute_ic_hac_stats
• metrics/conditional.py - conditional-IC t-test
• evaluation/binary_metrics.py - the two proportion z-tests (one-sided greater and
  two-sided branches)
• evaluation/event_analysis.py - the three event-study tests
• evaluation/factor/validation.py - the Ljung-Box residual test
• evaluation/factor/regularized.py - alpha and beta p-values
• visualization/signal/event_plots.py - the event-plot annotation
• visualization/backtest/tearsheet.py - the tearsheet z-test

Two-line form (4):

• evaluation/stats/deflated_sharpe_ratio.py - the DSR p-value in both entry points.
  probability is untouched; it is the DSR itself and is meant to saturate. Only
  p_value changes, to norm.sf(z_score).
• evaluation/trade_dashboard/tabs/stat_validation.py - the PSR p-value, which now takes
  the left tail of the z-score probabilistic_sharpe_ratio(..., return_components=True)
  already returns.

evaluation/factor/validation.py and the four two-line sites were not in the original
report; they are the same defect and are fixed here.

No decision changes: any statistic in the underflow zone is significant under any
threshold, so no verdict, winner or selection moves. What changes is that a reader no
longer sees a p-value of exactly zero, and stored p_value columns no longer carry 0.0.

Test

tests/test_evaluation/test_pvalue_tail_precision.py covers compute_ic_summary_stats,
compute_ic_hac_stats, Ljung-Box and the DSR numerically - each with a fixture whose
statistic lands past the cancellation point and short of where the true tail mass
underflows - and guards the rest with a static scan of src/.

The scan handles both spellings: it collects the names each file binds to a CDF value,
then flags 1 - <that name> wherever it appears. It reads code only, so a comment
explaining why a nearby line uses sf is prose, not the pattern.

Against main's source the numeric tests and the scan fail. On this branch all 6 pass,
and tests/test_evaluation tests/metrics tests/test_visualization tests/test_binary_metrics.py
is 3256 passed, 4 skipped.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[stefan-jansen]

Maintainer review is pending until 2026-08-12. The source fix is already on main and PR #36 isolates the missing regression guard; this PR will then be closed or reduced to any remaining non-duplicated coverage.

[stefan-jansen]

Next review: 2026-08-12.

[stefan-jansen]

Superseded by #36, merged as 1554783131273d6b25f3d517dfe2c30f94405d74.

Current main already computes upper-tail p-values with the survival function. PR #36 adds the missing public-behavior regression coverage for the SciPy path, asymptotic fallback, multiple-testing correction, and one-sided alternatives without carrying this branch's stale merge conflicts. All 36 required checks passed at the reviewed #36 head before merge.