Skip to content

Expand EvidenceBench with Data.gov and 2026 caselaw#2

Merged
blankey1337 merged 1 commit into
mainfrom
codex/evidencebench-data-gov
Jul 14, 2026
Merged

Expand EvidenceBench with Data.gov and 2026 caselaw#2
blankey1337 merged 1 commit into
mainfrom
codex/evidencebench-data-gov

Conversation

@blankey1337

Copy link
Copy Markdown
Member

What changed

  • freezes a 124-row public-domain DOJ/NIJ Post-PCAST caselaw snapshot cataloged by Data.gov
  • adds a deterministic 33-case published 2020–2024 selection and six published 2026 decisions from official court/GovInfo sources
  • expands the sealed holdout to 151 questions, including 55 caselaw questions
  • adds current reporter, slip-opinion, LEXIS, and docket citation normalization
  • publishes fresh aggregate results for all six OpenRouter model routes with source, recency, year, discipline, and proceeding dimensions

Why

The earlier 16-case module was too small to discriminate citation reliability on recent and obscure authority. This release more than triples caselaw coverage while preserving sealed items and reproducible public source commitments.

Validation

  • 19 unit tests pass
  • public development set validates
  • sealed 151-question holdout validates
  • six run artifacts contain exactly 151 unique IDs each
  • public secret scan found no key or database URL
  • git diff --check passes

@blankey1337
blankey1337 merged commit 08efe7b into main Jul 14, 2026
1 check passed
@blankey1337
blankey1337 deleted the codex/evidencebench-data-gov branch July 14, 2026 23:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants