Skip to content

test(dataset): offline unit tests for sample_minibatch and LCB parser (#88)#89

Open
luciferlive112116 wants to merge 1 commit into
mini-router:mainfrom
luciferlive112116:sn74-luciferlive112116-dataset-tests
Open

test(dataset): offline unit tests for sample_minibatch and LCB parser (#88)#89
luciferlive112116 wants to merge 1 commit into
mini-router:mainfrom
luciferlive112116:sn74-luciferlive112116-dataset-tests

Conversation

@luciferlive112116

Copy link
Copy Markdown
Contributor

Summary

  • Add offline unit tests for sample_minibatch (without/with replacement, validation, determinism) and load_tasks determinism.
  • Add tests for _parse_lcb_tests, including JSON-string payloads and the stdin/stdout alias pair.
  • Normalize stdout as an output alias in _parse_lcb_tests, mirroring existing stdin handling.

Fixes #88

Context

Continues the offline unit-test contribution line:

No overlap with open dataset PRs (#45 MMLU split, #66 loud toy fallback) or open test PRs (#73#75, #52#57).

Test plan

  • pytest tests/test_dataset.py -v (9 passed)
  • Pure stdlib / offline — no network or API keys required

Add pytest coverage for CMA minibatch sampling semantics and lock
_parse_lcb_tests behavior, including stdout as a symmetric alias for
stdin-style mirror rows.

Fixes mini-router#88
@github-actions github-actions Bot added miner Miner contribution PR train Training or routing changes labels Jul 9, 2026
@tmimmanuel tmimmanuel added the test Test code changes label Jul 12, 2026
@tmimmanuel

Copy link
Copy Markdown
Collaborator

The parser tweak in src/trinity/orchestration/dataset.py:330 is consistent with the existing LiveCodeBench test shape, and the new tests cover the intended behavior for sample_minibatch() and _parse_lcb_tests().

Residual risk: test_load_tasks_is_deterministic_and_truncates() still exercises load_tasks() directly, so it is only fully offline if the Hugging Face fetch path falls back as expected. That is acceptable here, but it leaves a small environment-dependent test gap.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

miner Miner contribution PR test Test code changes train Training or routing changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

test(dataset): offline unit tests for sample_minibatch; normalize stdout in LCB test parser

2 participants