You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Non-blocking follow-ups from the review of #29 (admina-redteam detection-efficacy suite). These do not block the merge of #29; tracking them here so they are not lost.
Corpus size / reporting — corpora are small (37/26/11 positives) and per-lang/per-tag cells are 1–3 samples. Show numerator/denominator per cell and mark the output indicative; grow the corpora (especially non-EN) over time.
Label provenance — labels are author-assigned in-repo. Add a one-line note that they are not third-party adjudicated.
CLI not shipped — scripts/redteam.py is not in [project.scripts], so a pip-installed user cannot run the CLI the docs reference. Add an entry point (e.g. admina redteam) or adjust the docs.
Gate fp invariant — the gate compares raw fp counts, sound only because the corpora are sealed. Add assert cur["fp_samples"] == base["fp_samples"], or compare rates.
Corpus edge cases — loop-en-009 (counter-reset labelled loop), non-Luhn test cards, and no hard negatives in the PII FPR set.
Non-blocking follow-ups from the review of #29 (admina-redteam detection-efficacy suite). These do not block the merge of #29; tracking them here so they are not lost.
scripts/redteam.pyis not in[project.scripts], so a pip-installed user cannot run the CLI the docs reference. Add an entry point (e.g.admina redteam) or adjust the docs.fpcounts, sound only because the corpora are sealed. Addassert cur["fp_samples"] == base["fp_samples"], or compare rates.loop-en-009(counter-reset labelledloop), non-Luhn test cards, and no hard negatives in the PII FPR set.