Skip to content

fix(control-testing): a control test could be created but never tested - #360

Merged
0xmanhnv merged 1 commit into
developfrom
fix/control-testing-record-result
Aug 4, 2026
Merged

fix(control-testing): a control test could be created but never tested#360
0xmanhnv merged 1 commit into
developfrom
fix/control-testing-record-result

Conversation

@0xmanhnv

@0xmanhnv 0xmanhnv commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

The Control Testing page imports useCreateControlTest and nothing else.

useRecordControlTestResult and useDeleteControlTest sit in use-simulation-api.ts, and the API has had PATCH /control-tests/{id}/result behind validation:write all along. The page just never called it. The row rendered status and last_tested_at and offered no action.

So a control test created through this page:

status untested, forever
last tested Never, forever
the pages own passed/failed/untested summary permanently 100% untested

control_tests has 0 rows on the live database. Recording effectiveness over time is the entire point of the feature.

The change

An Actions column with a Record-result dialog — outcome, evidence, notes.

It shows the controls expected_result while the tester picks an outcome, since that is exactly what they are comparing against and it was otherwise only visible in the create form.

Gated with <Can permission={Permission.PentestWrite}>, matching the servers middleware.Require(permission.PentestWrite). Both resolve to "validation:write"checked, not assumed; a UI gate that disagrees with the servers either hides a button that would work or shows one that 403s.

Why the vocabulary moved to its own module

Mirrors features/controls/vocabulary.ts, which exists because the compensating-controls form offered Preventive/Detective/Corrective/Compensating while the database accepted segmentation/identity/runtime/detection/otherzero overlap, every create a 500.

Control tests fail more quietly than that. control_tests has no CHECK constraint, so before api#417 the server stored whatever string arrived. A near-miss like passed persisted happily and then matched neither status === pass nor fail in this pages summary — a control that had been tested read as neither passed nor failed, with no error anywhere. Quieter is worse.

The tests assert:

  • the offered values against the backend list
  • that every backend status except untested is reachable — a status the API accepts with no way to record it is one the product claims and cannot deliver
  • that no near-miss (Pass, passed, PARTIAL, not-applicable) is offered
  • that no options label is its raw wire value (not_applicable in a dropdown is a leaked implementation detail)

Verified the tests fail by changing pass to passed:

× offers only recordable outcomes in the form
  the form offers "passed", which the API rejects
× never offers a near-miss the backend would reject

untested is deliberately not offered: it is the initial state, not an outcome. Recording it would stamp last_tested_at while setting the status back to untested — "tested, result: not tested".

Verification

  • tsc --noEmit — clean
  • eslint on the changed page and the new module — clean
  • vitest run902 tests, 56 files, all passing

Pairs with api#417, which stops an unknown status being stored at all. Either half is useful alone; together the value can only be set from a fixed list and only that list can be stored.

The Control Testing page imports useCreateControlTest and nothing else.
useRecordControlTestResult and useDeleteControlTest exist in
use-simulation-api.ts, and the API has PATCH /control-tests/{id}/result behind
validation:write — the page just never called it. The row rendered status and
last_tested_at and offered no action.

So a control test created through this page stayed 'untested' forever,
last_tested_at read 'Never', and the page's own passed/failed/untested summary
was permanently 100% untested. control_tests has 0 rows on the live database.
Recording effectiveness over time is the entire point of the feature.

Adds an Actions column with a Record-result dialog: outcome, evidence, notes.
It shows the control's expected_result while the tester chooses, since that is
what they are comparing against and it was otherwise only visible in the create
form. Gated with <Can permission={Permission.PentestWrite}>, matching the
server's middleware.Require(permission.PentestWrite) — both resolve to
"validation:write", checked rather than assumed.

The vocabulary moves to features/simulation/vocabulary.ts, mirroring
features/controls/vocabulary.ts, which exists because the compensating-controls
form offered Preventive/Detective/Corrective/Compensating while the database
accepted segmentation/identity/runtime/detection/other — zero overlap, every
create a 500.

Control tests fail more quietly than that: control_tests has no CHECK
constraint, so before api#417 the server stored whatever string arrived. A
near-miss like 'passed' persisted and then matched neither `status === 'pass'`
nor `'fail'` in this page's summary — a control that HAD been tested read as
neither passed nor failed, with no error anywhere.

The tests assert the offered values against the backend list, that every
backend status except 'untested' is reachable (a status with no way to record
it is one the product claims and cannot deliver), and that no near-miss is
offered. Verified they fail by changing 'pass' to 'passed': "the form offers
"passed", which the API rejects".

'untested' is deliberately not offered: it is the initial state, not an
outcome. Recording it would stamp last_tested_at while setting the status back
to untested, which reads as "tested, result: not tested".
@0xmanhnv
0xmanhnv merged commit 694b9d1 into develop Aug 4, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant