fix(simulation): a control test result was whatever string the client sent - #417
Merged
Merged
Conversation
… sent
RecordResult cast the incoming string straight into the entity and persisted it:
ct.RecordResult(simulation.ControlTestStatus(input.Status), ...) // no check
func (c *ControlTest) RecordResult(status ControlTestStatus, ...) {
c.status = status // no check
Nothing else validated it either. Unlike compensating_controls, the
control_tests table has NO CHECK constraint on status, so there was no layer
between a request field and the column.
The failure is quiet, which is what makes it worth fixing. A control recorded
as "Pass" or "passed" stores happily and then matches neither `status ===
'pass'` nor `'fail'` in the Control Testing page's own summary — so a control
that WAS tested reads as neither passed nor failed. Nobody gets an error;
the compliance number is just wrong.
RecordResult now returns an error for an unknown status and leaves the entity
untouched, so a rejected test does not leave a lastTestedAt behind implying a
test happened.
Note the caller compiled fine while ignoring the new return value — Go allows
that for a method call used as a statement, and `--new-from-rev` linting would
not flag an unchanged line. It is updated explicitly rather than left to a
linter that had no reason to look.
AllControlTestStatuses is the single list, with a test that fails if a constant
is declared without being added to it — otherwise a new status would be
silently unusable and the failure would look like a client bug.
Not addressed here: the Control Testing page has no way to record a result at
all. useRecordControlTestResult and useDeleteControlTest exist in
use-simulation-api.ts and the page imports neither, so a control test created
through the UI stays untested forever and the page's own passed/failed summary
is always 100% untested. That is a ui change and gets its own PR; this one makes
sure that when the call does arrive, a typo cannot become a stored status.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
There was no layer between a request field and the
control_tests.statuscolumn.And unlike
compensating_controls, thecontrol_teststable has no CHECK constraint onstatus. I checked:Why it matters — the failure is silent
A control recorded as
"Pass"or"passed"stores happily, then matches neitherstatus === passnorfailin the Control Testing pages own summary. A control that was tested reads as neither passed nor failed. No error anywhere; the compliance number is just wrong.This is the same shape as the compensating-controls bug (a vocabulary mismatch between what the client sends and what the platform understands) — except that one failed loudly with a 500, because the database refused it. Here the database accepts anything, so it fails quietly instead. Quieter is worse.
The fix
RecordResultreturns an error for an unknown status and leaves the entity untouched — a rejected test must not leave alastTestedAtbehind implying a test happened.AllControlTestStatuses()is the single list, with a test that fails if a constant is declared without being added to it. Otherwise a new status would be silently unusable and the failure would look like a client bug.One thing worth flagging about the change itself
The caller compiled fine while ignoring the new return value. Go allows that for a method call used as a statement, and
--new-from-revlinting would not have flagged it either, sincesimulation.go:273is not a line this PR would otherwise touch. So the build passing proved nothing here — the caller is updated explicitly.Verification
Rejection cases cover the realistic near-misses, not just nonsense:
"Pass","passed","PARTIAL","not-applicable","","untested ". Each asserts the error and that the entity was not mutated.GOWORK=off go build ./...— okGOWORK=off go test ./...againstapp_test— all greenGOWORK=off make lint-ci— cleanNot addressed here — but found, and it is the other half
The Control Testing page has no way to record a result at all.
useRecordControlTestResultanduseDeleteControlTestexist inuse-simulation-api.ts; the page imports neither (onlyuseCreateControlTest, line 279). The row rendersstatusandlast_tested_atand offers no action.So a control test created through the UI stays
untestedforever,last_tested_atshowsNever, and the pages own passed/failed/untested summary is permanently 100% untested.control_testshas 0 rows on the live database.That is a ui change and gets its own PR. This one makes sure that when the call does arrive, a typo cannot become a stored status.