Survey questions in the database, and candidate responses

[mikaalnaik]

Stacked on #93 — base is mikaal/voter-survey, not main, because main has no survey code and both branches touch election.rb, routes.rb and structure.sql. Merge #93 first; this diff shows only the new work.

Why

Survey questions lived in the tracker (surveyData.ts). They move here because candidates now answer surveys too, and an admin-entered candidate questionnaire has to render the question set as a form in this app. A question set that only exists in the front end can't do that.

This inverts a previously deliberate split — the old migration and model comments both said questions stay in the tracker so rewording one needs no migration here. Editing in the CMS needs no deploy at all, so that property is kept, but it's a real change of direction worth noticing in review.

Three tables

table
election_surveys	One per survey per election, told apart by audience. Toronto 2026 gets city-priorities (resident) and candidate-questionnaire.
election_survey_questions	One row per question, carrying its step. Steps are denormalised rather than given their own table — a step is a title, an intro and an order with no behaviour, so a table would only guarantee that two questions agree on a title, at the cost of another level of CRUD.
election_candidate_survey_responses	Separate from election_survey_responses. Residents are anonymous subscribers published in aggregate and replaced silently; a candidate's answers are attributed public statements entered by staff.

Questions are rows rather than one jsonb document because the CMS edits one question at a time — a document would make every edit a read-modify-write of the whole survey and give up per-field validation.

Resident and candidate question sets are independent, with no shared question_id. Comparing "residents said X / candidate said Y" will need a hand-maintained mapping; that was a deliberate call, but it's the thing most likely to drift, and the part I'd most want a second opinion on.

The migration is verified, not assumed

The 33 Toronto questions are seeded from JSON generated mechanically from surveyData.ts (not transcribed), committed at db/seeds/elections/toronto_2026_city_priorities.json as the migration record. I then diffed what the database serves against the original definition:

expected: 9 steps / 33 questions
actual:   9 steps / 33 questions

ROUND-TRIP IDENTICAL

Byte-identical, including all 26 ward options. Those aren't frozen into a row: the question stores options_source: "wards" and they resolve at serve time from the election's councillor races, so the list can't drift from the ward map and can't offer a ward with no council race behind it.

Candidate answers have no public write path

election_candidates carries an email, so an endpoint keyed on candidate email would let anyone publish positions in a candidate's name. Staff enter answers in the CMS (Elections → candidate → Enter questionnaire), and source / entered_by keep a published position traceable.

Two deliberate loosenesses there, both worth a look:

• An answer outside the offered options is kept — staff transcribe what candidates actually say, and refusing "supports with caveats" would mean losing the nuance or the response. Such an answer is offered back in the select as its own selected choice so it round-trips; clearing it stays possible, and stays deliberate, via "— no response —".
• An answer keyed to a since-deleted question is dropped, not rejected — a stale form shouldn't cost someone their edit. The model still validates unknown keys.

Two bugs found along the way

type is reserved. Naming the column type broke every read: ActiveRecord uses it for single-table inheritance and raised SubclassNotFound: failed to locate the subclass: 'text'. Writes succeeded, so the seed looked fine and only reads failed. Renamed to question_type, serialised back to type for the tracker. Chose the rename over self.inheritance_column = nil because a bare type column is a trap for the next person.

Transcribed answers were silently deleted (caught by Greptile, fixed in fea1144). A stored answer that wasn't among the options rendered as a blank select, posted blank, and the controller — which replaces the answers hash wholesale — dropped the key. So editing one question destroyed an attributed public statement on another, while the help text promised the opposite. The original tests missed it because they only exercised a direct submit, never render-then-save; the new test posts back what the rendered form would actually contain.

Verification

• 1124 tests, 0 failures — 41 new (20 model, 8 API, 13 admin).
• Rubocop clean, 704 files.
• Model guards exercised against real data: cross-election candidate rejected, unknown question_id rejected, question_id locked once published, duplicate option values rejected, publishing stamps published_at.
• Base merged in (c893cd9). The only conflict was the schema_migrations list in structure.sql; resolved as the union and verified by loading the resolved file into a fresh test database rather than trusting the splice.

Not in this PR

• The tracker still reads surveyData.ts. Questions are duplicated, not yet switched over, so nothing breaks until that lands separately. It also still needs a decision: API-only with an unavailable state, or permanent fallback to the committed set.
• No CMS CRUD for authoring questions. The candidate questionnaire is seeded empty and unpublished; questions can be authored as JSON + seed meanwhile, and the entry form renders a notice while it has none.

🤖 Generated with Claude Code

[greptile-apps]

Greptile Summary

This PR moves election survey definitions into database-backed models and adds staff-managed candidate questionnaire responses.

• Adds survey, question, and candidate-response tables with validation and publication state.
• Exposes published survey definitions through a read-only API.
• Adds an admin workflow for entering, publishing, and clearing candidate responses.
• Seeds the Toronto 2026 resident survey and an empty candidate questionnaire.
• Preserves transcribed out-of-options answers across unrelated form edits.

Confidence Score: 5/5

The PR appears safe to merge because the previously reported custom-answer data-loss path is fixed and no blocking failure remains.

No blocking failure remains.

Important Files Changed

Filename	Overview
app/views/admin/election_candidate_survey_responses/edit.html.erb	Renders candidate questionnaires and now round-trips stored out-of-options answers through a selected custom choice.
app/controllers/admin/election_candidate_survey_responses_controller.rb	Implements the admin-only candidate-response lifecycle and filters submitted answer keys against the selected survey.
app/models/warehouse/election_candidate_survey_response.rb	Defines response integrity, attribution, publication, and cross-election validation.
app/models/warehouse/election_survey_question.rb	Defines validated question types, dynamic ward options, serialization, and published identifier stability.
app/controllers/api/v1/election_surveys_controller.rb	Serves published survey definitions and supports draft previews for staff.
db/migrate/20260811000001_create_warehouse_election_surveys.rb	Creates normalized survey and question storage with uniqueness, type, and ordering constraints.
db/migrate/20260811000002_create_warehouse_election_candidate_survey_responses.rb	Creates attributed candidate-response storage with relational and publication constraints.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  Seed[Survey seed or CMS authoring] --> Survey[(Election survey)]
  Survey --> Questions[(Survey questions)]
  Questions --> PublicAPI[Read-only survey API]
  Survey --> AdminForm[Candidate questionnaire form]
  Candidate[Candidate] --> AdminForm
  AdminForm --> Response[(Candidate survey response)]
  Response -->|published| PublicSite[Public presentation]

Loading

_{Reviews (4): Last reviewed commit: "candidate survey db storage" | Re-trigger Greptile}