Labour Indicators API

A small REST service for storing labour-market indicator series and reading them back with year-over-year variation attached.

Live instance: not yet deployed. Once it is, the URL goes here and /health answers immediately. It will run on Render's free tier, so the first request after a quiet period takes around 30 seconds while the instance wakes up.

What this is. A deliberately small backend built to demonstrate the engineering apparatus around it: strict TypeScript, unit and integration tests against a real database, a CI pipeline that gates every push, a multi-stage container, and a real deployment. The domain is small on purpose. The tests, the pipeline and the design notes below are the substance.

What this is not. A statistical platform. There is no authentication, no multi-tenancy and no revision history beyond the archive state. Those are named here rather than implied away — see Limitations.

---

Table of contents

• Why this domain
• Quick start
• API
• Design decisions
• Testing strategy
• CI/CD
• Deployment
• Observability
• Limitations
• How AI was used

---

Why this domain

A generic CRUD service gives tests nothing to bite on. Indicator series have three rules that are genuinely worth testing, and each one has a plausible way to get it wrong:

1. Variation depends on the unit. A series already expressed as a percentage varies in percentage points; everything else varies in relative percent change. An unemployment rate moving from 6.0% to 7.5% is +1.5 pp, not +25%. Both numbers are computable from the same pair of values, and only one of them is the right answer. This is the single most common way a statistics dashboard ends up misinforming its readers.

2. Periods are not dates. A quarter is not a point in time. Storing 2024-Q1 as a Date makes it comparable to 2024-M03, which is meaningless, and the comparison would succeed silently.

3. Published figures are not editable. Once a number is published, someone may have cited it. The lifecycle enforces that: a published series can only be archived, never returned to draft.

Quick start

Everything below assumes Docker is running. Nothing else needs installing.

Run the whole stack

docker compose up --build

The API comes up on http://localhost:3000, backed by a PostgreSQL container. Migrations run automatically at boot.

curl http://localhost:3000/ready

Run the tests

npm ci
npm test

The unit suite needs nothing. The integration suite starts its own PostgreSQL container through Testcontainers, so Docker must be running — but you do not have to start a database yourself, and there is no test configuration to fill in.

npm run test:unit          # fast, no Docker
npm run test:integration   # real PostgreSQL, started and torn down per file
npm run test:coverage      # enforces the thresholds CI enforces

If Docker is not available, point the integration suite at any PostgreSQL you already have. It is truncated between tests, so give it a scratch database:

TEST_DATABASE_URL=postgres://user:pass@localhost:5432/scratch npm run test:integration

Develop locally

cp .env.example .env
docker compose up -d db
npm ci
npm run dev

API

Method	Path	Purpose
POST	/api/indicators	Create a draft indicator
GET	/api/indicators	List indicators (status, limit, offset)
GET	/api/indicators/:id	Read one indicator
POST	/api/indicators/:id/observations	Add an observation to a draft
PATCH	/api/indicators/:id/status	Move through the lifecycle
GET	/api/indicators/:id/series	Read the series with year-over-year variation
GET	/health	Liveness — never touches the database
GET	/ready	Readiness — checks the database

A complete example

# 1. Create a quarterly unemployment rate, as a draft
ID=$(curl -sS -X POST http://localhost:3000/api/indicators \
  -H 'Content-Type: application/json' \
  -d '{"code":"UNEMP_RATE","name":"Unemployment rate","unit":"percentage","frequency":"quarterly"}' \
  | node -pe "JSON.parse(require('fs').readFileSync(0)).id")

# 2. Add two observations, one year apart
curl -sS -X POST "http://localhost:3000/api/indicators/$ID/observations" \
  -H 'Content-Type: application/json' -d '{"period":"2023-Q1","value":6.0}'
curl -sS -X POST "http://localhost:3000/api/indicators/$ID/observations" \
  -H 'Content-Type: application/json' -d '{"period":"2024-Q1","value":7.5}'

# 3. Publish it
curl -sS -X PATCH "http://localhost:3000/api/indicators/$ID/status" \
  -H 'Content-Type: application/json' -d '{"status":"published"}'

# 4. Read the series
curl -sS "http://localhost:3000/api/indicators/$ID/series"

The last call returns:

{
  "indicator": { "code": "UNEMP_RATE", "unit": "percentage", "status": "published" },
  "variationKind": "percentage_points",
  "points": [
    {
      "period": "2023-Q1",
      "value": 6,
      "yearOverYear": null,
      "variationGap": "no_prior_year_observation"
    },
    {
      "period": "2024-Q1",
      "value": 7.5,
      "yearOverYear": { "kind": "percentage_points", "value": 1.5 },
      "variationGap": null
    }
  ]
}

Note variationKind at the top level and kind on every variation. A client that ignores them will render percentage points as if they were percent change. Making the unit of the answer explicit costs a few bytes and removes the entire class of mistake.

Errors

Every error has the same shape, so a client needs one code path:

{
  "error": {
    "code": "ILLEGAL_TRANSITION",
    "message": "Cannot move indicator from 'published' to 'draft': published figures cannot return to draft; archive it and publish a revision instead",
    "details": { "from": "published", "to": "draft" }
  }
}

Code	Status	When
VALIDATION_ERROR	400	Malformed request, or a value the unit's rules reject
NOT_FOUND	404	No indicator with that id
CONFLICT	409	Duplicate code, or a period the series already has
ILLEGAL_TRANSITION	409	A lifecycle rule refuses the change
INTERNAL_ERROR	500	A bug. Reported to Sentry, details withheld from the client

Design decisions

Variation is computed, never stored

Storing a variation column would create a second source of truth that has to be kept in step with the observations. Series are short and the arithmetic is trivial, so it is derived on read. If a series ever grew large enough for that to matter, a materialised view would be the next step — not a column the application has to remember to update.

Absence is data, not null

A point with no variation carries a variationGap saying why: no_prior_year_observation (the matching period is missing) or prior_value_is_zero (relative change from zero is undefined). A bare null would collapse two different situations into one, and a client cannot tell "we have no data" from "the question has no answer".

Periods are structured, and their text form is canonical

2024-M3 is rejected even though it is unambiguous to a human. Accepting it would give one period two spellings, and the UNIQUE (indicator_id, period) constraint compares the stored text — so the duplicate it exists to prevent would slip straight through.

Reads and writes that depend on each other share a transaction

Every state change locks the indicator row with SELECT ... FOR UPDATE and does the check and the write in one transaction. Without it, two simultaneous publish requests both read draft, both pass the transition check, and both succeed — the rule becomes advisory under exactly the conditions where it matters. There is an integration test that fires both requests concurrently and asserts one gets 200 and the other 409.

Uniqueness is enforced by the database, not by a prior read

Checking whether a code exists and then inserting is a race with a comfortable window. The unique index decides, and the 23505 SQLSTATE is translated into a 409. This is the only version that is correct when two requests arrive at once.

The domain layer knows nothing about HTTP

src/domain throws typed errors with no status codes in them. src/http/errorHandler.ts is the only file that maps them to responses. That is what lets the whole rule set be unit-tested without a server, and what would let it be reused from a CLI or a scheduled loader later.

Migrations run at boot

For a service this size it removes a whole class of "deployed, but the schema is one release behind" incidents, and it means the integration tests run the real migrator rather than a schema fixture that can drift. The trade-off is real: on a larger team, with migrations that take minutes and several instances rolling at once, this belongs in a release step instead. At this size the boot-time version is the better bargain, and the advisory lock in src/db/migrate.ts handles two instances starting together.

NUMERIC, with a type parser

Observation values are NUMERIC(18,6). node-postgres returns NUMERIC as a string by default to avoid silent precision loss — sensible, and a trap: every subsequent + becomes string concatenation. The parser is registered once in src/db/pool.ts, and an integration test asserts a decimal survives the round trip as a number.

Testing strategy

Two suites, split because they have genuinely different costs.

Unit (tests/unit, 75 tests, no I/O) covers the domain: period parsing and ordering, unit-specific value rules, lifecycle transitions, and the variation arithmetic including floating-point noise, division by zero, and the percentage-points-versus-percent-change distinction.

Integration (tests/integration, 47 tests) runs against a real PostgreSQL 16 started by Testcontainers, pinned to a fixed image tag. It covers what a mock cannot:

• the full API surface through Fastify's inject() — routing, validation, serialisation and error mapping all in the path
• transactional rollback: a unit of work that fails halfway leaves nothing behind, asserted by deliberately violating a constraint mid-transaction
• connection-pool hygiene: twelve consecutive rollbacks against a pool of ten do not leak a client
• concurrency: simultaneous publishes, simultaneous creates of the same code, and simultaneous observations for the same period each produce exactly one winner
• migration idempotence: a second run applies nothing
• database-level CHECK and ON DELETE CASCADE constraints

Testcontainers rather than a services: block in the workflow, because the same npm test then runs identically on a laptop and on the runner, and there is no second copy of the database configuration to drift out of sync.

The suite currently covers 98% of lines and 90% of branches. Thresholds (90% lines/statements/functions, 85% branches) are enforced by vitest.config.ts and set just below what the suite achieves, so an honest refactor has room while a real regression fails the build. src/index.ts is excluded: it wires the process together, and a test for it would only assert its own mocks.

CI/CD

.github/workflows/ci.yml runs on every push and pull request to main, in three parallel jobs:

Job	What it does
quality	Prettier check, ESLint (type-aware), tsc --noEmit
test	Unit + integration with coverage thresholds; uploads the HTML report
docker	Builds the image, brings the stack up, and drives the real API over HTTP

The docker job is the one worth pointing at. Building an image proves it compiles; it does not prove it runs. So the job starts the full stack with docker compose, waits for /ready (which also proves the boot-time migrations ran), then creates an indicator, posts two observations and asserts the API returns +1.5 pp — the real business rule, over the network, against the real container. On failure it dumps the service logs before tearing down.

Concurrent runs on the same branch cancel the previous one: a queue of stale runs tells you nothing and delays the result you want.

Deployment

Deployed on Render from render.yaml, which is committed so the deployment is reviewable in the diff and reproducible without anyone remembering what was clicked in a dashboard. autoDeploy is on, so main is what is running. Render polls /ready, not /health, so an instance whose database is unreachable is taken out of rotation rather than serving errors.

The container is a three-stage build: dependencies and compilation in the first, production-only npm ci --omit=dev in a second, and a runtime stage that copies just node_modules and dist. TypeScript, Vitest and Testcontainers never enter the final image. It runs as the unprivileged node user.

Observability

Sentry is wired through the Fastify error handler rather than its automatic integration, so exactly one event is sent per failed request and the code path is visible in src/http/errorHandler.ts. Only unhandled 5xx errors are reported — a 409 from a duplicate code is the API working correctly, and paging on it would train everyone to ignore the alerts.

Sentry is optional. With no SENTRY_DSN the service starts normally and errors go to the structured log. A service that refuses to boot without a monitoring account would be a worse service, and a reviewer cloning this repository does not have a DSN.

Logging is Pino via Fastify, JSON in production, silent under test.

Limitations

Stated plainly, because a reviewer will find them anyway:

• No authentication. Every endpoint is public. Real deployments of this shape need at least an API key on the write paths.
• No revision history. Archiving preserves the old series, but there is no explicit link from a revision to what it replaced.
• No aggregation. No rolling averages, no seasonal adjustment, no cross-indicator queries. Year-over-year variation is the only derived figure.
• Free-tier deployment. The instance sleeps when idle; the first request after a quiet period is slow.
• Coverage thresholds are floors, not proof. They stop the suite rotting. They do not mean every branch that matters is tested.

How AI was used

This repository was built with Claude Code as an active participant, and AI-WORKFLOW.md documents that honestly: what was delegated, what was written by hand, what the model got wrong and how it was caught, and how AI was used in testing, review and documentation specifically.

License

MIT — see LICENSE.