gitops-observability-sampleapp

A small, polyglot, three-service checkout flow that exists to generate realistic application traffic, structured logs, and error conditions for the gitops-observability-infra / gitops-observability-config lab to ingest and visualize.

This repo contains application source code and Dockerfiles only — Kubernetes manifests and the Argo CD Application that deploys these services live in gitops-observability-config, consistent with the repo-per-concern split used across this project.

Architecture

load-generator ──▶ frontend-gateway ──▶ order-service ──▶ inventory-service
  (Python)           (Node/Express)      (Python/FastAPI)   (Python/FastAPI)

A classic checkout flow, chosen because it's a shape any reviewer recognizes and because it naturally produces the log variety a log pipeline needs to be worth building: validation errors, a simulated payment decline, out-of-stock responses, and a genuine downstream timeout (inventory-service can be told to sleep past order-service's client timeout, so the timeout path is exercised for real, not mocked).

Services

Service	Language	Responsibility
frontend-gateway	Node.js / Express	Public entry point (BFF). Validates requests, calls order-service.
order-service	Python / FastAPI	Core business logic. Calls inventory-service; injects ~8% simulated payment declines and ~10% simulated downstream timeouts.
inventory-service	Python / FastAPI	Fake stock lookups. Two SKUs are pinned out-of-stock; supports a simulate_slow flag for the timeout scenario above.
load-generator	Python	Continuously calls frontend-gateway with a mix of valid and ~5% deliberately invalid requests.

Log schema

Every service logs structured JSON to stdout — this is what Fluent Bit tails and what makes OpenSearch queries/dashboards useful instead of regex-parsing free text:

{
  "timestamp": "2026-08-19T10:15:32.104Z",
  "level": "ERROR",
  "service": "order-service",
  "message": "inventory-service timed out",
  "request_id": "b3f1...",
  "http.method": "POST",
  "http.route": "/orders",
  "http.status_code": 504,
  "duration_ms": 1503.2,
  "error.type": "downstream_timeout"
}

request_id is generated at frontend-gateway (or taken from an inbound X-Request-Id header) and propagated through every downstream call, so a single checkout request's full path across all three services can be traced by filtering on one field once this lands in OpenSearch.

level is derived from HTTP status code at each service's logging middleware: >=500 → ERROR, >=400 → WARN, else INFO. Business-logic failures each carry an error.type (validation_error, payment_declined, out_of_stock, downstream_timeout, downstream_error, downstream_unreachable) — that field is what a "top errors by type" dashboard panel groups on.

Running locally (before touching Kubernetes)

docker compose up --build

Then, in another terminal:

curl -X POST http://localhost:3000/checkout \
  -H "Content-Type: application/json" \
  -d '{"items":[{"item_id":"sku-widget-1","qty":2}],"user_id":"user-1"}'

load-generator starts automatically with the rest of the stack, so docker compose logs -f on its own is enough to watch continuous traffic and a realistic mix of status codes without sending requests by hand.

Next steps

• Build and tag images, import into k3d — done, see gitops-observability-infra/RUNBOOK.md Step 4 for exact commands.
• Kubernetes Deployments/Services + an Argo CD Application — done, in gitops-observability-config: manifests under base/sampleapp/ and environments/local/sampleapp/, Application at apps/sampleapp/sampleapp.yaml.
• A provisioned OpenSearch Dashboards dashboard covering log ingestion rate, level:ERROR volume, and per-service breakdowns using the schema above — not yet done.