diff --git a/.gitattributes b/.gitattributes index 1ff0c42..b73659c 100644 --- a/.gitattributes +++ b/.gitattributes @@ -3,6 +3,16 @@ ############################################################################### * text=auto +############################################################################### +# Files consumed by Linux containers and CI must keep LF regardless of the +# platform they were edited on: a CRLF shell script fails with "\r: command +# not found", and a CRLF Dockerfile breaks its ENTRYPOINT. +############################################################################### +*.sh text eol=lf +Dockerfile text eol=lf +*.yml text eol=lf +*.yaml text eol=lf + ############################################################################### # Set default behavior for command prompt diff. # diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index a8ceb7e..6f89280 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -24,6 +24,8 @@ jobs: - name: Restore run: dotnet restore Stampede.Http.slnx + # The solution includes samples/, so a sample that stops compiling fails CI + # instead of rotting unnoticed. - name: Build run: dotnet build Stampede.Http.slnx -c Release --no-restore @@ -32,3 +34,25 @@ jobs: - name: Demo smoke test run: dotnet run -c Release --no-build --project Stampede.Http.Demo + + # Brings the whole sample stack up (origin + Redis + three clients + Prometheus + + # Grafana + Jaeger) and asserts the behaviour the sample README claims, using the + # origin's own request counters as the source of truth. + samples-smoke: + runs-on: ubuntu-latest + timeout-minutes: 25 + steps: + - uses: actions/checkout@v4 + + # KEEP_STACK leaves the containers up so the log-dump step below has something + # to read; the runner is thrown away either way. + - name: Run the sample stack smoke test + working-directory: samples + env: + KEEP_STACK: "1" + run: bash scripts/smoke-test.sh + + - name: Dump container logs on failure + if: failure() + working-directory: samples + run: docker compose logs --tail 200 diff --git a/README.md b/README.md index dcdd866..6963fe2 100644 --- a/README.md +++ b/README.md @@ -217,6 +217,19 @@ BenchmarkDotNet v0.15.2 · .NET 10 · Windows 11 · i7-12650H. --- +## Runnable sample + +[`samples/`](samples/) contains a full deployment — origin API, Redis, Polly, Prometheus, Grafana, Jaeger and a k6 load profile — plus a **control group**: a third instance of the same app with the Stampede.Http handlers removed, so the difference is measured rather than asserted. + +```bash +cd samples && docker compose up --build -d +docker compose logs -f client-a +``` + +It narrates itself: a ten-caller stampede, then twelve feature scenarios each verified against the origin's own request counters, then a steady-state loop feeding the dashboards. See the [sample README](samples/README.md). + +--- + ## Running the tests ```bash diff --git a/Stampede.Http.slnx b/Stampede.Http.slnx index 4bc1700..881bfbd 100644 --- a/Stampede.Http.slnx +++ b/Stampede.Http.slnx @@ -3,4 +3,8 @@ + + + + diff --git a/samples/README.md b/samples/README.md index 33c308f..eb82b16 100644 --- a/samples/README.md +++ b/samples/README.md @@ -1,76 +1,208 @@ # Stampede.Http — real-world sample -A complete, runnable deployment showing **Stampede.Http + Redis + Polly** working together over a real network, with **live metrics in Grafana**: +A complete, runnable deployment of **Stampede.Http + Redis + Polly**, with metrics, traces, a load generator, and — the part that makes it more than a demo — **a control group**. + +Three copies of the same application run the same workload against the same origin. Two have Stampede.Http in their outbound pipeline; one does not. Every claim below is the difference between them, measured at the origin. ``` -┌──────────┐ ┌─────────────┐ -│ client-a │──►│ Sample API │ client pipeline: -├──────────┤ │ (Kestrel) │ CachingMiddleware ← Redis-backed, shared -│ client-b │ └─────────────┘ └─ CoalescingHandler ← per process -└────┬─────┘ └─ Polly retry + timeout - │ └─ HttpClientHandler - ▼ -┌──────────┐ ┌────────────┐ ┌─────────┐ -│ Redis │ │ Prometheus │──►│ Grafana │ -└──────────┘ └─────△──────┘ └─────────┘ - │ scrapes /metrics from both clients + ┌──────────────────────────┐ + client-a ──┐ │ Sample API │ client pipeline: + client-b ──┼─────►│ (Kestrel) │ CachingMiddleware ← Redis-backed, shared + client-baseline ──┘ │ headers only — no │ └─ CoalescingHandler ← per process + ▲ │ │ Stampede.Http code │ └─ Polly retry + timeout + │ │ └────────────┬─────────────┘ └─ SocketsHttpHandler + │ ▼ │ + ┌──┴───┐ ┌──────┐ counts every request + │ k6 │ │Redis │ that actually arrived + └──────┘ └──────┘ │ + ┌───────────┐ │ ┌────────┐ + │Prometheus │◄──────┴──────►│ Jaeger │ + └─────△─────┘ └────────┘ + │ scrapes all three clients + the origin + ┌─────┴─────┐ + │ Grafana │ + └───────────┘ ``` -The API declares caching policy purely through standard headers (`Cache-Control`, `ETag`, `Vary`) — there is **zero Stampede.Http code on the server**. The client configures the whole pipeline in one statement: +The API declares caching policy purely through standard headers (`Cache-Control`, `ETag`, `Vary`, `Last-Modified`) — there is **zero Stampede.Http code on the server**. The client configures the whole pipeline in one place, [`PipelineRegistration.cs`](Stampede.Http.Sample.Client/PipelineRegistration.cs), and nothing else in the app knows the library exists. -```csharp -services.AddHttpClient("api", c => c.BaseAddress = new Uri(apiBase)) - .AddStampedeHttp( - configureCaching: o => o.DefaultTtl = TimeSpan.FromSeconds(5), - configureCoalescing: o => o.CoalescingTimeout = TimeSpan.FromSeconds(10)) - .UseDistributedCacheStore() // Redis via IDistributedCache - .AddResilienceHandler("sample-resilience", b => // Polly: retries + timeout - { - b.AddRetry(new HttpRetryStrategyOptions { MaxRetryAttempts = 2, BackoffType = DelayBackoffType.Exponential }); - b.AddTimeout(TimeSpan.FromSeconds(5)); - }); -``` +--- ## Run it ```bash -docker compose up --build +docker compose up --build -d ``` -Requires Docker. Watch the `client-a` / `client-b` logs, then open: - | | URL | |---|---| -| **Grafana dashboard** ("Stampede.Http — Live Metrics", auto-provisioned, no login needed) | http://localhost:3000/d/stampede-http-overview | +| **Grafana dashboard** (auto-provisioned, no login) | http://localhost:3000/d/stampede-http-overview | +| **Jaeger traces** | http://localhost:16686 | | Prometheus (raw queries, target health under **Status → Targets**) | http://localhost:9090 | -| client-a's own `/metrics` endpoint | http://localhost:9464/metrics | -| client-b's own `/metrics` endpoint | http://localhost:9465/metrics | +| Origin API | http://localhost:5080 | +| `client-a` — Stampede.Http, runs the feature tour | http://localhost:5081 | +| `client-b` — Stampede.Http, shares the Redis cache with `client-a` | http://localhost:5082 | +| `client-baseline` — **control group**, no Stampede.Http | http://localhost:5083 | > Grafana's anonymous-admin login is a convenience for this local demo only — never do that in a real deployment. -### Without Docker (except Redis) +Then watch the logs: ```bash -docker run -d -p 6379:6379 redis:7-alpine -dotnet run --project Stampede.Http.Sample.Api # listens on http://localhost:5080 -dotnet run --project Stampede.Http.Sample.Client +docker compose logs -f client-a ``` -(Set `ASPNETCORE_URLS=http://localhost:5080` for the API if your default differs.) The client still exposes Prometheus metrics on `http://localhost:9464/metrics`, but Prometheus/Grafana aren't running outside compose — point your own instance at it, or just read the values in your browser. +--- -## What to watch +## The first 90 seconds, narrated -| Moment | What it demonstrates | -|---|---| -| **Phase 1**: 10 concurrent `GET /slow` (2 s origin latency) finish in ~2 s total | Request coalescing — each instance's burst collapses into one origin call | -| `client-b`'s Phase 1 is instant if it starts after `client-a`'s | The Redis cache is **shared**: `client-b` hits the entry `client-a` stored | -| `GET /catalog` shows `Age: Ns` and ~0 ms | Fresh hits served from Redis, no network to the origin | -| After 10 s, one slow `/catalog` request, then fast again | Expiry → conditional revalidation (`If-None-Match`, 304) | -| `POST /catalog` line, then next `GET /catalog` refetches with a new version | Unsafe-method invalidation (RFC 9111 §4.4) — shared through Redis, both instances see it | -| `/flaky` keeps returning **200** while the origin is in its failure window | Polly retries the blips; when the outage persists, `stale-if-error` serves the last good response (look for `[served beyond max-age: stale window or revalidated entry]`) | -| `/feed` never blocks after the first fetch | `stale-while-revalidate` refreshes in the background | -| Periodic `stampede_http.*` metrics block in the console logs | The same counters the Grafana dashboard graphs in real time | -| Grafana's "cache hits vs misses" panel climbing while "coalescing deduplicated" stays flat between bursts | Steady-state traffic is dominated by cache hits; coalescing only fires during the Phase 1 stampede and the periodic origin-refresh moments | +`client-a` runs a scripted workload in three phases and explains itself as it goes. + +### Phase 1 — the stampede + +Ten concurrent callers hit `/slow`, an endpoint that takes the origin two seconds. + +``` +PHASE 1 — stampede: 10 concurrent GET /slow (the origin takes ~2 s per call) + 10 callers finished in 2039 ms — 10/10 OK. Coalescing collapsed this instance's + burst into a single origin call. +``` + +The same burst against `client-baseline` produces twenty origin calls. The smoke test asserts exactly that. + +### Phase 2 — the feature tour + +Twelve scenarios, each **verified against the origin's own request counters** rather than against the client's opinion of what happened. Real output from a clean run: + +``` +[ok] Vary: Accept-Language: 3 languages cold → 3 origin calls; the same 3 again → 0. +[ok] CoalesceKeyHeaders: X-Tenant-Id: 10 concurrent callers across 2 tenants → 2 origin calls. +[ok] Client conditional pass-through: If-None-Match against a fresh entry → 304, 0 origin calls. +[ok] CacheRequestPolicy.ForceRevalidate: origin answered 1 × 304, caller still got 200. +[ok] CacheRequestPolicy.BypassCache: Fresh entry ignored entirely → 1 origin call. +[ok] CacheRequestPolicy.NoStore: NoStore then a plain fetch → 2 origin calls: nothing was stored. +[ok] CoalescingRequestPolicy.BypassCoalescing: 5 concurrent callers → 5 origin calls when + bypassing, 1 when coalescing. +[ok] Cache-Control: only-if-cached: Nothing cached → 504 Gateway Timeout, 0 origin calls. +[ok] Cache-Control: immutable: cold → 1 origin call; a forced revalidation on top → 0. +[ok] CacheOptions.MaxBodySizeBytes: ~1.8 MB body fetched twice → 2 origin calls: too large to store. +[ok] CacheOptions.NormalizeQueryParameters: reordered parameters → 1 origin call. +[ok] Last-Modified revalidation: expired entry kept for revalidation → 1 × 304 to + If-Modified-Since, caller got 200 with no body transferred. +PHASE 2 — feature tour complete: 12/12 checks behaved as documented +``` + +The tour runs on `client-a` only (`Sample:Workload:FeatureTour`): its assertions are deltas on shared origin counters, so a second caller hitting the same endpoints would skew them. Don't run the k6 profile at the same time. + +### Phase 3 — steady state + +`/catalog` + `/feed` + `/flaky` every two seconds, forever, with a `POST /catalog` every tenth iteration and a burst of **8 concurrent `GET /slow`** every fifth. This is what feeds the dashboards. + +The burst is not decoration. The three probes are sequential, so on their own they never put two requests in flight and the coalescer correctly has nothing to do — leaving the coalescing panels at zero and the library's headline feature looking dead. Real inbound traffic overlaps; this models that. In the logs you can watch it alternate: + +``` +burst: 8 concurrent GET /slow -> 8 OK in 0 ms (slowest caller) ← entry fresh, all 8 from cache +burst: 8 concurrent GET /slow -> 8 OK in 2000 ms (slowest caller) ← entry expired: 1 origin call, 7 deduplicated +``` + +That second line is the cache stampede the library is named for, happening on schedule. + +--- + +## The measurement + +`client-baseline` is the same image, the same workload and the same Polly pipeline — with `Sample:Pipeline:Enabled=false`, which removes the two Stampede.Http handlers and nothing else. The origin tags every request it receives with the `X-Client` header the clients send, so the comparison is a single Prometheus query: + +```promql +sum by (client) (rate(sample_api_origin_requests_total[1m])) +``` + +The Grafana dashboard's top row turns that into four numbers: origin req/s per Stampede.Http client, origin req/s for the control client, and **origin load avoided** twice — once over all traffic and once with the failing endpoint excluded. All four restrict themselves to the endpoints every client exercises, so the feature tour's extra traffic cannot flatter the result. + +### How to read that percentage + +The headline reads around **37%**, and on its own it is misleading in both directions. The top row therefore carries **two** figures — *all traffic* and *healthy traffic*, identical selectors but for `/flaky` — so the pair isolates a single variable. Origin request rates from a real 5-minute window: + +| Endpoint | `client-a` | `client-b` | `a`+`b` | control | avoided, per client | +|---|---:|---:|---:|---:|---:| +| `/catalog` | 0.125 | 0.122 | 0.247 | 0.325 | **62%** | +| `/feed` | 0.146 | 0.146 | 0.292 | 0.295 | **51%** | +| `/flaky` | 0.441 | 0.454 | **0.895** | **0.485** | **8%** | +| `/slow` † | 0.027 | 0.027 | 0.054 | 0.461 | **94%** | + +† excluded from the headline — see below. + +Expect your own digits to differ by several points: `/flaky` alternates between healthy and failing on a 60-second cycle, which never divides evenly into a 5-minute rate window. The shape is stable; the digits are not. + +Which gives, depending on what you include: + +| Scenario | avoided | +|---|---:| +| All traffic — the headline | **~37%** | +| Excluding `/flaky` — sequential polling only | **~57%** | +| Excluding `/flaky`, including the concurrent `/slow` burst | **~73%** | +| `/slow` alone | **~94%** | + +**`/flaky` is not diluting the headline — it is actively penalising it.** It sends more traffic to the origin than the other three endpoints combined, and the cached clients send *more* of it than the control client does. That is not a defect: when the origin returns 503, every client has to ask before `stale-if-error` can rescue it, and Polly then retries twice. **Uncacheable failures scale with replica count; cache hits are shared.** `/flaky` is down for 20 seconds of every 60 — a harsher failure budget than any real dependency. + +**The TTLs are deliberately hostile.** For a cache that obeys origin headers, the ceiling is roughly `1 − poll interval / max-age`: + +| `max-age` | polled every | ceiling | +|---|---|---:| +| 5 s (`/feed`) | ~3 s | ~40–60% | +| 10 s (`/catalog`) | ~3 s | ~70% | +| 60 s (a realistic catalogue) | ~3 s | **~95%** | + +`/catalog` and `/feed` both land within a few points of it. `max-age=5` exists so expiry and revalidation are visible inside a 90-second demo, not because anything real is declared that way. Raise the numbers in [`CoreEndpoints.cs`](Stampede.Http.Sample.Api/Endpoints/CoreEndpoints.cs), rebuild the `api` service, and every line on the by-endpoint panel moves up. + +**The per-client normalisation hides the best result.** The dashboard divides the two clients' load by two to compare like with like. But they share one Redis cache, so they also share the refresh work: read the `a`+`b` column against the control column and **two instances cost the origin less than one uncached instance** on every cacheable endpoint. Excluding `/flaky`, the two replicas together generate 0.593 req/s against the control client's 1.081 — **45% less origin load from twice the application capacity**. On `/flaky`, for the reason above, the opposite holds. + +**And `/slow` is left out on purpose.** It is the one endpoint the steady state hits with 8 concurrent callers, so it is where coalescing rather than caching does the work — a 94% saving. Keeping it out of the headline is what makes ~37% a floor rather than a figure flattered by picking favourable traffic. The **Deduplicated requests** panel shows the same event from the client's side: flat while the entry is warm, spiking to ~7 the moment it expires. + +None of this is latency, either: + +``` +✓ { mode:stampede }...: avg=12.57ms med=752µs p(95)=1.37ms +``` + +against an origin that takes 300 ms to 2 s per call. + +--- + +## What demonstrates what + +| Endpoint | Headers it sets | What it shows | +|---|---|---| +| `/catalog` | `max-age=10, stale-if-error=60` + `ETag` | Fresh hits → conditional revalidation → `POST` invalidation (RFC 9111 §4.4), shared through Redis | +| `/feed` | `max-age=5, stale-while-revalidate=30` | Never blocks after the first fetch; refreshes in the background (RFC 5861 §3) | +| `/flaky` | `max-age=5, stale-if-error=120` | 503s for the first 20 s of every minute. Polly retries the blips; stale-if-error covers the rest (RFC 5861 §4) | +| `/slow` | `max-age=30` | 2 s of origin latency — the stampede showcase | +| `/ledger` | `max-age=10, must-revalidate` + `ETag` | Once stale it may **not** be served without checking the origin — contrast with `/flaky` | +| `/docs/{id}` | `max-age=5` + `Last-Modified` | `If-Modified-Since` revalidation, and `RevalidationGraceSeconds` keeping the entry alive to make it possible | +| `/greetings` | `max-age=30` + `Vary: Accept-Language` | One URL, one cache entry per language (RFC 9111 §4.1) | +| `/tenants/data` | `max-age=20` + `Vary: X-Tenant-Id` | Multi-tenancy: `Vary` on the server, `CoalesceKeyHeaders` on the client | +| `/assets/{id}` | `max-age=31536000, immutable` | Fresh immutable entries skip revalidation even when asked (RFC 8246) | +| `/bulk` | `max-age=300`, ~1.8 MB body | Perfectly cacheable and still declined: `MaxBodySizeBytes` | +| `/search` | `max-age=60` | `NormalizeQueryParameters` folding reordered query strings onto one entry | +| `/stats` | `no-store` | Live origin counters — the source of truth for every assertion here | + +--- + +## Drive it yourself + +### By hand + +[`samples.http`](samples.http) is a request collection for the VS Code REST Client, Visual Studio, or the JetBrains HTTP Client. It hits the Stampede.Http client and the control client side by side, with `/stats` calls in between so you can watch the origin counters move — or fail to. + +### Under load + +```bash +docker compose --profile load up k6 +``` + +[`load/stampede.js`](load/stampede.js) runs two identical arrival patterns — 40 VUs each, ramp / hold / drain — one against `client-a`, one against `client-baseline`, and prints the origin's counters at the end. The stampede then comes from real concurrent inbound HTTP rather than a scripted `Task.WhenAll`, which is why the client is a real ASP.NET Core service rather than a console loop. + +To let k6 be the only traffic, set `Sample__Workload__Enabled=false` on the clients first. ### The outage drill @@ -78,30 +210,81 @@ dotnet run --project Stampede.Http.Sample.Client docker compose stop api ``` -The clients keep receiving **200 OK** for `/catalog` and `/flaky` (stale-if-error window) instead of connection errors. Then: +The Stampede.Http clients keep answering **200 OK** for `/catalog` and `/flaky` out of the stale-if-error window. `client-baseline` starts returning connection errors immediately. Then: ```bash docker compose start api ``` -and watch them transparently return to fresh responses. +and watch them return to fresh responses with no intervention. + +--- + +## Observability + +### Metrics + +Every `stampede_http.*` instrument (the same ones in the [main README](../README.md#metrics)) is exported on each client's ordinary application port at `/metrics`, scraped every 5 seconds. The origin exports `sample_api.origin.requests`, tagged by endpoint, client and status. + +The dashboard is grouped into three sections: **Origin load** (the comparison), **Caching** (hits, misses, stale serving, revalidations, invalidations) and **Coalescing** (deduplication rate, in-flight calls, timeouts) — most panels broken down per client. + +> **Naming caveat:** Prometheus 3.x negotiates UTF-8 metric names with targets that support it, which the OTel exporter does. `prometheus.yml` sets `metric_name_escaping_scheme: underscores` globally so names stay in the classic form (`stampede_http_cache_hits_requests_total`) that the dashboard queries and this README use. + +### Traces + +Both the clients and the origin export OTLP traces to Jaeger. Search for the `sample-client` service and open a request that arrived during a burst: the coalesced call shows up as **one** outgoing HTTP span serving many inbound ones. That is the thing a counter cannot show you. + +### Runtime reconfiguration + +[`config/client.json`](config/client.json) is bind-mounted into every client and read with `reloadOnChange`. Edit `Stampede:Cache:DefaultTtl` while the stack is running, then: + +```bash +curl http://localhost:5081/api/config +``` + +The new value is live — no restart, no redeploy. That is `IOptionsMonitor` working through Stampede.Http's named options, keyed by the `HttpClient` name. Structural options (`MaxCacheSize`, `NormalizeQueryParameters`, `RevalidationGraceSeconds`) are read once at registration and deliberately do not move. -### Origin's point of view +> The compose file sets `DOTNET_USE_POLLING_FILE_WATCHER=true`: inotify events do not cross a Docker bind mount on Windows or macOS, so the file watcher would otherwise never fire. + +### Other introspection endpoints + +| Endpoint | What it returns | +|---|---| +| `GET /api/config` | The options actually in effect on this instance right now | +| `GET /api/counters` | This process's `stampede_http.*` instrument totals, as JSON | +| `GET /api/origin-stats` | The origin's counters, straight from the origin | +| `GET /metrics` | The same instruments in Prometheus exposition format | + +--- + +## Without Docker ```bash -curl http://localhost:5080/stats +docker run -d -p 6379:6379 redis:7-alpine # or set Sample:Pipeline:UseRedis=false + +dotnet run --project Stampede.Http.Sample.Api # http://localhost:5080 +dotnet run --project Stampede.Http.Sample.Client # http://localhost:5081 ``` -`no-store` live counters: how many requests actually reached the origin — across *all* client instances. Compare it with how many requests the clients have issued. +`launchSettings.json` also carries a **client (baseline, no Stampede.Http)** profile on port 5082 so you can run the comparison locally. Prometheus, Grafana and Jaeger aren't running outside compose; the clients still serve `/metrics`, and traces are simply not exported when `OTEL_EXPORTER_OTLP_ENDPOINT` is unset. + +--- + +## Automated verification -## Live metrics: Prometheus + Grafana +```bash +./scripts/smoke-test.sh # add KEEP_STACK=1 to leave the stack up afterwards +``` -Every `stampede_http.*` instrument (the same ones described in the [main README](../README.md#metrics)) is exported by each client via `OpenTelemetry.Exporter.Prometheus.HttpListener` on port `9464`, scraped by Prometheus every 5 seconds, and graphed by a pre-provisioned Grafana dashboard — no manual data source or dashboard setup required. +Brings the whole stack up and asserts the behaviour this README claims, using the origin's counters as the source of truth: the feature tour reports 12/12, a 20-caller burst costs at most one origin call, the same burst against the control client costs twenty, `Vary` keeps one entry per representation, and the instruments really are exported. -The dashboard has 10 panels: cache hits/misses/hit-ratio, coalescing deduplication (rate) and in-flight count, stale-if-error and stale-while-revalidate rates, revalidations and invalidations, and coalescing timeouts — each broken down **per client instance** so you can watch `client-a` and `client-b` side by side. That per-instance split is exactly what makes the earlier nuance visible: cache-hit panels for both instances rise together (shared Redis), while coalescing panels move independently (per-process). +This runs in CI on every push, and the sample projects are part of `Stampede.Http.slnx`, so neither the code nor the deployment can rot silently. -> **Naming caveat:** Prometheus 3.x negotiates UTF-8 metric names (e.g. `stampede_http.cache.hits_requests_total`, dots preserved) with targets that support it, which the OTel exporter does. `prometheus.yml` sets `metric_name_escaping_scheme: underscores` globally so names stay in the classic Prometheus form (`stampede_http_cache_hits_requests_total`) that the dashboard queries and this README use. +--- -## One nuance worth knowing +## What this sample does not claim -The **cache** (Redis) is shared across instances, but **coalescing is per process**: if both replicas miss the same key at the same instant, each makes its own origin call (2 total, not 1 — still not 20). Cross-instance request deduplication would require a distributed lock, which is out of scope for an HTTP client library. +- **Coalescing is per process.** The Redis cache is shared, but if both replicas miss the same key at the same instant, each makes its own origin call — 2, not 1, and still not 20. Cross-instance deduplication would need a distributed lock, which is out of scope for an HTTP client library. The dashboard makes this visible: cache-hit panels for `client-a` and `client-b` rise together; coalescing panels move independently. +- **The headline percentage is a property of the workload, not of the library.** It is bounded by `1 − interval / max-age`, and this sample deliberately runs 5–10 second TTLs against a 2-second poll so that expiry is visible in a short demo. Read [How to read that percentage](#how-to-read-that-percentage) before quoting the number anywhere, and model your own TTLs first. +- **Failures are not cacheable.** `stale-if-error` rescues the caller *after* the origin has failed; the request still goes out, and Polly retries it. Most of the origin traffic Stampede.Http cannot avoid in this sample is `/flaky` returning 503 — and unlike cacheable traffic, that cost scales with the number of replicas rather than being shared. +- **The origin is a toy.** It fabricates latency with `Task.Delay` and keeps its state in memory. Its job is to emit realistic headers, not to be a realistic service. diff --git a/samples/Stampede.Http.Sample.Api/Dockerfile b/samples/Stampede.Http.Sample.Api/Dockerfile index 7f786d0..6e82538 100644 --- a/samples/Stampede.Http.Sample.Api/Dockerfile +++ b/samples/Stampede.Http.Sample.Api/Dockerfile @@ -5,6 +5,10 @@ COPY samples/Stampede.Http.Sample.Api/ samples/Stampede.Http.Sample.Api/ RUN dotnet publish samples/Stampede.Http.Sample.Api -c Release -o /app FROM mcr.microsoft.com/dotnet/aspnet:10.0 +# curl is here only so docker compose can run a real HTTP healthcheck against /health. +RUN apt-get update \ + && apt-get install -y --no-install-recommends curl \ + && rm -rf /var/lib/apt/lists/* WORKDIR /app COPY --from=build /app . ENV ASPNETCORE_URLS=http://+:8080 diff --git a/samples/Stampede.Http.Sample.Api/Endpoints/CoreEndpoints.cs b/samples/Stampede.Http.Sample.Api/Endpoints/CoreEndpoints.cs new file mode 100644 index 0000000..ce1a1e1 --- /dev/null +++ b/samples/Stampede.Http.Sample.Api/Endpoints/CoreEndpoints.cs @@ -0,0 +1,140 @@ +namespace Stampede.Http.Sample.Api.Endpoints; + +/// +/// The core RFC 9111 / RFC 5861 scenarios: freshness, conditional revalidation, +/// unsafe-method invalidation, stale-while-revalidate, stale-if-error and the +/// slow endpoint used for the stampede. +/// +public static class CoreEndpoints +{ + /// Maps the core caching endpoints. + /// The route builder to map onto. + /// The same route builder, for chaining. + public static IEndpointRouteBuilder MapCoreEndpoints(this IEndpointRouteBuilder app) + { + // GET /catalog — max-age + ETag: fresh hits, then conditional revalidation + // (a 304 costs no body). stale-if-error keeps the entry usable during outages. + app.MapGet("/catalog", async (HttpRequest req, HttpResponse res, OriginState state) => + { + state.Count("GET /catalog"); + await Task.Delay(300); + + string etag = $"\"catalog-v{state.CatalogVersion}\""; + res.Headers.CacheControl = "public, max-age=10, stale-if-error=60"; + res.Headers.ETag = etag; + + if (req.Headers.IfNoneMatch.ToString().Contains(etag, StringComparison.Ordinal)) + { + state.Count("GET /catalog -> 304"); + return Results.StatusCode(StatusCodes.Status304NotModified); + } + + return Results.Json(new + { + version = state.CatalogVersion, + items = 120, + servedAt = DateTimeOffset.UtcNow, + }); + }); + + // POST /catalog — unsafe method: a 2xx response makes Stampede.Http evict the + // cached GET /catalog entry (RFC 9111 §4.4). With the Redis store that + // invalidation is shared by every client instance. + app.MapPost("/catalog", (OriginState state) => + { + state.Count("POST /catalog"); + state.BumpCatalogVersion(); + return Results.NoContent(); + }); + + // GET /feed — stale-while-revalidate: expiries are refreshed in the background, + // so callers never wait for the origin after the first fetch. + app.MapGet("/feed", async (HttpResponse res, OriginState state) => + { + state.Count("GET /feed"); + await Task.Delay(500); + res.Headers.CacheControl = "max-age=5, stale-while-revalidate=30"; + return Results.Json(new + { + generation = state.NextFeedGeneration(), + servedAt = DateTimeOffset.UtcNow, + }); + }); + + // GET /flaky — fails in bursts. Polly retries transient blips; when the outage + // outlasts the retries, stale-if-error shields the caller with the last good 200. + app.MapGet("/flaky", async (HttpResponse res, OriginState state) => + { + state.Count("GET /flaky"); + if (OriginState.FlakyIsDown()) + { + state.Count("GET /flaky -> 503"); + return Results.StatusCode(StatusCodes.Status503ServiceUnavailable); + } + + await Task.Delay(100); + res.Headers.CacheControl = "max-age=5, stale-if-error=120"; + return Results.Json(new { status = "healthy", servedAt = DateTimeOffset.UtcNow }); + }); + + // GET /slow — 2 s of origin latency: the stampede showcase. N concurrent + // cold-cache callers should produce a single origin call per client process. + app.MapGet("/slow", async (HttpResponse res, OriginState state) => + { + state.Count("GET /slow"); + await Task.Delay(2000); + res.Headers.CacheControl = "max-age=30"; + return Results.Json(new { report = "quarterly", servedAt = DateTimeOffset.UtcNow }); + }); + + // GET /ledger — max-age + must-revalidate + ETag: once stale the entry may NOT + // be served without checking the origin, so no stale window applies even under + // failure. Contrast with /flaky. + app.MapGet("/ledger", async (HttpRequest req, HttpResponse res, OriginState state) => + { + state.Count("GET /ledger"); + await Task.Delay(200); + + const string Etag = "\"ledger-2026-07\""; + res.Headers.CacheControl = "max-age=10, must-revalidate"; + res.Headers.ETag = Etag; + + if (req.Headers.IfNoneMatch.ToString().Contains(Etag, StringComparison.Ordinal)) + { + state.Count("GET /ledger -> 304"); + return Results.StatusCode(StatusCodes.Status304NotModified); + } + + return Results.Json(new { balance = 42_150.75m, closed = true }); + }); + + // GET /docs/{id} — validator is Last-Modified rather than ETag, so expiry + // triggers an If-Modified-Since revalidation. + app.MapGet("/docs/{id}", (string id, HttpRequest req, HttpResponse res, OriginState state) => + { + state.Count("GET /docs/{id}"); + + // Stable per-document timestamp so If-Modified-Since can match. + DateTimeOffset lastModified = new DateTimeOffset(2026, 7, 1, 12, 0, 0, TimeSpan.Zero) + .AddMinutes(id.GetHashCode(StringComparison.Ordinal) % 60); + + res.Headers.CacheControl = "max-age=5"; + res.Headers.LastModified = lastModified.ToString("R", System.Globalization.CultureInfo.InvariantCulture); + + if (DateTimeOffset.TryParse( + req.Headers.IfModifiedSince, + System.Globalization.CultureInfo.InvariantCulture, + System.Globalization.DateTimeStyles.AssumeUniversal, + out DateTimeOffset since) + && lastModified <= since) + { + state.Count("GET /docs/{id} -> 304"); + return Results.StatusCode(StatusCodes.Status304NotModified); + } + + return Results.Json(new { id, title = $"Document {id}", lastModified }); + }); + + return app; + } +} diff --git a/samples/Stampede.Http.Sample.Api/Endpoints/DiagnosticsEndpoints.cs b/samples/Stampede.Http.Sample.Api/Endpoints/DiagnosticsEndpoints.cs new file mode 100644 index 0000000..5eee88f --- /dev/null +++ b/samples/Stampede.Http.Sample.Api/Endpoints/DiagnosticsEndpoints.cs @@ -0,0 +1,45 @@ +namespace Stampede.Http.Sample.Api.Endpoints; + +/// +/// Endpoints the origin uses to describe itself: liveness, live counters and a +/// counter reset used by the automated smoke test. +/// +public static class DiagnosticsEndpoints +{ + /// Maps the diagnostics endpoints. + /// The route builder to map onto. + /// The same route builder, for chaining. + public static IEndpointRouteBuilder MapDiagnosticsEndpoints(this IEndpointRouteBuilder app) + { + // GET /health — compose healthcheck target. Never cached. + app.MapGet("/health", (HttpResponse res) => + { + res.Headers.CacheControl = "no-store"; + return Results.Ok(new { status = "healthy" }); + }); + + // GET /stats — no-store: live origin-side counters. This is how you see how + // many requests actually reached the origin across ALL client instances. + app.MapGet("/stats", (HttpResponse res, OriginState state) => + { + res.Headers.CacheControl = "no-store"; + return Results.Json(new + { + flakyIsDown = OriginState.FlakyIsDown(), + catalogVersion = state.CatalogVersion, + counters = state.Snapshot(), + }); + }); + + // POST /stats/reset — clears the counters so a measurement window can start + // from zero. Used by scripts/smoke-test.sh. + app.MapPost("/stats/reset", (HttpResponse res, OriginState state) => + { + res.Headers.CacheControl = "no-store"; + state.ResetCounters(); + return Results.NoContent(); + }); + + return app; + } +} diff --git a/samples/Stampede.Http.Sample.Api/Endpoints/VariantEndpoints.cs b/samples/Stampede.Http.Sample.Api/Endpoints/VariantEndpoints.cs new file mode 100644 index 0000000..d680135 --- /dev/null +++ b/samples/Stampede.Http.Sample.Api/Endpoints/VariantEndpoints.cs @@ -0,0 +1,99 @@ +using System.Text; + +namespace Stampede.Http.Sample.Api.Endpoints; + +/// +/// Endpoints that exercise content negotiation (Vary), multi-tenancy, +/// immutable, the body-size ceiling and query-parameter normalisation. +/// +public static class VariantEndpoints +{ + private static readonly string[] BulkPayload = Enumerable.Range(0, 20_000) + .Select(i => $"row-{i:D6}-{new string('x', 80)}") + .ToArray(); + + /// Maps the variant / policy-showcase endpoints. + /// The route builder to map onto. + /// The same route builder, for chaining. + public static IEndpointRouteBuilder MapVariantEndpoints(this IEndpointRouteBuilder app) + { + // GET /greetings — Vary: Accept-Language. One URL, one cached variant per + // language (RFC 9111 §4.1). Before v2.2.0 these overwrote each other. + app.MapGet("/greetings", async (HttpRequest req, HttpResponse res, OriginState state) => + { + state.Count("GET /greetings"); + await Task.Delay(200); + + string lang = req.Headers.AcceptLanguage.ToString(); + string greeting = lang.StartsWith("es", StringComparison.OrdinalIgnoreCase) ? "¡Hola!" + : lang.StartsWith("fr", StringComparison.OrdinalIgnoreCase) ? "Bonjour !" + : "Hello!"; + + res.Headers.CacheControl = "max-age=30"; + res.Headers.Vary = "Accept-Language"; + return Results.Json(new { greeting, language = lang, servedAt = DateTimeOffset.UtcNow }); + }); + + // GET /tenants/data — the multi-tenant case: one URL, tenant chosen by header. + // Vary: X-Tenant-Id gives each tenant its own cache entry, and the client adds + // X-Tenant-Id to CoalesceKeyHeaders so concurrent bursts are deduplicated + // per tenant instead of collapsing two tenants into one response. + app.MapGet("/tenants/data", async (HttpRequest req, HttpResponse res, OriginState state) => + { + string tenant = req.Headers["X-Tenant-Id"].ToString(); + tenant = string.IsNullOrWhiteSpace(tenant) ? "public" : tenant; + + state.Count($"GET /tenants/data [{tenant}]"); + await Task.Delay(1000); + + res.Headers.CacheControl = "max-age=20"; + res.Headers.Vary = "X-Tenant-Id"; + return Results.Json(new { tenant, seats = tenant.Length * 10, servedAt = DateTimeOffset.UtcNow }); + }); + + // GET /assets/{id} — Cache-Control: immutable (RFC 8246). A fresh immutable + // entry skips revalidation even when the caller asks for one, which is exactly + // what you want for content-addressed assets. + app.MapGet("/assets/{id}", (string id, HttpResponse res, OriginState state) => + { + state.Count("GET /assets/{id}"); + res.Headers.CacheControl = "public, max-age=31536000, immutable"; + res.Headers.ETag = $"\"asset-{id}\""; + return Results.Json(new { id, bytes = 4096, servedAt = DateTimeOffset.UtcNow }); + }); + + // GET /bulk — a ~1.8 MB body with a perfectly good max-age. It still never gets + // cached: it exceeds CacheOptions.MaxBodySizeBytes (1 MB by default), so the + // cache declines to store it rather than blowing up memory. + app.MapGet("/bulk", (HttpResponse res, OriginState state) => + { + state.Count("GET /bulk"); + res.Headers.CacheControl = "max-age=300"; + + StringBuilder sb = new(BulkPayload.Length * 90); + foreach (string row in BulkPayload) + { + _ = sb.AppendLine(row); + } + + return Results.Text(sb.ToString(), "text/plain"); + }); + + // GET /search — the same logical query can arrive with its parameters in any + // order. With CacheOptions.NormalizeQueryParameters the client folds + // ?a=1&b=2 and ?b=2&a=1 onto one entry; without it they are two misses. + app.MapGet("/search", (HttpRequest req, HttpResponse res, OriginState state) => + { + state.Count("GET /search"); + res.Headers.CacheControl = "max-age=60"; + + var query = req.Query + .OrderBy(kv => kv.Key, StringComparer.Ordinal) + .ToDictionary(kv => kv.Key, kv => kv.Value.ToString(), StringComparer.Ordinal); + + return Results.Json(new { query, servedAt = DateTimeOffset.UtcNow }); + }); + + return app; + } +} diff --git a/samples/Stampede.Http.Sample.Api/OriginMetrics.cs b/samples/Stampede.Http.Sample.Api/OriginMetrics.cs new file mode 100644 index 0000000..b77ab04 --- /dev/null +++ b/samples/Stampede.Http.Sample.Api/OriginMetrics.cs @@ -0,0 +1,59 @@ +using System.Diagnostics; +using System.Diagnostics.Metrics; + +namespace Stampede.Http.Sample.Api; + +/// +/// Origin-side instrumentation. This is the control instrument for the whole sample: +/// it counts requests that actually reached the origin, broken down by endpoint and by +/// which client sent them (via the X-Client header the clients attach). +/// +/// +/// Comparing this counter for a Stampede.Http-enabled client against the one running a +/// bare is what turns "caching helps" into a number. +/// +public sealed class OriginMetrics : IDisposable +{ + /// Name of the meter published by the sample origin. + public const string MeterName = "Stampede.Http.Sample.Api"; + + private readonly Meter _meter; + private readonly Counter _requests; + private readonly Histogram _duration; + + /// Initialises the origin meter and its instruments. + public OriginMetrics() + { + _meter = new Meter(MeterName); + // No unit on the counter: the OTel Prometheus exporter appends the unit to the + // metric name, and "sample_api_origin_requests_requests_total" helps nobody. + _requests = _meter.CreateCounter( + "sample_api.origin.requests", + description: "Requests that actually reached the origin."); + _duration = _meter.CreateHistogram( + "sample_api.origin.duration", + unit: "ms", + description: "Origin-side handling time."); + } + + /// Records one request that reached the origin. + /// Route pattern (e.g. /catalog). + /// Value of the X-Client request header, or unknown. + /// Response status code the origin produced. + /// Origin-side handling time in milliseconds. + public void Record(string endpoint, string client, int statusCode, double elapsedMs) + { + TagList tags = new() + { + { "endpoint", endpoint }, + { "client", client }, + { "status", statusCode }, + }; + + _requests.Add(1, tags); + _duration.Record(elapsedMs, tags); + } + + /// + public void Dispose() => _meter.Dispose(); +} diff --git a/samples/Stampede.Http.Sample.Api/OriginState.cs b/samples/Stampede.Http.Sample.Api/OriginState.cs new file mode 100644 index 0000000..ec7a3f6 --- /dev/null +++ b/samples/Stampede.Http.Sample.Api/OriginState.cs @@ -0,0 +1,41 @@ +using System.Collections.Concurrent; + +namespace Stampede.Http.Sample.Api; + +/// +/// Mutable origin state shared by the endpoints: request counters (surfaced by +/// /stats), the catalog version bumped by POST /catalog, and the feed +/// generation incremented on every origin fetch. +/// +public sealed class OriginState +{ + private readonly ConcurrentDictionary _counters = new(StringComparer.Ordinal); + private int _catalogVersion = 1; + private int _feedGeneration; + + /// Gets the current catalog version, bumped by POST /catalog. + public int CatalogVersion => Volatile.Read(ref _catalogVersion); + + /// Bumps the catalog version and returns the new value. + public int BumpCatalogVersion() => Interlocked.Increment(ref _catalogVersion); + + /// Increments and returns the feed generation. + public int NextFeedGeneration() => Interlocked.Increment(ref _feedGeneration); + + /// Increments the named counter surfaced by /stats. + public void Count(string key) => _counters.AddOrUpdate(key, 1, static (_, v) => v + 1); + + /// Returns a snapshot of all counters, ordered by key. + public IReadOnlyDictionary Snapshot() => + _counters.OrderBy(kv => kv.Key, StringComparer.Ordinal) + .ToDictionary(kv => kv.Key, kv => kv.Value, StringComparer.Ordinal); + + /// Clears every counter — used by the smoke test to isolate a measurement window. + public void ResetCounters() => _counters.Clear(); + + /// + /// /flaky is "down" for the first 20 seconds of every 60-second window, so the + /// outage is reproducible without anyone having to stop a container. + /// + public static bool FlakyIsDown() => Environment.TickCount64 / 1000 % 60 < 20; +} diff --git a/samples/Stampede.Http.Sample.Api/Program.cs b/samples/Stampede.Http.Sample.Api/Program.cs index c8183ef..9440a90 100644 --- a/samples/Stampede.Http.Sample.Api/Program.cs +++ b/samples/Stampede.Http.Sample.Api/Program.cs @@ -1,115 +1,70 @@ -using System.Collections.Concurrent; +using System.Diagnostics; +using OpenTelemetry.Metrics; +using OpenTelemetry.Resources; +using OpenTelemetry.Trace; +using Stampede.Http.Sample.Api; +using Stampede.Http.Sample.Api.Endpoints; // --------------------------------------------------------------------------- // Sample origin API — every endpoint drives Stampede.Http purely through // standard HTTP caching headers. There is no Stampede.Http code here: the // origin declares policy, the client-side cache obeys it. +// +// The origin also exports its own metrics (/metrics) and traces (OTLP), so the +// dashboards can show the number that actually matters: how much traffic the +// clients kept off the origin. // --------------------------------------------------------------------------- var builder = WebApplication.CreateBuilder(args); -var app = builder.Build(); - -var counters = new ConcurrentDictionary(); -int catalogVersion = 1; -int feedGeneration = 0; - -void Count(string key) => counters.AddOrUpdate(key, 1, (_, v) => v + 1); -// /flaky is "down" for the first 20 seconds of every 60-second window. -bool FlakyIsDown() => Environment.TickCount64 / 1000 % 60 < 20; +builder.Services.AddSingleton(); +builder.Services.AddSingleton(); -// GET /catalog — max-age + ETag: demonstrates fresh hits and conditional -// revalidation (304 costs no body). stale-if-error keeps the entry usable -// during outages. -app.MapGet("/catalog", async (HttpRequest req, HttpResponse res) => -{ - Count("GET /catalog"); - await Task.Delay(300); - - string etag = $"\"catalog-v{catalogVersion}\""; - res.Headers.CacheControl = "public, max-age=10, stale-if-error=60"; - res.Headers.ETag = etag; +string? otlpEndpoint = builder.Configuration["OTEL_EXPORTER_OTLP_ENDPOINT"]; - if (req.Headers.IfNoneMatch.ToString().Contains(etag)) +builder.Services.AddOpenTelemetry() + .ConfigureResource(r => r.AddService( + serviceName: "sample-api", + serviceInstanceId: Environment.MachineName)) + .WithMetrics(m => m + .AddMeter(OriginMetrics.MeterName) + .AddAspNetCoreInstrumentation() + .AddPrometheusExporter()) + .WithTracing(t => { - Count("GET /catalog -> 304"); - return Results.StatusCode(StatusCodes.Status304NotModified); - } - - return Results.Json(new { version = catalogVersion, items = 120, servedAt = DateTimeOffset.UtcNow }); -}); - -// POST /catalog — unsafe method: a 2xx response makes Stampede.Http evict the -// cached GET /catalog entry (RFC 9111 §4.4). With the Redis store this -// invalidation is shared by every client instance. -app.MapPost("/catalog", () => -{ - Count("POST /catalog"); - Interlocked.Increment(ref catalogVersion); - return Results.NoContent(); -}); + t.AddAspNetCoreInstrumentation(o => + // Scrapes and healthchecks would drown out the interesting spans. + o.Filter = ctx => !ctx.Request.Path.StartsWithSegments("/metrics") + && !ctx.Request.Path.StartsWithSegments("/health")); + + if (!string.IsNullOrWhiteSpace(otlpEndpoint)) + { + t.AddOtlpExporter(); + } + }); -// GET /feed — stale-while-revalidate: expiries are refreshed in the -// background; callers never wait for the origin after the first fetch. -app.MapGet("/feed", async (HttpResponse res) => -{ - Count("GET /feed"); - await Task.Delay(500); - int gen = Interlocked.Increment(ref feedGeneration); - res.Headers.CacheControl = "max-age=5, stale-while-revalidate=30"; - return Results.Json(new { generation = gen, servedAt = DateTimeOffset.UtcNow }); -}); +var app = builder.Build(); -// GET /flaky — fails in bursts. Polly retries transient blips; when the -// outage outlasts the retries, stale-if-error shields the caller with the -// last good 200. -app.MapGet("/flaky", async (HttpResponse res) => +// Every request that gets here is, by definition, a request the client cache did +// not absorb. Attribute it to the endpoint and to the calling client. +app.Use(async (ctx, next) => { - Count("GET /flaky"); - if (FlakyIsDown()) - { - Count("GET /flaky -> 503"); - return Results.StatusCode(StatusCodes.Status503ServiceUnavailable); - } + long start = Stopwatch.GetTimestamp(); + await next(ctx); - await Task.Delay(100); - res.Headers.CacheControl = "max-age=5, stale-if-error=120"; - return Results.Json(new { status = "healthy", servedAt = DateTimeOffset.UtcNow }); -}); - -// GET /slow — 2 s of origin latency: the stampede showcase. N concurrent -// cold-cache callers should produce a single origin call per client process. -app.MapGet("/slow", async (HttpResponse res) => -{ - Count("GET /slow"); - await Task.Delay(2000); - res.Headers.CacheControl = "max-age=30"; - return Results.Json(new { report = "quarterly", servedAt = DateTimeOffset.UtcNow }); -}); + string endpoint = (ctx.GetEndpoint() as RouteEndpoint)?.RoutePattern.RawText ?? ctx.Request.Path.ToString(); + string client = ctx.Request.Headers["X-Client"].ToString(); -// GET /greetings — Vary: Accept-Language: one URL, one cached variant per -// language. -app.MapGet("/greetings", (HttpRequest req, HttpResponse res) => -{ - Count("GET /greetings"); - string lang = req.Headers.AcceptLanguage.ToString(); - string greeting = lang.StartsWith("es", StringComparison.OrdinalIgnoreCase) ? "¡Hola!" : "Hello!"; - res.Headers.CacheControl = "max-age=30"; - res.Headers.Vary = "Accept-Language"; - return Results.Json(new { greeting, language = lang }); + ctx.RequestServices.GetRequiredService().Record( + endpoint, + string.IsNullOrWhiteSpace(client) ? "unknown" : client, + ctx.Response.StatusCode, + Stopwatch.GetElapsedTime(start).TotalMilliseconds); }); -// GET /stats — no-store: live origin-side counters, never cached. This is -// how the client (and you) can see how many requests actually reached the -// origin across ALL client instances. -app.MapGet("/stats", (HttpResponse res) => -{ - res.Headers.CacheControl = "no-store"; - return Results.Json(new - { - flakyIsDown = FlakyIsDown(), - counters = counters.OrderBy(kv => kv.Key).ToDictionary(kv => kv.Key, kv => kv.Value), - }); -}); +app.MapCoreEndpoints(); +app.MapVariantEndpoints(); +app.MapDiagnosticsEndpoints(); +app.MapPrometheusScrapingEndpoint(); app.Run(); diff --git a/samples/Stampede.Http.Sample.Api/Properties/launchSettings.json b/samples/Stampede.Http.Sample.Api/Properties/launchSettings.json new file mode 100644 index 0000000..b87f288 --- /dev/null +++ b/samples/Stampede.Http.Sample.Api/Properties/launchSettings.json @@ -0,0 +1,13 @@ +{ + "$schema": "https://json.schemastore.org/launchsettings.json", + "profiles": { + "api": { + "commandName": "Project", + "launchBrowser": false, + "applicationUrl": "http://localhost:5080", + "environmentVariables": { + "ASPNETCORE_ENVIRONMENT": "Development" + } + } + } +} diff --git a/samples/Stampede.Http.Sample.Api/Stampede.Http.Sample.Api.csproj b/samples/Stampede.Http.Sample.Api/Stampede.Http.Sample.Api.csproj index a3a34b6..d9f6912 100644 --- a/samples/Stampede.Http.Sample.Api/Stampede.Http.Sample.Api.csproj +++ b/samples/Stampede.Http.Sample.Api/Stampede.Http.Sample.Api.csproj @@ -4,6 +4,17 @@ net10.0 enable enable + Stampede.Http.Sample.Api + + NU5104 + + + + + + + + diff --git a/samples/Stampede.Http.Sample.Api/appsettings.json b/samples/Stampede.Http.Sample.Api/appsettings.json new file mode 100644 index 0000000..3408cda --- /dev/null +++ b/samples/Stampede.Http.Sample.Api/appsettings.json @@ -0,0 +1,10 @@ +{ + "Logging": { + "LogLevel": { + "Default": "Information", + "Microsoft.AspNetCore": "Warning", + "Microsoft.Hosting.Lifetime": "Information" + } + }, + "AllowedHosts": "*" +} diff --git a/samples/Stampede.Http.Sample.Client/Dockerfile b/samples/Stampede.Http.Sample.Client/Dockerfile index c371003..1780d4a 100644 --- a/samples/Stampede.Http.Sample.Client/Dockerfile +++ b/samples/Stampede.Http.Sample.Client/Dockerfile @@ -6,7 +6,13 @@ COPY Stampede.Http/ Stampede.Http/ COPY samples/Stampede.Http.Sample.Client/ samples/Stampede.Http.Sample.Client/ RUN dotnet publish samples/Stampede.Http.Sample.Client -c Release -o /app -FROM mcr.microsoft.com/dotnet/runtime:10.0 +FROM mcr.microsoft.com/dotnet/aspnet:10.0 +# curl is here only so docker compose can run a real HTTP healthcheck against /healthz. +RUN apt-get update \ + && apt-get install -y --no-install-recommends curl \ + && rm -rf /var/lib/apt/lists/* WORKDIR /app COPY --from=build /app . +ENV ASPNETCORE_URLS=http://+:8080 +EXPOSE 8080 ENTRYPOINT ["dotnet", "Stampede.Http.Sample.Client.dll"] diff --git a/samples/Stampede.Http.Sample.Client/Endpoints/BffEndpoints.cs b/samples/Stampede.Http.Sample.Client/Endpoints/BffEndpoints.cs new file mode 100644 index 0000000..9e80180 --- /dev/null +++ b/samples/Stampede.Http.Sample.Client/Endpoints/BffEndpoints.cs @@ -0,0 +1,117 @@ +using Microsoft.Extensions.Options; +using Stampede.Http.Caching; +using Stampede.Http.Options; +using Stampede.Http.Sample.Client.Workload; + +namespace Stampede.Http.Sample.Client.Endpoints; + +/// +/// The client's own HTTP surface. Every endpoint is a thin pass-through to the origin, which +/// is what makes the sample driveable by a real load generator or by curl instead of only by +/// the scripted workload — the stampede then comes from actual concurrent inbound traffic. +/// +public static class BffEndpoints +{ + /// Origin paths exposed verbatim under /api, one per caching behaviour. + private static readonly string[] PassThroughPaths = ["/catalog", "/feed", "/flaky", "/slow", "/ledger", "/bulk"]; + + /// Maps the client's endpoints. + /// The route builder to map onto. + /// The same route builder, for chaining. + public static IEndpointRouteBuilder MapBffEndpoints(this IEndpointRouteBuilder app) + { + app.MapGet("/healthz", () => Results.Ok(new { status = "healthy" })); + + RouteGroupBuilder api = app.MapGroup("/api"); + + // Straight pass-throughs — one per caching behaviour. + foreach (string path in PassThroughPaths) + { + _ = api.MapGet(path, (OriginClient client, CancellationToken ct) => + client.GetAsync(path, cancellationToken: ct)); + } + + // Content negotiation: same URL, one cached representation per language. + _ = api.MapGet("/greetings", (OriginClient client, string? lang, CancellationToken ct) => + client.GetAsync( + "/greetings", + r => r.Headers.TryAddWithoutValidation("Accept-Language", lang ?? "en-GB"), + ct)); + + // Multi-tenant: the tenant travels in a header that is both a Vary field at the + // origin and a coalescing key on the client. + _ = api.MapGet("/tenants/{tenant}", (OriginClient client, string tenant, CancellationToken ct) => + client.GetAsync("/tenants/data", r => r.Headers.Add("X-Tenant-Id", tenant), ct)); + + _ = api.MapGet("/assets/{id}", (OriginClient client, string id, CancellationToken ct) => + client.GetAsync($"/assets/{id}", cancellationToken: ct)); + + _ = api.MapGet("/docs/{id}", (OriginClient client, string id, CancellationToken ct) => + client.GetAsync($"/docs/{id}", cancellationToken: ct)); + + _ = api.MapGet("/search", (OriginClient client, HttpRequest request, CancellationToken ct) => + client.GetAsync($"/search{request.QueryString}", cancellationToken: ct)); + + // The mutation that evicts the shared GET entry for every instance (RFC 9111 §4.4). + _ = api.MapPost("/catalog", async (OriginClient client, CancellationToken ct) => + Results.Ok(new { originStatus = await client.PostAsync("/catalog", ct) })); + + // What the origin actually received, straight from the origin's own counters. + _ = api.MapGet("/origin-stats", (OriginClient client, CancellationToken ct) => + client.GetOriginCountersAsync(ct)); + + // The Stampede.Http instrument totals for this process, in JSON. The same values are + // exported for Prometheus at /metrics. + _ = api.MapGet("/counters", (StampedeCounters counters) => counters.Snapshot()); + + // The effective, currently-loaded options. Edit samples/config/client.json while the + // stack is running and refresh this: IOptionsMonitor picks the change up with no + // restart. Structural options (MaxCacheSize, NormalizeQueryParameters, + // RevalidationGraceSeconds) are read once at registration and will not move. + _ = api.MapGet("/config", ( + IOptions sample, + IOptionsMonitor cache, + IOptionsMonitor coalescing) => + { + if (!sample.Value.Pipeline.Enabled) + { + // The control group has no handlers registered, so the options objects would + // report library defaults that nothing is actually using. + return Results.Json(new + { + instance = sample.Value.Instance, + stampedeEnabled = false, + note = "Control group: no Stampede.Http handlers are registered on this instance.", + }); + } + + CacheOptions c = cache.Get(PipelineRegistration.ClientName); + CoalescerOptions q = coalescing.Get(PipelineRegistration.ClientName); + + return Results.Json(new + { + instance = sample.Value.Instance, + stampedeEnabled = true, + cache = new + { + c.DefaultTtl, + c.MaxBodySizeBytes, + c.DefaultStaleIfErrorSeconds, + c.DefaultStaleWhileRevalidateSeconds, + c.RevalidationGraceSeconds, + c.NormalizeQueryParameters, + c.MaxCacheSize, + }, + coalescing = new + { + q.Enabled, + q.CoalescingTimeout, + q.MaxResponseBodyBytes, + q.CoalesceKeyHeaders, + }, + }); + }); + + return app; + } +} diff --git a/samples/Stampede.Http.Sample.Client/OriginClient.cs b/samples/Stampede.Http.Sample.Client/OriginClient.cs new file mode 100644 index 0000000..aafec21 --- /dev/null +++ b/samples/Stampede.Http.Sample.Client/OriginClient.cs @@ -0,0 +1,124 @@ +using System.Diagnostics; +using System.Net; + +namespace Stampede.Http.Sample.Client; + +/// +/// The outcome of a single call to the origin, as observed by the caller. Everything the +/// sample wants to show — was it a hit, how old was it, how long did it take — is visible +/// from standard response metadata, which is the point: the caller never touches a cache key. +/// +/// Request path. +/// Status code as seen by the caller. +/// Wall-clock time for the call, including cache lookup. +/// Value of the Age response header; means the response came straight from the origin. +/// Truncated response body, for the logs. +public sealed record ProbeResult( + string Path, + int StatusCode, + long ElapsedMs, + double? AgeSeconds, + string Body) +{ + /// Whether the response was served from the cache rather than the origin. + public bool FromCache => AgeSeconds is not null; + + /// A short human-readable rendering used in the workload logs. + public string Describe() + { + string age = AgeSeconds is null ? "fresh from origin" : $"Age: {AgeSeconds.Value:F0}s"; + string stale = AgeSeconds is > 10 ? " [beyond max-age: stale window or revalidated entry]" : string.Empty; + return $"{Path,-22} -> {StatusCode} {ElapsedMs,5} ms ({age}){stale} {Body}"; + } +} + +/// +/// Typed client for the sample origin. The Stampede.Http pipeline sits underneath this class +/// and is entirely invisible to it — no cache keys, no TTLs, no invalidation calls. +/// +public sealed class OriginClient(HttpClient http) +{ + /// Maximum number of body characters kept for logging. + private const int BodySnippetLength = 60; + + /// Gets the underlying client, for the rare call that needs full control. + public HttpClient Http { get; } = http; + + /// + /// Issues a GET and summarises the result. can set request + /// headers or Stampede.Http per-request policies via . + /// + /// Request path, relative to the configured base address. + /// Optional hook to customise the request before it is sent. + /// Cancellation token. + /// A summary of what the caller observed. + public async Task GetAsync( + string path, + Action? configure = null, + CancellationToken cancellationToken = default) + { + long start = Stopwatch.GetTimestamp(); + + using HttpRequestMessage request = new(HttpMethod.Get, path); + configure?.Invoke(request); + + using HttpResponseMessage response = await Http.SendAsync(request, cancellationToken).ConfigureAwait(false); + + string body = response.StatusCode == HttpStatusCode.NotModified + ? "(304, no body)" + : await response.Content.ReadAsStringAsync(cancellationToken).ConfigureAwait(false); + + if (body.Length > BodySnippetLength) + { + body = body[..BodySnippetLength] + "…"; + } + + return new ProbeResult( + path, + (int)response.StatusCode, + (long)Stopwatch.GetElapsedTime(start).TotalMilliseconds, + response.Headers.Age?.TotalSeconds, + body.ReplaceLineEndings(" ")); + } + + /// Reads the ETag the origin (or the cache) currently reports for a resource. + /// Request path. + /// Cancellation token. + /// The entity tag, or when the resource carries none. + public async Task GetETagAsync(string path, CancellationToken cancellationToken = default) + { + using HttpResponseMessage response = await Http.GetAsync(path, cancellationToken).ConfigureAwait(false); + return response.Headers.ETag?.ToString(); + } + + /// Issues the unsafe request that triggers RFC 9111 §4.4 invalidation. + /// Request path. + /// Cancellation token. + /// The status code returned by the origin. + public async Task PostAsync(string path, CancellationToken cancellationToken = default) + { + using HttpResponseMessage response = await Http.PostAsync(path, content: null, cancellationToken).ConfigureAwait(false); + return (int)response.StatusCode; + } + + /// + /// Reads the origin's own request counters. /stats is no-store, so this always + /// reflects reality rather than a cached snapshot. + /// + /// Cancellation token. + /// Counter name to value, as reported by the origin. + public async Task> GetOriginCountersAsync(CancellationToken cancellationToken = default) + { + OriginStats? stats = await Http.GetFromJsonAsync("/stats", cancellationToken).ConfigureAwait(false); + return stats?.Counters ?? new Dictionary(StringComparer.Ordinal); + } + + /// Shape of the origin's /stats payload. + /// Whether /flaky is currently in its failure window. + /// Current catalog version. + /// Per-endpoint request counters. + public sealed record OriginStats( + bool FlakyIsDown, + int CatalogVersion, + Dictionary Counters); +} diff --git a/samples/Stampede.Http.Sample.Client/PipelineRegistration.cs b/samples/Stampede.Http.Sample.Client/PipelineRegistration.cs new file mode 100644 index 0000000..1295f3b --- /dev/null +++ b/samples/Stampede.Http.Sample.Client/PipelineRegistration.cs @@ -0,0 +1,102 @@ +using Microsoft.Extensions.Http.Resilience; +using Polly; +using Stampede.Http.Caching; +using Stampede.Http.Extensions; +using Stampede.Http.Options; + +namespace Stampede.Http.Sample.Client; + +/// +/// Assembles the outbound pipeline. This is the only file in the client that knows +/// Stampede.Http exists. +/// +public static class PipelineRegistration +{ + /// Named the sample uses; also the options key for both option classes. + public const string ClientName = "origin"; + + /// + /// Registers and its handler pipeline: + /// + /// CachingMiddleware ← RFC 9111, entries shared across instances via Redis + /// └─ CoalescingHandler ← per-process request deduplication + /// └─ Polly ← retry with exponential backoff + per-attempt timeout + /// └─ SocketsHttpHandler + /// + /// When is the two Stampede.Http + /// handlers are omitted and everything else stays identical — that instance is the control group. + /// + /// The service collection. + /// Application configuration, used for hot-reloadable option overrides. + /// Eagerly bound sample options. + /// The same service collection, for chaining. + public static IServiceCollection AddOriginClient( + this IServiceCollection services, + IConfiguration configuration, + SampleOptions options) + { + PipelineOptions pipeline = options.Pipeline; + + IHttpClientBuilder builder = services.AddHttpClient(ClientName, http => + { + http.BaseAddress = new Uri(options.ApiBaseUrl); + + // Lets the origin attribute its load to a specific client instance, which is what + // makes the "with vs without Stampede.Http" dashboard panel possible. + http.DefaultRequestHeaders.Add("X-Client", options.Instance); + }); + + if (pipeline.Enabled) + { + _ = builder.AddStampedeHttp( + configureCaching: cache => + { + cache.DefaultTtl = pipeline.DefaultTtl; + + // Structural options: read once, when the store and key builder are created. + cache.NormalizeQueryParameters = pipeline.NormalizeQueryParameters; + cache.MaxCacheSize = pipeline.MaxCacheSize; + }, + configureCoalescing: coalescing => + { + coalescing.CoalescingTimeout = pipeline.CoalescingTimeout; + + // One URL, many tenants: keep each tenant's burst in its own coalescing group. + coalescing.CoalesceKeyHeaders = pipeline.CoalesceKeyHeaders; + + // Deliberately above CacheOptions.MaxBodySizeBytes — see the remarks on + // PipelineOptions.MaxResponseBodyBytes for why the two limits differ. + coalescing.MaxResponseBodyBytes = pipeline.MaxResponseBodyBytes; + }); + + // Runtime-tuneable overrides, layered on top of the lambdas above. `config/` is a + // mounted directory watched with reloadOnChange, so editing DefaultTtl there takes + // effect on the next request — no restart. Structural options are not reloadable. + _ = services.Configure(ClientName, configuration.GetSection("Stampede:Cache")); + _ = services.Configure(ClientName, configuration.GetSection("Stampede:Coalescing")); + + if (pipeline.UseRedis && !string.IsNullOrWhiteSpace(options.RedisConnection)) + { + _ = services.AddStackExchangeRedisCache(redis => redis.Configuration = options.RedisConnection); + _ = builder.UseDistributedCacheStore(); + } + } + + // Polly goes on last so it sits BELOW the coalescer: a retry storm is coalesced too. + // The baseline instance gets the identical resilience pipeline, so the only difference + // measured between instances is Stampede.Http itself. + _ = builder.AddResilienceHandler("origin-resilience", resilience => + { + _ = resilience.AddRetry(new HttpRetryStrategyOptions + { + MaxRetryAttempts = 2, + Delay = TimeSpan.FromMilliseconds(200), + BackoffType = DelayBackoffType.Exponential, + }); + + _ = resilience.AddTimeout(TimeSpan.FromSeconds(5)); + }); + + return services; + } +} diff --git a/samples/Stampede.Http.Sample.Client/Program.cs b/samples/Stampede.Http.Sample.Client/Program.cs index 1c7dc1e..e65b3bf 100644 --- a/samples/Stampede.Http.Sample.Client/Program.cs +++ b/samples/Stampede.Http.Sample.Client/Program.cs @@ -1,206 +1,91 @@ -using Microsoft.Extensions.DependencyInjection; -using Microsoft.Extensions.Http.Resilience; -using OpenTelemetry; using OpenTelemetry.Metrics; -using Polly; -using Stampede.Http.Extensions; +using OpenTelemetry.Resources; +using OpenTelemetry.Trace; using Stampede.Http.Metrics; -using System.Diagnostics; -using System.Diagnostics.Metrics; +using Stampede.Http.Sample.Client; +using Stampede.Http.Sample.Client.Endpoints; +using Stampede.Http.Sample.Client.Workload; // --------------------------------------------------------------------------- -// Sample client — a realistic Stampede.Http pipeline: +// Sample client — an ordinary ASP.NET Core service that happens to call another +// service. The Stampede.Http pipeline lives entirely in PipelineRegistration; +// nothing else in this app knows it exists. // -// CachingMiddleware (RFC 9111, entries shared across instances via Redis) -// └─ CoalescingHandler (per-process request deduplication) -// └─ Polly (retry with exponential backoff + per-attempt timeout) +// CachingMiddleware ← RFC 9111, entries shared across instances via Redis +// └─ CoalescingHandler ← per-process request deduplication +// └─ Polly ← retry with exponential backoff + per-attempt timeout // └─ SocketsHttpHandler → the sample API // -// Run two replicas (docker compose does) and watch the Redis-backed cache be -// shared while coalescing stays per-process. +// The same image runs three ways in docker compose: two Stampede.Http instances +// (client-a, client-b) and one control instance with the handlers removed +// (client-baseline), so the dashboards can show the difference as a number. // --------------------------------------------------------------------------- -string instance = Environment.GetEnvironmentVariable("INSTANCE") ?? Environment.MachineName; -string apiBase = Environment.GetEnvironmentVariable("API__BASEURL") ?? "http://localhost:5080"; -string redisConn = Environment.GetEnvironmentVariable("REDIS__CONNECTION") ?? "localhost:6379"; +var builder = WebApplication.CreateBuilder(args); -Log($"starting — api: {apiBase} redis: {redisConn}", ConsoleColor.Cyan); +// A mounted directory rather than a mounted file: editing a bind-mounted file on the host +// usually replaces the inode, and the container would keep reading the old one. +builder.Configuration.AddJsonFile("config/client.json", optional: true, reloadOnChange: true); -// -- In-process metrics: aggregate every stampede_http.* instrument ---------- -var metrics = new Dictionary(StringComparer.Ordinal); -using var listener = new MeterListener(); -listener.InstrumentPublished = (instrument, l) => +builder.Logging.ClearProviders(); +builder.Logging.AddSimpleConsole(o => { - if (instrument.Meter.Name == StampedeHttpMetrics.MeterName) - l.EnableMeasurementEvents(instrument); -}; -listener.SetMeasurementEventCallback((instrument, measurement, _, _) => -{ - lock (metrics) - { - metrics.TryGetValue(instrument.Name, out long prev); - metrics[instrument.Name] = prev + measurement; - } + o.SingleLine = true; + o.TimestampFormat = "HH:mm:ss "; }); -listener.Start(); - -// -- Prometheus scrape endpoint: the same "Stampede.Http" meter, exported via -// OpenTelemetry so Prometheus/Grafana can graph it in real time. ------------- -int metricsPort = int.TryParse(Environment.GetEnvironmentVariable("METRICS__PORT"), out int parsedPort) ? parsedPort : 9464; -// Must be a real resolvable hostname/IP — the Prometheus exporter's Host option is validated -// via UriBuilder (rejects "+"/"*") and .NET's HttpListener itself refuses a literal "0.0.0.0". -// docker-compose sets METRICS__HOST to each container's own `hostname` (e.g. "client-a"), -// which Docker's embedded DNS also resolves to that same container from other containers — -// so this is reachable AND unprivileged. Local (non-container) runs keep "localhost" below. -string metricsHost = Environment.GetEnvironmentVariable("METRICS__HOST") ?? "localhost"; -using MeterProvider meterProvider = Sdk.CreateMeterProviderBuilder() - .AddMeter(StampedeHttpMetrics.MeterName) - .AddPrometheusHttpListener(o => - { - o.Host = metricsHost; - o.Port = metricsPort; - }) - .Build(); -Log($"Prometheus metrics exposed — host: {metricsHost} port: {metricsPort} path: /metrics", ConsoleColor.Cyan); - -// -- The pipeline ------------------------------------------------------------ -var services = new ServiceCollection(); -services.AddStackExchangeRedisCache(o => o.Configuration = redisConn); - -services.AddHttpClient("api", c => c.BaseAddress = new Uri(apiBase)) - .AddStampedeHttp( - configureCaching: o => o.DefaultTtl = TimeSpan.FromSeconds(5), - configureCoalescing: o => o.CoalescingTimeout = TimeSpan.FromSeconds(10)) - .UseDistributedCacheStore() - .AddResilienceHandler("sample-resilience", b => +builder.Services.AddOptions() + .Bind(builder.Configuration.GetSection(SampleOptions.SectionName)) + .ValidateDataAnnotations() + .ValidateOnStart(); + +// Also needed eagerly: the pipeline shape is decided at registration time. +SampleOptions sample = builder.Configuration.GetSection(SampleOptions.SectionName).Get() ?? new SampleOptions(); + +builder.Services.AddOriginClient(builder.Configuration, sample); +builder.Services.AddSingleton(); +builder.Services.AddTransient(); +builder.Services.AddHostedService(); + +string? otlpEndpoint = builder.Configuration["OTEL_EXPORTER_OTLP_ENDPOINT"]; + +builder.Services.AddOpenTelemetry() + .ConfigureResource(r => r + .AddService(serviceName: "sample-client", serviceInstanceId: sample.Instance) + .AddAttributes([new KeyValuePair("stampede.enabled", sample.Pipeline.Enabled)])) + .WithMetrics(m => m + .AddMeter(StampedeHttpMetrics.MeterName) + .AddAspNetCoreInstrumentation() + .AddHttpClientInstrumentation() + .AddPrometheusExporter()) + .WithTracing(t => { - // Polly sits BELOW the coalescer: a retry storm is coalesced too. - b.AddRetry(new HttpRetryStrategyOptions - { - MaxRetryAttempts = 2, - Delay = TimeSpan.FromMilliseconds(200), - BackoffType = DelayBackoffType.Exponential, - }); - b.AddTimeout(TimeSpan.FromSeconds(5)); - }); - -await using var provider = services.BuildServiceProvider(); -var client = provider.GetRequiredService().CreateClient("api"); - -// -- Wait for the API -------------------------------------------------------- -for (int attempt = 1; ; attempt++) -{ - try - { - using var ping = await client.GetAsync("/stats"); - if (ping.IsSuccessStatusCode) break; - } - catch when (attempt < 30) - { - } - - if (attempt >= 30) - { - Log("API not reachable after 30 attempts — giving up.", ConsoleColor.Red); - return 1; - } - - await Task.Delay(1000); -} - -Log("API is up.", ConsoleColor.Cyan); - -// -- Phase 1: the stampede --------------------------------------------------- -Log("PHASE 1 — stampede: 10 concurrent GET /slow (origin takes ~2 s each)", ConsoleColor.Yellow); -var sw = Stopwatch.StartNew(); -var burst = Enumerable.Range(0, 10).Select(_ => client.GetAsync("/slow")).ToArray(); -var burstResponses = await Task.WhenAll(burst); -sw.Stop(); -Log($" 10 callers finished in {sw.ElapsedMilliseconds} ms — " + - $"{burstResponses.Count(r => r.IsSuccessStatusCode)}/10 OK. " + - "Coalescing collapsed this instance's burst into (at most) one origin call.", ConsoleColor.Green); - -// -- Phase 2: steady-state loop ---------------------------------------------- -Log("PHASE 2 — steady state: /catalog + /feed + /flaky every 2 s. " + - "Try `docker compose stop api` and watch stale-if-error shield the 200s.", ConsoleColor.Yellow); + t.AddAspNetCoreInstrumentation(o => + o.Filter = ctx => !ctx.Request.Path.StartsWithSegments("/metrics") + && !ctx.Request.Path.StartsWithSegments("/healthz")); -int iteration = 0; -while (true) -{ - iteration++; - - await Probe("/catalog"); - await Probe("/feed"); - await Probe("/flaky"); + // The outgoing spans are the interesting ones: a coalesced burst produces a single + // child span for N inbound requests, which is the whole story in one screenshot. + t.AddHttpClientInstrumentation(); - // Every 10th iteration: mutate the catalog. The 2xx POST makes - // CachingMiddleware evict the shared /catalog entry (RFC 9111 §4.4), - // so the next GET — from ANY instance — refetches and sees a new ETag. - if (iteration % 10 == 0) - { - try - { - using var post = await client.PostAsync("/catalog", content: null); - Log($" POST /catalog -> {(int)post.StatusCode} — cached entry invalidated for every instance", ConsoleColor.Magenta); - } - catch (Exception ex) + if (!string.IsNullOrWhiteSpace(otlpEndpoint)) { - Log($" POST /catalog failed: {ex.GetBaseException().Message}", ConsoleColor.Red); + t.AddOtlpExporter(); } - } - - if (iteration % 8 == 0) - { - PrintMetrics(); - } - - await Task.Delay(2000); -} - -async Task Probe(string path) -{ - var probeSw = Stopwatch.StartNew(); - try - { - using var response = await client!.GetAsync(path); - probeSw.Stop(); + }); - TimeSpan? age = response.Headers.Age; - string ageText = age is null ? "fresh from origin" : $"Age: {age.Value.TotalSeconds:F0}s"; - string body = await response.Content.ReadAsStringAsync(); - if (body.Length > 60) body = body[..60] + "…"; +var app = builder.Build(); - var color = response.IsSuccessStatusCode - ? (age is { TotalSeconds: > 10 } ? ConsoleColor.DarkYellow : ConsoleColor.Green) - : ConsoleColor.Red; - string staleHint = age is { TotalSeconds: > 10 } ? " [served beyond max-age: stale window or revalidated entry]" : string.Empty; +app.MapBffEndpoints(); +app.MapPrometheusScrapingEndpoint(); - Log($" GET {path,-9} -> {(int)response.StatusCode} {probeSw.ElapsedMilliseconds,5} ms ({ageText}){staleHint} {body}", color); - } - catch (Exception ex) - { - probeSw.Stop(); - Log($" GET {path,-9} -> EXCEPTION after {probeSw.ElapsedMilliseconds} ms: {ex.GetBaseException().Message}", ConsoleColor.Red); - } -} +app.Logger.LogInformation( + "Starting {Instance} — origin: {ApiBaseUrl}, Stampede.Http: {Enabled}, Redis: {Redis}", + sample.Instance, + sample.ApiBaseUrl, + sample.Pipeline.Enabled ? "enabled" : "DISABLED (control group)", + sample.Pipeline is { Enabled: true, UseRedis: true } && !string.IsNullOrWhiteSpace(sample.RedisConnection) + ? sample.RedisConnection + : "in-memory store"); -void PrintMetrics() -{ - lock (metrics) - { - if (metrics.Count == 0) return; - Log(" ── stampede_http metrics (this instance) ─────────────────", ConsoleColor.Cyan); - foreach (var (name, value) in metrics.OrderBy(kv => kv.Key)) - { - Log($" {name,-50} {value,6}", ConsoleColor.Cyan); - } - } -} - -void Log(string message, ConsoleColor color) -{ - Console.ForegroundColor = color; - Console.WriteLine($"[{instance}] {message}"); - Console.ResetColor(); -} +app.Run(); diff --git a/samples/Stampede.Http.Sample.Client/Properties/launchSettings.json b/samples/Stampede.Http.Sample.Client/Properties/launchSettings.json new file mode 100644 index 0000000..a73d2e0 --- /dev/null +++ b/samples/Stampede.Http.Sample.Client/Properties/launchSettings.json @@ -0,0 +1,35 @@ +{ + "$schema": "https://json.schemastore.org/launchsettings.json", + "profiles": { + "client": { + "commandName": "Project", + "launchBrowser": false, + "applicationUrl": "http://localhost:5081", + "environmentVariables": { + "ASPNETCORE_ENVIRONMENT": "Development", + "Sample__Instance": "local" + } + }, + "client (no redis)": { + "commandName": "Project", + "launchBrowser": false, + "applicationUrl": "http://localhost:5081", + "environmentVariables": { + "ASPNETCORE_ENVIRONMENT": "Development", + "Sample__Instance": "local", + "Sample__Pipeline__UseRedis": "false" + } + }, + "client (baseline, no Stampede.Http)": { + "commandName": "Project", + "launchBrowser": false, + "applicationUrl": "http://localhost:5082", + "environmentVariables": { + "ASPNETCORE_ENVIRONMENT": "Development", + "Sample__Instance": "local-baseline", + "Sample__Pipeline__Enabled": "false", + "Sample__Workload__FeatureTour": "false" + } + } + } +} diff --git a/samples/Stampede.Http.Sample.Client/SampleOptions.cs b/samples/Stampede.Http.Sample.Client/SampleOptions.cs new file mode 100644 index 0000000..b644077 --- /dev/null +++ b/samples/Stampede.Http.Sample.Client/SampleOptions.cs @@ -0,0 +1,120 @@ +using System.ComponentModel.DataAnnotations; + +namespace Stampede.Http.Sample.Client; + +/// +/// Everything about this instance that is worth changing without a rebuild. +/// Bound from appsettings.json plus environment variables, and validated at startup. +/// +public sealed class SampleOptions +{ + /// Configuration section this class binds to. + public const string SectionName = "Sample"; + + /// Instance name used in logs, metrics and the X-Client header sent to the origin. + [Required] + public string Instance { get; set; } = Environment.MachineName; + + /// Base address of the sample origin API. + [Required] + public string ApiBaseUrl { get; set; } = "http://localhost:5080"; + + /// Redis connection string used as the shared second-level cache. + public string? RedisConnection { get; set; } + + /// Pipeline configuration. + public PipelineOptions Pipeline { get; set; } = new(); + + /// Background workload configuration. + public WorkloadOptions Workload { get; set; } = new(); +} + +/// +/// How the outbound pipeline is assembled. Turning off produces the +/// control group: an identical app making identical calls with a bare . +/// +public sealed class PipelineOptions +{ + /// + /// Whether Stampede.Http is in the pipeline at all. yields the + /// baseline instance the dashboards compare against. + /// + public bool Enabled { get; set; } = true; + + /// Whether to use Redis (via IDistributedCache) instead of the in-memory store. + public bool UseRedis { get; set; } = true; + + /// Fallback freshness lifetime. Hot-reloadable — see config/client.json. + public TimeSpan DefaultTtl { get; set; } = TimeSpan.FromSeconds(5); + + /// How long a coalesced waiter waits before falling back to its own request. + public TimeSpan CoalescingTimeout { get; set; } = TimeSpan.FromSeconds(10); + + /// + /// Request headers folded into the coalescing key, so concurrent bursts for different + /// tenants are deduplicated independently rather than collapsing into one response. + /// + /// + /// Left empty on purpose: the configuration binder concatenates bound array elements onto + /// a non-empty default, so a default here would duplicate every value in appsettings.json. + /// + public string[] CoalesceKeyHeaders { get; set; } = []; + + /// + /// Largest body the coalescer will buffer while sharing one response among waiters. + /// + /// + /// This is a different ceiling from , + /// and they fail differently: exceeding the cache's limit silently declines to store the + /// response, while exceeding the coalescer's throws for every waiter. It is set above the + /// cache limit here so /bulk can demonstrate the first without tripping the second. + /// + public long MaxResponseBodyBytes { get; set; } = 4 * 1024 * 1024; + + /// Sort query parameters before building the cache key. + public bool NormalizeQueryParameters { get; set; } = true; + + /// Total byte ceiling for the in-memory store; ignored when is set. + public long? MaxCacheSize { get; set; } +} + +/// Which parts of the scripted workload this instance runs. +public sealed class WorkloadOptions +{ + /// Master switch — set to to leave the app purely request-driven (k6, curl). + public bool Enabled { get; set; } = true; + + /// Number of concurrent callers in the opening stampede. + [Range(1, 1000)] + public int StampedeSize { get; set; } = 10; + + /// + /// Whether to run the narrated feature tour. Enable it on exactly one instance: + /// it asserts against the origin's own counters, which every other caller perturbs. + /// + public bool FeatureTour { get; set; } + + /// Delay between steady-state iterations. + public TimeSpan Interval { get; set; } = TimeSpan.FromSeconds(2); + + /// Iterations between POST /catalog mutations. + [Range(1, 1000)] + public int MutateEvery { get; set; } = 10; + + /// Iterations between concurrent bursts on /slow. + /// + /// Without this the steady state issues one request at a time and there is, correctly, + /// nothing to deduplicate — the coalescing panels sit at zero and the library's headline + /// feature looks dead. Real inbound traffic is concurrent; this models that. + /// + /// The burst deliberately targets /slow, which is excluded from the origin-load + /// comparison, so adding it does not inflate the headline percentage. + /// + /// + [Range(1, 1000)] + public int BurstEvery { get; set; } = 5; + + /// Number of concurrent callers in each steady-state burst. + [Range(1, 1000)] + public int BurstSize { get; set; } = 8; +} diff --git a/samples/Stampede.Http.Sample.Client/Stampede.Http.Sample.Client.csproj b/samples/Stampede.Http.Sample.Client/Stampede.Http.Sample.Client.csproj index 29eeb15..6a99d77 100644 --- a/samples/Stampede.Http.Sample.Client/Stampede.Http.Sample.Client.csproj +++ b/samples/Stampede.Http.Sample.Client/Stampede.Http.Sample.Client.csproj @@ -1,10 +1,10 @@ - + - Exe net10.0 enable enable + Stampede.Http.Sample.Client NU5104 @@ -12,9 +12,12 @@ - + + + + - + diff --git a/samples/Stampede.Http.Sample.Client/Workload/FeatureTour.cs b/samples/Stampede.Http.Sample.Client/Workload/FeatureTour.cs new file mode 100644 index 0000000..7852eea --- /dev/null +++ b/samples/Stampede.Http.Sample.Client/Workload/FeatureTour.cs @@ -0,0 +1,360 @@ +using System.Net.Http.Headers; +using Stampede.Http.Caching; +using Stampede.Http.Coalescing; + +namespace Stampede.Http.Sample.Client.Workload; + +/// +/// A narrated walk through every Stampede.Http behaviour that a running system can +/// actually demonstrate, each step verified against the origin's own request counters +/// rather than against the client's belief about what happened. +/// +/// +/// Enable this on exactly one instance (Sample:Workload:FeatureTour). The assertions +/// are deltas on shared origin counters, so any other caller hitting the same endpoints at +/// the same time — a second instance, or the k6 load profile — will skew them. +/// +/// Typed client for the origin. +/// Logger used for the narration. +public sealed class FeatureTour(OriginClient client, ILogger logger) +{ + private int _checks; + private int _passed; + + /// Runs every scenario in order. + /// Cancellation token. + public async Task RunAsync(CancellationToken cancellationToken) + { + logger.LogInformation("PHASE 2 — feature tour: each step is verified against the origin's own counters"); + + (string Name, Func Run)[] scenarios = + [ + ("Vary: Accept-Language", VaryByLanguageAsync), + ("CoalesceKeyHeaders: X-Tenant-Id", TenantCoalescingAsync), + ("Client conditional pass-through", ConditionalPassThroughAsync), + ("CacheRequestPolicy.ForceRevalidate", ForceRevalidateAsync), + ("CacheRequestPolicy.BypassCache", BypassCacheAsync), + ("CacheRequestPolicy.NoStore", NoStoreAsync), + ("CoalescingRequestPolicy.BypassCoalescing", BypassCoalescingAsync), + ("Cache-Control: only-if-cached", OnlyIfCachedAsync), + ("Cache-Control: immutable", ImmutableAsync), + ("CacheOptions.MaxBodySizeBytes", BodySizeCeilingAsync), + ("CacheOptions.NormalizeQueryParameters", QueryNormalizationAsync), + ("Last-Modified revalidation (RevalidationGraceSeconds)", LastModifiedRevalidationAsync), + ]; + + foreach ((string name, Func run) in scenarios) + { + try + { + await run(cancellationToken).ConfigureAwait(false); + } + catch (Exception ex) when (ex is not OperationCanceledException) + { + // One broken scenario must not take the process down — the remaining + // steps still have something to say. + _checks++; + logger.LogError(ex, " [!!] {Scenario}: threw {Exception}", name, ex.GetBaseException().Message); + } + } + + logger.LogInformation("PHASE 2 — feature tour complete: {Passed}/{Total} checks behaved as documented", _passed, _checks); + } + + // -- Scenarios ----------------------------------------------------------- + + // Vary: Accept-Language — one URL, one entry per language. Before v2.2.0 these + // representations overwrote each other and content-negotiated endpoints never hit. + private async Task VaryByLanguageAsync(CancellationToken ct) + { + string[] languages = ["en-GB", "es-ES", "fr-FR"]; + + int cold = await OriginDeltaAsync("GET /greetings", async () => + { + foreach (string language in languages) + { + _ = await GetAsync("/greetings", language, ct).ConfigureAwait(false); + } + }, ct).ConfigureAwait(false); + + int warm = await OriginDeltaAsync("GET /greetings", async () => + { + foreach (string language in languages) + { + _ = await GetAsync("/greetings", language, ct).ConfigureAwait(false); + } + }, ct).ConfigureAwait(false); + + Check("Vary: Accept-Language", + cold == 3 && warm == 0, + $"3 languages cold → {cold} origin calls; the same 3 again → {warm}. Each variant has its own entry."); + + Task GetAsync(string path, string language, CancellationToken token) => + client.GetAsync(path, r => r.Headers.AcceptLanguage.Add(new StringWithQualityHeaderValue(language)), token); + } + + // Vary + CoalesceKeyHeaders — the multi-tenant case. Ten concurrent callers across two + // tenants must produce exactly two origin calls: not ten (no dedup) and not one + // (tenant-a receiving tenant-b's data). + private async Task TenantCoalescingAsync(CancellationToken ct) + { + const string TenantA = "acme"; + const string TenantB = "globex"; + + int calls = await OriginDeltaAsync([$"GET /tenants/data [{TenantA}]", $"GET /tenants/data [{TenantB}]"], async () => + { + IEnumerable> burst = + [ + .. Enumerable.Range(0, 5).Select(_ => Fetch(TenantA)), + .. Enumerable.Range(0, 5).Select(_ => Fetch(TenantB)), + ]; + + _ = await Task.WhenAll(burst).ConfigureAwait(false); + }, ct).ConfigureAwait(false); + + Check("CoalesceKeyHeaders: X-Tenant-Id", + calls == 2, + $"10 concurrent callers across 2 tenants → {calls} origin calls. Per-tenant coalescing, no cross-tenant bleed."); + + Task Fetch(string tenant) => + client.GetAsync("/tenants/data", r => r.Headers.Add("X-Tenant-Id", tenant), ct); + } + + // RFC 9111 §4.3.2 — a client-supplied If-None-Match that matches a fresh entry is + // answered with 304 by the cache itself; the origin is never consulted. + private async Task ConditionalPassThroughAsync(CancellationToken ct) + { + _ = await client.GetAsync("/ledger", cancellationToken: ct).ConfigureAwait(false); + string? etag = await client.GetETagAsync("/ledger", ct).ConfigureAwait(false); + + ProbeResult? conditional = null; + int calls = await OriginDeltaAsync("GET /ledger", async () => + { + conditional = await client.GetAsync( + "/ledger", + r => r.Headers.TryAddWithoutValidation("If-None-Match", etag), + ct).ConfigureAwait(false); + }, ct).ConfigureAwait(false); + + Check("Client conditional pass-through", + calls == 0 && conditional?.StatusCode == 304, + $"If-None-Match against a fresh entry → {conditional?.StatusCode}, {calls} origin calls."); + } + + // ForceRevalidate — behaves like a request `no-cache`: the entry is fresh, but the cache + // still asks the origin, which answers 304 and costs no body. + private async Task ForceRevalidateAsync(CancellationToken ct) + { + _ = await client.GetAsync("/ledger", cancellationToken: ct).ConfigureAwait(false); + + ProbeResult? forced = null; + int notModified = await OriginDeltaAsync("GET /ledger -> 304", async () => + { + forced = await client.GetAsync( + "/ledger", + r => r.Options.Set(CacheRequestPolicy.ForceRevalidate, true), + ct).ConfigureAwait(false); + }, ct).ConfigureAwait(false); + + Check("CacheRequestPolicy.ForceRevalidate", + notModified == 1 && forced?.StatusCode == 200, + $"Fresh entry revalidated anyway → origin answered {notModified} × 304, caller still got {forced?.StatusCode}."); + } + + // BypassCache — no lookup, no storage. The origin always sees a full request. + private async Task BypassCacheAsync(CancellationToken ct) + { + _ = await client.GetAsync("/ledger", cancellationToken: ct).ConfigureAwait(false); + + int calls = await OriginDeltaAsync("GET /ledger", async () => + { + _ = await client.GetAsync( + "/ledger", + r => r.Options.Set(CacheRequestPolicy.BypassCache, true), + ct).ConfigureAwait(false); + }, ct).ConfigureAwait(false); + + Check("CacheRequestPolicy.BypassCache", + calls == 1, + $"Fresh entry ignored entirely → {calls} origin call."); + } + + // NoStore — the response is served to the caller but never written to the cache, so the + // next plain request for the same URL is still a miss. + private async Task NoStoreAsync(CancellationToken ct) + { + string path = UniqueSearchPath("nostore"); + + int calls = await OriginDeltaAsync("GET /search", async () => + { + _ = await client.GetAsync(path, r => r.Options.Set(CacheRequestPolicy.NoStore, true), ct).ConfigureAwait(false); + _ = await client.GetAsync(path, cancellationToken: ct).ConfigureAwait(false); + }, ct).ConfigureAwait(false); + + Check("CacheRequestPolicy.NoStore", + calls == 2, + $"Fetch with NoStore then a plain fetch of the same URL → {calls} origin calls: nothing was stored."); + } + + // BypassCoalescing — the escape hatch. Same burst, opted out of deduplication. + private async Task BypassCoalescingAsync(CancellationToken ct) + { + string bypassed = UniqueSearchPath("bypass-coalescing"); + string coalesced = UniqueSearchPath("coalesced"); + + int withBypass = await OriginDeltaAsync("GET /search", async () => + { + _ = await Task.WhenAll(Enumerable.Range(0, 5).Select(_ => client.GetAsync( + bypassed, + r => r.Options.Set(CoalescingRequestPolicy.BypassCoalescing, true), + ct))).ConfigureAwait(false); + }, ct).ConfigureAwait(false); + + int withCoalescing = await OriginDeltaAsync("GET /search", async () => + { + _ = await Task.WhenAll(Enumerable.Range(0, 5).Select(_ => client.GetAsync(coalesced, cancellationToken: ct))) + .ConfigureAwait(false); + }, ct).ConfigureAwait(false); + + Check("CoalescingRequestPolicy.BypassCoalescing", + withBypass == 5 && withCoalescing == 1, + $"5 concurrent callers → {withBypass} origin calls when bypassing, {withCoalescing} when coalescing."); + } + + // RFC 9111 §5.2.1.7 — only-if-cached must never reach the network: with nothing cached + // the cache answers 504 rather than fetching. + private async Task OnlyIfCachedAsync(CancellationToken ct) + { + string path = UniqueSearchPath("only-if-cached"); + + ProbeResult? result = null; + int calls = await OriginDeltaAsync("GET /search", async () => + { + result = await client.GetAsync( + path, + r => r.Headers.CacheControl = new CacheControlHeaderValue { OnlyIfCached = true }, + ct).ConfigureAwait(false); + }, ct).ConfigureAwait(false); + + Check("Cache-Control: only-if-cached", + calls == 0 && result?.StatusCode == 504, + $"Nothing cached for that URL → {result?.StatusCode} Gateway Timeout, {calls} origin calls."); + } + + // RFC 8246 — a fresh immutable entry is not revalidated, even when the caller demands it. + private async Task ImmutableAsync(CancellationToken ct) + { + // A fresh id per run: these entries carry max-age=31536000 and live in Redis, so an + // asset cached by an earlier run of this tour would still be warm a restart later. + // (That is the feature working, but it makes "cold fetch" mean nothing here.) + string path = $"/assets/tour-{Guid.NewGuid():N}"; + + int cold = await OriginDeltaAsync("GET /assets/{id}", async () => + { + _ = await client.GetAsync(path, cancellationToken: ct).ConfigureAwait(false); + _ = await client.GetAsync(path, cancellationToken: ct).ConfigureAwait(false); + }, ct).ConfigureAwait(false); + + int forced = await OriginDeltaAsync("GET /assets/{id}", async () => + { + _ = await client.GetAsync(path, r => r.Options.Set(CacheRequestPolicy.ForceRevalidate, true), ct) + .ConfigureAwait(false); + }, ct).ConfigureAwait(false); + + Check("Cache-Control: immutable", + cold == 1 && forced == 0, + $"Two cold fetches → {cold} origin call; a forced revalidation on top → {forced}. Immutable entries skip revalidation."); + } + + // MaxBodySizeBytes — a response can be perfectly cacheable and still be declined for + // being too large. Every request for it goes to the origin. + private async Task BodySizeCeilingAsync(CancellationToken ct) + { + int calls = await OriginDeltaAsync("GET /bulk", async () => + { + _ = await client.GetAsync("/bulk", cancellationToken: ct).ConfigureAwait(false); + _ = await client.GetAsync("/bulk", cancellationToken: ct).ConfigureAwait(false); + }, ct).ConfigureAwait(false); + + Check("CacheOptions.MaxBodySizeBytes", + calls == 2, + $"A ~1.8 MB body with max-age=300, fetched twice → {calls} origin calls: too large to store (1 MB ceiling)."); + } + + // NormalizeQueryParameters — the same logical query with its parameters reordered must + // not split into two cache entries. + private async Task QueryNormalizationAsync(CancellationToken ct) + { + string tag = Guid.NewGuid().ToString("N")[..8]; + + int calls = await OriginDeltaAsync("GET /search", async () => + { + _ = await client.GetAsync($"/search?alpha=1&beta=2&tour={tag}", cancellationToken: ct).ConfigureAwait(false); + _ = await client.GetAsync($"/search?tour={tag}&beta=2&alpha=1", cancellationToken: ct).ConfigureAwait(false); + }, ct).ConfigureAwait(false); + + Check("CacheOptions.NormalizeQueryParameters", + calls == 1, + $"The same query with reordered parameters → {calls} origin call."); + } + + // Last-Modified + RevalidationGraceSeconds — the entry outlives its freshness so that + // expiry can be settled with If-Modified-Since instead of a full refetch. + private async Task LastModifiedRevalidationAsync(CancellationToken ct) + { + const string Path = "/docs/tour"; + + _ = await client.GetAsync(Path, cancellationToken: ct).ConfigureAwait(false); + + // max-age=5 on this endpoint; wait it out so the next request finds a stale entry. + await Task.Delay(TimeSpan.FromSeconds(6), ct).ConfigureAwait(false); + + ProbeResult? revalidated = null; + int notModified = await OriginDeltaAsync("GET /docs/{id} -> 304", async () => + { + revalidated = await client.GetAsync(Path, cancellationToken: ct).ConfigureAwait(false); + }, ct).ConfigureAwait(false); + + Check("Last-Modified revalidation (RevalidationGraceSeconds)", + notModified == 1 && revalidated?.StatusCode == 200, + $"Expired entry kept for revalidation → origin answered {notModified} × 304 to If-Modified-Since, caller got {revalidated?.StatusCode} with no body transferred."); + } + + // -- Plumbing ------------------------------------------------------------ + + private static string UniqueSearchPath(string scenario) => + $"/search?tour={scenario}-{Guid.NewGuid():N}"; + + private Task OriginDeltaAsync(string counter, Func action, CancellationToken ct) => + OriginDeltaAsync([counter], action, ct); + + /// + /// Runs and reports how much the given origin counters moved. + /// The origin is the source of truth here: the client cannot lie about traffic it never sent. + /// + private async Task OriginDeltaAsync(string[] counters, Func action, CancellationToken ct) + { + IReadOnlyDictionary before = await client.GetOriginCountersAsync(ct).ConfigureAwait(false); + await action().ConfigureAwait(false); + IReadOnlyDictionary after = await client.GetOriginCountersAsync(ct).ConfigureAwait(false); + + return counters.Sum(c => Value(after, c) - Value(before, c)); + + static int Value(IReadOnlyDictionary source, string key) => + source.TryGetValue(key, out int value) ? value : 0; + } + + private void Check(string scenario, bool asExpected, string detail) + { + _checks++; + if (asExpected) + { + _passed++; + logger.LogInformation(" [ok] {Scenario}: {Detail}", scenario, detail); + } + else + { + logger.LogWarning(" [??] {Scenario}: {Detail} (not the documented outcome — is another caller hitting these endpoints?)", scenario, detail); + } + } +} diff --git a/samples/Stampede.Http.Sample.Client/Workload/StampedeCounters.cs b/samples/Stampede.Http.Sample.Client/Workload/StampedeCounters.cs new file mode 100644 index 0000000..9559b75 --- /dev/null +++ b/samples/Stampede.Http.Sample.Client/Workload/StampedeCounters.cs @@ -0,0 +1,59 @@ +using System.Diagnostics.Metrics; +using Stampede.Http.Metrics; + +namespace Stampede.Http.Sample.Client.Workload; + +/// +/// Keeps a running total of every stampede_http.* instrument in this process, so the +/// container logs and GET /api/metrics can show the same numbers Grafana graphs — +/// handy when you are looking at a terminal rather than a dashboard. +/// +/// +/// This is deliberately separate from the OpenTelemetry pipeline: it demonstrates that the +/// meter is plain and can be consumed by anything. +/// +public sealed class StampedeCounters : IDisposable +{ + private readonly Dictionary _totals = new(StringComparer.Ordinal); + private readonly MeterListener _listener; + private readonly Lock _gate = new(); + + /// Starts listening to the Stampede.Http meter. + public StampedeCounters() + { + _listener = new MeterListener + { + InstrumentPublished = (instrument, listener) => + { + if (instrument.Meter.Name == StampedeHttpMetrics.MeterName) + { + listener.EnableMeasurementEvents(instrument); + } + }, + }; + + _listener.SetMeasurementEventCallback((instrument, measurement, _, _) => + { + lock (_gate) + { + _totals.TryGetValue(instrument.Name, out long previous); + _totals[instrument.Name] = previous + measurement; + } + }); + + _listener.Start(); + } + + /// Returns a snapshot of the accumulated instrument totals, ordered by name. + public IReadOnlyDictionary Snapshot() + { + lock (_gate) + { + return _totals.OrderBy(kv => kv.Key, StringComparer.Ordinal) + .ToDictionary(kv => kv.Key, kv => kv.Value, StringComparer.Ordinal); + } + } + + /// + public void Dispose() => _listener.Dispose(); +} diff --git a/samples/Stampede.Http.Sample.Client/Workload/WorkloadService.cs b/samples/Stampede.Http.Sample.Client/Workload/WorkloadService.cs new file mode 100644 index 0000000..7e086e1 --- /dev/null +++ b/samples/Stampede.Http.Sample.Client/Workload/WorkloadService.cs @@ -0,0 +1,240 @@ +using System.Diagnostics; +using Microsoft.Extensions.Options; + +namespace Stampede.Http.Sample.Client.Workload; + +/// +/// The scripted traffic that makes the sample tell a story on its own, without anyone +/// having to drive it: an opening stampede, the narrated feature tour, then a steady-state +/// loop that runs until the container is stopped. +/// +/// +/// For request-driven load instead, set Sample:Workload:Enabled=false and use the +/// k6 profile (docker compose --profile load up k6) or the endpoints in samples.http. +/// +/// Used to resolve the typed client per unit of work. +/// Sample options. +/// Local mirror of the Stampede.Http instruments. +/// Logger. +public sealed class WorkloadService( + IServiceScopeFactory scopeFactory, + IOptions options, + StampedeCounters counters, + ILogger logger) : BackgroundService +{ + private static readonly string[] SteadyStatePaths = ["/catalog", "/feed", "/flaky"]; + + private readonly SampleOptions _options = options.Value; + + /// + protected override async Task ExecuteAsync(CancellationToken stoppingToken) + { + WorkloadOptions workload = _options.Workload; + + if (!workload.Enabled) + { + logger.LogInformation("Scripted workload disabled — this instance only serves requests."); + return; + } + + try + { + if (!await WaitForOriginAsync(stoppingToken).ConfigureAwait(false)) + { + return; + } + + await RunStampedeAsync(workload.StampedeSize, stoppingToken).ConfigureAwait(false); + + if (workload.FeatureTour) + { + await WithClientAsync((_, tour, ct) => tour.RunAsync(ct), stoppingToken).ConfigureAwait(false); + } + + await RunSteadyStateAsync(workload, stoppingToken).ConfigureAwait(false); + } + catch (OperationCanceledException) when (stoppingToken.IsCancellationRequested) + { + // Normal shutdown: docker compose stop / Ctrl+C. + logger.LogInformation("Workload stopped."); + } + catch (Exception ex) + { + // An unhandled BackgroundService exception stops the host by default. The scripted + // workload is a demo aid, not the reason this service exists — keep serving requests. + logger.LogError(ex, "Workload failed; the HTTP surface stays up."); + } + } + + /// Polls the origin until it answers, so the sample survives any container start order. + private async Task WaitForOriginAsync(CancellationToken ct) + { + const int MaxAttempts = 30; + + for (int attempt = 1; attempt <= MaxAttempts; attempt++) + { + try + { + using IServiceScope scope = scopeFactory.CreateScope(); + OriginClient client = scope.ServiceProvider.GetRequiredService(); + using HttpResponseMessage response = await client.Http.GetAsync("/health", ct).ConfigureAwait(false); + + if (response.IsSuccessStatusCode) + { + logger.LogInformation("Origin is up at {BaseUrl}.", _options.ApiBaseUrl); + return true; + } + } + catch (Exception ex) when (ex is HttpRequestException or TaskCanceledException && !ct.IsCancellationRequested) + { + // Origin not listening yet. + } + + await Task.Delay(TimeSpan.FromSeconds(1), ct).ConfigureAwait(false); + } + + logger.LogError("Origin not reachable after {Attempts} attempts — giving up.", MaxAttempts); + return false; + } + + /// Phase 1 — the thundering herd: N concurrent cold-cache callers for a 2 s endpoint. + private async Task RunStampedeAsync(int size, CancellationToken ct) + { + logger.LogInformation( + "PHASE 1 — stampede: {Size} concurrent GET /slow (the origin takes ~2 s per call)", size); + + long start = Stopwatch.GetTimestamp(); + + ProbeResult[] results = await WithClientAsync( + (client, _, token) => Task.WhenAll( + Enumerable.Range(0, size).Select(_ => client.GetAsync("/slow", cancellationToken: token))), + ct).ConfigureAwait(false); + + TimeSpan elapsed = Stopwatch.GetElapsedTime(start); + + logger.LogInformation( + " {Size} callers finished in {Elapsed} ms — {Ok}/{Size} OK. {Verdict}", + size, + (long)elapsed.TotalMilliseconds, + results.Count(r => r.StatusCode == 200), + size, + _options.Pipeline.Enabled + ? "Coalescing collapsed this instance's burst into a single origin call." + : "No coalescing on this instance — every caller went to the origin."); + } + + /// Phase 3 — steady state: the loop that feeds the dashboards. + private async Task RunSteadyStateAsync(WorkloadOptions workload, CancellationToken ct) + { + logger.LogInformation( + "PHASE 3 — steady state: /catalog + /feed + /flaky every {Interval}s, plus a burst of " + + "{BurstSize} concurrent GET /slow every {BurstEvery} iterations. " + + "Try `docker compose stop api` and watch stale-if-error keep the 200s coming.", + workload.Interval.TotalSeconds, + workload.BurstSize, + workload.BurstEvery); + + for (int iteration = 1; !ct.IsCancellationRequested; iteration++) + { + int currentIteration = iteration; + + await WithClientAsync(async (client, _, token) => + { + foreach (string path in SteadyStatePaths) + { + logger.LogInformation(" {Probe}", (await Probe(client, path, token).ConfigureAwait(false)).Describe()); + } + + // Every Nth iteration: a burst of concurrent callers for one resource. The three + // probes above are sequential, so on their own they give the coalescer nothing to + // do — deduplication needs requests to overlap in time, which is what real inbound + // traffic does and a polling loop does not. + if (currentIteration % workload.BurstEvery == 0) + { + ProbeResult[] burst = await Task.WhenAll( + Enumerable.Range(0, workload.BurstSize) + .Select(_ => Probe(client, "/slow", token))).ConfigureAwait(false); + + logger.LogInformation( + " burst: {Size} concurrent GET /slow -> {Ok} OK in {Elapsed} ms (slowest caller)", + workload.BurstSize, + burst.Count(r => r.StatusCode == 200), + burst.Max(r => r.ElapsedMs)); + } + + // Every Nth iteration: mutate the catalog. The 2xx POST makes CachingMiddleware + // evict the shared /catalog entry (RFC 9111 §4.4), so the next GET — from ANY + // instance — refetches and sees a new ETag. + if (currentIteration % workload.MutateEvery == 0) + { + int status = await client.PostAsync("/catalog", token).ConfigureAwait(false); + logger.LogInformation( + " POST /catalog -> {Status} — cached entry invalidated for every instance", status); + } + + return 0; + }, ct).ConfigureAwait(false); + + if (iteration % 8 == 0) + { + LogCounters(); + } + + await Task.Delay(workload.Interval, ct).ConfigureAwait(false); + } + } + + private static async Task Probe(OriginClient client, string path, CancellationToken ct) + { + try + { + return await client.GetAsync(path, cancellationToken: ct).ConfigureAwait(false); + } + catch (Exception ex) when (ex is not OperationCanceledException) + { + return new ProbeResult(path, 0, 0, null, $"EXCEPTION: {ex.GetBaseException().Message}"); + } + } + + private void LogCounters() + { + IReadOnlyDictionary snapshot = counters.Snapshot(); + if (snapshot.Count == 0) + { + return; + } + + logger.LogInformation(" ── stampede_http instruments (this instance) ──"); + foreach ((string name, long value) in snapshot) + { + logger.LogInformation(" {Name,-52} {Value,6}", name, value); + } + } + + /// + /// Resolves the typed client from a fresh scope for each unit of work. A + /// is a singleton, and holding a typed client for the + /// lifetime of the process would pin one HttpMessageHandler past its rotation window. + /// + private async Task WithClientAsync( + Func> work, + CancellationToken ct) + { + using IServiceScope scope = scopeFactory.CreateScope(); + return await work( + scope.ServiceProvider.GetRequiredService(), + scope.ServiceProvider.GetRequiredService(), + ct).ConfigureAwait(false); + } + + private async Task WithClientAsync( + Func work, + CancellationToken ct) + { + _ = await WithClientAsync(async (client, tour, token) => + { + await work(client, tour, token).ConfigureAwait(false); + return 0; + }, ct).ConfigureAwait(false); + } +} diff --git a/samples/Stampede.Http.Sample.Client/appsettings.json b/samples/Stampede.Http.Sample.Client/appsettings.json new file mode 100644 index 0000000..d9ff9cf --- /dev/null +++ b/samples/Stampede.Http.Sample.Client/appsettings.json @@ -0,0 +1,38 @@ +{ + "Logging": { + "LogLevel": { + "Default": "Information", + "Microsoft.AspNetCore": "Warning", + "Microsoft.Hosting.Lifetime": "Information", + "System.Net.Http.HttpClient": "Warning", + "Polly": "Warning" + } + }, + "AllowedHosts": "*", + + "Sample": { + "Instance": "local", + "ApiBaseUrl": "http://localhost:5080", + "RedisConnection": "localhost:6379", + + "Pipeline": { + "Enabled": true, + "UseRedis": true, + "DefaultTtl": "00:00:05", + "CoalescingTimeout": "00:00:10", + "CoalesceKeyHeaders": [ "X-Tenant-Id" ], + "NormalizeQueryParameters": true, + "MaxResponseBodyBytes": 4194304 + }, + + "Workload": { + "Enabled": true, + "StampedeSize": 10, + "FeatureTour": true, + "Interval": "00:00:02", + "MutateEvery": 10, + "BurstEvery": 5, + "BurstSize": 8 + } + } +} diff --git a/samples/config/client.json b/samples/config/client.json new file mode 100644 index 0000000..c7d93ee --- /dev/null +++ b/samples/config/client.json @@ -0,0 +1,23 @@ +{ + // This file is bind-mounted into every client container and read with reloadOnChange. + // Edit it while the stack is running, then hit http://localhost:5081/api/config: + // the new values are live, with no restart and no redeploy. That is IOptionsMonitor + // working through Stampede.Http's named options, keyed by the HttpClient name. + // + // Runtime-tuneable: DefaultTtl, MaxBodySizeBytes, DefaultStaleIfErrorSeconds, + // DefaultStaleWhileRevalidateSeconds, Coalescing.Enabled, + // Coalescing.CoalescingTimeout, Coalescing.MaxResponseBodyBytes + // Read once at startup (structural): MaxCacheSize, NormalizeQueryParameters, + // RevalidationGraceSeconds + "Stampede": { + "Cache": { + "DefaultTtl": "00:00:05", + "DefaultStaleIfErrorSeconds": 0, + "DefaultStaleWhileRevalidateSeconds": 0 + }, + "Coalescing": { + "Enabled": true, + "CoalescingTimeout": "00:00:10" + } + } +} diff --git a/samples/docker-compose.yml b/samples/docker-compose.yml index 96dc2bf..016204a 100644 --- a/samples/docker-compose.yml +++ b/samples/docker-compose.yml @@ -1,59 +1,110 @@ name: stampede-http-sample +# --------------------------------------------------------------------------- +# Three clients run the identical image against the identical origin: +# +# client-a, client-b Stampede.Http enabled, sharing one Redis cache +# client-baseline the same app with the handlers removed — the control group +# +# Everything the dashboards claim is a comparison between those two groups, +# measured at the origin itself. +# --------------------------------------------------------------------------- + +x-client-base: &client-base + build: + context: .. + dockerfile: samples/Stampede.Http.Sample.Client/Dockerfile + volumes: + # A directory, not a single file: editing a bind-mounted file on the host replaces + # the inode and the container would keep reading the old one. + - ./config:/app/config:ro + healthcheck: + test: ["CMD", "curl", "-fsS", "http://localhost:8080/healthz"] + interval: 5s + timeout: 3s + retries: 10 + start_period: 10s + depends_on: + api: + condition: service_healthy + redis: + condition: service_healthy + services: redis: image: redis:7-alpine ports: - "6379:6379" + healthcheck: + test: ["CMD", "redis-cli", "ping"] + interval: 5s + timeout: 3s + retries: 10 api: build: context: .. dockerfile: samples/Stampede.Http.Sample.Api/Dockerfile + environment: + - OTEL_EXPORTER_OTLP_ENDPOINT=http://jaeger:4317 ports: - "5080:8080" + healthcheck: + test: ["CMD", "curl", "-fsS", "http://localhost:8080/health"] + interval: 5s + timeout: 3s + retries: 10 + start_period: 10s - # Two named instances (rather than `deploy.replicas`) so Prometheus has stable, - # individually addressable scrape targets and each instance's metrics port can be - # published to a predictable host port. - # - # METRICS__HOST is set to this container's own `hostname` (not a wildcard like "+"/"*"): - # the Prometheus HttpListener exporter's Host/Port options go through UriBuilder, which - # rejects wildcard syntax, and .NET's cross-platform HttpListener itself refuses a literal - # "0.0.0.0" too. Binding to the container's own resolvable hostname is what actually works - # unprivileged AND is reachable from other containers, since Docker's embedded DNS resolves - # that same name to this container's address on the compose network. + # Runs the narrated feature tour. Exactly one instance may do so: the tour verifies + # itself against the origin's shared counters, which any other caller would skew. client-a: - build: - context: .. - dockerfile: samples/Stampede.Http.Sample.Client/Dockerfile + <<: *client-base hostname: client-a environment: - - INSTANCE=client-a - - API__BASEURL=http://api:8080 - - REDIS__CONNECTION=redis:6379 - - METRICS__HOST=client-a + - Sample__Instance=client-a + - Sample__Workload__FeatureTour=true + - Sample__ApiBaseUrl=http://api:8080 + - Sample__RedisConnection=redis:6379 + - OTEL_EXPORTER_OTLP_ENDPOINT=http://jaeger:4317 + # inotify events do not cross a Docker bind mount on Windows or macOS, so the + # reloadOnChange watcher over ./config would never fire. Polling does work. + - DOTNET_USE_POLLING_FILE_WATCHER=true ports: - - "9464:9464" - depends_on: - - api - - redis + - "5081:8080" client-b: - build: - context: .. - dockerfile: samples/Stampede.Http.Sample.Client/Dockerfile + <<: *client-base hostname: client-b environment: - - INSTANCE=client-b - - API__BASEURL=http://api:8080 - - REDIS__CONNECTION=redis:6379 - - METRICS__HOST=client-b + - Sample__Instance=client-b + - Sample__Workload__FeatureTour=false + - Sample__ApiBaseUrl=http://api:8080 + - Sample__RedisConnection=redis:6379 + - OTEL_EXPORTER_OTLP_ENDPOINT=http://jaeger:4317 + # inotify events do not cross a Docker bind mount on Windows or macOS, so the + # reloadOnChange watcher over ./config would never fire. Polling does work. + - DOTNET_USE_POLLING_FILE_WATCHER=true ports: - - "9465:9464" - depends_on: - - api - - redis + - "5082:8080" + + # The control group: same code, same Polly pipeline, same workload — no Stampede.Http. + # Its origin-request rate is the "before" number every claim in the README is measured against. + client-baseline: + <<: *client-base + hostname: client-baseline + environment: + - Sample__Instance=client-baseline + - Sample__Pipeline__Enabled=false + - Sample__Workload__FeatureTour=false + - Sample__ApiBaseUrl=http://api:8080 + - Sample__RedisConnection=redis:6379 + - OTEL_EXPORTER_OTLP_ENDPOINT=http://jaeger:4317 + # inotify events do not cross a Docker bind mount on Windows or macOS, so the + # reloadOnChange watcher over ./config would never fire. Polling does work. + - DOTNET_USE_POLLING_FILE_WATCHER=true + ports: + - "5083:8080" prometheus: image: prom/prometheus:v3.13.1 @@ -64,6 +115,7 @@ services: depends_on: - client-a - client-b + - client-baseline grafana: image: grafana/grafana:13.1.1 @@ -80,3 +132,28 @@ services: - "3000:3000" depends_on: - prometheus + + # Traces show what counters cannot: one origin span serving a whole burst of callers. + jaeger: + image: jaegertracing/all-in-one:1.65.0 + environment: + - COLLECTOR_OTLP_ENABLED=true + ports: + - "16686:16686" + + # Opt-in load generator: `docker compose --profile load up k6`. + # Drives real concurrent HTTP traffic into the clients instead of the scripted workload. + k6: + image: grafana/k6:0.55.0 + profiles: ["load"] + volumes: + - ./load:/scripts:ro + environment: + - STAMPEDE_TARGET=http://client-a:8080 + - BASELINE_TARGET=http://client-baseline:8080 + command: ["run", "/scripts/stampede.js"] + depends_on: + client-a: + condition: service_healthy + client-baseline: + condition: service_healthy diff --git a/samples/grafana/dashboards/stampede-http.json b/samples/grafana/dashboards/stampede-http.json index c8b0c52..8c40376 100644 --- a/samples/grafana/dashboards/stampede-http.json +++ b/samples/grafana/dashboards/stampede-http.json @@ -1,156 +1,1025 @@ { "uid": "stampede-http-overview", "title": "Stampede.Http — Live Metrics", - "tags": ["stampede-http"], + "tags": [ + "stampede-http", + "sample" + ], "timezone": "browser", "schemaVersion": 39, - "version": 1, "refresh": "5s", - "time": { "from": "now-15m", "to": "now" }, + "time": { + "from": "now-15m", + "to": "now" + }, "panels": [ { "id": 1, - "type": "stat", - "title": "Cache hits (total)", - "gridPos": { "x": 0, "y": 0, "w": 6, "h": 4 }, - "datasource": { "type": "prometheus", "uid": "stampede-http-prometheus" }, - "targets": [ - { "expr": "sum(stampede_http_cache_hits_requests_total)", "legendFormat": "hits", "refId": "A" } - ], - "fieldConfig": { "defaults": { "color": { "mode": "thresholds" }, "thresholds": { "steps": [{ "color": "green" }] } }, "overrides": [] }, - "options": { "reduceOptions": { "calcs": ["lastNotNull"] } } + "type": "row", + "title": "Origin load — Stampede.Http vs the control group", + "gridPos": { + "x": 0, + "y": 0, + "w": 24, + "h": 1 + }, + "collapsed": false, + "panels": [] }, { "id": 2, "type": "stat", - "title": "Cache misses (total)", - "gridPos": { "x": 6, "y": 0, "w": 6, "h": 4 }, - "datasource": { "type": "prometheus", "uid": "stampede-http-prometheus" }, + "title": "Origin req/s caused by one Stampede.Http client", + "gridPos": { + "x": 0, + "y": 1, + "w": 6, + "h": 5 + }, + "datasource": { + "type": "prometheus", + "uid": "stampede-http-prometheus" + }, "targets": [ - { "expr": "sum(stampede_http_cache_misses_requests_total)", "legendFormat": "misses", "refId": "A" } + { + "expr": "sum(rate(sample_api_origin_requests_total{client=~\"client-a|client-b\", endpoint=~\"/catalog|/feed|/flaky\"}[5m])) / 2", + "legendFormat": "stampede", + "refId": "A", + "instant": true + } ], - "fieldConfig": { "defaults": { "color": { "mode": "thresholds" }, "thresholds": { "steps": [{ "color": "orange" }] } }, "overrides": [] }, - "options": { "reduceOptions": { "calcs": ["lastNotNull"] } } + "fieldConfig": { + "defaults": { + "color": { + "mode": "thresholds" + }, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": null + } + ] + }, + "unit": "reqps", + "decimals": 2 + }, + "overrides": [] + }, + "options": { + "reduceOptions": { + "calcs": [ + "lastNotNull" + ] + }, + "textMode": "auto" + }, + "description": "Rate of requests that actually reached the origin, averaged across client-a and client-b, counting only the endpoints every client exercises (/catalog, /feed, /flaky)." }, { "id": 3, "type": "stat", - "title": "Cache hit ratio", - "gridPos": { "x": 12, "y": 0, "w": 6, "h": 4 }, - "datasource": { "type": "prometheus", "uid": "stampede-http-prometheus" }, + "title": "Origin req/s caused by the control client", + "gridPos": { + "x": 6, + "y": 1, + "w": 6, + "h": 5 + }, + "datasource": { + "type": "prometheus", + "uid": "stampede-http-prometheus" + }, "targets": [ { - "expr": "100 * sum(stampede_http_cache_hits_requests_total) / (sum(stampede_http_cache_hits_requests_total) + sum(stampede_http_cache_misses_requests_total))", - "legendFormat": "hit ratio", - "refId": "A" + "expr": "sum(rate(sample_api_origin_requests_total{client=\"client-baseline\", endpoint=~\"/catalog|/feed|/flaky\"}[5m]))", + "legendFormat": "baseline", + "refId": "A", + "instant": true } ], "fieldConfig": { "defaults": { - "unit": "percent", - "decimals": 1, - "min": 0, - "max": 100, - "color": { "mode": "thresholds" }, - "thresholds": { "steps": [{ "color": "red", "value": 0 }, { "color": "yellow", "value": 40 }, { "color": "green", "value": 70 }] } + "color": { + "mode": "thresholds" + }, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": null + } + ] + }, + "unit": "reqps", + "decimals": 2 }, "overrides": [] }, - "options": { "reduceOptions": { "calcs": ["lastNotNull"] } } + "options": { + "reduceOptions": { + "calcs": [ + "lastNotNull" + ] + }, + "textMode": "auto" + }, + "description": "Identical app, identical workload, identical Polly pipeline — without Stampede.Http." }, { "id": 4, "type": "stat", - "title": "Deduplicated by coalescing (total)", - "description": "Requests that reused an in-flight response instead of triggering their own origin call — the stampede protection at work.", - "gridPos": { "x": 18, "y": 0, "w": 6, "h": 4 }, - "datasource": { "type": "prometheus", "uid": "stampede-http-prometheus" }, + "title": "Origin load avoided (all traffic)", + "gridPos": { + "x": 12, + "y": 1, + "w": 6, + "h": 5 + }, + "datasource": { + "type": "prometheus", + "uid": "stampede-http-prometheus" + }, "targets": [ - { "expr": "sum(stampede_http_coalescing_deduplicated_requests_total)", "legendFormat": "deduplicated", "refId": "A" } + { + "expr": "100 * (1 - (sum(rate(sample_api_origin_requests_total{client=~\"client-a|client-b\", endpoint=~\"/catalog|/feed|/flaky\"}[5m])) / 2) / (sum(rate(sample_api_origin_requests_total{client=\"client-baseline\", endpoint=~\"/catalog|/feed|/flaky\"}[5m]))))", + "legendFormat": "avoided", + "refId": "A", + "instant": true + } ], - "fieldConfig": { "defaults": { "color": { "mode": "thresholds" }, "thresholds": { "steps": [{ "color": "blue" }] } }, "overrides": [] }, - "options": { "reduceOptions": { "calcs": ["lastNotNull"] } } + "fieldConfig": { + "defaults": { + "color": { + "mode": "thresholds" + }, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "red", + "value": null + }, + { + "color": "orange", + "value": 25 + }, + { + "color": "green", + "value": 50 + } + ] + }, + "unit": "percent", + "decimals": 1 + }, + "overrides": [] + }, + "options": { + "reduceOptions": { + "calcs": [ + "lastNotNull" + ] + }, + "textMode": "auto" + }, + "description": "How much of the control group's origin traffic Stampede.Http absorbed. Give the stack a minute to settle before reading this — and read it with the panel to the right. Expect roughly 40-55%, bounded by (1 - poll interval / max-age): this sample runs 5-10 s TTLs against a 2 s poll so expiry is visible in a short demo. Almost half of what remains is /flaky returning 503, which no cache can avoid — stale-if-error rescues the caller after the origin has failed, so the request still goes out. This figure also ignores the two biggest wins: the stampede (20 concurrent callers -> 1 origin call) and latency. Finally, it is normalised per client: because client-a and client-b share one Redis cache they also share the refresh work, so together they cost the origin LESS on /catalog and /feed than the single uncached control client does." }, { "id": 5, - "type": "timeseries", - "title": "Cache hits vs misses (rate, per instance)", - "description": "Both client-a and client-b share the same Redis-backed cache — a hit on one instance often reflects an entry stored by the other.", - "gridPos": { "x": 0, "y": 4, "w": 12, "h": 8 }, - "datasource": { "type": "prometheus", "uid": "stampede-http-prometheus" }, + "type": "stat", + "title": "Origin load avoided (healthy traffic)", + "gridPos": { + "x": 18, + "y": 1, + "w": 6, + "h": 5 + }, + "datasource": { + "type": "prometheus", + "uid": "stampede-http-prometheus" + }, "targets": [ - { "expr": "sum by (instance) (rate(stampede_http_cache_hits_requests_total[1m]))", "legendFormat": "{{instance}} hits/s", "refId": "A" }, - { "expr": "sum by (instance) (rate(stampede_http_cache_misses_requests_total[1m]))", "legendFormat": "{{instance}} misses/s", "refId": "B" } + { + "expr": "100 * (1 - (sum(rate(sample_api_origin_requests_total{client=~\"client-a|client-b\", endpoint=~\"/catalog|/feed\"}[5m])) / 2) / (sum(rate(sample_api_origin_requests_total{client=\"client-baseline\", endpoint=~\"/catalog|/feed\"}[5m]))))", + "legendFormat": "avoided", + "refId": "A", + "instant": true + } ], - "fieldConfig": { "defaults": { "unit": "reqps", "custom": { "drawStyle": "line", "fillOpacity": 10 } }, "overrides": [] }, - "options": { "legend": { "displayMode": "list", "placement": "bottom" }, "tooltip": { "mode": "multi" } } + "fieldConfig": { + "defaults": { + "color": { + "mode": "thresholds" + }, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "red", + "value": null + }, + { + "color": "orange", + "value": 25 + }, + { + "color": "green", + "value": 50 + } + ] + }, + "unit": "percent", + "decimals": 1 + }, + "overrides": [] + }, + "options": { + "reduceOptions": { + "calcs": [ + "lastNotNull" + ] + }, + "textMode": "auto" + }, + "description": "The same measurement with /flaky excluded — identical in every other respect, so the pair isolates exactly one variable: whether the origin is failing. Expect roughly 55%. /flaky is not diluting the number to its left, it is actively penalising it: uncacheable 503s are the single largest source of origin traffic in this sample, and they scale with replica count while cache hits are shared. Neither figure includes /slow, where the concurrent burst lands and coalescing pushes the saving above 90%; add it to both selectors and this reads around 73%." }, { "id": 6, "type": "timeseries", - "title": "Coalescing: deduplicated requests (rate, per instance)", - "description": "Coalescing is per-process: each instance's burst is deduplicated independently, it is NOT shared across instances like the cache is.", - "gridPos": { "x": 12, "y": 4, "w": 12, "h": 8 }, - "datasource": { "type": "prometheus", "uid": "stampede-http-prometheus" }, + "title": "Origin requests/s by client", + "gridPos": { + "x": 0, + "y": 6, + "w": 8, + "h": 8 + }, + "datasource": { + "type": "prometheus", + "uid": "stampede-http-prometheus" + }, "targets": [ - { "expr": "sum by (instance) (rate(stampede_http_coalescing_deduplicated_requests_total[1m]))", "legendFormat": "{{instance}}", "refId": "A" } + { + "expr": "sum by (client) (rate(sample_api_origin_requests_total[1m]))", + "legendFormat": "{{client}}", + "refId": "A" + } ], - "fieldConfig": { "defaults": { "unit": "reqps", "custom": { "drawStyle": "line", "fillOpacity": 10 } }, "overrides": [] }, - "options": { "legend": { "displayMode": "list", "placement": "bottom" }, "tooltip": { "mode": "multi" } } + "fieldConfig": { + "defaults": { + "unit": "reqps", + "custom": { + "drawStyle": "line", + "fillOpacity": 10, + "lineWidth": 2, + "stacking": { + "mode": "none", + "group": "A" + } + } + }, + "overrides": [] + }, + "options": { + "legend": { + "displayMode": "list", + "placement": "bottom" + }, + "tooltip": { + "mode": "multi", + "sort": "desc" + } + }, + "description": "Measured at the origin, attributed via the X-Client header. The baseline line sitting above the others is the entire value proposition of the library." }, { "id": 7, "type": "timeseries", - "title": "Stale responses served (rate, per instance)", - "description": "stale-if-error (RFC 5861 §4) shields callers during origin outages; stale-while-revalidate (§3) serves instantly and refreshes in the background.", - "gridPos": { "x": 0, "y": 12, "w": 12, "h": 8 }, - "datasource": { "type": "prometheus", "uid": "stampede-http-prometheus" }, + "title": "Origin requests/s by endpoint", + "gridPos": { + "x": 8, + "y": 6, + "w": 8, + "h": 8 + }, + "datasource": { + "type": "prometheus", + "uid": "stampede-http-prometheus" + }, "targets": [ - { "expr": "sum by (instance) (rate(stampede_http_cache_stale_errors_served_requests_total[1m]))", "legendFormat": "{{instance}} stale-if-error/s", "refId": "A" }, - { "expr": "sum by (instance) (rate(stampede_http_cache_stale_while_revalidate_served_requests_total[1m]))", "legendFormat": "{{instance}} stale-while-revalidate/s", "refId": "B" } + { + "expr": "sum by (endpoint) (rate(sample_api_origin_requests_total[1m]))", + "legendFormat": "{{endpoint}}", + "refId": "A" + } ], - "fieldConfig": { "defaults": { "unit": "reqps", "custom": { "drawStyle": "line", "fillOpacity": 10 } }, "overrides": [] }, - "options": { "legend": { "displayMode": "list", "placement": "bottom" }, "tooltip": { "mode": "multi" } } + "fieldConfig": { + "defaults": { + "unit": "reqps", + "custom": { + "drawStyle": "line", + "fillOpacity": 10, + "lineWidth": 2, + "stacking": { + "mode": "normal", + "group": "A" + } + } + }, + "overrides": [] + }, + "options": { + "legend": { + "displayMode": "list", + "placement": "bottom" + }, + "tooltip": { + "mode": "multi", + "sort": "desc" + } + }, + "description": "Which resources still cost origin calls. /slow should be near-flat once its 30 s entry is warm; /flaky never settles, because its 503s are not cacheable." }, { "id": 8, "type": "timeseries", - "title": "Coalescing in-flight requests (current, per instance)", - "description": "Current number of origin calls in flight through the coalescer. Should return to 0 shortly after every burst.", - "gridPos": { "x": 12, "y": 12, "w": 12, "h": 8 }, - "datasource": { "type": "prometheus", "uid": "stampede-http-prometheus" }, + "title": "Origin load avoided by endpoint", + "gridPos": { + "x": 16, + "y": 6, + "w": 8, + "h": 8 + }, + "datasource": { + "type": "prometheus", + "uid": "stampede-http-prometheus" + }, "targets": [ - { "expr": "sum by (instance) (stampede_http_coalescing_inflight_requests)", "legendFormat": "{{instance}}", "refId": "A" } + { + "expr": "100 * (1 - (sum by (endpoint) (rate(sample_api_origin_requests_total{client=~\"client-a|client-b\", endpoint=~\"/catalog|/feed|/flaky\"}[5m])) / 2) / sum by (endpoint) (rate(sample_api_origin_requests_total{client=\"client-baseline\", endpoint=~\"/catalog|/feed|/flaky\"}[5m])))", + "legendFormat": "{{endpoint}}", + "refId": "A" + } ], - "fieldConfig": { "defaults": { "unit": "short", "custom": { "drawStyle": "line", "fillOpacity": 10 } }, "overrides": [] }, - "options": { "legend": { "displayMode": "list", "placement": "bottom" }, "tooltip": { "mode": "multi" } } + "fieldConfig": { + "defaults": { + "unit": "percent", + "custom": { + "drawStyle": "line", + "fillOpacity": 0, + "lineWidth": 2, + "stacking": { + "mode": "none", + "group": "A" + } + } + }, + "overrides": [] + }, + "options": { + "legend": { + "displayMode": "list", + "placement": "bottom" + }, + "tooltip": { + "mode": "multi", + "sort": "desc" + } + }, + "description": "The headline percentage, split by resource — this is the panel that explains it. /catalog (max-age=10) and /feed (max-age=5) sit near their theoretical ceiling of 1 - 2 s poll / max-age. /flaky sits far below because a third of every minute it returns 503, and failures always reach the origin. Raise the TTLs in the origin's Cache-Control headers and every line here moves up." }, { "id": 9, + "type": "row", + "title": "Caching", + "gridPos": { + "x": 0, + "y": 14, + "w": 24, + "h": 1 + }, + "collapsed": false, + "panels": [] + }, + { + "id": 10, "type": "timeseries", - "title": "Revalidations & unsafe-method invalidations (per instance)", - "description": "Revalidations: conditional If-None-Match/If-Modified-Since requests on stale entries. Invalidations: cache entries evicted by a successful POST/PUT/DELETE/PATCH (RFC 9111 §4.4) — shared across instances via Redis.", - "gridPos": { "x": 0, "y": 20, "w": 18, "h": 8 }, - "datasource": { "type": "prometheus", "uid": "stampede-http-prometheus" }, + "title": "Cache hits vs misses (rate, per client)", + "gridPos": { + "x": 0, + "y": 15, + "w": 12, + "h": 8 + }, + "datasource": { + "type": "prometheus", + "uid": "stampede-http-prometheus" + }, "targets": [ - { "expr": "sum by (instance) (increase(stampede_http_cache_revalidations_requests_total[1m]))", "legendFormat": "{{instance}} revalidations", "refId": "A" }, - { "expr": "sum by (instance) (increase(stampede_http_cache_invalidations_entries_total[1m]))", "legendFormat": "{{instance}} invalidations", "refId": "B" } + { + "expr": "sum by (client) (rate(stampede_http_cache_hits_requests_total[1m]))", + "legendFormat": "{{client}} hits/s", + "refId": "A" + }, + { + "expr": "sum by (client) (rate(stampede_http_cache_misses_requests_total[1m]))", + "legendFormat": "{{client}} misses/s", + "refId": "B" + } ], - "fieldConfig": { "defaults": { "unit": "short", "custom": { "drawStyle": "bars", "fillOpacity": 40 } }, "overrides": [] }, - "options": { "legend": { "displayMode": "list", "placement": "bottom" }, "tooltip": { "mode": "multi" } } + "fieldConfig": { + "defaults": { + "unit": "reqps", + "custom": { + "drawStyle": "line", + "fillOpacity": 10, + "lineWidth": 2, + "stacking": { + "mode": "none", + "group": "A" + } + } + }, + "overrides": [] + }, + "options": { + "legend": { + "displayMode": "list", + "placement": "bottom" + }, + "tooltip": { + "mode": "multi", + "sort": "desc" + } + }, + "description": "client-a and client-b share one Redis-backed cache — a hit on one instance often reflects an entry the other stored." }, { - "id": 10, + "id": 11, + "type": "timeseries", + "title": "Stale responses served (rate, per client)", + "gridPos": { + "x": 12, + "y": 15, + "w": 12, + "h": 8 + }, + "datasource": { + "type": "prometheus", + "uid": "stampede-http-prometheus" + }, + "targets": [ + { + "expr": "sum by (client) (rate(stampede_http_cache_stale_errors_served_requests_total[1m]))", + "legendFormat": "{{client}} stale-if-error/s", + "refId": "A" + }, + { + "expr": "sum by (client) (rate(stampede_http_cache_stale_while_revalidate_served_requests_total[1m]))", + "legendFormat": "{{client}} stale-while-revalidate/s", + "refId": "B" + } + ], + "fieldConfig": { + "defaults": { + "unit": "reqps", + "custom": { + "drawStyle": "line", + "fillOpacity": 10, + "lineWidth": 2, + "stacking": { + "mode": "none", + "group": "A" + } + } + }, + "overrides": [] + }, + "options": { + "legend": { + "displayMode": "list", + "placement": "bottom" + }, + "tooltip": { + "mode": "multi", + "sort": "desc" + } + }, + "description": "stale-if-error spikes while /flaky is in its failure window, or while the api container is stopped." + }, + { + "id": 12, + "type": "timeseries", + "title": "Revalidations & unsafe-method invalidations (per client)", + "gridPos": { + "x": 0, + "y": 23, + "w": 12, + "h": 8 + }, + "datasource": { + "type": "prometheus", + "uid": "stampede-http-prometheus" + }, + "targets": [ + { + "expr": "sum by (client) (increase(stampede_http_cache_revalidations_requests_total[1m]))", + "legendFormat": "{{client}} revalidations", + "refId": "A" + }, + { + "expr": "sum by (client) (increase(stampede_http_cache_invalidations_entries_total[1m]))", + "legendFormat": "{{client}} invalidations", + "refId": "B" + } + ], + "fieldConfig": { + "defaults": { + "unit": "short", + "custom": { + "drawStyle": "line", + "fillOpacity": 10, + "lineWidth": 2, + "stacking": { + "mode": "none", + "group": "A" + } + } + }, + "overrides": [] + }, + "options": { + "legend": { + "displayMode": "list", + "placement": "bottom" + }, + "tooltip": { + "mode": "multi", + "sort": "desc" + } + }, + "description": "Revalidations are conditional requests that came back 304 — an expiry settled without transferring a body. Invalidations are entries evicted by POST /catalog." + }, + { + "id": 13, + "type": "stat", + "title": "Cache hit ratio", + "gridPos": { + "x": 12, + "y": 23, + "w": 3, + "h": 4 + }, + "datasource": { + "type": "prometheus", + "uid": "stampede-http-prometheus" + }, + "targets": [ + { + "expr": "100 * sum(stampede_http_cache_hits_requests_total) / (sum(stampede_http_cache_hits_requests_total) + sum(stampede_http_cache_misses_requests_total))", + "legendFormat": "hit ratio", + "refId": "A", + "instant": true + } + ], + "fieldConfig": { + "defaults": { + "color": { + "mode": "thresholds" + }, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": null + } + ] + }, + "unit": "percent", + "decimals": 1 + }, + "overrides": [] + }, + "options": { + "reduceOptions": { + "calcs": [ + "lastNotNull" + ] + }, + "textMode": "auto" + }, + "description": "Client-side view: the share of outbound requests the cache answered without touching the network. Related to the origin-load panels above, but not the same measurement — a revalidation that comes back 304 is a miss here and still saves the origin a body." + }, + { + "id": 14, + "type": "stat", + "title": "Cache hits (total)", + "gridPos": { + "x": 15, + "y": 23, + "w": 3, + "h": 4 + }, + "datasource": { + "type": "prometheus", + "uid": "stampede-http-prometheus" + }, + "targets": [ + { + "expr": "sum(stampede_http_cache_hits_requests_total)", + "legendFormat": "hits", + "refId": "A", + "instant": true + } + ], + "fieldConfig": { + "defaults": { + "color": { + "mode": "thresholds" + }, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": null + } + ] + }, + "unit": "short" + }, + "overrides": [] + }, + "options": { + "reduceOptions": { + "calcs": [ + "lastNotNull" + ] + }, + "textMode": "auto" + } + }, + { + "id": 15, + "type": "stat", + "title": "Cache misses (total)", + "gridPos": { + "x": 18, + "y": 23, + "w": 3, + "h": 4 + }, + "datasource": { + "type": "prometheus", + "uid": "stampede-http-prometheus" + }, + "targets": [ + { + "expr": "sum(stampede_http_cache_misses_requests_total)", + "legendFormat": "misses", + "refId": "A", + "instant": true + } + ], + "fieldConfig": { + "defaults": { + "color": { + "mode": "thresholds" + }, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": null + } + ] + }, + "unit": "short" + }, + "overrides": [] + }, + "options": { + "reduceOptions": { + "calcs": [ + "lastNotNull" + ] + }, + "textMode": "auto" + } + }, + { + "id": 16, + "type": "stat", + "title": "Revalidations (total)", + "gridPos": { + "x": 21, + "y": 23, + "w": 3, + "h": 4 + }, + "datasource": { + "type": "prometheus", + "uid": "stampede-http-prometheus" + }, + "targets": [ + { + "expr": "sum(stampede_http_cache_revalidations_requests_total)", + "legendFormat": "revalidations", + "refId": "A", + "instant": true + } + ], + "fieldConfig": { + "defaults": { + "color": { + "mode": "thresholds" + }, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": null + } + ] + }, + "unit": "short" + }, + "overrides": [] + }, + "options": { + "reduceOptions": { + "calcs": [ + "lastNotNull" + ] + }, + "textMode": "auto" + } + }, + { + "id": 17, + "type": "row", + "title": "Coalescing", + "gridPos": { + "x": 0, + "y": 31, + "w": 24, + "h": 1 + }, + "collapsed": false, + "panels": [] + }, + { + "id": 18, + "type": "timeseries", + "title": "Deduplicated requests (per 2 min, per client)", + "gridPos": { + "x": 0, + "y": 32, + "w": 12, + "h": 8 + }, + "datasource": { + "type": "prometheus", + "uid": "stampede-http-prometheus" + }, + "targets": [ + { + "expr": "sum by (client) (increase(stampede_http_coalescing_deduplicated_requests_total[2m]))", + "legendFormat": "{{client}}", + "refId": "A" + } + ], + "fieldConfig": { + "defaults": { + "unit": "short", + "custom": { + "drawStyle": "line", + "fillOpacity": 0, + "lineWidth": 2, + "stacking": { + "mode": "none", + "group": "A" + } + } + }, + "overrides": [] + }, + "options": { + "legend": { + "displayMode": "list", + "placement": "bottom" + }, + "tooltip": { + "mode": "multi", + "sort": "desc" + } + }, + "description": "Counted over a window rather than as a rate, because deduplication is inherently bursty: it can only happen when identical requests overlap in time AND the entry is not already fresh in the cache. The steady-state workload fires a burst of 8 concurrent GET /slow every few iterations, so this stays flat while the 30 s entry is warm and spikes to ~7 the moment it expires — that expiry is precisely the cache stampede the library is named for. Coalescing is per process, so client-a and client-b spike independently even though they share a cache." + }, + { + "id": 19, + "type": "timeseries", + "title": "In-flight coalesced origin calls (per client)", + "gridPos": { + "x": 12, + "y": 32, + "w": 12, + "h": 8 + }, + "datasource": { + "type": "prometheus", + "uid": "stampede-http-prometheus" + }, + "targets": [ + { + "expr": "sum by (client) (stampede_http_coalescing_inflight_requests)", + "legendFormat": "{{client}}", + "refId": "A" + } + ], + "fieldConfig": { + "defaults": { + "unit": "short", + "custom": { + "drawStyle": "line", + "fillOpacity": 10, + "lineWidth": 2, + "stacking": { + "mode": "none", + "group": "A" + } + } + }, + "overrides": [] + }, + "options": { + "legend": { + "displayMode": "list", + "placement": "bottom" + }, + "tooltip": { + "mode": "multi", + "sort": "desc" + } + }, + "description": "How many origin calls are open right now. A burst of N callers should keep this at 1. It is a gauge sampled every 5 s, so short bursts can fall between scrapes and leave it reading zero — the panel to the left is the reliable one." + }, + { + "id": 20, + "type": "stat", + "title": "Deduplicated (total)", + "gridPos": { + "x": 0, + "y": 40, + "w": 8, + "h": 4 + }, + "datasource": { + "type": "prometheus", + "uid": "stampede-http-prometheus" + }, + "targets": [ + { + "expr": "sum(stampede_http_coalescing_deduplicated_requests_total)", + "legendFormat": "deduplicated", + "refId": "A", + "instant": true + } + ], + "fieldConfig": { + "defaults": { + "color": { + "mode": "thresholds" + }, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": null + } + ] + }, + "unit": "short" + }, + "overrides": [] + }, + "options": { + "reduceOptions": { + "calcs": [ + "lastNotNull" + ] + }, + "textMode": "auto" + } + }, + { + "id": 21, "type": "stat", "title": "Coalescing timeouts (5m)", - "description": "Waiters that gave up on the shared in-flight response and fell back to an independent request. Zero is expected unless CoalescingTimeout is exceeded under heavy load.", - "gridPos": { "x": 18, "y": 20, "w": 6, "h": 8 }, - "datasource": { "type": "prometheus", "uid": "stampede-http-prometheus" }, + "gridPos": { + "x": 8, + "y": 40, + "w": 8, + "h": 4 + }, + "datasource": { + "type": "prometheus", + "uid": "stampede-http-prometheus" + }, "targets": [ - { "expr": "sum(increase(stampede_http_coalescing_timeouts_requests_total[5m]) or vector(0))", "legendFormat": "timeouts", "refId": "A" } + { + "expr": "sum(increase(stampede_http_coalescing_timeouts_requests_total[5m]) or vector(0))", + "legendFormat": "timeouts", + "refId": "A", + "instant": true + } ], - "fieldConfig": { "defaults": { "color": { "mode": "thresholds" }, "thresholds": { "steps": [{ "color": "green", "value": 0 }, { "color": "red", "value": 1 }] } }, "overrides": [] }, - "options": { "reduceOptions": { "calcs": ["lastNotNull"] } } + "fieldConfig": { + "defaults": { + "color": { + "mode": "thresholds" + }, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": null + }, + { + "color": "red", + "value": 1 + } + ] + }, + "unit": "short" + }, + "overrides": [] + }, + "options": { + "reduceOptions": { + "calcs": [ + "lastNotNull" + ] + }, + "textMode": "auto" + }, + "description": "Waiters that gave up on the winner and issued their own request. Should stay at zero unless the origin is slower than CoalescingTimeout." + }, + { + "id": 22, + "type": "stat", + "title": "Invalidations (total)", + "gridPos": { + "x": 16, + "y": 40, + "w": 8, + "h": 4 + }, + "datasource": { + "type": "prometheus", + "uid": "stampede-http-prometheus" + }, + "targets": [ + { + "expr": "sum(stampede_http_cache_invalidations_entries_total)", + "legendFormat": "invalidations", + "refId": "A", + "instant": true + } + ], + "fieldConfig": { + "defaults": { + "color": { + "mode": "thresholds" + }, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": null + } + ] + }, + "unit": "short" + }, + "overrides": [] + }, + "options": { + "reduceOptions": { + "calcs": [ + "lastNotNull" + ] + }, + "textMode": "auto" + } } ] } diff --git a/samples/load/stampede.js b/samples/load/stampede.js new file mode 100644 index 0000000..2d43d25 --- /dev/null +++ b/samples/load/stampede.js @@ -0,0 +1,105 @@ +// --------------------------------------------------------------------------- +// Load profile for the Stampede.Http sample. +// +// docker compose --profile load up k6 +// +// Two identical arrival patterns run side by side: one against the client with +// Stampede.Http in its pipeline, one against the control instance without it. +// Because both drive the same origin, the origin's own counters at the end of the +// run are a controlled measurement rather than a marketing claim. +// --------------------------------------------------------------------------- + +import http from 'k6/http'; +import { check, sleep } from 'k6'; +import { Counter } from 'k6/metrics'; + +const STAMPEDE = __ENV.STAMPEDE_TARGET || 'http://localhost:5081'; +const BASELINE = __ENV.BASELINE_TARGET || 'http://localhost:5083'; + +// Served straight from the client cache — no origin involvement. +const cacheHits = new Counter('client_cache_hits'); + +const LANGUAGES = ['en-GB', 'es-ES', 'fr-FR']; +const TENANTS = ['acme', 'globex', 'initech']; + +const STAGES = [ + { duration: '20s', target: 40 }, // ramp into the stampede + { duration: '60s', target: 40 }, // steady state + { duration: '20s', target: 0 }, // drain +]; + +export const options = { + scenarios: { + stampede: { + executor: 'ramping-vus', + exec: 'browse', + stages: STAGES, + env: { TARGET: STAMPEDE }, + tags: { mode: 'stampede' }, + }, + baseline: { + executor: 'ramping-vus', + exec: 'browse', + stages: STAGES, + env: { TARGET: BASELINE }, + tags: { mode: 'baseline' }, + startTime: '0s', + }, + }, + thresholds: { + // The clients must stay up under load regardless of which pipeline they run. + http_req_failed: ['rate<0.05'], + // Both limits are deliberately loose — they exist so the summary prints the two + // latency distributions side by side. That contrast, and the origin counters in + // teardown(), are the actual output of this run. + 'http_req_duration{mode:stampede}': ['p(95)<3000'], + 'http_req_duration{mode:baseline}': ['p(95)<10000'], + }, +}; + +/** One virtual user's browsing session against whichever client this scenario targets. */ +export function browse() { + const target = __ENV.TARGET; + const roll = Math.random(); + + let response; + if (roll < 0.55) { + response = http.get(`${target}/api/catalog`, { tags: { endpoint: 'catalog' } }); + } else if (roll < 0.7) { + response = http.get(`${target}/api/feed`, { tags: { endpoint: 'feed' } }); + } else if (roll < 0.82) { + const lang = LANGUAGES[Math.floor(Math.random() * LANGUAGES.length)]; + response = http.get(`${target}/api/greetings?lang=${lang}`, { tags: { endpoint: 'greetings' } }); + } else if (roll < 0.94) { + const tenant = TENANTS[Math.floor(Math.random() * TENANTS.length)]; + response = http.get(`${target}/api/tenants/${tenant}`, { tags: { endpoint: 'tenants' } }); + } else { + // The expensive one. With coalescing a burst of these collapses into a single + // origin call; without it, every caller waits out the origin's 2 s. + response = http.get(`${target}/api/slow`, { tags: { endpoint: 'slow' } }); + } + + check(response, { 'status is 200': (r) => r.status === 200 }); + + // The client echoes the origin's Age header back in its JSON payload; a non-null + // value means the caller never touched the origin. + try { + if (response.json('ageSeconds') !== null) { + cacheHits.add(1); + } + } catch { + // Non-JSON body (an error page under load) — not worth failing the run over. + } + + sleep(Math.random() * 0.5); +} + +/** Prints the origin's own view of the run: how much traffic each client actually caused. */ +export function teardown() { + const stats = http.get(`${STAMPEDE}/api/origin-stats`); + console.log('Origin counters after the run (shared across every client):'); + console.log(JSON.stringify(stats.json(), null, 2)); + console.log( + 'Compare in Prometheus: sum by (client) (rate(sample_api_origin_requests_total[1m]))' + ); +} diff --git a/samples/prometheus/prometheus.yml b/samples/prometheus/prometheus.yml index 94a8014..84a645a 100644 --- a/samples/prometheus/prometheus.yml +++ b/samples/prometheus/prometheus.yml @@ -7,10 +7,28 @@ global: metric_name_escaping_scheme: underscores scrape_configs: + # The clients expose /metrics on their normal application port — no side-channel + # listener, no wildcard-binding gymnastics. The `mode` label is what lets a single + # dashboard query compare Stampede.Http instances against the control group. - job_name: stampede-http-clients static_configs: - - targets: - - client-a:9464 - - client-b:9464 + - targets: ["client-a:8080"] + labels: + client: client-a + mode: stampede + - targets: ["client-b:8080"] + labels: + client: client-b + mode: stampede + - targets: ["client-baseline:8080"] + labels: + client: client-baseline + mode: baseline + + # The origin counts what actually reached it, tagged with the X-Client header the + # clients send. This is the measurement that matters. + - job_name: sample-api + static_configs: + - targets: ["api:8080"] labels: app: stampede-http-sample diff --git a/samples/samples.http b/samples/samples.http new file mode 100644 index 0000000..c9cb1cb --- /dev/null +++ b/samples/samples.http @@ -0,0 +1,123 @@ +### Stampede.Http sample — hands-on request collection +# +# Works with the VS Code REST Client, Visual Studio's .http editor and JetBrains +# HTTP Client. Start the stack first: +# +# docker compose up --build -d +# +# @origin is the sample API: no Stampede.Http anywhere, it only sets headers. +# @client is the app with Stampede.Http in its outbound pipeline. +# @control is the identical app without it — send the same request to both and +# compare what the origin's counters do. + +@origin = http://localhost:5080 +@client = http://localhost:5081 +@control = http://localhost:5083 + + +### ─── The origin's own view ──────────────────────────────────────────────── +### Live counters of what actually reached the origin. Re-read this between any +### two requests below to see whether they cost an origin call. +GET {{origin}}/stats + +### Reset the counters so you can measure a clean window. +POST {{origin}}/stats/reset + + +### ─── Freshness and revalidation ─────────────────────────────────────────── +### First call is a miss (~300 ms). Repeat within 10 s: instant, with an Age header. +GET {{client}}/api/catalog + +### After 10 s the entry is stale but retained (RevalidationGraceSeconds), so this +### expiry is settled with If-None-Match → 304, no body transferred. Watch +### "GET /catalog -> 304" climb in /stats. +GET {{client}}/api/catalog + +### Mutate. The 2xx response evicts the cached GET entry (RFC 9111 §4.4) — and +### because the store is Redis, it is evicted for client-b too. +POST {{client}}/api/catalog + +### ...so this one refetches and comes back with a new version. +GET {{client}}/api/catalog + + +### ─── Stale extensions (RFC 5861) ────────────────────────────────────────── +### stale-while-revalidate: never blocks after the first fetch. The generation +### number moves in the background while you keep getting instant responses. +GET {{client}}/api/feed + +### stale-if-error: /flaky returns 503 for the first 20 s of every minute. Polly +### retries the blips; when the outage outlasts the retries the cache serves the +### last good 200. Try it repeatedly across a minute boundary. +GET {{client}}/api/flaky + +### must-revalidate: once stale this entry may NOT be served without checking the +### origin, even under failure. Contrast with /api/flaky above. +GET {{client}}/api/ledger + + +### ─── Coalescing ─────────────────────────────────────────────────────────── +### The origin takes 2 s. Fire this several times at once (in separate editor +### tabs, or with the k6 profile) — one origin call serves all of them. +GET {{client}}/api/slow + +### Same request against the control instance: every caller pays the full 2 s and +### the origin sees every one of them. +GET {{control}}/api/slow + + +### ─── Vary: one URL, many representations ────────────────────────────────── +GET {{client}}/api/greetings?lang=en-GB + +### A different language is a different cache entry, not an overwrite. +GET {{client}}/api/greetings?lang=es-ES + +### And a third. +GET {{client}}/api/greetings?lang=fr-FR + + +### ─── Multi-tenancy: Vary + CoalesceKeyHeaders ───────────────────────────── +### X-Tenant-Id is a Vary field at the origin and a coalescing key on the client, +### so concurrent bursts stay separated per tenant. The origin takes 1 s. +GET {{client}}/api/tenants/acme + +### +GET {{client}}/api/tenants/globex + + +### ─── Immutable, size ceiling, query normalization ───────────────────────── +### Cache-Control: immutable — cached for a year and never revalidated. +GET {{client}}/api/assets/logo-v3 + +### ~1.8 MB with a perfectly good max-age, and still never stored: it exceeds +### CacheOptions.MaxBodySizeBytes. Every call reaches the origin. +GET {{client}}/api/bulk + +### These two are the same cache entry (NormalizeQueryParameters = true) — +### only the first costs an origin call. +GET {{client}}/api/search?alpha=1&beta=2 + +### +GET {{client}}/api/search?beta=2&alpha=1 + + +### ─── Last-Modified rather than ETag ─────────────────────────────────────── +### max-age=5. Call it, wait six seconds, call it again: the expiry is settled +### with If-Modified-Since → 304. +GET {{client}}/api/docs/handbook + + +### ─── Introspection ──────────────────────────────────────────────────────── +### Effective options right now. Edit samples/config/client.json (for example set +### Stampede:Cache:DefaultTtl to "00:00:30") and re-run this: the value changes +### with no restart. Structural options will not move — that is by design. +GET {{client}}/api/config + +### This process's stampede_http.* instrument totals, as JSON. +GET {{client}}/api/counters + +### The same instruments in Prometheus exposition format. +GET {{client}}/metrics + +### And the origin's. +GET {{origin}}/metrics diff --git a/samples/scripts/smoke-test.sh b/samples/scripts/smoke-test.sh new file mode 100755 index 0000000..bfb118b --- /dev/null +++ b/samples/scripts/smoke-test.sh @@ -0,0 +1,197 @@ +#!/usr/bin/env bash +# --------------------------------------------------------------------------- +# End-to-end smoke test for the sample stack. +# +# ./scripts/smoke-test.sh # brings the stack up, asserts, tears it down +# KEEP_STACK=1 ./scripts/smoke-test.sh # leaves it running for inspection +# +# This runs in CI so the sample cannot rot silently: it asserts the behaviour the +# README claims, using the origin's own request counters as the source of truth. +# --------------------------------------------------------------------------- +set -euo pipefail + +cd "$(dirname "$0")/.." + +ORIGIN=http://localhost:5080 +CLIENT=http://localhost:5081 +CONTROL=http://localhost:5083 + +failures=0 + +log() { printf '\n\033[1;36m==> %s\033[0m\n' "$*"; } +pass() { printf ' \033[0;32m[pass]\033[0m %s\n' "$*"; } +fail() { printf ' \033[0;31m[FAIL]\033[0m %s\n' "$*"; failures=$((failures + 1)); } + +cleanup() { + if [[ "${KEEP_STACK:-0}" == "1" ]]; then + log "KEEP_STACK=1 — leaving the stack running" + return + fi + log "Tearing the stack down" + docker compose down -v --remove-orphans >/dev/null 2>&1 || true +} +trap cleanup EXIT + +# Poll a condition until it holds or the budget runs out. +# +# Metric endpoints do not become correct at the instant a container reports healthy: +# an OpenTelemetry counter emits nothing until it has recorded its first measurement, +# and the exporter caches scrape responses briefly. Asserting on that with a single +# request is a race, and it is the race that first broke this script in CI. +retry_until() { + local deadline=$((SECONDS + $1)); shift + while ! "$@"; do + if (( SECONDS >= deadline )); then + return 1 + fi + sleep 2 + done +} + +# Whether an endpoint's body contains a pattern. Used as the predicate for retry_until. +body_contains() { + curl -fsS "$1" 2>/dev/null | grep -q "$2" +} + +# Whether Prometheus reports the origin's scrape target as up. +prometheus_origin_target_up() { + curl -fsS --get "http://localhost:9090/api/v1/query" \ + --data-urlencode 'query=up{job="sample-api"}' 2>/dev/null \ + | grep -q '"value":\[[0-9.]*,"1"\]' +} + +# Value of a numeric JSON property, 0 when absent. The keys here contain spaces and +# slashes but never quotes or nesting, so grep is enough — and keeps this script +# dependent on nothing but curl. +json_number() { + local url=$1 key=$2 value + value=$(curl -fsS "$url" | grep -o "\"${key}\":[0-9]\+" | head -1 | cut -d: -f2) + echo "${value:-0}" +} + +# Counter value for a given endpoint from the origin's /stats. +origin_count() { + json_number "$ORIGIN/stats" "$1" +} + +# Fire N concurrent GETs at a URL and wait for all of them. +burst() { + local url=$1 count=$2 + seq "$count" | xargs -P "$count" -I{} curl -fsS -o /dev/null "$url" || true +} + +log "Building and starting the stack" +docker compose up -d --build --wait --wait-timeout 300 + +log "Waiting for the opening stampede and the feature tour to finish" +deadline=$((SECONDS + 180)) +until docker compose logs client-a 2>/dev/null | grep -q "feature tour complete"; do + if (( SECONDS > deadline )); then + fail "client-a never finished the feature tour" + docker compose logs --tail 60 client-a + break + fi + sleep 3 +done + +# --------------------------------------------------------------------------- +log "1. The feature tour verified every documented behaviour" +tour_line=$(docker compose logs client-a 2>/dev/null | grep -o "feature tour complete: [0-9]*/[0-9]* checks" | tail -1 || true) +if [[ -z "$tour_line" ]]; then + fail "no feature tour summary in client-a's logs" +elif [[ "$tour_line" =~ ([0-9]+)/([0-9]+) ]] && [[ "${BASH_REMATCH[1]}" == "${BASH_REMATCH[2]}" ]]; then + pass "$tour_line" +else + fail "$tour_line — some scenario did not behave as documented" + docker compose logs client-a 2>/dev/null | grep "\[??\]" || true +fi + +# --------------------------------------------------------------------------- +log "2. Coalescing collapses a concurrent burst into one origin call" +before=$(origin_count "GET /slow") +burst "$CLIENT/api/slow" 20 +after=$(origin_count "GET /slow") +delta=$((after - before)) +# 0 is also correct: the entry may already be warm from the scripted workload. And the +# workload's own /slow burst can land inside this window and contribute one call of its +# own, so allow 2 — still decisive against the 20 the control instance produces below. +if (( delta <= 2 )); then + pass "20 concurrent callers → $delta origin call(s)" +else + fail "20 concurrent callers → $delta origin calls (expected at most 2)" +fi + +# --------------------------------------------------------------------------- +log "3. The control instance shows what that costs without Stampede.Http" +before=$(origin_count "GET /slow") +burst "$CONTROL/api/slow" 20 +after=$(origin_count "GET /slow") +control_delta=$((after - before)) +if (( control_delta >= 10 )); then + pass "20 concurrent callers with no coalescing → $control_delta origin calls" +else + fail "control instance produced only $control_delta origin calls (expected ≥ 10) — is Stampede.Http leaking into the baseline?" +fi + +# --------------------------------------------------------------------------- +log "4. Vary keeps one cache entry per representation" +before=$(origin_count "GET /greetings") +for lang in en-GB es-ES fr-FR; do + curl -fsS -o /dev/null "$CLIENT/api/greetings?lang=$lang" + curl -fsS -o /dev/null "$CLIENT/api/greetings?lang=$lang" +done +after=$(origin_count "GET /greetings") +delta=$((after - before)) +if (( delta <= 3 )); then + pass "3 languages fetched twice each → $delta origin call(s)" +else + fail "3 languages fetched twice each → $delta origin calls (expected at most 3)" +fi + +# --------------------------------------------------------------------------- +log "5. The instruments are exported for Prometheus" +if retry_until 60 body_contains "$CLIENT/metrics" "stampede_http"; then + pass "client exposes stampede_http.* on /metrics" +else + fail "no stampede_http instruments on $CLIENT/metrics after 60 s" + printf ' what it returned instead:\n' + curl -fsS "$CLIENT/metrics" 2>&1 | head -5 | sed 's/^/ /' +fi + +if retry_until 60 body_contains "$ORIGIN/metrics" "sample_api_origin_requests"; then + pass "origin exposes its request counter on /metrics" +else + fail "no origin request counter on $ORIGIN/metrics after 60 s" + printf ' what it returned instead:\n' + curl -fsS "$ORIGIN/metrics" 2>&1 | head -5 | sed 's/^/ /' +fi + +# Stronger than "Prometheus answers queries": assert the origin target is actually up, +# which is what the dashboards depend on. Prometheus scrapes every 5 s, so give it room +# to have completed a first round. +if retry_until 60 prometheus_origin_target_up; then + pass "Prometheus is scraping the origin" +else + fail "Prometheus has no healthy sample-api target after 60 s" + printf ' target health:\n' + curl -fsS "http://localhost:9090/api/v1/targets?state=active" 2>&1 \ + | tr ',' '\n' | grep -E '"(scrapeUrl|health|lastError)"' | sed 's/^/ /' | head -20 +fi + +# --------------------------------------------------------------------------- +log "6. The cache is actually being hit" +retry_until 60 body_contains "$CLIENT/api/counters" '"stampede_http.cache.hits"' || true +hits=$(json_number "$CLIENT/api/counters" "stampede_http.cache.hits") +if (( hits > 0 )); then + pass "client-a served $hits requests from cache" +else + fail "client-a reports zero cache hits" +fi + +# --------------------------------------------------------------------------- +if (( failures == 0 )); then + log "All smoke checks passed" +else + log "$failures smoke check(s) failed" +fi +exit $(( failures > 0 ? 1 : 0 ))