The aggregates in the stats service answer the continuous questions (who's crawling, how much, from where, which probe families). The remaining questions from #1458 need the raw caddy logs in S3, not the bounded rollups:
- ip-range clusters: which agents share /24s or ASNs, and whether the anonymous "(more)" agents cluster with a named one
- back-off: which AI scrapers stop after a 403, and how quickly; which keep hammering
- request rate shapes per agent (bursts vs steady), and which routes each one prefers beyond the top-level counts
Output is a duckdb or notebook pass over logs/source=caddy/; anything worth watching continuously gets promoted into the rollups, and findings feed the caddy block rules.
The aggregates in the stats service answer the continuous questions (who's crawling, how much, from where, which probe families). The remaining questions from #1458 need the raw caddy logs in S3, not the bounded rollups:
Output is a duckdb or notebook pass over
logs/source=caddy/; anything worth watching continuously gets promoted into the rollups, and findings feed the caddy block rules.