chore(clickhouse): 30d TTL on the trace-list rollup tables

[arseniycodes]

Follow-up to #136. The rollup tables (otel_traces_recent, otel_traces_summary) were created without a TTL, so they'd grow unbounded — and otel_traces_recent holds one row per span (the full span history). Give both the same 30-day TTL as otel_traces (toDateTime(<ts>) + toIntervalDay(30)) so the derived data ages out in lockstep with the spans it summarizes.

• migrations/006_otel_traces_rollup_ttl.sql — ALTER … MODIFY TTL for existing clusters (idempotent; ttl_only_drop_parts = 1 keeps enforcement to whole-part drops).
• Bootstrap schema updated inline so fresh clusters get the TTL at creation.

No app code change.

---

Note

Low Risk
Infrastructure-only ClickHouse retention on derived tables; read path already falls back to raw traces when rollups don’t cover the window.

Overview
Adds 30-day retention on the trace-list rollup tables otel_traces_recent and otel_traces_summary so they no longer grow without bound (especially otel_traces_recent, which stores one row per span).

otel_traces_recent expires rows 30 days after each span’s ts. otel_traces_summary uses end_unix_nano (last span end), not trace start, so long-running traces don’t lose summary stats while recent rows for the same trace_id still exist.

Existing HA clusters get idempotent ALTER … MODIFY TTL in 006_otel_traces_rollup_ttl.sql; fresh installs pick up the same rules in ha-replicated-otel.sql, still with ttl_only_drop_parts = 1. No application changes—older windows that rollups no longer cover keep using the existing raw otel_traces fallback via the coverage gate.

^{Reviewed by Cursor Bugbot for commit 2c3fe52. Bugbot is set up for automated code reviews on this repo. Configure here.}

---

Summary by cubic

Add a 30-day TTL to the ClickHouse rollup tables otel_traces_recent and otel_traces_summary to cap storage. The summary TTL anchors on the trace end so fast-path stats stay consistent; older windows fall back to the raw scan.

• Migration
  • Adds infra/clickhouse/migrations/006_otel_traces_rollup_ttl.sql to ALTER ... MODIFY TTL on superlog.otel_traces_recent and superlog.otel_traces_summary across the superlog_ha cluster.
  • TTL rules: toDateTime(ts) + toIntervalDay(30) on otel_traces_recent; toDateTime(fromUnixTimestamp64Nano(end_unix_nano)) + toIntervalDay(30) on otel_traces_summary.
  • Bootstrap schema updated so new clusters get the TTL; keeps ttl_only_drop_parts = 1 for whole-part drops.

^{Written for commit 2c3fe52. Summary will update on new commits.}

[cursor]

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

Want higher recall? High effort reviews run extra passes and find more bugs. A team admin can switch effort levels in the Cursor dashboard.

^{Reviewed by Cursor Bugbot for commit dc51437. Configure here.}

[cubic-dev-ai]

3 issues found across 2 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="infra/clickhouse/schema/ha-replicated-otel.sql">

<violation number="1" location="infra/clickhouse/schema/ha-replicated-otel.sql:476">
P2: Trace-list rollups are now hard-capped at 30 days, but the repository’s `otel_traces` source table definition still has no TTL. That means older traces can still exist in `otel_traces` while `otel_traces_recent`/`otel_traces_summary` have already dropped them, causing the rollup-backed list path to under-report historical traces relative to source data. Consider adding/updating the source table TTL in the same schema/migration set (or documenting/enforcing it in code-managed SQL) so retention actually stays in lockstep.</violation>
</file>

<sub>Reply with feedback, questions, or to request a fix.

Fix all with cubic | Re-trigger cubic</sub>

[cubic-dev-ai]

P2: Trace-list rollups are now hard-capped at 30 days, but the repository’s otel_traces source table definition still has no TTL. That means older traces can still exist in otel_traces while otel_traces_recent/otel_traces_summary have already dropped them, causing the rollup-backed list path to under-report historical traces relative to source data. Consider adding/updating the source table TTL in the same schema/migration set (or documenting/enforcing it in code-managed SQL) so retention actually stays in lockstep.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At infra/clickhouse/schema/ha-replicated-otel.sql, line 476:

<comment>Trace-list rollups are now hard-capped at 30 days, but the repository’s `otel_traces` source table definition still has no TTL. That means older traces can still exist in `otel_traces` while `otel_traces_recent`/`otel_traces_summary` have already dropped them, causing the rollup-backed list path to under-report historical traces relative to source data. Consider adding/updating the source table TTL in the same schema/migration set (or documenting/enforcing it in code-managed SQL) so retention actually stays in lockstep.</comment>

<file context>
@@ -473,6 +473,7 @@ CREATE TABLE IF NOT EXISTS superlog.otel_traces_recent ON CLUSTER superlog_ha
 ENGINE = ReplicatedMergeTree('/clickhouse/{cluster}/tables/{shard}/{database}/{table}', '{replica}')
 PARTITION BY toDate(ts)
 ORDER BY (project_id, ts)
+TTL toDateTime(ts) + toIntervalDay(30)
 SETTINGS index_granularity = 8192, ttl_only_drop_parts = 1
 ;
</file context>

[arseniycodes]

Intentional, and now spelled out in the migration comment. We are deliberately not adding a TTL to the open-core otel_traces (005’s rationale stands). Capping only the derived tables does not under-report: the read path gates on the recent index covering the window start (traceRollupCoversWindow) and falls back to the raw otel_traces scan for anything older than the rollup retains. So beyond the derived retention the list is served the slow-but-complete way from source, rather than silently truncated — retention stays correct without forcing a source-table TTL.

[cubic-dev-ai]

The parent comment was wrong here: the derived-table TTL is intentional, and traceRollupCoversWindow already falls back to the raw otel_traces scan, so older windows aren’t silently truncated. The source table TTL doesn’t need to be added for correctness in this PR.