feat: add DuckDB telemetry backend for self-hosted mode (#203)#254
feat: add DuckDB telemetry backend for self-hosted mode (#203)#254jstojiljkovic wants to merge 20 commits into
Conversation
|
We have too many separate backends that our folder structure makes no sense anymore. Instead of doing We should do 3 folders: repositories/telemetry/duckdb/something.go This will allow us to (based on tags) switch between them We should also update the tags we use when building the projects, right now they are kinda confusing, we should probably do something like oltp_sqlite This would make tags clear and allow us to implement more backends like victoria metrics/logs/traces easily |
@dusanstanojeviccs Agreed, separated everything with clean tags, so in the future we can easily add new backends |
This reverts commit 33a00f3.
Adds DuckDB as an opt-in telemetry store for self-hosted deployments (
-tags duckdb, CGO required). The main DB stays SQLite (relational/config); only the telemetry DB becomes DuckDB (columnar), written via the Appender API.Closes #203.
Three mutually-exclusive backends, selected at build time:
pgch!pgch && !duckdbduckdb && !pgchBenchmarks (CCX13 · €16.49/mo · vs SQLite)
Read: largest fill level the dashboard stays usable (≤ 5 s, no timeout):
DuckDB holds 10–100× more queryable rows and flips SQLite's logs cliff.
Write: max sustainable items/sec:
Disk: ~6× smaller than SQLite (auto FSST/Dictionary compression).