Skip to content

adaptive_export passthrough fails on http2_messages.beta every cycle when the beta table is not registered in Vizier #60

@entlein

Description

@entlein

Summary

In passthrough mode (ADAPTIVE_PASSTHROUGH=1) adaptive_export iterates its builtin
table list and queries each one via PxL. http2_messages.beta fails PxL
compilation every cycle
with Table 'http2_messages.beta' not found, so HTTP/2 +
gRPC is never exported on Viziers where that beta table is not registered, and a
warning is logged on every passthrough pass per node.

Evidence

Rig: k3s, Vizier 0.14.19-aeprod-clean4 (registered, healthy). AE log, every passthrough cycle:

level=warning msg="ADAPTIVE_PASSTHROUGH: pixie query failed"
  error="pixieapi: PxL compilation: Compilation failed: \n2:18 Table 'http2_messages.beta' not found."
  table=http2_messages.beta

Live px run probes against the same Vizier:

table queried result
http2_messages.beta Table not found
http2_messages Table not found
http2_events Table not found
grpc_events Table not found

Every other builtin table compiles fine (http_events, dns_events, conn_stats,
redis_events, mysql_events, pgsql_events, cql_events, mongodb_events,
amqp_events, mux_events, tls_events, and notably kafka_events.beta — the other
dotted beta table — all work).

The table name is correct (not a typo, not missing from the DDL)

Stirling defines the table verbatim as http2_messages.beta:

src/stirling/source_connectors/socket_tracer/http2_messages_table.h:56

DataTableSchema("http2_messages.beta", "HTTP2 messages events", kHTTP2MessagesElements);

And adaptive_export already includes it in both the builtin list and the forensic_db DDL:

  • src/vizier/services/adaptive_export/internal/pxl/tables.go:63{Name: "http2_messages.beta", Protocol: "HTTP/2 + gRPC"}
  • src/vizier/services/adaptive_export/internal/clickhouse/schema.sql:111CREATE TABLE IF NOT EXISTS forensic_db.http2_messages.beta (...)

So the ClickHouse side already exists and the name matches the Stirling source. This is
not a missing-DDL or wrong-name issue.

Root cause

http2_messages.beta is a beta data table that is not present in this Vizier's schema
(the http2 source / beta table is not registered by default — unlike kafka_events.beta,
which is). AE queries it unconditionally and emits a hard warning every cycle.

Impact

  • HTTP/2 + gRPC traffic is not exported when the beta table is not registered in Vizier.
  • One warning per passthrough cycle per node (log noise).
  • Non-fatal (the query failure is handled as a warning; other tables continue).

Proposed fix (pick one)

  1. Probe the Vizier schema once (or on first failure) and skip builtin tables that are
    absent from it — graceful, recommended.
  2. Gate http2_messages.beta behind a config flag (e.g. ADAPTIVE_ENABLE_HTTP2_BETA,
    default off) so it is only queried where the beta table is enabled.
  3. Downgrade the per-cycle "table not found" warning to debug for known-absent tables, and
    document the Vizier flag required to register the http2 beta table.

The name correctness and DDL presence are confirmed; the open decision is which of the
above to implement.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions