feat: add sending of metrics directly to otel collector via prom-client bridge - #1714
Draft
mutambaraf wants to merge 2 commits into
Draft
feat: add sending of metrics directly to otel collector via prom-client bridge#1714mutambaraf wants to merge 2 commits into
mutambaraf wants to merge 2 commits into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This PR introduces the functionality to send metrics directly to otel collector via a prom-client bridge. By introducing
PromClientMetricProducer, a custom OpenTelemetryMetricProducerthat bridges metrics already collected viaprom-client(HTTP request counts/durations, GraphQL request/error/response-size stats, Node.js default process metrics) into the OpenTelemetry SDK's export pipeline. At each collection cycle it readsprom-client's global registry (register.getMetricsAsJSON()) and converts each metric, counters, gauges, histograms, and summaries into OTel'sMetricDatashape.tracing.tsnow registers aPeriodicExportingMetricReaderwith this producer and anOTLPMetricExporter, gated behindOTEL_EXPORTER_OTLP_METRICS_ENDPOINT(mirroring how traces/logs are already conditionally enabled). The new@opentelemetry/exporter-metrics-otlp-httpdependency provides that exporter and exports on 60s interval.Motivation and Context
This change is required to improve the efficiency of sending metrics directly to otel collector. Metrics currently only reach observability tooling via a pull-based
/metricsendpoint for direct Prometheus scraping, separate from traces and logs, which already export through OTel/OTLP to Alloy. .Changes
How Has This Been Tested?
Fixes Jira Issue
Part of: https://github.com/isisbusapps/observability/issues/150
Tests included/Docs Updated?
Manual testing