fix(o11y): record output request + HEC ACK-poll latency in ms (PIPE-1404) - #292
Open
Dylan-M wants to merge 1 commit into
Open
Conversation
This was referenced Aug 11, 2026
feat(o11y): per-instance metrics via injectable MeterProvider, drop package globals (PIPE-1066)
#285
Open
Open
Contributor
Author
2 tasks
…404) Assisted-by: Claude Opus 4.8
Dylan-M
force-pushed
the
dylanmyers/pipe-1404-output-latency-ms-buckets
branch
from
August 14, 2026 19:22
b24bf34 to
792856e
Compare
Dylan-M
force-pushed
the
dylanmyers/pipe-1066-lifecycle-metrics
branch
from
August 14, 2026 19:22
be2b515 to
c52f955
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Proposed Change
blitz.output.request_latencyandblitz.output.hec.ack_poll_latencywere declared in seconds and recorded via.Seconds(). Both use OTel's default histogram buckets (0, 5, 10, 25, ... 10000), where the first boundary is 5. Output latencies almost always run well under 5 seconds, so every sample landed in the first bucket. The histograms carried no distribution signal.Both now record in milliseconds, where the default buckets fit. A new
output.DurationMillishelper converts a duration to fractional milliseconds. It stays float so a sub-millisecond sample is not truncated to zero. Truncating would also understate the histogram sum. The request-latency feeders (tcp, otlp_grpc, hec) and the HEC ACK-poll site record through it. Bothmetric.yamlunits change fromstoms.Stacked on #291.
How the reviewer can validate
Manual:
make buildnc -lk 127.0.0.1 5140 >/dev/null &curl -s localhost:9100/metrics | grep request_latencyblitz_blitz_output_request_latency_milliseconds_*(previously..._seconds_*). On a localhost sink, sub-5ms sends correctly sit in the first bucket. Any latency above 5ms now lands in a higher bucket, where in seconds it would have collapsed into the first.The HEC ACK-poll histogram (
blitz_blitz_output_hec_ack_poll_latency_milliseconds_*) validates the same way against a HEC endpoint with ack enabled.Automated:
go test ./output/....TestDurationMilliscovers the conversion including sub-millisecond preservation. Manual-reader tests assert themsunit on bothblitz.output.request_latencyandblitz.output.hec.ack_poll_latency.Checklist