Skip to content

[superlog] Don't mark ingest.queue_send span as ERROR for empty/oversized client bodies#150

Open
superlog-app[bot] wants to merge 1 commit into
mainfrom
superlog/fix-empty-body-otel-span-noise
Open

[superlog] Don't mark ingest.queue_send span as ERROR for empty/oversized client bodies#150
superlog-app[bot] wants to merge 1 commit into
mainfrom
superlog/fix-empty-body-otel-span-noise

Conversation

@superlog-app

@superlog-app superlog-app Bot commented Jul 6, 2026

Copy link
Copy Markdown

Summary

The proxy's ingest.queue_send OTel span was marking ALL exceptions as ERROR before rethrowing, including EmptyBodyError and PayloadTooLargeError. These are expected client-side conditions (empty OTLP payload, body over the size cap) that the outer handleIngestBodyError handler converts into correct 4xx HTTP responses — so no data is lost and the client is informed. The ERROR span label was a false positive that caused Superlog's worker to fingerprint each occurrence as an issue and open recurring incidents.

The pre-auth Content-Length: 0 guard added in #129 fast-rejects the most common form of empty bodies, but explicitly does not cover chunked/undeclared-length empty bodies, which still reach captureBody and trigger this path.

The fix skips recordException/setStatus(ERROR) on the ingest.queue_send span when the error is EmptyBodyError or PayloadTooLargeError. Unexpected server-side errors still record as ERROR as before. An alternative approach would be to silence the incident entirely via Superlog's noise-classification feature, but fixing the span label is cleaner and avoids masking real future errors that happen to share the same span name.

Incident on Superlog


Was this PR helpful? Leave feedback — goes straight to the Superlog team.


Summary by cubic

Stop marking the ingest.queue_send OTel span as ERROR for empty or oversized client bodies to eliminate false error noise and recurring incidents while keeping 4xx paths clean.

  • Bug Fixes
    • Skip recordException and setStatus(ERROR) for EmptyBodyError and PayloadTooLargeError.
    • Preserve ERROR reporting for unexpected server-side faults.
    • Handles empty chunked bodies that bypass the pre-auth Content-Length: 0 check.

Written for commit 5966cd6. Summary will update on new commits.

Review in cubic

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant