Skip to content

JSON log records can emit a bare NaN/Infinity token, which strict parsers reject #1496

Description

@groupthinking

Summary

StructuredFormatter._format_json calls json.dumps without allow_nan=False. allow_nan defaults to True, so a non-finite float in performance_ms or correlation_id is emitted as a bare NaN / Infinity / -Infinity token.

Those tokens are not valid JSON. RFC 8259 has no numeric literal for them — they are a CPython extension. A strict parser rejects the record, which means the record is effectively lost at the consumer even though it was written to the sink intact.

Reproduction

Verified on origin/main @ 715cbf5 (i.e. this is pre-existing, not introduced by the #1452 follow-up).

logger.info("probe", extra={"request_id": float("nan")})
raw tail: ction": "<module>", "process": 17924, "correlation_id": NaN}
contains bare NaN token: True

Parsed with the constant hook disabled, which is what a strict parser does:

json.loads(raw, parse_constant=lambda c: (_ for _ in ()).throw(ValueError("strict rejects " + c)))
STRICT parse: REJECTED -> strict rejects NaN

Why it hides

json.loads accepts NaN by default, so every round-trip test written in Python passes and the record looks fine locally. It breaks in the consumer: Go's encoding/json, Jackson without ALLOW_NON_NUMERIC_NUMBERS, and most log pipelines reject it. JSON_LOGGING defaults to "true" in production_config.py:72, so the JSON path is the production path.

Reachability

Low severity, same shape as #1452. Not attacker-reachable, and not currently reachable at all:

  • correlation_id comes from record.request_id and from header values — strings.
  • performance_ms is built as a formatted string (f"{record.duration * 1000:.2f}ms"), so it is never a raw float today. Note float("nan") formats to the string "nanms" through that path, which is ugly but valid JSON.
  • Reaching this needs a future call site passing a non-finite float directly.

Like #1452, this is a latent gap plus a correctness claim worth closing while the surrounding code is fresh, not an incident.

Relationship to #1452 and #1488

Found during the step-5 red-team pass on #1488 (the #1452 follow-up). Filing separately rather than folding it in, for two reasons:

  1. It is a different failure modedefault=str in _format_json does not deliver its stated guarantee — a record can still be lost (follow-up to #1439) #1452 is "the record never reaches the sink"; this is "the record reaches the sink but the consumer cannot parse it". fix(logging): never lose a JSON log record to a serialization error #1488's fallback tiers do not touch it, because NaN does not make json.dumps raise.
  2. PR Governance requires exactly one closing reference per PR, and fix(logging): never lose a JSON log record to a serialization error #1488 already closes default=str in _format_json does not deliver its stated guarantee — a record can still be lost (follow-up to #1439) #1452.

Filing it once so it is tracked rather than re-derived — the waste #1452 itself documented.

Suggested fix

allow_nan=False makes json.dumps raise ValueError on a non-finite float, which #1488's tier-2 fallback then catches and degrades cleanly — so after #1488 lands, this is close to a one-word change plus a test. Doing it before #1488 would instead turn a malformed record into a lost one, so the ordering matters.

Acceptance criteria

  1. A record with float("nan"), float("inf") or float("-inf") in an enrichment field parses under a strict parser.
  2. A regression test asserts the emitted record contains no bare NaN/Infinity token, and fails on the pre-fix implementation.
  3. The record is still emitted — a non-finite float must not become a dropped record.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions