Skip to content

_format_json still loses records after #1491 — four holes, nine PRs, none merged #1505

Description

@groupthinking

#1452 closed with #1491 (8517bf8), which wrapped _format_json's json.dumps in a fallback so a bad enrichment costs its own value rather than the whole record.

The fallback does not yet deliver that. Three inputs still cost the record — or its validity — on current main. All three reproduced through a real StreamHandler, healthy → poisoned → healthy.

1. The retention filter is forgeable

if isinstance(value, (str, int, float, bool, type(None)))

isinstance consults value.__class__, which an object can forge as a property returning str. Such a value passes the filter, reaches the fallback json.dumps — which still passes default=str — and its raising __str__ propagates. logging swallows it via Handler.handleError and drops the record, which is the exact outcome the fallback exists to prevent.

Observed: 2 of 3 records reach the sink.

2. serialization_error can raise inside the recovery

safe["serialization_error"] = f"{type(exc).__name__}: {exc}"

{exc} calls str(exc). The exception being described can itself originate in a call site's __str__, so it may be an instance of a class whose __str__ also raises — failing inside the handler for the failure, one level down.

Observed: 2 of 3 records reach the sink.

3. Non-finite floats never reach the fallback

default is not consulted for float('nan') / float('inf'), so they bypass the fallback entirely and json renders them as the JavaScript literals NaN / Infinity. These are not valid JSON.

Python's own json.loads accepts them, which is why the existing tests did not catch this:

RAW: {..., "level": "INFO", "performance_ms": NaN}
json.loads(...)                       -> accepted
json.loads(..., parse_constant=raise) -> ValueError: not valid JSON: NaN

A strict downstream consumer rejects the whole record — the same loss as dropping it here, only moved to the consumer where it is harder to diagnose. performance_ms is one of the two enrichment fields, so this is reachable.

Acceptance criteria

  • A value forging __class__ does not cost its record.
  • An exception whose own __str__ raises does not cost its record; serialization_error degrades rather than propagating.
  • A non-finite enrichment yields a record that a strict JSON parser accepts (one that rejects NaN/Infinity), with the offending value dropped rather than the record.
  • A well-behaved finite float still keeps its value — the guard must not widen silently.
  • Regression tests in the shape of the existing ones in tests/unit/test_logging_config_crlf.py, each failing on the pre-fix implementation.

Severity

Low, and not attacker-reachable — every live call site passes a scalar, and an attacker-supplied header is a string. This closes the gap before a future call site opens it, and makes the fallback deliver the guarantee its comment already claims. The CWE-117 work in #1429 / #1439 is unaffected; the degraded path must still escape attacker content, which is already pinned by an existing test.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions