Skip to content

waste.py unused_variable: two systematic false-positive shapes (module constants, instance attributes) — 130/130 findings on requests, 12/12 on stdlib sqlite3 #25

Description

@CSLewin

Thanks for releasing SlopCodeBench — we've vendored the detectors verbatim
(pin 06b5c06) as a code-quality instrument and calibrated them against
known-good corpora. Two systematic false-positive shapes in waste.py's
unused_variable detector surfaced; both are cheap to reproduce.

Shape 1 — module-scope constants read only inside functions

A constant assigned at module scope and read only inside function bodies is
reported as unused. The detector sees the definition but not the
function-scope reads. Verified against unmodified upstream at 06b5c06
(name-identical output to our vendored copy). Minimal shape:

JOBS_DIR = Path("jobs")        # flagged unused

def next_queued():
    return sorted(JOBS_DIR.glob("*.json"))   # the read

Shape 2 — instance attributes read from other methods

self.x = ... is treated as a variable definition local to the method where
it's written; reads from other methods are invisible, so ordinary
cross-method state is reported as unused. On class-heavy code this shape
dwarfs shape 1. Verified against unmodified upstream at 06b5c06: the
minimal snippet below is flagged (_abort), and a 1,300-line class-heavy
production file produced 113 unused-variable findings — name-identical,
line-for-line, to our vendored copy — dominated by this shape plus shape 1.
Minimal shape:

class Daemon:
    def stop(self):
        self._abort = True        # flagged unused

    def loop(self):
        while not self._abort:    # the read, in another method
            ...

Thirty-second reproduction at scale

Run the detector over requests (we used 2.33.1) and stdlib sqlite3:

  • requests: the unused_variable detector reports 130 findings, and all
    130 are these two shapes
    — zero true positives. (Our full sweep also runs
    detectors of our own that are not ports of yours; their findings are
    excluded here.)
  • stdlib sqlite3 (decades-reviewed): 12 unused_variable findings, 12/12
    quirk-shaped
    , 0 real.

Reproducible with the same corpora; happy to share the raw sweep output for
both on request.

Why it matters for the benchmark

For files heavy in module constants or instance attributes, these shapes
dominate raw tier-1 density (~97% of findings on two of our OO files),
which inverts comparisons: requests scores raw density 6.6 vs 6.8 for our
known-crufty corpus — indistinguishable — while quirk-excluded density
separates them cleanly. Anyone using the published metrics comparatively
inherits this.

We deliberately did not patch our vendored copy (comparability with your
published numbers is the point of vendoring verbatim); we bucket these
shapes downstream instead. Happy to share the bucketing signatures or the
sweep configs if useful.


Written by Fable 5/high effort, read and approved by a human.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions