Skip to content

feat: drop clj-holmes; replace its detections with semgrep rules - #6

Merged
arootroatch merged 1 commit into
masterfrom
drop-clj-holmes
Jul 28, 2026
Merged

feat: drop clj-holmes; replace its detections with semgrep rules#6
arootroatch merged 1 commit into
masterfrom
drop-clj-holmes

Conversation

@arootroatch

Copy link
Copy Markdown
Collaborator

clj-holmes read only *.clj — silently skipping .cljs and .cljc and
rejecting reader conditionals outright — had been unmaintained since October
2022
, and produced three failures in its first week of real use:

  1. --fail-on-result with -t sarif exited 3 on zero findings
  2. findings went to a file, so a red build printed no reason (PR fix: clj-holmes crash on large repos; make semgrep findings visible #5)
  3. a progrock integer overflow computing an ETA crashed the scan with exit
    255 and no SARIF
    , blocking a production deploy on cleancoders.com (PR fix: clj-holmes crash on large repos; make semgrep findings visible #5)

Measured, not assumed

A fixture covering all eight sink types clj-holmes claims to detect:

sink clj-holmes replacement rules
read-string cc-read-string
clojure.xml/parse cc-clojure-xml-xxe
Blowfish cc-weak-crypto
DESede cc-weak-crypto
AES/ECB cc-weak-crypto
SSLContext "SSL" cc-insecure-tls
permissive HostnameVerifier cc-insecure-tls
MD5 cc-weak-crypto
total 6/8 8/8

Zero false positives on the safe corpus for both. clj-holmes ships rules for
Blowfish and DESede and matches neither.

Semgrep independently already ran clojure.lang.security.use-of-md5,
use-of-sha1, and documentbuilderfactory-xxe, so the overlap was larger than
the table suggests.

Why the alias argument no longer applies

clj-holmes was originally chosen for namespace-alias resolution. Every detection
being replaced is Java interop (Cipher, MessageDigest, SSLContext,
HostnameVerifier) or clojure.core/clojure.xml — there is no alias to
resolve in any of them. The capability didn't apply to what it was covering.

The replacements also cover .cljs and .cljc, which clj-holmes never read —
so weak crypto in a .cljc namespace was invisible until now.

Honest costs

  • cc-clojure-xml-xxe is severity: WARNING. It fires on every XML parse
    call: proving the factory was hardened needs dataflow neither engine has. Same
    treatment as cc-path-traversal and cc-generic-catch, so it feeds
    /security-audit triage rather than gating merges. This is a precision
    regression against clj-holmes, which had a pattern-not for the setFeature
    calls, and it is recorded in the README.
  • cc-weak-crypto and cc-insecure-tls use pattern-regex for algorithm
    and protocol names, because semgrep does not bind metavariables inside Clojure
    string literals — the same limitation that shaped cc-shell-exec. Textual
    matching can fire inside a comment. Also in the README limitations.
  • correctness/schema-require-typo is lost. Non-security, clj-kondo territory.

Verification

  • 16 rules tagged; fixture corpus passes both directions
  • All 16 metadata.class values cross-check clean against the clojure-security
    skill index — the check that caught the cc-shell-exec/dynamic-eval drift
  • Against cleancoders.com: identical 7 findings / 3 blocking, so the new
    rules add no noise to the repo that exposed the crash
  • shellcheck, actionlint, zizmor (0 findings), clj-kondo on fixtures: clean

Seven scanners, 10 jobs. v1 needs retagging after merge.

🤖 Generated with Claude Code

clj-holmes read only *.clj — silently skipping .cljs and .cljc and rejecting
reader conditionals outright — had been unmaintained since Oct 2022, and
produced three failures in its first week of real use:

  1. --fail-on-result with -t sarif exited 3 on ZERO findings
  2. findings went to a file, so a red build printed no reason
  3. a progrock int overflow computing an ETA crashed the scan with exit 255 and
     no SARIF, blocking a production deploy on cleancoders.com

Its five unique detections become four semgrep rules: cc-read-string,
cc-weak-crypto, cc-insecure-tls (all ERROR) and cc-clojure-xml-xxe (WARNING).

Measured on a fixture covering all eight sink types, the replacements catch 8
where clj-holmes caught 6 — it ships rules for Blowfish and DESede and matches
neither. Zero false positives on the safe corpus for both. The replaced
detections are Java interop or clojure.core, so semgrep's inability to resolve
namespace aliases does not apply to any of them, and they now cover .cljs and
.cljc as well.

cc-clojure-xml-xxe is WARNING because it fires on every parse call: proving the
factory was hardened needs dataflow neither engine has. Same treatment as
cc-path-traversal and cc-generic-catch, so it feeds /security-audit triage
rather than gating merges.

cc-weak-crypto and cc-insecure-tls use pattern-regex for algorithm and protocol
names because semgrep does not bind metavariables inside Clojure string
literals — the same limitation that shaped cc-shell-exec. Recorded in the
README limitations.

Verified against cleancoders.com: identical 7 findings / 3 blocking, so the new
rules add no noise to the repo that exposed the crash.

Seven scanners now, 16 cc-* rules. All 16 classes cross-check clean against the
clojure-security skill index.
@arootroatch
arootroatch merged commit 4967c75 into master Jul 28, 2026
10 checks passed
@arootroatch
arootroatch deleted the drop-clj-holmes branch July 28, 2026 17:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant