feat: drop clj-holmes; replace its detections with semgrep rules - #6
Merged
Conversation
clj-holmes read only *.clj — silently skipping .cljs and .cljc and rejecting
reader conditionals outright — had been unmaintained since Oct 2022, and
produced three failures in its first week of real use:
1. --fail-on-result with -t sarif exited 3 on ZERO findings
2. findings went to a file, so a red build printed no reason
3. a progrock int overflow computing an ETA crashed the scan with exit 255 and
no SARIF, blocking a production deploy on cleancoders.com
Its five unique detections become four semgrep rules: cc-read-string,
cc-weak-crypto, cc-insecure-tls (all ERROR) and cc-clojure-xml-xxe (WARNING).
Measured on a fixture covering all eight sink types, the replacements catch 8
where clj-holmes caught 6 — it ships rules for Blowfish and DESede and matches
neither. Zero false positives on the safe corpus for both. The replaced
detections are Java interop or clojure.core, so semgrep's inability to resolve
namespace aliases does not apply to any of them, and they now cover .cljs and
.cljc as well.
cc-clojure-xml-xxe is WARNING because it fires on every parse call: proving the
factory was hardened needs dataflow neither engine has. Same treatment as
cc-path-traversal and cc-generic-catch, so it feeds /security-audit triage
rather than gating merges.
cc-weak-crypto and cc-insecure-tls use pattern-regex for algorithm and protocol
names because semgrep does not bind metavariables inside Clojure string
literals — the same limitation that shaped cc-shell-exec. Recorded in the
README limitations.
Verified against cleancoders.com: identical 7 findings / 3 blocking, so the new
rules add no noise to the repo that exposed the crash.
Seven scanners now, 16 cc-* rules. All 16 classes cross-check clean against the
clojure-security skill index.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
clj-holmes read only
*.clj— silently skipping.cljsand.cljcandrejecting reader conditionals outright — had been unmaintained since October
2022, and produced three failures in its first week of real use:
--fail-on-resultwith-t sarifexited 3 on zero findingsprogrockinteger overflow computing an ETA crashed the scan with exit255 and no SARIF, blocking a production deploy on cleancoders.com (PR fix: clj-holmes crash on large repos; make semgrep findings visible #5)
Measured, not assumed
A fixture covering all eight sink types clj-holmes claims to detect:
read-stringcc-read-stringclojure.xml/parsecc-clojure-xml-xxecc-weak-cryptocc-weak-cryptocc-weak-cryptoSSLContext "SSL"cc-insecure-tlsHostnameVerifiercc-insecure-tlscc-weak-cryptoZero false positives on the safe corpus for both. clj-holmes ships rules for
Blowfish and DESede and matches neither.
Semgrep independently already ran
clojure.lang.security.use-of-md5,use-of-sha1, anddocumentbuilderfactory-xxe, so the overlap was larger thanthe table suggests.
Why the alias argument no longer applies
clj-holmes was originally chosen for namespace-alias resolution. Every detection
being replaced is Java interop (
Cipher,MessageDigest,SSLContext,HostnameVerifier) orclojure.core/clojure.xml— there is no alias toresolve in any of them. The capability didn't apply to what it was covering.
The replacements also cover
.cljsand.cljc, which clj-holmes never read —so weak crypto in a
.cljcnamespace was invisible until now.Honest costs
cc-clojure-xml-xxeisseverity: WARNING. It fires on every XML parsecall: proving the factory was hardened needs dataflow neither engine has. Same
treatment as
cc-path-traversalandcc-generic-catch, so it feeds/security-audittriage rather than gating merges. This is a precisionregression against clj-holmes, which had a
pattern-notfor thesetFeaturecalls, and it is recorded in the README.
cc-weak-cryptoandcc-insecure-tlsusepattern-regexfor algorithmand protocol names, because semgrep does not bind metavariables inside Clojure
string literals — the same limitation that shaped
cc-shell-exec. Textualmatching can fire inside a comment. Also in the README limitations.
correctness/schema-require-typois lost. Non-security, clj-kondo territory.Verification
metadata.classvalues cross-check clean against theclojure-securityskill index — the check that caught the
cc-shell-exec/dynamic-evaldriftrules add no noise to the repo that exposed the crash
Seven scanners, 10 jobs.
v1needs retagging after merge.🤖 Generated with Claude Code