Access control, warnings that count, an installer that can update (3.5.0) - #9
Conversation
…te (3.5.0) The engine was made honest in 3.4.0. This is about what it never looked at. Access Control is a new category. The tool shipped three versions with no authentication check at all, and the same afternoon an independent review of a live app found a send endpoint reachable by any signed-in user and an owner guard returning true when its env var was unset. The audit had graded that repo 'A, strong evidence of controls'. auth-3 now quotes the exact line and the grade is D. Warnings cost points. Eight checks printed a severity badge and subtracted nothing: 55 points shown but not counted. The installer can deliver a correction. It printed 'already installed' and exited 0 without comparing anything, so the going-live rule fixed in 3.2.1 could never reach anyone who installed before it. boy-scout-cleanup gets the verification discipline its sibling /refactor already had. All five prompt-only skills fixed from independent testing. 68 tests.
|
Warning Review limit reached
Next review available in: 8 minutes Limit details: You’ve used the included review currently available. You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. How can I continue?Wait for the limit to reset, then comment An organization admin can change what happens after included review limits in Billing. How do review limits work?CodeRabbit enforces per-developer PR review limits within each organization. For paid Pro and Pro+ reviews, CodeRabbit uses a developer's included PR review attempts over the past 7 days to set the current hourly allowance. At typical activity levels, the full plan allowance applies. Higher sustained activity can lower the allowance until earlier attempts leave the 7-day window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (19)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
1 participant
3.4.0 made the engine honest about what it found. This is about what it was never looking at.
1. There was no authentication check. At all.
Thirty-one checks across eight categories, and not one looked at access control. I grepped every check module for
auth,authoriz,permission,rbac,session: three hits, all stray words inside recommendation prose.The same afternoon, an independent review of a live application found:
This audit had graded that repository "A, strong evidence of controls."
New
Access Controlcategory:auth-1is there an authentication mechanism, and which oneauth-2do request handlers consult an identity, or serve anyone who reaches the URL. Reported as a text match: mentioning a guard is not being guardedauth-3does a permission check grant access when its own environment variable is unsetauth-3needed real work. My first regex looked forif (!process.env.X) return trueand found nothing, because real code reads the variable into a local first. It now learns the env-derived local names per file, then looks for the fail-open branch on them. On the live repo above it returns:That repository went from A, 90 to D, 83.
2. Fifty-five points of findings were displayed but never scored
Eight checks reported as
warnwith a severity badge, three of them MEDIUM, andCategory.scoreonly subtracted onfail. A repository could show three MEDIUM findings and score 100 in those categories. A warning now costs half of the same finding failing.3. The toolkit could not deliver its own corrections
/add-clean-codeprinted "already installed" and exited 0 without comparing anything. The going-live rule was wrong in 3.1.0 and corrected in 3.2.1 after a reviewer said "wait until launch day is too late". Anyone who installed before that correction still has the wrong rule, permanently.The managed block now carries a version. A re-run diffs it and exits 3;
--updatereplaces only what is between the markers;--checkreports without touching anything. Verified: user content above the block survives.The 3.2.1 edit also left a dangling reference ("Before that day", "On that day", after the first bullet stopped naming a day). Fixed.
4 and 5. The skills
boy-scout-cleanupclaimed "behavior-preserving" while/refactor, its sibling with identical risk, carried the discipline that makes the claim meaningful. A tester's mutation test found three of four real behavior changes passing a green suite. It now requires an undo to exist, requires proving the suite covers the behavior being touched, and requires saying which evidence was used.All five prompt-only skills were run once against a real repository by an independent tester. Every one returned "partly works". Each finding is fixed: the scaffold that invented a stack silently and failed on first run, the handoff whose empty template sections invited fiction, the review that promised to run the coach and never mentioned it, the review that never said where its output goes.
Verified