Create cflite_pr.yml for basic ClusterFuzzLite functionality - #13
Closed
ikaruscareer wants to merge 2 commits into
Closed
Create cflite_pr.yml for basic ClusterFuzzLite functionality#13ikaruscareer wants to merge 2 commits into
ikaruscareer wants to merge 2 commits into
Conversation
ikaruscareer
added a commit
that referenced
this pull request
Aug 13, 2026
…nitisation, workflow hardening) - Fix suppressed findings incorrectly affecting numeric score (#1) - Remove duplicate CC_HOOK_SHELL_EXEC category mapping (#2) - Add concurrency group and timeout-minutes to community-scan (#3, #4) - Document private artifact visibility limitation (#5) - Sanitize safeai_score in public summaries (#6) - Replace overly broad base64 pattern with explicit token patterns (#7, #8) - Redact secrets from file paths in sanitize_location (#9) - Add _severity_index/_safe_line helpers for safe sort keys (#10, #11) - Add scorecard JSON schema document (#12) - Document GITHUB_TOKEN scope in workflow (#13) - Remove unused security-events: write permission (#14) - Sanitize set_output values against newline injection (#15)
ikaruscareer
added a commit
that referenced
this pull request
Aug 13, 2026
* feat: add SafeAI Security Scorecard and Community Scan programme - Add deterministic scorecard engine (safeai/scorecard.py) with JSON, Markdown, and GitHub job-summary writers; schema version 1. - Extend CLI scan flags (--scorecard, --scorecard-json, --scorecard-md, --scorecard-summary, --scorecard-fail-under) and wire into postprocess. - Harden GitHub Action: scorecard-summary maps to GITHUB_STEP_SUMMARY instead of literal 'true'; validate no control chars; numeric fail-under. - Add SafeAI Community Scan programme: targets.yml, methodology, disclosure policy, report schema, Jinja2 templates, and sanitise/manifest/render/ validation scripts. - Add community-scan.yml (read-only matrix scan, pinned SHAs, private vs public artifact separation, no external publication) and validate-community-scan.yml (manifest, workflow-safety, sanitisation). - Add tests for scorecard (48) and community-scan sanitisation/disclosure. - .gitignore: exclude generated scorecard artifacts and reports dir. * chore: implement SafeAI community research workflow v1 This commit implements the full SafeAI community research programme: Infrastructure: - community-scans/targets.yml: 5-target allowlist (n8n, langchain, crewai, llama-index, langgraph) - community-scans/requirements.in + requirements.txt: pip-compile hash-locked deps (4 direct + transitive) - community-scans/targets-schema.json + report-schema.json: JSON schema validation for targets and provenance manifests Core SafeAI Action: - safeai/scorecard.py: deterministic 0-10 scoring engine with JSON/Markdown/summary writers - safeai/cmd/cli.py: --scorecard, --scorecard-json, --scorecard-summary, --scorecard-fail-under flags - safeai/cmd/postprocess.py: scorecard generation, fail-under gate, output lifecycle - action.yml: input/output contract (scorecard*, scorecard-path, safeai-version) - scripts/safeai-action.py: driver with version provenance, scorecard summary mapping Scan Pipeline: - scripts/resolve_meta.py: resolves target ref to pinned SHA via GitHub API - scripts/build_scan_manifest.py: generates provenance manifest per report-schema.json - scripts/sanitise_report.py: hardened redaction (control chars, HTML, URLs, dangerous schemes, Markdown metacharacters, paths, truncation) - scripts/render_reddit_draft.py: Jinja2 template rendering merging manifest provenance fields Workflows: - .github/workflows/community-scan.yml: read-only matrix scan with pinned SHAs (ikacarere/SafeAI@full-sha), ref resolution, artifact separation (private vs public), failure propagation, artifact uploads - .github/workflows/validate-community-scan.yml: manifest/schema/dependency validation, safety checks (exact security-policy hostname match, no token to HTML), test deps Security & Safety: - Exact security-policy hostname matching (not prefix substrings) - No Authorization header sent to HTML pages - --fail-on-missing-policy enforced - Sanitization strips javascript:, data:, vbscript: URIs; escapes Markdown; truncates to 280 chars - Safety heuristic refinements: checks target-src specifically, not loose "target" substring - Hash-locked deps installed with --require-hashes in both workflows Test & Verification: - 446 tests passed, 1 skipped across full suite - 17 community-san test cases - Ruff clean across all directories - YAML/JSON/schema validation passed - Hash-locked requirements dry-run verified Provenance: manifest records resolved SHA, requested ref, SafeAI version, action commit, disclosure status — validated against report-schema.json and rendered into Reddit/draft/public summaries. No automated external publication: no Reddit API calls, no fork/issue/PR creation in target repos, no code execution from scanned repos. * fix(ci): resolve 7 failing PR checks - validate-community-scan.yml / community-scan.yml: replace invalid actions/setup-python SHA (a26af69...) with the correct v5.3.0 SHA (0b93645e9fea7318ecaed2b359559ac225c90a2b) so the validate jobs no longer fail to resolve the action. - validate-community-scan.yml: drop --fail-on-missing-policy so a target's unreachable GitHub security-policy URL is recorded for human review rather than hard-failing the manifest validation job. - requirements-dev.txt: add jinja2 + jsonschema (and transitive deps) to the dev lockfile so the main pytest suite (tests/) and the GitHub Action test suite can import the community-scan test modules. - community-scans/requirements.txt + requirements.in: bump urllib3 2.2.2 -> 2.7.0 to clear the high-severity advisories flagged by Dependency Review (GHSA-gm62-xv2j-4w53, GHSA-2xpw-w6gg-jr37, GHSA-38jv-5279-wg99, GHSA-qccp-gfcp-xxvc, etc.). * fix(deps): pin typing-extensions and bump jinja2 to clear CI failures - requirements.in / pyproject.toml: explicitly pin typing-extensions>=4.4.0. pip-compile ran on Python 3.14 omitted it (referencing only needs it on Python < 3.13), but CI runs on 3.12 where referencing==0.37.0 requires it; with --require-hashes the missing unpinned package broke the install. - requirements.in: bump jinja2 3.1.4 -> 3.1.6 to clear the moderate sandbox-breakout advisories (GHSA-q2x7-8rv6-6q7h, GHSA-gmj6-6f8f-6699, GHSA-cpwx-vrp4-4pq7) flagged by Dependency Review. - Regenerate requirements-dev.txt and community-scans/requirements.txt with the corrected, hash-locked pins. * fix(validate-targets): detect public repos via 'private' field The GitHub REST API repo object has no top-level 'public' field, so data.get('public') was always falsy and every target was wrongly reported as 'not public', failing the manifest validation job. Check 'private' instead (and tolerate a 'visibility' string). Add unit tests covering public/private resolution with a mocked API. * chore: bump version to 1.6.0 * fix(ci): make action-test hermetic by skipping PyPI install of unpublished version * fix(ci): restore local wheel install before running action-test * refactor: address architectural review (version source of truth, hermetic install, output/validation hardening) * fix(ci): derive action-test version from package source of truth * fix: address architectural review findings (scorecard correctness, sanitisation, workflow hardening) - Fix suppressed findings incorrectly affecting numeric score (#1) - Remove duplicate CC_HOOK_SHELL_EXEC category mapping (#2) - Add concurrency group and timeout-minutes to community-scan (#3, #4) - Document private artifact visibility limitation (#5) - Sanitize safeai_score in public summaries (#6) - Replace overly broad base64 pattern with explicit token patterns (#7, #8) - Redact secrets from file paths in sanitize_location (#9) - Add _severity_index/_safe_line helpers for safe sort keys (#10, #11) - Add scorecard JSON schema document (#12) - Document GITHUB_TOKEN scope in workflow (#13) - Remove unused security-events: write permission (#14) - Sanitize set_output values against newline injection (#15) * test: add schema validation tests for scorecard JSON output * fix: harden scorecard against malformed input and correct schema drift - compute_score: use _safe_line for line key so a non-numeric line value cannot crash scoring (previously hardened elsewhere but missed here) - build_scorecard: guard fail_on threshold with fail-closed fallback so an unrecognised fail-on value blocks any active finding instead of raising ValueError or failing open - scorecard-schema.json: replace phantom 'outcome' policy field with the actual emitted fields (fail_on, fail_on_new, fail_on_escalation, scorecard_fail_under); add additionalProperties:false and tightened required/top_findings so future drift fails validation - add regression tests for both crash fixes and schema drift guard
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.