feat(mapping): ATR cross-reference for CCC GenAI threats - #1168
Conversation
✅ Deploy Preview for common-cloud-controls canceled.
|
|
Note: the |
|
Updated: rather than a separate PR, I've wrapped the ten pre-existing over-length controls.yaml remarks (from #986) into folded plain scalars in this PR (second commit). The parsed values are unchanged — delivery-toolkit compile and cue vet still pass for both controls and threats — and yaml-lint is now clean repo-wide. |
|
Hey @eeee2345 — Sorry for remaining quiet on this one. We've been slowed down by a few housekeeping issues, especially a CI overhaul which has been a thorn for reviewers recently. I suspect that it will be good to address this proposal after we close out #1129, which aims to upgrade this repo to the latest Gemara approach for external mappings. |
|
@eddie-knight — thanks for the context, and no apology needed; a CI overhaul is a real reason. Sequencing behind #1129 makes sense to me. This PR is external mapping data, so it is exactly the shape #1129 is changing — rebasing it onto the current format now would just be work I would throw away once the Gemara approach lands. I will leave it as is rather than churn the diff, and rework it against the new format when #1129 closes. No action needed from you in the meantime. Ping me when #1129 is in and I will bring this up to the new shape. |
|
This branch has conflicts ... |
Adds a standalone mapping document relating CCC GenAI platform threats to Agent Threat Rules detection rules, in the layout finos#1129 introduced for external mappings. 25 references across seven threats (TH01, TH02, TH03, TH04, TH06, TH07, TH08), drawn from 24 distinct ATR rules. TH05 (Model Overreliance), TH09 (Lack of Explainability) and TH10 (Model Version Drift) are deliberately unmapped. ATR is runtime attack detection and has no genuine coverage for those governance and quality threats; mapping them would assert coverage that does not exist. target-reference entry-type is Control rather than Vector, matching threats-ccm-v4.yaml: ATLAS, CWE and the OWASP LLM Top 10 are attack taxonomies, while ATR rules are detections. The mapping is one-way and lossy — an ATR rule id indicates detection coverage adjacent to a threat, not an equivalence.
13661ce to
6981353
Compare
|
@eddie-knight @zigmax — #1129 landed on 23 Jul, which was the sequencing you asked me to wait for, so this is unblocked. I have rebuilt the branch on current @zigmax — the conflict you saw on 20 Jul is gone. Worth recording why, because the signal was confusing: GitHub reported the PR mergeable while a rebase conflicted. The branch carried four commits, the first of which edited Re-verified today rather than trusting the green from July, since #1129 changed the shape this file lives in and a green result does not survive that. With the pinned toolchain (
A gap I ran into, which is not this PR's problemWhile reproducing CI locally I found that the So I ran the step that is missing: All five pass — Adding |
Adds a standalone mapping document relating CCC GenAI platform threats to Agent Threat Rules (ATR) detection rules, in the layout #1129 introduced.
Rebuilt as a single commit on current
main. The branch previously carried four commits, the first of which added inlineexternal-mappingstothreats.yaml— the field #1129 removed — so replaying it conflicted even though the merge result did not. The net change is unchanged: one new file, +112 lines, nothing else touched.What this adds
catalogs/ai-ml/gen-ai/mappings/threats-agent-threat-rules.yaml— 25 references across seven threats, drawn from 24 distinct ATR rules:TH05 (Model Overreliance), TH09 (Lack of Explainability) and TH10 (Model Version Drift) are deliberately unmapped. ATR is runtime attack detection and has no genuine coverage for those governance and quality threats; mapping them would assert coverage that does not exist.
target-reference.entry-typeisControlrather thanVector, matchingthreats-ccm-v4.yaml. ATLAS, CWE and the OWASP LLM Top 10 are attack taxonomies; ATR rules are detections.The mapping is one-way and lossy — an ATR rule id indicates detection coverage adjacent to a threat, not an equivalence. Same caveat as the metadata note in #986.
Verification
Re-run today against current
mainrather than relying on the earlier green, with the toolchain this repo pins (cue v0.16.0,gemara v1.2.0):Every ATR rule id referenced was re-checked against the current published ruleset (3.5.11): all 24 exist, none is deprecated. They were last verified when this PR was opened, and rules do get deprecated, so the check was repeated rather than assumed.
One thing worth flagging separately
The
Compile and validate touched catalog YAMLsjob loops overcapabilities threats controlsonly.delivery-toolkitalready supports--type mappings, but CI never invokes it, so no mapping document undercatalogs/*/*/mappings/is schema-validated — including the four that #1129 merged.Nothing is currently wrong: I ran the missing step against all five documents in this catalog and they all pass. It is a latent gap rather than a live failure, and adding
mappingsto that asset loop would close it. Happy to send that as its own PR if it is wanted; it is out of scope here.