Skip to content

feat(mapping): ATR cross-reference for CCC GenAI threats - #1168

Open
eeee2345 wants to merge 1 commit into
finos:mainfrom
eeee2345:feat/atr-threats-mapping
Open

feat(mapping): ATR cross-reference for CCC GenAI threats#1168
eeee2345 wants to merge 1 commit into
finos:mainfrom
eeee2345:feat/atr-threats-mapping

Conversation

@eeee2345

@eeee2345 eeee2345 commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

Adds a standalone mapping document relating CCC GenAI platform threats to Agent Threat Rules (ATR) detection rules, in the layout #1129 introduced.

Rebuilt as a single commit on current main. The branch previously carried four commits, the first of which added inline external-mappings to threats.yaml — the field #1129 removed — so replaying it conflicted even though the merge result did not. The net change is unchanged: one new file, +112 lines, nothing else touched.

What this adds

catalogs/ai-ml/gen-ai/mappings/threats-agent-threat-rules.yaml — 25 references across seven threats, drawn from 24 distinct ATR rules:

Threat Refs
TH01 Prompt Injection 3
TH02 Data Poisoning 4
TH03 Sensitive Information Disclosure 2
TH04 Insecure / Unreliable Model Output 2
TH06 Unintended Action by a Model-Based Agent 5
TH07 Insecure Plugin 6
TH08 Model Tampering 3

TH05 (Model Overreliance), TH09 (Lack of Explainability) and TH10 (Model Version Drift) are deliberately unmapped. ATR is runtime attack detection and has no genuine coverage for those governance and quality threats; mapping them would assert coverage that does not exist.

target-reference.entry-type is Control rather than Vector, matching threats-ccm-v4.yaml. ATLAS, CWE and the OWASP LLM Top 10 are attack taxonomies; ATR rules are detections.

The mapping is one-way and lossy — an ATR rule id indicates detection coverage adjacent to a threat, not an equivalence. Same caveat as the metadata note in #986.

Verification

Re-run today against current main rather than relying on the earlier green, with the toolchain this repo pins (cue v0.16.0, gemara v1.2.0):

delivery-toolkit compile --build-target ai-ml/gen-ai --type threats   → ok
cue vet -d '#ThreatCatalog'  github.com/gemaraproj/gemara@v1.2.0 …    → ok
delivery-toolkit compile --build-target ai-ml/gen-ai --type mappings  → 5 documents
cue vet -d '#MappingDocument' github.com/gemaraproj/gemara@v1.2.0 …   → all 5 pass

Every ATR rule id referenced was re-checked against the current published ruleset (3.5.11): all 24 exist, none is deprecated. They were last verified when this PR was opened, and rules do get deprecated, so the check was repeated rather than assumed.

One thing worth flagging separately

The Compile and validate touched catalog YAMLs job loops over capabilities threats controls only. delivery-toolkit already supports --type mappings, but CI never invokes it, so no mapping document under catalogs/*/*/mappings/ is schema-validated — including the four that #1129 merged.

Nothing is currently wrong: I ran the missing step against all five documents in this catalog and they all pass. It is a latent gap rather than a live failure, and adding mappings to that asset loop would close it. Happy to send that as its own PR if it is wanted; it is out of scope here.

@eeee2345
eeee2345 requested a review from a team as a code owner July 10, 2026 22:54
@netlify

netlify Bot commented Jul 10, 2026

Copy link
Copy Markdown

Deploy Preview for common-cloud-controls canceled.

Name Link
🔨 Latest commit 6981353
🔍 Latest deploy log https://app.netlify.com/projects/common-cloud-controls/deploys/6a7adfb141003d0008af0897

@eeee2345

Copy link
Copy Markdown
Contributor Author

Note: the yaml-lint failure is pre-existing on main — all ten flagged lines are in catalogs/ai-ml/gen-ai/controls.yaml (long remarks from #986), none in this PR. The threats.yaml change here is yamllint-clean (all added lines <= 120 chars). Happy to send a separate PR wrapping the pre-existing controls.yaml remarks if that's useful.

@eeee2345

Copy link
Copy Markdown
Contributor Author

Updated: rather than a separate PR, I've wrapped the ten pre-existing over-length controls.yaml remarks (from #986) into folded plain scalars in this PR (second commit). The parsed values are unchanged — delivery-toolkit compile and cue vet still pass for both controls and threats — and yaml-lint is now clean repo-wide.

@eddie-knight

Copy link
Copy Markdown
Collaborator

Hey @eeee2345 — Sorry for remaining quiet on this one. We've been slowed down by a few housekeeping issues, especially a CI overhaul which has been a thorn for reviewers recently.

I suspect that it will be good to address this proposal after we close out #1129, which aims to upgrade this repo to the latest Gemara approach for external mappings.

@eeee2345

Copy link
Copy Markdown
Contributor Author

@eddie-knight — thanks for the context, and no apology needed; a CI overhaul is a real reason.

Sequencing behind #1129 makes sense to me. This PR is external mapping data, so it is exactly the shape #1129 is changing — rebasing it onto the current format now would just be work I would throw away once the Gemara approach lands. I will leave it as is rather than churn the diff, and rework it against the new format when #1129 closes.

No action needed from you in the meantime. Ping me when #1129 is in and I will bring this up to the new shape.

@zigmax

zigmax commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

This branch has conflicts ...

Adds a standalone mapping document relating CCC GenAI platform threats to
Agent Threat Rules detection rules, in the layout finos#1129 introduced for
external mappings.

25 references across seven threats (TH01, TH02, TH03, TH04, TH06, TH07,
TH08), drawn from 24 distinct ATR rules.

TH05 (Model Overreliance), TH09 (Lack of Explainability) and TH10 (Model
Version Drift) are deliberately unmapped. ATR is runtime attack detection
and has no genuine coverage for those governance and quality threats;
mapping them would assert coverage that does not exist.

target-reference entry-type is Control rather than Vector, matching
threats-ccm-v4.yaml: ATLAS, CWE and the OWASP LLM Top 10 are attack
taxonomies, while ATR rules are detections.

The mapping is one-way and lossy — an ATR rule id indicates detection
coverage adjacent to a threat, not an equivalence.
@eeee2345
eeee2345 force-pushed the feat/atr-threats-mapping branch from 13661ce to 6981353 Compare August 11, 2026 08:39
@eeee2345

Copy link
Copy Markdown
Contributor Author

@eddie-knight @zigmax#1129 landed on 23 Jul, which was the sequencing you asked me to wait for, so this is unblocked. I have rebuilt the branch on current main and rewritten the description, which had gone stale in a way worth naming: it still described adding inline external-mappings, the field #1129 removed.

@zigmax — the conflict you saw on 20 Jul is gone. Worth recording why, because the signal was confusing: GitHub reported the PR mergeable while a rebase conflicted. The branch carried four commits, the first of which edited threats.yaml in the old inline format, so replaying it hit #1129's removal head-on even though the merge result was clean. It is now one commit adding one file, so there is nothing left to conflict with.

Re-verified today rather than trusting the green from July, since #1129 changed the shape this file lives in and a green result does not survive that. With the pinned toolchain (cue v0.16.0, gemara v1.2.0):

  • delivery-toolkit compile --type threats for ai-ml/gen-ai, then cue vet -d '#ThreatCatalog' — passes
  • all 24 referenced ATR rule ids re-checked against the published ruleset — all exist, none deprecated

A gap I ran into, which is not this PR's problem

While reproducing CI locally I found that the Compile and validate touched catalog YAMLs job iterates capabilities threats controls and never mappings. delivery-toolkit does support --type mappings — it is just not called. The effect is that no document under catalogs/*/*/mappings/ is schema-validated by CI, including the four #1129 merged.

So I ran the step that is missing:

delivery-toolkit compile --build-target ai-ml/gen-ai --type mappings --catalogs-dir catalogs --output-dir out --version v0.0.0-ci
# compiled ai-ml/gen-ai mappings (5 document(s))

for f in out/ai-ml/gen-ai/mappings/*.yaml; do
  cue vet -d '#MappingDocument' github.com/gemaraproj/gemara@v1.2.0 "$f"
done

All five pass — threats-agent-threat-rules (this PR), threats-finos-aigf, threats-mitre-atlas, threats-owasp-llm-top10, threats-saif. So this is a latent gap rather than a live failure: nothing is broken today, but nothing would catch it if it were, and #1129's own note that it "does not yet address publication of the new artifact type" suggests the artifact is still only half-wired.

Adding mappings to that asset loop looks like a small change. I have not touched it here because it is unrelated to this PR and belongs to whoever owns the workflow — say the word and I will send it separately.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants