Skip to content
5 changes: 3 additions & 2 deletions .claude/skills/agentic-contribution-skill/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -244,8 +244,9 @@ mkdir -p <pack>/skills/<skill-name>/docs/
- `docs/external-resources.md` - Any external docs/links/KB articles mentioned by user
3. **Update <pack>/AGENTS.md**: Add intent routing entry
4. **Create <pack>/mcps.json**: If new MCP server needed (use `${ENV_VAR}` format)
5. **Update marketplace/rh-agentic-collection.yml** in [agentic-catalog](https://github.com/RHEcosystemAppEng/agentic-catalog): If new pack (register pack for Lola installation)
6. **Create pack structure**: If new pack (README.md, AGENTS.md, skills/ directory)
5. **Compass manifests**: Run **compass-manifest-maintenance** (`.claude/skills/compass-manifest-maintenance/`) for registered packs — skill `catalog-info.yaml`, Location targets, bidirectional `dependsOn`/`dependencyOf` on plugin and MCP manifests
6. **Update marketplace/rh-agentic-collection.yml** in [agentic-catalog](https://github.com/RHEcosystemAppEng/agentic-catalog): If new pack (register pack for Lola installation)
7. **Create pack structure**: If new pack (README.md, AGENTS.md, skills/ directory)

Generate SKILL.md following the mandatory section template in SKILL_DESIGN_PRINCIPLES.md (already consulted in Phase 3). If SKILL.md becomes too long, move detailed content to `docs/` with references in main file.

Expand Down
203 changes: 203 additions & 0 deletions .claude/skills/compass-manifest-maintenance/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,203 @@
---
name: compass-manifest-maintenance
description: |
Author and maintain Compass catalog-info.yaml manifests for agentic packs (skills, plugins, Locations, MCP inverse relations). Use when:
- Adding or updating a skill and its Compass manifest
- Registering an existing pack in Compass (manifests only; not creating a new agentic pack)
- MCP usage or orchestration changed in SKILL.md
- Auditing bidirectional dependsOn/dependencyOf drift against repo Compass conventions

File-based only: Read/Glob/Grep/Bash. For `.catalog/` marketplace metadata use create-collection.

NOT for: `.catalog/` metadata (use create-collection) or automated Compass registration.
model: inherit
color: magenta
license: Apache-2.0
allowed-tools: Read Glob Grep Bash
---

# /compass-manifest-maintenance Skill

**Audience:** Maintainers updating `catalog-info.yaml` manifests for Red Hat Compass (Backstage) registration.

**Goal:** Keep Compass manifests aligned with golden sources (`SKILL.md`, `mcps.json`, `AGENTS.md`)

## Prerequisites

**Required MCP Servers:** None — file-based skill. Do not use Compass MCP tools (`validate-entity`, `register-entity`, `get-catalog-entity`, `query-entity-graph`).

- Repository root as cwd.
- Read [CLAUDE.md](../../CLAUDE.md) Compass / Backstage Manifests section.
- Read [references/relationship-rules.md](references/relationship-rules.md) and [references/mcp-mapping.md](references/mcp-mapping.md).
- Templates: [assets/](assets/).

**Verification:**
```bash
test -f CLAUDE.md && echo "✓ repo root" || echo "✗ wrong directory"
```

**Human Notification Protocol:** If drift audit finds violations, report file path and fix per workflow §1.

**Security:** Never display or expose credentials or token values.

## When to Use This Skill

**Use when:**
- New skill needs `skills/<name>/catalog-info.yaml` and Location / inverse updates.
- Skill `allowed-tools` or orchestration changed → MCP or skill `dependsOn` must change.
- Register an existing pack in Compass (plugin, Location, root index, `system.yaml` manifests).
- Drift audit: skill on disk without manifest, stale `dependencyOf`, or dangling entity refs.
- After `agentic-contribution-skill` creates a skill (run before opening PR).

**Do not use when:** `.catalog/collection.yaml` (use **create-collection** instead), generic Compass platform tasks, or automated Compass registration.

## Workflow

**MCP Tool:** None — file-based; uses Read, Glob, Grep, Bash only (see `allowed-tools` in frontmatter).

**Parameters:** N/A — no MCP tools; inputs are `<pack>`, `<skill-name>`, and on-disk manifest paths.

### 1. Add or update a skill manifest

1. **Resolve** `<pack>` and `<skill-name>` — confirm `<pack>/skills/<skill-name>/SKILL.md` exists.

2. **Read golden sources** (precedence):
- `<pack>/<pack>-plugin.yaml` — `spec.lifecycle` (default for new skill manifest; see [relationship-rules.md](references/relationship-rules.md) Lifecycle)
- `SKILL.md` frontmatter: `name`, `description`, `allowed-tools`
- `SKILL.md` body: `Required MCP Servers`, `/skill-name` invocations, Dependencies, validator prerequisites
- `<pack>/mcps.json` — server keys (map via [mcp-mapping.md](references/mcp-mapping.md))
- `<pack>/AGENTS.md` — tags/disciplines hints only (not orchestration inference)

3. **Derive `dependsOn`** for the skill manifest:
- Always: `airesource:ai5-marketplace/<pack>`
- MCP: only servers the skill **actually uses** (tool prefixes + body); see [mcp-mapping.md](references/mcp-mapping.md)
- Other skills: orchestration or documented `/other-skill` invocations in `SKILL.md`
- Script-only skills: no `mcpserver:` entries

4. **Set `spec.lifecycle`** (do not hardcode `beta`):
- Read `spec.lifecycle` from `<pack>/<pack>-plugin.yaml` — use as the **default** for the skill.
- **Human in the loop:** ask whether to change it. The skill may match the plugin or use a **less mature** value only (e.g. plugin `beta` → skill `development` is OK; plugin `development` → skill `beta` is **not** allowed).
- See [relationship-rules.md](references/relationship-rules.md) Lifecycle.

5. **Write** `<pack>/skills/<skill-name>/catalog-info.yaml` from [assets/skill-catalog-info.yaml](assets/skill-catalog-info.yaml):
- `namespace: ai5-marketplace`
- `labels.distribution: external`
- `agents: []`
- `lifecycle:` value from step 4 (typically matches the plugin)
- `owner: group:redhat/ai5-marketplace`
- `backstage.io/source-location` → GitHub `main` branch SKILL.md URL
- `dependencyOf`: list orchestrators that `dependsOn` this skill (scan pack or update when editing orchestrator)

6. **Update inverse manifests** ([relationship-rules.md](references/relationship-rules.md)):
- `<pack>/catalog-info.yaml` — `./skills/<skill-name>/catalog-info.yaml` in `spec.targets`
- `<pack>/<pack>-plugin.yaml` — `dependencyOf: airesource:ai5-marketplace/<skill-name>`
- Each `mcpserver:` in skill `dependsOn` → matching `mcps/*.yaml` `dependencyOf`
- Each skill in skill `dependsOn` → that skill's `dependencyOf` includes orchestrator

7. **Reconcile plugin MCP deps** — plugin `dependsOn` = union of all `mcpserver:` refs across pack skill manifests.

### 2. Register a new pack in Compass

1. Create `<pack>/<pack>-plugin.yaml` from [assets/plugin-catalog-info.yaml](assets/plugin-catalog-info.yaml) with **`spec.lifecycle: development`** (default for new packs; confirm with user before raising maturity).
2. Create `<pack>/catalog-info.yaml` from [assets/pack-location.yaml](assets/pack-location.yaml).
3. Add `./<pack>/catalog-info.yaml` to root `catalog-info.yaml`.
4. Add `airesource:ai5-marketplace/<pack>` to `system.yaml` `spec.dependencyOf`.
5. Run **create-collection** for `<pack>/.catalog/` separately.

### 3. Drift / compliance audit

Run `uv run python scripts/validate_compass_manifests.py` (or `make validate-compass-manifests`). It enforces the same structural rules as this table:

| Check | Rule |
|-------|------|
| Roster parity | Every `skills/*/SKILL.md` has `catalog-info.yaml` and pack Location target |
| Inverse parity | Plugin `dependencyOf` = full skill set; each skill `dependsOn` includes plugin |
| MCP inverse | Each skill `mcpserver:` in `dependsOn` → MCP `dependencyOf` includes skill |
| Skill inverse | Each skill→skill `dependsOn` → target `dependencyOf` includes source |
| Dangling refs | Every ref resolves to on-disk manifest or documented canonical MCP |
| Forbidden | No `partOf`/`hasPart` on AiResource/MCPServer; no redundant `dependsOn: system:default/agentic-plugins` on plugins |
| Namespace | Refs use `ai5-marketplace` except `mcpserver:redhat/*` and `default/agentic-plugins` |

Report violations with file path and fix per workflow §1. Do not weaken checks.

### 4. Validate (file-based)

1. Compare YAML against [assets/](assets/) and a known-good manifest in the same pack.
2. Run `uv run python scripts/validate_compass_manifests.py` (included in `make validate` and `make validate-structure`).
3. Tier 1 on this skill if edited: `uv run python scripts/validate_skills_tier1.py .claude/skills/compass-manifest-maintenance/SKILL.md`.
4. Post-merge: maintainer may register in Compass UI manually (out of scope for this skill).

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

isn't there any drift that we can prevent adding a CI validation script? (we already have many iof them in the scripts/ folder)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense. I figured this was out of scope for this PR since it's just the skill and docs, but I can add a scripts/validate_compass_manifests.py script for it. Let me know if you'd rather I include it here or open a separate ticket.


**Error Handling:**
- If skill on disk has no manifest → create from `assets/skill-catalog-info.yaml` and add Location target.
- If inverse `dependencyOf` missing on plugin or MCP → update per [relationship-rules.md](references/relationship-rules.md).
- If MCP ref cannot be resolved → grep pack manifests; do not invent new `mcpserver:` refs.

## Self-review checklist

- [ ] Skill `spec.lifecycle` matches plugin default or a less mature value (never above the plugin).
- [ ] Manifest conventions: `agents: []`, `labels.distribution: external`, `namespace: ai5-marketplace`, `owner: group:redhat/ai5-marketplace`.
- [ ] MCP deps derived from `SKILL.md` usage, not copied from sibling skills.
- [ ] Every new `dependsOn` has matching `dependencyOf` on the target entity.
- [ ] Pack Location lists every skill manifest path.
- [ ] Plugin `dependencyOf` lists every skill in the pack.
- [ ] No `partOf`/`hasPart` on custom kinds.

## Dependencies

### Required MCP Servers

None — file-based manifest maintenance only.

### Required MCP Tools

None — uses Read, Glob, Grep, Bash.

### Related Skills

- **create-collection** — for `.catalog/` marketplace metadata (not Compass manifests)
- **agentic-contribution-skill** — run before this skill when creating a new skill

### Reference Documentation

- [CLAUDE.md](../../CLAUDE.md) — entity kinds, namespaces, reference formats
- `scripts/validate_compass_manifests.py` — CI roster and bidirectional ref checks
- [references/relationship-rules.md](references/relationship-rules.md)
- [references/mcp-mapping.md](references/mcp-mapping.md)

## Common Issues

- **COMPASS-1288 drift** — updated skill `dependsOn` but forgot plugin or MCP `dependencyOf`.
- **Blanket MCP deps** — declaring every MCP in `mcps.json` instead of per-skill `allowed-tools` usage.
- **Orchestration gaps** — `remediation`-style skills missing skill→skill edges or inverse `dependencyOf` on depended skills.
- **Canonical vs owned** — Lightspeed and Security use `mcpserver:redhat/...`; do not register duplicates in `mcps/`.
- **Unregistered packs** — `rh-developer`, `rh-ai-engineer`, `rh-automation` exist on disk but are not in root Location until explicitly added.

## Example usage

```bash
# CI structural validation (roster + bidirectional refs)
uv run python scripts/validate_compass_manifests.py
# or: make validate-compass-manifests

# Roster: skills on disk missing from Location (example: rh-sre)
comm -23 \
<(find rh-sre/skills -name SKILL.md | sed 's|.*/skills/||;s|/SKILL.md||' | sort) \
<(grep -oP 'skills/\K[^/]+(?=/catalog-info)' rh-sre/catalog-info.yaml | sort)

# Tier 1 lint for this maintenance skill
uv run python scripts/validate_skills_tier1.py .claude/skills/compass-manifest-maintenance/SKILL.md

# Full repo validation
make validate
```

Audit all four registered packs:

```bash
for pack in ocp-admin rh-sre rh-virt rh-basic; do
echo "=== $pack ==="
comm -23 \
<(find "$pack/skills" -name SKILL.md 2>/dev/null | sed 's|.*/skills/||;s|/SKILL.md||' | sort) \
<(grep -oP 'skills/\K[^/]+(?=/catalog-info)' "$pack/catalog-info.yaml" 2>/dev/null | sort)
done
```
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
# Template: <pack>/catalog-info.yaml (Location)
apiVersion: backstage.io/v1alpha1
kind: Location
metadata:
name: <pack>
title: <Pack Title> Agentic Pack
description: >
Points to all AI skill catalog entities in the <pack> agentic pack.
annotations:
agentic-plugins.redhat.com/version: "1.0"
agentic-plugins.redhat.com/repo: "agentic-plugins"
agentic.redhat.com/lola-module: "true"
agentic.redhat.com/lola-module-name: <pack>
agentic.redhat.com/repository: https://github.com/RHEcosystemAppEng/agentic-plugins
agentic.redhat.com/path: <pack>
spec:
targets:
- ./<pack>-plugin.yaml
- ./skills/<skill-name>/catalog-info.yaml
# List every skill manifest under skills/
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
# Template: <pack>/<pack>-plugin.yaml
# Replace <pack>, title, description, tags, disciplines, categories, dependencyOf, dependsOn (MCP union).
# New packs: spec.lifecycle defaults to development (confirm with user before raising maturity).
apiVersion: backstage.io/v1alpha1
kind: AiResource
metadata:
name: <pack>
namespace: ai5-marketplace
title: <Pack Title>
description: >
<Pack description from README or plugin manifest.>
tags:
- ai-skill
- <pack-tag>
annotations:
agentic-plugins.redhat.com/version: "1.0"
agentic-plugins.redhat.com/repo: "agentic-plugins"
agentic.redhat.com/lola-module: "true"
agentic.redhat.com/lola-module-name: <pack>
agentic.redhat.com/repository: https://github.com/RHEcosystemAppEng/agentic-plugins
agentic.redhat.com/path: <pack>
github.com/project-slug: RHEcosystemAppEng/agentic-plugins
links:
- url: https://github.com/RHEcosystemAppEng/agentic-plugins
title: Source Repository
icon: github
spec:
type: plugin

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

New skill in an existing pack: copy spec.lifecycle from the parent *-plugin.yaml. Do not hardcode lifecycle: beta in assets/skill-catalog-info.yaml, the compass-manifest-maintenance workflow, or CLAUDE.md.

New pack: assets/plugin-catalog-info.yaml and the skill template must use the same default (development is the safer new-pack value).

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@RedTanny I will take care of it in a separate PR for that ticket: https://redhat.atlassian.net/browse/APPENG-6307
Ok ?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rhartuv
sure no problem

lifecycle: development
owner: group:redhat/ai5-marketplace
system: default/agentic-plugins
disciplines:
- <discipline>
categories:
- <category>
agents: []
dependsOn:
# Union of mcpserver refs used by any skill in this pack (omit if none)
# - mcpserver:ai5-marketplace/<owned-mcp>
# - mcpserver:redhat/<canonical-mcp>
dependencyOf:
- airesource:ai5-marketplace/<skill-name>
# Every skill in the pack — bidirectional with skill dependsOn plugin
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
# Template: skills/<skill-name>/catalog-info.yaml
# Replace <pack>, <skill-name>, title, description, tags, disciplines, categories, dependsOn.
# lifecycle: copy from <pack>/<pack>-plugin.yaml spec.lifecycle (default); may downgrade only — see relationship-rules.md
apiVersion: backstage.io/v1alpha1
kind: AiResource
metadata:
name: <skill-name>
namespace: ai5-marketplace
title: <Skill Title>
description: >
<Multi-line description from SKILL.md frontmatter description, trimmed for catalog.>
labels:
distribution: external
annotations:
backstage.io/source-location: >-
url:https://github.com/RHEcosystemAppEng/agentic-plugins/blob/main/<pack>/skills/<skill-name>/SKILL.md
tags:
- ai-skill
- <domain-tag>
links:
- url: https://github.com/RHEcosystemAppEng/agentic-plugins
title: Source Repository
icon: github
spec:
type: skill
lifecycle: <plugin-lifecycle> # from <pack>/<pack>-plugin.yaml; same or less mature than plugin
owner: group:redhat/ai5-marketplace
disciplines:
- <discipline>
categories:
- <category>
agents: []
dependsOn:
- airesource:ai5-marketplace/<pack>
# Add mcpserver: and airesource: (other skills) only when SKILL.md uses them
dependencyOf: []
# dependencyOf: list orchestration skills that dependsOn this skill (COMPASS-1288)
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
# Deriving MCP `dependsOn` for skill manifests

Map **only** MCP servers the skill **actually uses**. Never copy a sibling manifest and never declare every key in `<pack>/mcps.json`.

## Step 1 — Detect MCP usage in the skill

From `<pack>/skills/<skill>/SKILL.md`:

- `allowed-tools` in frontmatter (tool names / prefixes)
- `Required MCP Servers` / `Required MCP Tools` in the body
- Workflow steps that name an MCP server key or tool

If none of the above reference MCP tools → **no** `mcpserver:` in `dependsOn` (script-only, `WebFetch`, or public API skills).

## Step 2 — Match usage to `mcps.json` keys

Read `<pack>/mcps.json`. Note which **server keys** the skill documents (e.g. `openshift-administration`, `lightspeed-mcp`, `aap-mcp-job-management`).

When `allowed-tools` lists tools but SKILL.md does not name a key, infer the server from context in the skill body (which MCP block documents those tools).

## Step 3 — Resolve each key to a Compass entity ref

For each MCP the skill uses:

### A. Owned MCP (manifest in this repo)

1. List `mcps/*.yaml` (exclude `catalog-info.yaml`).
2. Open manifests and read `metadata.name` and `metadata.namespace` (typically `ai5-marketplace`).
3. Entity ref: `mcpserver:<namespace>/<metadata.name>`.
4. **Deduplication:** Several `mcps.json` keys may share one Compass entity (same image/technology). Compare `mcps.json` entries (image URL, upstream repo in manifest `links`) to pick the correct owned manifest — do not create one ref per key if they map to the same `mcps/*.yaml`.
5. Confirm the skill’s tools appear in that manifest’s `spec.primitives` (or match the server described in SKILL.md) when ambiguous.

### B. Canonical MCP (registered upstream, not in `mcps/`)

If the pack references an MCP that **no** file under `mcps/` covers:

1. Search existing skill manifests in the same pack (or repo) for the same `mcps.json` key or the same upstream product.
2. Reuse the `mcpserver:` ref already used there (often `mcpserver:redhat/<name>`).
3. Do **not** add a duplicate MCPServer manifest in `mcps/`.

When unsure, grep the repo for the `mcps.json` key or tool prefix in other `catalog-info.yaml` files under the same pack.

## Step 4 — Update inverse relations

For each `mcpserver:` added to the skill’s `dependsOn`:

- Add `airesource:ai5-marketplace/<skill-name>` to that MCP manifest’s `spec.dependencyOf` (owned MCPs only — canonical MCPs are maintained by their owners).

## Step 5 — Plugin MCP union

The pack plugin’s `dependsOn` MCP list = **union** of all `mcpserver:` refs across skill manifests in that pack — not the full set of keys in `mcps.json`.

## Sanity checks

- Fewer MCP refs than keys in `mcps.json` is normal (per-skill usage).
- More MCP refs than the skill uses is wrong (blanket template deps).
- Canonical refs (`mcpserver:redhat/...`) do not get new files under `mcps/`.
Loading
Loading