feat(workstream-c): add cheatsheet categorizer and grouping

[shreeshtripurwarcomp23-coder]

• Implement categorize_cheatsheet() with 29-label controlled taxonomy
• Implement group_cheatsheets() with stable sha256-based group IDs
• Deterministic keyword/rule baseline, no LLM dependency
• LLM-optional path with safe fallback on failure
• Validate all CheatsheetRecord fields in post_init
• 50 tests covering all acceptance criteria from RFC Issue C

[coderabbitai]

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

• @coderabbitai resume to resume automatic reviews.
• @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

• ▶️ Resume reviews
• 🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yml

Review profile: CHILL

Plan: Pro

Run ID: af50adff-c2d1-4dfd-aa58-5c95b33567c7

📥 Commits

Reviewing files that changed from the base of the PR and between 2592552 and 6ca2839.

📒 Files selected for processing (1)

• application/utils/external_project_parsers/parsers/cheatsheet_categorizer.py

🚧 Files skipped from review as they are similar to previous changes (1)

• application/utils/external_project_parsers/parsers/cheatsheet_categorizer.py

---

Summary by CodeRabbit

• New Features
  • Added Workstream C cheatsheet categorization with a controlled label vocabulary and an “uncategorized” fallback.
  • Implemented deterministic label extraction from title/headings and category hints, with an opt-in AI-assisted path and safe fallback when AI output is unusable.
  • Added grouping of cheatsheets by canonical label sets, including stable group IDs and consistent ordering.
• Tests
  • Added end-to-end and unit tests covering taxonomy rules, deterministic outputs, fallback behavior, AI-path handling, and grouping semantics.

Walkthrough

Adds a cheatsheet categorizer with controlled taxonomy labels, deterministic and LLM-assisted categorization, stable grouping by label set, and a unittest suite covering taxonomy integrity, categorization, validation, and grouping.

Changes

Cheatsheet Categorization System

Layer / File(s)	Summary
Taxonomy and group contract application/utils/external_project_parsers/parsers/cheatsheet_categorizer.py, application/tests/test_cheatsheet_categorizer.py	Defines TAXONOMY, UNCATEGORIZED, and deterministic group IDs; tests verify taxonomy integrity and group ID stability.
Categorization helpers and validation application/utils/external_project_parsers/parsers/cheatsheet_categorizer.py, application/tests/test_cheatsheet_categorizer.py	Implements deterministic matching, LLM validation, and fallback behavior; tests cover mappings, uncategorized results, LLM outcomes, and label validation.
Grouping by label set application/utils/external_project_parsers/parsers/cheatsheet_categorizer.py, application/tests/test_cheatsheet_categorizer.py	Groups records by canonical label sets and returns sorted groups; tests cover membership, stable IDs, empty input, and direct execution.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Suggested reviewers: northdpole, pa04rth, paoga87, robvanderveer

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 42.42% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly summarizes the main change: adding cheatsheet categorization and grouping.
Description check	✅ Passed	The description is directly related to the implemented categorizer, grouping, fallback behavior, and tests.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@application/tests/test_cheatsheet_categorizer.py`:
- Around line 42-48: The _make_record helper function creates CheatsheetRecord
instances with an empty summary string, but CheatsheetRecord.__post_init__
enforces that summary must be non-empty, causing construction to fail before
test assertions run. Replace the empty string assignment `summary=""` in the
_make_record function with a minimal non-empty placeholder value (such as a
single space, period, or descriptive placeholder text like "Test summary") to
satisfy the validation requirement.

In
`@application/utils/external_project_parsers/parsers/cheatsheet_categorizer.py`:
- Around line 194-210: The __post_init__ method in CheatsheetRecord currently
validates only string-typed fields but does not validate the element types of
list-typed fields (headings and category_hints). This causes runtime crashes
when non-string items reach code expecting to call " ".join() on these fields.
Add validation in __post_init__ to ensure that headings and category_hints are
lists containing only string elements, raising a ValueError with a clear message
if any element is not a string, so that parser input validation fails fast at
construction time rather than later during string joining operations.
- Around line 366-381: The `_validate_labels` function currently allows
`uncategorized` to be returned alongside other valid labels, which violates the
sentinel semantics where `uncategorized` should only be returned when it is the
sole valid label. Modify the function to add logic after building the deduped
list: if `uncategorized` (or the appropriate constant reference from TAXONOMY)
is present in the result AND there are other valid labels alongside it, remove
the `uncategorized` entry from the returned list. This ensures that
`uncategorized` is only returned when no other categories match, preserving its
role as a fallback indicator and preventing inconsistent downstream grouping and
UX.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yml

Review profile: CHILL

Plan: Pro

Run ID: ca9bd7d2-9738-4eb1-9bf6-5c1b32faf0db

📥 Commits

Reviewing files that changed from the base of the PR and between e853cd3 and f91b995.

📒 Files selected for processing (2)

• application/tests/test_cheatsheet_categorizer.py
• application/utils/external_project_parsers/parsers/cheatsheet_categorizer.py

[coderabbitai]

Actionable comments posted: 1

♻️ Duplicate comments (1)

application/utils/external_project_parsers/parsers/cheatsheet_categorizer.py (1)

328-345: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Return validated LLM labels in canonical order.

categorize_cheatsheet() documents a sorted result, but _validate_labels() preserves the LLM’s order. That makes the LLM path non-deterministic for the same label set and breaks the public contract.

Suggested fix

     real = [lbl for lbl in deduped if lbl != UNCATEGORIZED]
-    return real if real else deduped
+    return sorted(real if real else deduped)

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@application/utils/external_project_parsers/parsers/cheatsheet_categorizer.py`
around lines 328 - 345, The `_validate_labels` helper in
`cheatsheet_categorizer.py` preserves the LLM’s input order, which conflicts
with the documented sorted output from `categorize_cheatsheet()`. Update
`_validate_labels()` to return approved labels in the canonical `TAXONOMY` order
after filtering and deduping, while still stripping `UNCATEGORIZED` when real
labels exist. Keep the fix localized to `_validate_labels` so the LLM path
matches the deterministic contract everywhere it is used.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In
`@application/utils/external_project_parsers/parsers/cheatsheet_categorizer.py`:
- Around line 280-291: `group_cheatsheets()` is currently keying the `bucket` by
the truncated `CheatsheetGroup.make_group_id(labels)` value, which can merge
distinct label sets on collision; change the grouping map to use the canonical
sorted unique labels as the key while still deriving `group_id` from
`CheatsheetGroup.make_group_id` for the created `CheatsheetGroup`. Update the
`bucket` population logic in `group_cheatsheets()` so groups are formed strictly
by label set identity, and keep `CheatsheetGroup(group_id=...,
labels=sorted(labels))` as the stored object.

---

Duplicate comments:
In
`@application/utils/external_project_parsers/parsers/cheatsheet_categorizer.py`:
- Around line 328-345: The `_validate_labels` helper in
`cheatsheet_categorizer.py` preserves the LLM’s input order, which conflicts
with the documented sorted output from `categorize_cheatsheet()`. Update
`_validate_labels()` to return approved labels in the canonical `TAXONOMY` order
after filtering and deduping, while still stripping `UNCATEGORIZED` when real
labels exist. Keep the fix localized to `_validate_labels` so the LLM path
matches the deterministic contract everywhere it is used.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yml

Review profile: CHILL

Plan: Pro

Run ID: a607e96a-00c8-4fb8-9d70-f8a4dd5baeaa

📥 Commits

Reviewing files that changed from the base of the PR and between f8472d4 and 12e041f.

📒 Files selected for processing (2)

• application/tests/test_cheatsheet_categorizer.py
• application/utils/external_project_parsers/parsers/cheatsheet_categorizer.py

🚧 Files skipped from review as they are similar to previous changes (1)

• application/tests/test_cheatsheet_categorizer.py

[Abhijeet2409]

Hey @shreeshtripurwarcomp23-coder, great work! I found a few points that might be worth looking into:

1. The categorizer uses record.category_hints as input for categorization, but Module B intentionally leaves this field empty. My understanding is that Module C is expected to categorize the records and populate these category hints. Currently, it returns the generated labels separately but does not populate the record's category_hints field.

2. The categorizer currently does not use record.summary; it only uses the title, headings, and category_hints (which are empty upstream). In cases where the title or headings provide limited context, the summary could potentially help with categorization. However, with the current simple keyword-matching approach, it may need to be handled carefully to avoid overly broad matches.

3. The CheatsheetGroup dataclass currently has no basic validation. Since it can also be used directly, invalid group_ids or labels outside the taxonomy can be passed. It might be worth adding some basic validation.

[shreeshtripurwarcomp23-coder]

@Abhijeet2409 Thank you so much! I'll look into this.

[shreeshtripurwarcomp23-coder]

@coderabbitai review

[coderabbitai]

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.