Enhance documentation extraction capabilities for code-parser - #2
Open
M97Chahboun wants to merge 1 commit into
Open
Enhance documentation extraction capabilities for code-parser#2M97Chahboun wants to merge 1 commit into
M97Chahboun wants to merge 1 commit into
Conversation
- Add support for extracting documentation comments from Python, TypeScript, and Dart. - Update README and command documentation to reflect new functionality. - Introduce `parse-docs` command for extracting structured documentation. - Modify data structures to include optional documentation fields for classes and methods.
📝 WalkthroughWalkthroughThe PR adds documentation comment extraction to the code parser, supporting Python docstrings, TypeScript JSDoc, and Dart doc comments. It extends output models with optional Changes
Sequence Diagram(s)sequenceDiagram
participant Parser as Code Parser
participant Python as Python Extractor
participant TypeScript as TypeScript Extractor
participant Dart as Dart Extractor
participant Output as Output Formatter
Parser->>Python: Parse Python AST
Python->>Python: Extract docstrings (classes/methods)
Python->>Output: Return ClassInfo/MethodInfo with doc field
Parser->>TypeScript: Parse TypeScript
TypeScript->>TypeScript: Extract JSDoc from preceding comments
TypeScript->>Output: Return ClassInfo/MethodInfo with doc field
Parser->>Dart: Tokenize Dart code
Dart->>Dart: Recognize and clean doc comments (/// and /** */)
Dart->>Output: Return ClassInfo/MethodInfo with doc field
Output->>Output: Aggregate results with documentation
Output->>Output: Serialize to JSON with doc fields
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Poem
🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
code-parser-commands/README.md (1)
41-48:⚠️ Potential issue | 🟡 MinorManual install section missing
parse-docs.md.The manual installation example lists six
.mdfiles but doesn't include the newparse-docs.mdcommand file.📝 Proposed fix to add parse-docs.md
cp index.md ~/.claude/commands/ cp parse-find.md ~/.claude/commands/ cp parse-read.md ~/.claude/commands/ +cp parse-docs.md ~/.claude/commands/ cp parse-edit.md ~/.claude/commands/ cp parse-audit.md ~/.claude/commands/ cp parse-stats.md ~/.claude/commands/🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@code-parser-commands/README.md` around lines 41 - 48, Update the manual install example in README.md to include the new parse-docs.md file: add a cp parse-docs.md ~/.claude/commands/ line alongside the existing cp index.md, parse-find.md, parse-read.md, parse-edit.md, parse-audit.md, and parse-stats.md entries so the manual install copies the new command file too.
🧹 Nitpick comments (4)
src/main.rs (3)
119-133: Raw string prefix not handled in docstring cleaning.The function strips quotes but doesn't handle raw string prefixes (
r"""..."""orr'''...'''). While uncommon for docstrings, this could leave anrcharacter at the start.♻️ Proposed fix to handle raw string prefixes
fn clean_python_docstring(s: &str) -> String { let s = s.trim(); + // Remove raw string prefix if present + let s = s.strip_prefix('r').unwrap_or(s); // Remove triple quotes let s = s.strip_prefix("\"\"\"").unwrap_or(s); let s = s.strip_prefix("\'\'\'").unwrap_or(s);🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@src/main.rs` around lines 119 - 133, clean_python_docstring currently strips triple and single quotes but ignores raw string prefixes, which can leave a leading 'r' (or 'R') before the quotes; update the function clean_python_docstring to detect and remove an optional raw-string prefix ('r' or 'R') before handling strip_prefix/strip_suffix for triple-quoted (r""" or r''') and single-quoted forms, i.e., first check for and strip a leading 'r'/'R' when it is immediately followed by quotes, then proceed with the existing triple-quote and single-quote removal and trimming so raw-prefixed docstrings are normalized correctly.
673-679: Redundantpending_doc = Noneafter.take()already consumed it.Line 677 sets
pending_doc = None, but line 673 already usedpending_doc.take()which returns the value and sets the Option toNone. This line is harmless but unnecessary.♻️ Remove redundant assignment
if let Some((method_name, start_line, end_i, doc)) = detect_dart_member(tokens, i, pending_doc.take()) { // Find the end line — either at semicolon or matching brace let end = find_member_end(tokens, end_i); methods.push(MethodInfo { name: method_name, line_start: start_line, line_end: end.0, doc }); - pending_doc = None; i = end.1; continue; }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@src/main.rs` around lines 673 - 679, The assignment pending_doc = None is redundant because pending_doc.take() in the detect_dart_member call already consumes the Option and sets it to None; remove the extra assignment by deleting the pending_doc = None line following the detect_dart_member branch (referencing detect_dart_member(...) and pending_doc.take()) so the code still sets i = end.1, pushes MethodInfo, and continues without the unnecessary reset.
998-1002: Commented-out assertions indicate a known limitation.The comments acknowledge that doc comments before methods with return types may not be captured correctly. This is a documentation of known behavior, but consider either:
- Fixing the limitation if it's a common pattern
- Adding this to the project's Limitations section in README
Would you like me to help investigate the root cause of this limitation or open an issue to track it?
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@src/main.rs` around lines 998 - 1002, The commented-out assertions around classes[0].methods.iter().find(|m| m.name == "fetchUser") indicate a known parser limitation where doc comments preceding methods with return types aren’t being attached (see fetch.doc); either fix the parser that attaches docs (search for the method that extracts member docs—e.g., the function responsible for parsing method/comments and adjust it to associate leading doc comments even when a return type token follows) or remove these commented assertions and add a clear entry to the README Limitations section documenting the behavior (referencing fetchUser and fetch.doc) and create an issue to track a future parser fix.code-parser-commands/parse-docs.md (1)
107-120: API doc generation jq filter may fail on missing docs.The jq filter at lines 114-116 uses
(.doc // "No documentation")which handles null docs, but if a class has no methods, the inner.methods[]iteration will produce no output for that class's methods section, potentially resulting in incomplete markdown.♻️ Consider adding a check for empty methods
(.classes[] | "## " + .name + " (" + .kind + ")\n\n" + (.doc // "No documentation") + "\n\n" + "### Methods\n\n" + - (.methods[] | - "#### " + .name + "\n\n" + - (.doc // "No documentation") + "\n" - ) + (if (.methods | length) > 0 then + (.methods[] | + "#### " + .name + "\n\n" + + (.doc // "No documentation") + "\n" + ) + else + "*No methods*\n" + end) )🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@code-parser-commands/parse-docs.md` around lines 107 - 120, The jq pipeline can drop the Methods section when a class has no methods because `.methods[]` yields nothing; update the filter around the "### Methods" block to handle empty or missing `.methods` (e.g., use `.methods // []` and emit a fallback like "No methods" when the list is empty, or wrap with an if/then check using `if (.methods | length) == 0 then "No methods" else (.methods[] | ... ) end`) so the "### Methods" header and a clear "No methods" message always appear; keep the existing `(.doc // "No documentation")` handling for docs.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@code-parser-commands/parse-docs.md`:
- Around line 21-34: The script incorrectly uses the reserved environment
variable ${PATH:-.} which will expand to the system executable search path;
update the command invocation in this file to use a positional or non-reserved
variable instead (e.g., replace ${PATH:-.} with ${1:-.} or ${TARGET_PATH:-.}) so
the CLI receives the intended directory argument; locate the occurrence of
${PATH:-.} in the example command and change it to the chosen safe variable name
in the code block.
- Around line 49-55: The jq pipeline loses the class name because after
.[0].classes[] | select(.name == $class) you descend into .methods[] so .name
refers to the method; capture the class name before iterating methods (e.g.,
bind the matched class to a variable with as $cls or use the existing $class
input) and then inside .methods[] | select(.name == $method) construct the
object using the captured class (use $cls or $class for the class field) and the
method's .name and .doc so {class: $cls, method: .name, doc: .doc}.
---
Outside diff comments:
In `@code-parser-commands/README.md`:
- Around line 41-48: Update the manual install example in README.md to include
the new parse-docs.md file: add a cp parse-docs.md ~/.claude/commands/ line
alongside the existing cp index.md, parse-find.md, parse-read.md, parse-edit.md,
parse-audit.md, and parse-stats.md entries so the manual install copies the new
command file too.
---
Nitpick comments:
In `@code-parser-commands/parse-docs.md`:
- Around line 107-120: The jq pipeline can drop the Methods section when a class
has no methods because `.methods[]` yields nothing; update the filter around the
"### Methods" block to handle empty or missing `.methods` (e.g., use `.methods
// []` and emit a fallback like "No methods" when the list is empty, or wrap
with an if/then check using `if (.methods | length) == 0 then "No methods" else
(.methods[] | ... ) end`) so the "### Methods" header and a clear "No methods"
message always appear; keep the existing `(.doc // "No documentation")` handling
for docs.
In `@src/main.rs`:
- Around line 119-133: clean_python_docstring currently strips triple and single
quotes but ignores raw string prefixes, which can leave a leading 'r' (or 'R')
before the quotes; update the function clean_python_docstring to detect and
remove an optional raw-string prefix ('r' or 'R') before handling
strip_prefix/strip_suffix for triple-quoted (r""" or r''') and single-quoted
forms, i.e., first check for and strip a leading 'r'/'R' when it is immediately
followed by quotes, then proceed with the existing triple-quote and single-quote
removal and trimming so raw-prefixed docstrings are normalized correctly.
- Around line 673-679: The assignment pending_doc = None is redundant because
pending_doc.take() in the detect_dart_member call already consumes the Option
and sets it to None; remove the extra assignment by deleting the pending_doc =
None line following the detect_dart_member branch (referencing
detect_dart_member(...) and pending_doc.take()) so the code still sets i =
end.1, pushes MethodInfo, and continues without the unnecessary reset.
- Around line 998-1002: The commented-out assertions around
classes[0].methods.iter().find(|m| m.name == "fetchUser") indicate a known
parser limitation where doc comments preceding methods with return types aren’t
being attached (see fetch.doc); either fix the parser that attaches docs (search
for the method that extracts member docs—e.g., the function responsible for
parsing method/comments and adjust it to associate leading doc comments even
when a return type token follows) or remove these commented assertions and add a
clear entry to the README Limitations section documenting the behavior
(referencing fetchUser and fetch.doc) and create an issue to track a future
parser fix.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: ecad1496-cc27-4960-9164-7bfc6a9ad6a9
📒 Files selected for processing (6)
README.mdcode-parser-commands/README.mdcode-parser-commands/index.mdcode-parser-commands/install-commands.shcode-parser-commands/parse-docs.mdsrc/main.rs
Comment on lines
+21
to
+34
| ```bash | ||
| # All docs from path | ||
| code-parser ${PATH:-.} --format json | jq ' | ||
| .[] | { | ||
| file: .file, | ||
| language: .language, | ||
| classes: [.classes[] | select(.doc != null) | { | ||
| name: .name, | ||
| kind: .kind, | ||
| doc: .doc, | ||
| methods: [.methods[] | select(.doc != null) | {name, doc}] | ||
| }] | ||
| } | select(.classes | length > 0) | ||
| ' |
There was a problem hiding this comment.
Critical: ${PATH:-.} will use system PATH variable, not the command argument.
Line 23 uses ${PATH:-.} but PATH is a reserved system environment variable containing executable search paths. This will produce unexpected results. Use a different variable name or $1.
🐛 Proposed fix
```bash
# All docs from path
-code-parser ${PATH:-.} --format json | jq '
+code-parser ${1:-.} --format json | jq '
.[] | {
file: .file,Or use a more descriptive variable like ${TARGET_PATH:-.} with appropriate setup.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@code-parser-commands/parse-docs.md` around lines 21 - 34, The script
incorrectly uses the reserved environment variable ${PATH:-.} which will expand
to the system executable search path; update the command invocation in this file
to use a positional or non-reserved variable instead (e.g., replace ${PATH:-.}
with ${1:-.} or ${TARGET_PATH:-.}) so the CLI receives the intended directory
argument; locate the occurrence of ${PATH:-.} in the example command and change
it to the chosen safe variable name in the code block.
Comment on lines
+49
to
+55
| # Specific method doc | ||
| code-parser FILE --format json | jq --arg class "$CLASSNAME" --arg method "$METHODNAME" ' | ||
| .[0].classes[] | select(.name == $class) | | ||
| .methods[] | select(.name == $method) | | ||
| {class: .name, method: .name, doc: .doc} | ||
| ' | ||
| ``` |
There was a problem hiding this comment.
Incorrect field reference in method doc extraction.
Line 53 creates {class: .name, method: .name, doc: .doc} but at this point in the jq pipeline, we're inside a method object (after select(.name == $method)). The .name here refers to the method's name, not the class name. The class name is lost.
🐛 Proposed fix to capture class name
# Specific method doc
code-parser FILE --format json | jq --arg class "$CLASSNAME" --arg method "$METHODNAME" '
- .[0].classes[] | select(.name == $class) |
- .methods[] | select(.name == $method) |
- {class: .name, method: .name, doc: .doc}
+ .[0].classes[] | select(.name == $class) |
+ . as $cls |
+ .methods[] | select(.name == $method) |
+ {class: $cls.name, method: .name, doc: .doc}
'📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| # Specific method doc | |
| code-parser FILE --format json | jq --arg class "$CLASSNAME" --arg method "$METHODNAME" ' | |
| .[0].classes[] | select(.name == $class) | | |
| .methods[] | select(.name == $method) | | |
| {class: .name, method: .name, doc: .doc} | |
| ' | |
| ``` | |
| # Specific method doc | |
| code-parser FILE --format json | jq --arg class "$CLASSNAME" --arg method "$METHODNAME" ' | |
| .[0].classes[] | select(.name == $class) | | |
| . as $cls | | |
| .methods[] | select(.name == $method) | | |
| {class: $cls.name, method: .name, doc: .doc} | |
| ' |
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@code-parser-commands/parse-docs.md` around lines 49 - 55, The jq pipeline
loses the class name because after .[0].classes[] | select(.name == $class) you
descend into .methods[] so .name refers to the method; capture the class name
before iterating methods (e.g., bind the matched class to a variable with as
$cls or use the existing $class input) and then inside .methods[] | select(.name
== $method) construct the object using the captured class (use $cls or $class
for the class field) and the method's .name and .doc so {class: $cls, method:
.name, doc: .doc}.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
parse-docscommand for extracting structured documentation.Summary by CodeRabbit
Release Notes
New Features
/parse-docscommand to extract and display documentation comments from source code/indexcommand to include documentation in output for documented classes and methodsDocumentation