Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions IMPLEMENTATION_PLAN.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,10 @@ priorities.

### Completed maintenance

- [x] **Issue #64 — path-shaped operands after native verb chains.**
Stop the Bash and PowerShell native greedy passes before a token that
matches the shared path-shape rules. Preserve that token as a resolved
argument without a command dictionary or a public API change.
- [x] **Issue #52 — hyphenated PowerShell parameters/native options.**
Preserve internal hyphens, apply bash-compatible native
`--flag=value` splitting and path classification, keep colon binding
Expand Down
17 changes: 12 additions & 5 deletions SPEC.POWERSHELL.md
Original file line number Diff line number Diff line change
Expand Up @@ -393,14 +393,21 @@ cmdlets, so the bash greedy walk does not apply to them.

When the first token is neither cmdlet-shaped nor a known alias (`git`,
`dotnet`, `npm`, `kubectl`, `python`, ...) it is a **native command**.
Native commands reuse the bash greedy verb-chain walk (`SPEC.md` §6.1):
append the first token, then walk consecutive verb-like Word tokens,
transparently consuming flag-with-value pairs, stopping at the first
non-verb-like token, flag, operator, quoted string, or opaque token. The
verb-like predicate is the bash predicate **unchanged** (`SPEC.md` §6.1:
Native commands reuse the bash greedy verb-chain walk (`SPEC.md` §6.1).
The parser appends the first token and then walks consecutive verb-like Word
tokens. The walk transparently consumes flag-with-value pairs. It stops at a
path-shaped token, non-verb-like token, flag, operator, quoted string, or
opaque token.

The path-shape test uses `BashResolver.LooksLikePath`. Both native parsers
therefore share one boundary. The verb-like predicate is the bash predicate
**unchanged** (`SPEC.md` §6.1:
`Kind == Word`, length `[1, 64]`, first char ASCII lowercase `[a-z]`,
remaining chars `[a-z0-9._-]`).

The path-shape boundary does not apply to the first native command token.
For example, `deploy.sh status` has verb tokens `deploy.sh` and `status`.

Keeping the predicate **case-sensitive** — not relaxing it to accept an
uppercase first char — is deliberate. The leading-lowercase rule is the only
signal that stops the greedy walk at a capitalized identifier
Expand Down
25 changes: 20 additions & 5 deletions SPEC.md
Original file line number Diff line number Diff line change
Expand Up @@ -394,7 +394,8 @@ verb_chain := verb_like_word (FW_pair? verb_like_word)*
// greedy walk per §6.1; FW_pair is a
// flag-with-value pair owned by word_0
// (transparent to the walk); stops at
// the first non-verb-like token. For
// the first path-shaped or non-verb-like
// token. For
// word_0 ∈ FileVerbs, exactly 1 token.
arg := word | flag | quoted_string
flag := "-" letter+ | "--" word
Expand Down Expand Up @@ -545,9 +546,9 @@ These are **data**, not logic. Implement as `static readonly` collections.
### 6.1 Verb-chain extraction (greedy heuristic)

Per issue #27 (locked in v0.1.4-alpha), the parser does not consult a
static arity table. Instead, it walks consecutive verb-like Word tokens
from the start of the clause and stops at the first token that doesn't
look like a subcommand. This naturally scales to unknown CLIs
static arity table. It walks consecutive verb-like Word tokens from the
clause start. The walk stops before a path-shaped or non-verb-like token.
This rule naturally scales to unknown CLIs
(`freshdesk ticket list`, `kubectl get pods`, `dotnet ef migrations add`)
without curated table entries.

Expand All @@ -568,13 +569,17 @@ paths (`/`, `\`, `~`), env-var refs (`$VAR`), URLs (`://`), globs
(`* ? [`), and user-named identifiers (uppercase first char like
`InitialCreate`).

The walk also rejects a token that matches the §8 path-shape heuristic.
This rule applies even when the lexical predicate accepts the token.

#### Walk algorithm

For a clause whose first token is a Word `firstVerb`:

1. Append `firstVerb` to the verb chain (it does not need to satisfy
`IsVerbLikeToken` — bare commands like `Curl` or `_init` are still
commands).
commands). Do not apply the path-shape boundary at command position.
A command such as `deploy.sh` or `./deploy.sh` remains the first verb.
2. Iterate the remaining tokens in order. For each token `t`:
- If `t.Kind != Word`: **stop**.
- If `t` is a flag (`IsFlagWord`):
Expand All @@ -585,6 +590,8 @@ For a clause whose first token is a Word `firstVerb`:
and continue walking.
- Otherwise: **stop**.
- If `firstVerb ∈ FileVerbs`: **stop** (1-token carveout — see below).
- If `BashResolver.LooksLikePath(t.Value)`: **stop**. The argument pass
uses the same classifier and preserves the token as a path argument.
- If `!IsVerbLikeToken(t)`: **stop**.
- Otherwise: append `t.Value` to the verb chain and continue.

Expand Down Expand Up @@ -617,6 +624,10 @@ on for zone-gate evaluation.
| `kubectl get pods my-pod` | `[kubectl, get, pods, my-pod]` | `[]` |
| `aws s3 cp src dst` | `[aws, s3, cp, src, dst]` | `[]` (bare-word path args over-extract) |
| `dotnet ef migrations add InitialCreate` | `[dotnet, ef, migrations, add]` | `[InitialCreate]` (stops at uppercase) |
| `deploy.sh status` | `[deploy.sh, status]` | `[]` (command position wins) |
| `git diff install-skills.sh` | `[git, diff]` | `[install-skills.sh]` (path-shaped operand) |
| `kubectl apply deployment.yaml` | `[kubectl, apply]` | `[deployment.yaml]` (path-shaped operand) |
| `tool plugin.sh list` | `[tool]` | `[plugin.sh, list]` (path evidence wins) |
| `cat /etc/passwd` | `[cat]` | `[/etc/passwd]` (FileVerb carveout) |
| `cat README` | `[cat]` | `[README]` (FileVerb carveout preserves IsPath) |
| `ls -la /tmp` | `[ls]` | `[-la, /tmp]` (FileVerb carveout) |
Expand Down Expand Up @@ -655,6 +666,10 @@ False-negative (re-prompt) is recoverable. False-positive (silent
destructive grant) is not. Narrow-by-default favors the recoverable
failure mode.

The path-shape boundary requires no command dictionary. It uses the same
curated evidence as argument classification. A rare extension-shaped
subcommand becomes a path argument because the stronger path evidence wins.

### 6.2 CWD verbs

Verbs whose first non-flag positional arg becomes the cwd for subsequent
Expand Down
2 changes: 2 additions & 0 deletions openspec/changes/preserve-path-shaped-operands/.openspec.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
schema: spec-driven
created: 2026-08-04
71 changes: 71 additions & 0 deletions openspec/changes/preserve-path-shaped-operands/design.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
## Context

The parser builds a greedy native verb chain from lowercase word tokens. A lowercase filename can therefore leave no argument or path metadata.

Three users define the required behavior:

- A security-policy author needs a file operand and its directory scope before a persistent grant.
- An audit-tool author needs the parser's path result without a duplicate extension table.
- A private-CLI author needs unknown lowercase subcommand chains to remain intact.

The public API is locked. Both shell parsers use the same greedy native-command policy.

## Goals / Non-Goals

**Goals:**

- Preserve a path-shaped token as an argument after the command token.
- Use the existing path-shape rules as the canonical boundary.
- Keep Bash and PowerShell native-command results equivalent.
- Preserve plain lowercase subcommand chains.

**Non-Goals:**

- Define the grammar of Git or another command.
- Add source positions or an ordered public token list.
- Solve the relative-position problem from issue #62.
- Expand the file-extension table.

## Decisions

### Use the existing path-shape classifier as a verb boundary

The greedy pass will stop before a token when `BashResolver.LooksLikePath` returns `true`.

This rule applies only after the first command token. A path-shaped command name remains the command name.

The argument pass already calls the same classifier. It will therefore emit the token with `IsPath` and `Resolved` metadata.

### Apply the rule to both native-command parsers

The Bash parser and the PowerShell native-command path must make the same boundary decision.

Both paths will call the same `BashResolver.LooksLikePath` method. This choice prevents extension-table drift.

### Keep the public AST unchanged

Issue #64 does not require a new public type. The current `Arg` record already carries the required path result.

An ordered public token API remains a possible answer for issue #62. That larger API is outside this change.

### Preserve the existing greedy default

Plain lowercase words still extend the verb chain. Commands such as `freshdesk ticket list` keep their current result.

The parser will not use a command dictionary. The existing path evidence supplies the only new boundary.

## Risks / Trade-offs

- A legitimate subcommand can have a known file suffix. The curated path table limits this case, and path evidence wins for security consumers.
- The parsed verb becomes shorter for affected commands. Regression tests will lock the intentional result.
- The shared rule can change PowerShell native-command output. Equivalent tests will cover both shell parsers.

## Migration Plan

This change requires no consumer API migration. Consumers receive richer argument data after a package update.

A revert restores the prior parser behavior if the rule causes an unexpected regression.

## Open Questions

Issue #62 can later add lexical provenance. That proposal must remain compatible with this parser boundary.
27 changes: 27 additions & 0 deletions openspec/changes/preserve-path-shaped-operands/proposal.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
## Why

The greedy verb pass can consume a lowercase filename as a verb token. Security consumers then lose the path and its directory scope.

## What Changes

- Stop a native verb chain before a token that matches the existing path-shape rules.
- Return that token as an argument with the existing path metadata.
- Apply the same native-command rule to Bash and PowerShell.
- Preserve greedy verb chains for lowercase words that do not have a path shape.
- Add no command dictionary and make no public API change.

## Capabilities

### New Capabilities

- `path-shaped-operands`: Preserve lowercase file operands and their resolved directory scope after multi-token commands.

### Modified Capabilities

None.

## Impact

The change affects the Bash and PowerShell native verb passes. It also affects their unit tests, corpora, and parser specifications.

The public AST remains unchanged. The change adds no dependency and no native library.
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
## ADDED Requirements

### Requirement: A path-shaped operand terminates a native verb chain
The parser SHALL stop a native verb chain before each later word token that matches the existing path-shape rules.

#### Scenario: Bash file operand after a multi-token command
- **WHEN** Bash parses `git diff install-skills.sh` with `/home/user/repo` as the working directory
- **THEN** the verb tokens are `git` and `diff`
- **THEN** `install-skills.sh` is an argument with `IsPath` set to `true`
- **THEN** the resolved path is `/home/user/repo/install-skills.sh`

#### Scenario: PowerShell native file operand after a multi-token command
- **WHEN** PowerShell parses `git diff install-skills.sh` with `/home/user/repo` as the working directory
- **THEN** the verb tokens are `git` and `diff`
- **THEN** `install-skills.sh` is an argument with `IsPath` set to `true`
- **THEN** the resolved path is `/home/user/repo/install-skills.sh`

#### Scenario: A path-shaped command name remains the command
- **WHEN** Bash parses `deploy.sh status`
- **THEN** `deploy.sh` remains the first verb token

#### Scenario: A PowerShell native path-shaped command name remains the command
- **WHEN** PowerShell parses `deploy.sh status`
- **THEN** `deploy.sh` remains the first verb token

### Requirement: Path classification uses one canonical rule
The native verb pass SHALL use the same path-shape rules as the argument path classifier.

#### Scenario: An unknown command has a file operand
- **WHEN** Bash parses `acme inspect report.json` with `/home/user/repo` as the working directory
- **THEN** the verb tokens are `acme` and `inspect`
- **THEN** `report.json` is a resolved path argument

#### Scenario: A real non-Git command has a file operand
- **WHEN** Bash parses `kubectl apply deployment.yaml` with `/home/user/repo` as the working directory
- **THEN** the verb tokens are `kubectl` and `apply`
- **THEN** `deployment.yaml` is a resolved path argument

#### Scenario: An explicit separator gives equivalent path metadata
- **WHEN** Bash parses `git diff -- install-skills.sh` with `/home/user/repo` as the working directory
- **THEN** `install-skills.sh` has the same path classification and resolved value as the form without `--`

### Requirement: Plain lowercase subcommands keep the greedy behavior
The parser SHALL keep each later lowercase word in the verb chain when the word has no path shape.

#### Scenario: Unknown private CLI subcommands
- **WHEN** Bash parses `freshdesk ticket list --status open`
- **THEN** the verb tokens are `freshdesk`, `ticket`, and `list`
- **THEN** `--status` and `open` remain arguments

#### Scenario: Bare Git values remain narrow by default
- **WHEN** Bash parses `git push origin main`
- **THEN** the verb tokens are `git`, `push`, `origin`, and `main`

#### Scenario: Path evidence wins over an extension-shaped subcommand
- **WHEN** Bash parses `tool plugin.sh list`
- **THEN** the only verb token is `tool`
- **THEN** `plugin.sh` is a path argument
- **THEN** `list` is a non-path argument
20 changes: 20 additions & 0 deletions openspec/changes/preserve-path-shaped-operands/tasks.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
## 1. Contract

- [x] 1.1 Update the Bash verb-chain specification with the path-shape boundary and use cases.
- [x] 1.2 Update the PowerShell native-command specification with the same boundary.

## 2. Parser

- [x] 2.1 Stop the Bash greedy verb pass before a path-shaped later token.
- [x] 2.2 Stop the PowerShell native verb pass before the same token shape.

## 3. Verification

- [x] 3.1 Add Bash tests for file operands, flags, separators, command names, and plain subcommands.
- [x] 3.2 Add equivalent PowerShell native-command tests.
- [x] 3.3 Add corpus coverage for the issue #64 reproduction.

## 4. Completion

- [x] 4.1 Update `IMPLEMENTATION_PLAN.md` with the completed issue #64 work.
- [x] 4.2 Run the OpenSpec check, build, tests, header check, and Slopwatch.
Original file line number Diff line number Diff line change
Expand Up @@ -864,7 +864,11 @@ private static ClauseResult ParseClauseSegment(
continue;
}

if (fileVerbCarveout || !BashVerbs.IsVerbLikeToken(t))
// Path evidence wins before the lexical verb heuristic.
// The argument pass uses the same classifier.
if (fileVerbCarveout
|| BashResolver.LooksLikePath(t.Value)
|| !BashVerbs.IsVerbLikeToken(t))
{
break;
}
Expand Down
13 changes: 5 additions & 8 deletions src/ShellSyntaxTree/Internal/Bash/Verbs/BashVerbs.cs
Original file line number Diff line number Diff line change
Expand Up @@ -128,14 +128,11 @@ internal static readonly IReadOnlyDictionary<string, HashSet<string>>
/// </summary>
/// <remarks>
/// Strict allow-list (leading <c>[a-z]</c>, body <c>[a-z0-9._-]</c>)
/// over the more obvious negation-of-LooksLikePath because it stays
/// conservative for unknown shapes: a token like <c>readme.md</c>
/// satisfies the allow-list and would extend an unknown CLI's verb
/// chain, but the FileVerb carveout in <c>BashCommandParser</c>
/// short-circuits the common case (<c>cat readme.md</c>) before the
/// allow-list ever runs. Quoted strings are excluded so the user's
/// intent to treat bytes literally is preserved. The 64-char bound
/// is a defensive cap against pathological inputs.
/// remains independent from path classification. The caller applies
/// <c>BashResolver.LooksLikePath</c> first so a token such as
/// <c>readme.md</c> remains an argument. Quoted strings are excluded so
/// the user's intent to treat bytes literally is preserved. The
/// 64-char bound is a defensive cap against pathological inputs.
/// </remarks>
internal static bool IsVerbLikeToken(in BashToken token)
{
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -710,7 +710,10 @@ private static ClassifiedVerb ClassifyVerb(List<PwshToken> body, int start)
continue;
}

if (t.Kind != PwshTokenKind.Word || !PwshVerbs.IsNativeVerbLikeToken(t.Value))
// Keep native-command boundaries equal across both shells.
if (t.Kind != PwshTokenKind.Word
|| BashResolver.LooksLikePath(t.Value)
|| !PwshVerbs.IsNativeVerbLikeToken(t.Value))
{
break;
}
Expand Down
3 changes: 2 additions & 1 deletion src/ShellSyntaxTree/Internal/Pwsh/Verbs/PwshVerbs.cs
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,8 @@ internal static bool IsPwshHost(string? verb) =>
/// <em>case-sensitive</em>: the leading-lowercase rule is the only
/// signal that stops the greedy walk at a capitalized identifier
/// (<c>dotnet ef migrations add InitialCreate</c> stops at
/// <c>InitialCreate</c>).
/// <c>InitialCreate</c>). The caller first rejects tokens that match the
/// shared Bash path-shape rules.
/// </summary>
internal static bool IsNativeVerbLikeToken(string value)
{
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
{
"name": "Path-shaped operand: git diff lowercase shell file",
"input": "git diff install-skills.sh",
"expected": {
"isUnparseable": false,
"clauses": [
{
"operator": "None",
"verb": ["git", "diff"],
"args": [
{
"raw": "install-skills.sh",
"kind": "Literal",
"isPath": true,
"isFlag": false,
"resolved": "/work/install-skills.sh"
}
],
"redirects": [],
"isSubshell": false,
"isCommandStringWrapped": false
}
]
},
"notes": "Issue #64: the path-shape rule stops the greedy verb pass before a lowercase file operand."
}
Loading