Preserve source-ordered clause elements for issue #62

IMPLEMENTATION_PLAN.md

@@ -37,7 +37,7 @@ priorities.
       the 64 KiB cap (§11).
 - [x] **11. Multi-shell corpus runner + PII audit** — directory-routed by
       `Corpus/<shell>/`.
-- [x] **12. PowerShell corpus** — 211 entries under `Corpus/powershell/`,
+- [x] **12. PowerShell corpus** — 273 entries under `Corpus/powershell/`,
       every §13 category minimum exceeded.
 - [x] **13. `pwsh` validation gate + `tools/PwshCorpusTool`** —
       `PwshOracleTests` enforces the §13 oracle matrix + the `PwshAliases`
@@ -54,6 +54,39 @@ priorities.
       Recurse into provably static `Invoke-Expression` / `iex` payloads,
       safe-fail computed and pipeline-fed code, share the existing recursion
       limits, and preserve current-scope PowerShell location attribution.
+- [x] **Issue #62 — source-ordered clause elements.** Added the additive
+      `Clause.Elements` provenance view for Bash and PowerShell with exact raw
+      spelling, decoded values, source spans when available, verb-relative
+      argument placement, path facts, redirects, and conservative wrapper-span
+      handling. Authored order is authoritative; element roles and
+      `PrecedingVerbElementCount` explicitly mirror the greedy parser
+      projection rather than executable semantics. Paired Bash/PowerShell
+      corpus cases cover Git `-c`/`-C`, multiple occurrences, and a valueless
+      option that stops the greedy walk. Existing projection shapes and
+      synthetic cwd attribution remain compatible; native options that differ
+      only by case receive corrected metadata. The post-implementation option
+      audit explicitly covers Wget `-o` / `-O`, curl `-d` / `-D` / `-o` /
+      `-O`, Git `-c` / `-C`, and tar `-c` / `-C` / `-f` / `-F`; paired corpus
+      cases pin Wget log/document output, curl data/header-output and `@file`
+      semantics, and tar helper-command safe-fail behavior in both shells.
+      Adversarial review added deterministic coverage for quoted inline native
+      fragment runs (including unquoted prefixes and mixed-quote safe-fail),
+      PowerShell backtick-decoded colon bindings, native file-verb boundaries,
+      and outer redirects on PowerShell command wrappers, including empty
+      payloads. The
+      corpus runner now verifies direct authored-token coverage even for legacy
+      entries without explicit element expectations. Docker `-v`
+      remains explicitly context-sensitive: the generic table supports
+      `docker run`, while consumers use authored elements for global-option
+      interpretation.
+      Command-string provenance is integrated with the later
+      `Invoke-Expression` recursion work: static expansion clears unmappable
+      outer spans, while dynamic payloads retain conservative source-aligned
+      elements. Nested and dynamic `bash -c` cases pin the equivalent Bash
+      boundary. Consumer guidance separates strict authored-stream matching
+      from general executable-aware normalization; Netclaw can use the latter
+      for reusable approvals without treating parser verb roles as semantic
+      command boundaries.
 - [x] **Issue #64 — path-shaped operands after native verb chains.**
       Stop the Bash and PowerShell native greedy passes before a token that
       matches the shared path-shape rules. Preserve that token as a resolved
@@ -70,14 +103,18 @@ priorities.
       corpus. Review follow-ups shipped with it: the equals-form split moved
       to a shared `NativeFlagSyntax` so the two parsers can't drift, a colon
       value under an `=`-bearing parameter name safe-fails to `DynamicSkip`,
-      and `-?` lexes as one parameter token.
+      `-?` lexes as one parameter token, and native option tables now use
+      ordinal spelling while PowerShell cmdlet parameters remain
+      case-insensitive.
 
 ### 15. Release 0.2.0 (alpha → beta → stable) — SPEC.PWSH §15 / §17
 
 - [x] Tag `0.2.0-alpha`; `publish_nuget.yml` produced
       `ShellSyntaxTree.0.2.0-alpha.nupkg` and it is live on nuget.org
       (released 2026-05-20).
 - [x] `0.2.0-beta.1` so Netclaw validates the parser + the breaking rename
+- [ ] Publish the next `0.2.0` prerelease with the additive issue #62
+      `Clause.Elements` provenance surface and migration guidance
 - [ ] Promote to stable `0.2.0` after Netclaw validation
 
 ### 16. Netclaw v0.2.0 integration — SPEC.PWSH §17 #9
@@ -105,6 +142,9 @@ priorities.
 - PowerShell script-level constructs — control flow, `function` / `class` /
   `enum` definitions, `param()` / `begin` / `process` / `end` blocks,
   `.ps1` file parsing (`SPEC.POWERSHELL.md` §18).
+- [Issue #69](https://github.com/Aaronontheweb/ShellSyntaxTree/issues/69) —
+  extract shared native argument-fragment classification before adding a third
+  shell or another fragment rule; keep shell tokenization and parsing local.
 - Extract a shared lexer/parser core now that two parsers exist — the seam
   can be designed from real duplication (`SPEC.POWERSHELL.md` §18); the
   path-normalization helpers duplicated between `BashResolver` and

PROJECT_CONTEXT.md

@@ -19,6 +19,9 @@ The output is a `ParsedCommand` containing:
   when known, and explicit `DynamicSkip` marking for unresolved env vars
   / unexpanded globs
 - redirect operators (`>`, `>>`, `<`, `2>`, `2>>`)
+- source-ordered clause elements with exact spelling, decoded values, source
+  spans when available, and coordinates relative to parser-classified verb
+  elements; executable-specific semantics remain consumer-owned
 - Bash `cd <dir> && cmd` and PowerShell `Set-Location <dir>; cmd`
   propagation — the target is attributed to subsequent clauses
 - recursion into `bash -c`, `pwsh -Command`, and `pwsh -EncodedCommand` so
@@ -39,7 +42,9 @@ open-source autonomous operations agent. Netclaw's POSIX approval gate consumes
 ShellSyntaxTree's Bash parser to decompose approval units, identify candidate
 verbs and directories, propagate cwd context, inspect redirects, and fail
 closed when parsing is uncertain. Its PowerShell integration is the remaining
-v0.2.0 downstream acceptance item. See
+v0.2.0 downstream acceptance item. Netclaw is expected to use the consumer
+guide's general executable-aware matching path for supported commands, with
+strict authored-stream matching as the fallback for unrecognized shapes. See
 [`docs/CONSUMER_GUIDE.md`](./docs/CONSUMER_GUIDE.md) for the public consumer
 algorithm and immutable Netclaw examples.
 
@@ -83,8 +88,9 @@ zero-native-deps .NET parser sized to what security gates actually need.
   and `.ps1` file-content parsing.
 - Performance optimization beyond "fast enough to invoke per shell call
   without noticeable latency" (~1 ms typical).
-- Full IDE-style source mapping. Security-motivated token provenance is under
-  active design in issue #62.
+- Full IDE-style source mapping. `Clause.Elements` provides security-motivated
+  provenance for significant clause leaves, not a lossless concrete syntax
+  tree.
 
 ### Versioning
 

README.md

@@ -16,7 +16,9 @@ dotnet add package ShellSyntaxTree --version 0.2.0-beta.1
 ```
 
 The `0.2.0-beta.1` prerelease is required for PowerShell support. The latest
-stable `0.1.x` package supports Bash only.
+stable `0.1.x` package supports Bash only. The public surface documented below
+tracks the `dev` branch; `Clause.Elements` is scheduled for the next `0.2.0`
+prerelease and is not present in `beta.1`.
 
 ## What you get
 
@@ -120,12 +122,14 @@ public sealed record BashParserOptions : ShellParserOptions;
 public sealed record PwshParserOptions : ShellParserOptions;
 
 public sealed record ParsedCommand { /* Source, Clauses, IsUnparseable, … */ }
-public sealed record Clause        { /* Operator, Verb, Args, Redirects, IsSubshell, IsCommandStringWrapped */ }
+public sealed record Clause        { /* Operator, Verb, Args, Redirects, Elements, IsSubshell, IsCommandStringWrapped */ }
+public sealed record ClauseElement { /* Raw, Value, Role, source span, verb-relative position, path facts */ }
 public sealed record VerbChain     { /* Tokens, Joined, CanonicalVerb, IsDynamic */ }
 public sealed record Arg           { /* Raw, Resolved, Kind, IsPath, IsCwdAttribution, IsFlag */ }
 public sealed record Redirect      { /* Direction, Target, IsDynamicSkip */ }
 
 public enum ArgKind            { Literal, EnvVar, Glob, Tilde, DynamicSkip }
+public enum ClauseElementRole  { Verb, Argument, Redirect }
 public enum RedirectDirection  { In, Out, Append, ErrOut, ErrAppend }
 public enum CompoundOperator   { None, AndIf, OrIf, Sequence, Pipe }
 ```

SPEC.POWERSHELL.md

@@ -75,9 +75,11 @@ syntax (§5) are all PowerShell 7 semantics. The `pwsh` validation oracle
 
 ## 2. Public API Surface
 
-The shared interface, AST records, and enums are defined in **`SPEC.md` §2**
-and are unchanged. PowerShell adds the following to namespace
-`ShellSyntaxTree`; everything else is internal.
+The shared interface, AST records, and enums are defined in **`SPEC.md` §2**.
+The additive `Clause.Elements`, `ClauseElement`, and `ClauseElementRole`
+provenance surface applies identically to both parsers. PowerShell adds the
+following parser types to namespace `ShellSyntaxTree`; everything else is
+internal.
 
 ```csharp
 namespace ShellSyntaxTree;
@@ -117,14 +119,17 @@ public sealed class PwshParser : IShellParser
 }
 ```
 
-Two shared types gain a change (see §3):
+The shared v0.2 AST gains the following changes (see §3):
 
 - `VerbChain` gains an additive `string? CanonicalVerb` field.
 - `VerbChain` gains an additive `bool IsDynamic` field.
+- `Clause` gains the additive `Elements` provenance view shared with Bash;
+  `ClauseElement` and `ClauseElementRole` define its entries.
 - `Clause.IsBashCWrapped` is renamed `Clause.IsCommandStringWrapped`.
 
 **Versioning.** `PwshParser`, `PwshParserOptions`, `ShellParserOptions`,
-`VerbChain.CanonicalVerb`, and `VerbChain.IsDynamic` are additive. The
+`VerbChain.CanonicalVerb`, `VerbChain.IsDynamic`, `Clause.Elements`,
+`ClauseElement`, and `ClauseElementRole` are additive. The
 `Clause` field rename and the `BashParserOptions` reparenting are
 **breaking**; `SPEC.md` Appendix A permits a breaking AST change on a `0.x`
 minor bump when `RELEASE_NOTES.md` carries the old→new mapping and Netclaw is
@@ -136,9 +141,47 @@ never throws on a well-formed string, exactly like `BashParser`.
 
 ## 3. AST Reference
 
-The AST records and enums are defined in **`SPEC.md` §3** and are emitted
-unchanged by `PwshParser` — a consumer walks a PowerShell `ParsedCommand`
-exactly as it walks a bash one. Two deltas:
+The AST records and enums are defined in **`SPEC.md` §3** and are emitted by
+`PwshParser` under the same shared contract — a consumer walks a PowerShell
+`ParsedCommand` exactly as it walks a bash one. PowerShell has the following
+deltas and provenance rules:
+
+### `Clause.Elements` PowerShell rules
+
+PowerShell parameters, native options, quoted/here-string values, and opaque
+dynamic regions each occupy their authored position in `Clause.Elements`.
+The leading call operator in `& command` and grouping parentheses are shell
+syntax rather than verb/argument/redirect leaves and do not appear.
+
+Inline parameter forms remain one source element. For `-Path:C:\repo`, the
+element's `Raw` and `Value` describe the full parameter token while `Kind`,
+`IsPath`, and `Resolved` describe the bound `C:\repo` value. Native
+`--flag=value` follows the same rule as Bash. Backtick escapes in an inline
+bound value are decoded before `Value` and path metadata are produced. Adjacent
+native fragments such as `--data='@C:\payload file'` form one element because
+PowerShell passes them to the executable as one argument. The complete
+contiguous fragment run is consumed. Resolver-sensitive syntax inside a
+single-quoted fragment mixed with expandable fragments safe-fails as
+`DynamicSkip` rather than being expanded.
+
+Clauses recursively surfaced from `pwsh -Command` and
+`pwsh -EncodedCommand` retain inner `Raw` and `Value` but have null
+`SourceStart` and `SourceLength`: quote/backtick processing, script-block
+stripping, and base64 decoding do not provide a generally exact map into the
+outer `ParsedCommand.Source`.
+An outer redirect authored after a `pwsh -Command` or `-EncodedCommand`
+payload remains on the surfaced wrapped clause with its exact outer source
+span; only decoded inner elements have null spans.
+
+`ClauseElement.Role` and `PrecedingVerbElementCount` mirror the shared greedy
+native verb projection. They are AST coordinates, not native-executable
+semantic boundaries. A PowerShell consumer applies executable-specific grammar
+to the complete authored element order exactly as a Bash consumer does.
+
+PowerShell cmdlet names, aliases, and parameter names remain
+case-insensitive. Native option spelling is ordinal and reuses the shared Bash
+native tables unchanged: PowerShell does not make a native executable's `-c`
+and `-C` options equivalent.
 
 ### `VerbChain.CanonicalVerb` (new, additive)
 
@@ -672,11 +715,21 @@ positionals are paths," exactly as `SPEC.md` §7.
 
 Native commands reuse the bash per-verb rules table verbatim — `git`,
 `curl`, `tar`, etc. behave identically to `SPEC.md` §7 (`curl` / `wget`:
-the first positional is a URL; the `-o` / `-O` value is a path). This
+the first positional is a URL; curl `-o` / `-D` values and Wget `-o` / `-O`
+values are paths, while curl `-d` data is non-path unless `@file` requests a
+file read; `@-` denotes stdin). Tar `-F` / `--info-script` /
+`--new-volume-script` values execute commands and therefore safe-fail as
+`DynamicSkip`, not paths. This
 includes hyphenated option names and the bash `--flag=value` split: the
 flag and value surface as separate args, and a curated flag's value receives
 the same path classification in both parsers. Native `--flag:value` has no
-cmdlet-binding semantics and remains verbatim.
+cmdlet-binding semantics and remains verbatim. PowerShell still owns outer
+tokenization: spaced curl operands beginning with `@` should be quoted because
+`@name` is splatting and bare `@-` is a parse error. Use forms such as
+`-d "@request.json"` / `-d "@-"`, or bind a file inline as
+`--data=@request.json`, so the native command receives one value. An equals
+prefix adjacent to a quoted value, such as `--data='@C:\payload file'`, is
+also one native argument and one clause element.
 
 ---
 
@@ -853,8 +906,8 @@ not just one quoted token. The parser handles all three real forms:
 - **Script block** — `pwsh -Command { Remove-Item C:\tmp\x }`. Parse the
   script-block *interior* (braces stripped) as a fresh `ParsedCommand`.
 - **Bare / multi-token** — `pwsh -Command Remove-Item C:\tmp\x`. Take the
-  verbatim source slice from the first token after `-Command` to the end of
-  the statement and parse *that* as a fresh `ParsedCommand`.
+  verbatim source slice from the first token after `-Command` through the last
+  command token and parse *that* as a fresh `ParsedCommand`.
 
 In every form the inner clauses surface inline, each with
 `IsCommandStringWrapped = true`. **Not** recognizing the bare/multi-token
@@ -865,6 +918,12 @@ yields `IsUnparseable = true`** (e.g. a `-Command` payload that decodes to a
 control-flow script), sets the outer `ParsedCommand.IsUnparseable = true` so
 the whole command routes to safe-fail (`SPEC.md` §10).
 
+A terminal redirect belongs to the outer PowerShell invocation, not the child
+command string. The parser appends each such redirect to the last surfaced
+inner clause's `Redirects` and `Elements`; its outer source span remains exact.
+Non-redirect arguments after a quoted, script-block, colon-bound, or encoded
+payload are not modeled and set `IsUnparseable=true` rather than disappearing.
+
 `pwsh -File script.ps1` is **not** recursion — the file content is not
 available to the parser. It parses as an ordinary clause with `script.ps1`
 as a path arg.
@@ -911,6 +970,9 @@ leaves the required payload missing. Dynamic inline values remain opaque.
 For a static payload, the parser consumes the outer expression clause and
 surfaces the inner clauses inline with `IsCommandStringWrapped = true`. The
 first inner clause takes the operator that preceded the outer expression.
+Surfaced `Clause.Elements` retain their inner raw and decoded values but have
+null source spans because their offsets cannot be mapped exactly into the
+outer `ParsedCommand.Source`.
 The parse increments the same depth counter used by `pwsh -Command` and
 `-EncodedCommand`, and the payload passes through the same 64 KiB input cap.
 
@@ -924,7 +986,10 @@ The parser never evaluates variables, interpolation, concatenation,
 subexpressions, script blocks, arrays, or other computed expressions. When a
 direct computed payload has a source expression, the outer expression clause
 remains and the entire payload source slice becomes one
-`Arg { Kind=DynamicSkip, IsPath=false, Resolved=null }`. Pipeline input,
+`Arg { Kind=DynamicSkip, IsPath=false, Resolved=null }`. Its authored
+`Clause.Elements` retain the expression verb, an optional separate `-Command`
+parameter, and one source-aligned `DynamicSkip` payload region. An inline form
+such as `-Command:$code` remains one authored parameter element. Pipeline input,
 missing payloads, and ambiguous parameter binding set
 `ParsedCommand.IsUnparseable = true`; an incoming pipeline is dynamic even
 when an explicit literal argument also appears. These rules prevent a clean,