Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 15 additions & 3 deletions IMPLEMENTATION_PLAN.md
Original file line number Diff line number Diff line change
Expand Up @@ -274,9 +274,21 @@ priorities.
cases pin exact syntax, command ancestry, spans, completeness, and
literal-versus-expanding behavior; real-Bash output and parse-only
oracles independently pin the bounded semantic boundary.
- [ ] Extend Bash substitution discovery to iterables with the complete
`for ... in` vertical slice, then add the Bash substitution cases to the
Netclaw approval matrix.
- [x] Deliver the static-value Bash `for ... in` slice: locked structural
nodes and spans, iterator `$()` discovery, condition-free body
occurrences, exact/finite/pattern/unknown value domains, quote-proved
effective arguments, nested distinct-name correlation, fixed
candidate/depth limits,
strict executable-corpus facts, and real-Bash oracles. Compatibility
leaves preserve authored dynamic operands. Loop binding and cwd
mutation fail closed, loops reached after recognized prior shell-state
mutation fail closed, and occurrence cwd remains Unknown.
- [ ] Design and implement structure-aware Bash abstract-state analysis before
enabling cwd-changing loop bodies or claiming the complete `for ... in`
vertical slice. The parse-order attribution model cannot soundly publish
occurrence cwd across pipelines, conditional lists, substitutions, and
repeated iterations. Keep OpenSpec task 6.5 open, then add the remaining
loop cases and Netclaw approval matrix after that design is reviewed.
- [ ] Complete PowerShell `$()` discovery in `foreach` expressions and add the
Netclaw approval-matrix cases. The simple-command slice is delivered for
ordinary, adjacent, quoted, here-string, redirect, standalone,
Expand Down
18 changes: 11 additions & 7 deletions SPEC.md
Original file line number Diff line number Diff line change
Expand Up @@ -1070,9 +1070,11 @@ discovered subset.
The lexer produces tokens consumed by the parser. Token kinds:

- **WORD** — sequence of non-whitespace, non-operator, non-quote chars.
Example: `git`, `/etc/foo`, `--force`, `~/path`, `$VAR`. Simple
parameter expansion `${VAR}` (no `//` slash) is absorbed into a Word
token; the resolver in §8 decides `Kind`.
Example: `git`, `/etc/foo`, `--force`, `~/path`, `$VAR`. A braced
parameter is absorbed only when its body is a simple shell identifier,
positional parameter, or special parameter. Parameter operators are
unparseable because their operands can contain hidden execution; the
resolver in §8 decides `Kind` for accepted simple forms.
- **QUOTED_STRING** — single- or double-quoted string. The lexer strips
the quote delimiters from the token value. Example: `"hello world"`
becomes the token value `hello world`.
Expand All @@ -1096,8 +1098,9 @@ The lexer produces tokens consumed by the parser. Token kinds:
Expanding-heredoc substitutions use the same opaque fragment semantics but
remain attached to the delimiter token rather than entering the ordinary
command-token stream.
- **UNPARSEABLE_SENTINEL** — `$((expr))` arithmetic expansion or
`${var//pat/repl}` complex parameter expansion. The lexer skips past
- **UNPARSEABLE_SENTINEL** — `$((expr))` arithmetic expansion or any
operator-bearing parameter expansion such as `${var:-$(cmd)}` or
`${var//pat/repl}`. The lexer skips past
the matching close (`))` or `}` respectively) and emits a sentinel
whose reason names the rejected construct. The parser consumes this
token by setting outer `ParsedCommand.IsUnparseable = true` (see §11).
Expand Down Expand Up @@ -1729,8 +1732,9 @@ Conditions that produce `IsUnparseable = true`:
- Process substitution (`<(cmd)`, `>(cmd)`).
- Arithmetic expansion `$((expr))` (per §1 non-goal; lexer emits an
UNPARSEABLE_SENTINEL token; parser sets the outer flag).
- Complex parameter expansion `${var//pat/repl}` (per §1 non-goal; same
mechanism).
- Operator-bearing parameter expansion such as `${var:-$(cmd)}` or
`${var//pat/repl}` (per §1 non-goal; same mechanism). Only simple braced
identifiers, positional parameters, and special parameters are accepted.
- Recursion depth exceeded on `bash -c` chains (>5 levels).

**Diagnostic precedence.** When multiple conditions could fire on a
Expand Down
10 changes: 10 additions & 0 deletions openspec/changes/v0-3-structured-shell-analysis/design.md
Original file line number Diff line number Diff line change
Expand Up @@ -422,6 +422,16 @@ scope-isolated groups do not leak state. Branches join their possible exit
states; loops include the zero-iteration path unless shell semantics prove at
least one iteration.

Implementation must run this as a structure-aware abstract-state pass over the
proved syntax tree, not by exposing the compatibility parser's mutable
parse-order cwd attribution. Parse order is not execution-state order for
pipelines or conditional lists, and one symbolic loop-body parse cannot prove
the cwd of later iterations. The compatibility attribution path remains a
v0.2 leaf-construction detail. Until the abstract pass lands, loop cwd mutation
fails closed, recognized shell-state mutation before or inside a loop fails
closed, nested reuse of an active Bash binding name fails closed, and
occurrence `WorkingDirectory` stays `Unknown`.

Bash command substitution executes in an isolated subshell state. State changes
affect later commands inside that substitution but never the containing command
or following outer commands. PowerShell `$()` evaluates in the current runspace
Expand Down
15 changes: 11 additions & 4 deletions openspec/changes/v0-3-structured-shell-analysis/tasks.md
Original file line number Diff line number Diff line change
Expand Up @@ -68,11 +68,18 @@

## 6. Bash For-In Vertical Slice

- [ ] 6.1 Parse Bash `for name in literal...; do ...; done` into the locked structural nodes.
- [ ] 6.2 Emit condition-free loop-body occurrences and conservative compatibility clauses.
- [ ] 6.3 Derive exact and finite literal binding domains within the locked candidate cap.
- [ ] 6.4 Substitute a bounded binding only where Bash quoting proves argument boundaries.
- [x] 6.1 Parse Bash `for name in literal...; do ...; done` into the locked structural nodes.
- [x] 6.2 Emit condition-free loop-body occurrences and conservative compatibility clauses.
- [x] 6.3 Derive exact and finite literal binding domains within the locked candidate cap.
- [x] 6.4 Substitute a bounded binding only where Bash quoting proves argument boundaries.
- [ ] 6.5 Propagate and conservatively join cwd and supported binding state across zero-or-more loop execution.
- The first static-value slice deliberately leaves occurrence cwd Unknown
and rejects loop shell-state mutation, nested active-binding reuse, or
loops reached after recognized prior shell-state mutation. A separate
structure-aware abstract-state pass is required
before enabling cwd-changing bodies;
mutable parse-order attribution is unsound across pipelines, `&&` / `||`,
substitutions, and repeated iterations.
- [ ] 6.6 Cover empty iterables, separators, multiline bodies, redirects, pipelines, nested loops, and wrapper boundaries.
- [ ] 6.7 Add adversarial cases for option injection, mutation, unquoted expansion, indirect expansion, substitutions, and cap overflow.
- [ ] 6.8 Add sanitized Bash corpus entries and Netclaw allow/prompt/deny integration cases.
Expand Down
72 changes: 48 additions & 24 deletions src/ShellSyntaxTree/Internal/Bash/Lexing/BashLexer.cs
Original file line number Diff line number Diff line change
Expand Up @@ -844,14 +844,9 @@ private static bool TryConsumeComplexParamExpansion(
ReadOnlySpan<char> src, int start, List<BashToken> tokens, out int afterBrace)
{
// src[start] = '$', src[start+1] = '{'. We need to find the matching
// '}' and decide: simple ${VAR} -> false (let word reader take it);
// ${...//...} or any other "complex" form -> emit UnparseableSentinel.
//
// For v0.1 we treat the presence of a slash inside the braces as the
// single signal of "complex param expansion" (per the locked
// interpretation #2 in the OpenSpec change). Other operators inside
// ${...} (like ${X-default}, ${X#prefix}) fall through to the word
// reader; a future PR can tighten this if needed.
// '}' and decide: a simple variable, positional, or special parameter
// falls through to the word reader. Operators can themselves contain
// executable substitutions, so every other body fails closed.
var openBrace = start + 1;
var scan = OpaqueRegionScanner.Scan(src, openBrace, '{', '}');
if (!scan.Closed)
Expand All @@ -870,20 +865,9 @@ private static bool TryConsumeComplexParamExpansion(
var endInclusive = scan.EndIndex;
var bodyStart = openBrace + 1;
var bodyEnd = endInclusive; // exclusive of '}'
var hasSlash = false;
for (var k = bodyStart; k < bodyEnd; k++)
var body = src.Slice(bodyStart, bodyEnd - bodyStart);
if (IsSimpleBracedParameterName(body))
{
if (src[k] == '/')
{
hasSlash = true;
break;
}
}

if (!hasSlash)
{
// Simple ${VAR} (or ${X-default} etc.). Caller will fall through
// to the word reader and absorb it as part of a Word token.
afterBrace = -1;
return false;
}
Expand All @@ -895,7 +879,7 @@ private static bool TryConsumeComplexParamExpansion(
null,
start,
length,
"complex parameter expansion '${var//pat/repl}' not supported in v0.1"));
"complex parameter expansion is not supported in v0.3"));
afterBrace = start + length;
return true;
}
Expand Down Expand Up @@ -1101,9 +1085,9 @@ private static bool TryAppendBashExpansion(

expansionLength = scan.EndIndex - start + 1;
name = src.Slice(start + 2, expansionLength - 3).ToString();
if (name.Length == 0 || name.IndexOf('/') >= 0)
if (!IsSimpleBracedParameterName(name.AsSpan()))
{
error = "complex parameter expansion '${var//pat/repl}' not supported in v0.1";
error = "complex parameter expansion is not supported in v0.3";
index += expansionLength;
return true;
}
Expand Down Expand Up @@ -1173,6 +1157,46 @@ private static bool IsAllAsciiDigits(string value)
return true;
}

private static bool IsSimpleBracedParameterName(ReadOnlySpan<char> value)
{
if (value.Length == 0)
{
return false;
}

if (value.Length == 1 &&
value[0] is '?' or '$' or '#' or '-' or '!' or '@' or '*')
{
return true;
}

var allDigits = true;
for (var index = 0; index < value.Length; index++)
{
allDigits &= value[index] is >= '0' and <= '9';
}

if (allDigits)
{
return true;
}

if (!IsBashIdentifierStart(value[0]))
{
return false;
}

for (var index = 1; index < value.Length; index++)
{
if (!IsBashIdentifierContinuation(value[index]))
{
return false;
}
}

return true;
}

private static bool IsBashIdentifierStart(char value) =>
value == '_' || value is >= 'A' and <= 'Z' or >= 'a' and <= 'z';

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -395,7 +395,8 @@ private static bool TryDetectAnomaly(IReadOnlyList<BashToken> tokens, out string

if (nextIsVerbSlot
&& t.Kind == BashTokenKind.Word
&& BashVerbs.ControlFlowKeywords.Contains(t.Value))
&& BashVerbs.ControlFlowKeywords.Contains(t.Value)
&& t.Value is not ("for" or "do" or "done"))
{
reason = $"control-flow keyword '{t.Value}' is not supported in v0.1";
return true;
Expand Down Expand Up @@ -558,6 +559,7 @@ private static ClauseResult ParseClauseSegment(
// Path evidence wins before the lexical verb heuristic.
// The argument pass uses the same classifier.
if (fileVerbCarveout
|| BashVerbs.ControlFlowKeywords.Contains(t.Value)
|| BashResolver.LooksLikePath(t.Value)
|| !BashVerbs.IsVerbLikeToken(t))
{
Expand Down
Loading