Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
51 changes: 51 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
.PHONY: all build test clean build-ts build-go test-ts test-go clean-ts clean-go publish-go tags-go tidy-go reset

all: build test

build: build-ts build-go

test: test-ts test-go

clean: clean-ts clean-go

# TypeScript
build-ts:
npm run build

test-ts:
npm test

clean-ts:
rm -rf dist dist-test

# Go
build-go:
cd go && go build ./...

test-go:
cd go && go test ./...

clean-go:
cd go && go clean -cache

# Publish Go module: make publish-go V=0.1.7
publish-go: test-go
@test -n "$(V)" || (echo "Usage: make publish-go V=x.y.z" && exit 1)
sed -i '' 's/^const Version = ".*"/const Version = "$(V)"/' go/expr.go
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Use GNU-compatible in-place sed syntax

The publish-go target currently uses sed -i '' ..., which is BSD/macOS syntax and fails on GNU sed (common in Linux CI and many developer environments), so make publish-go V=... aborts before the commit/tag steps run. GNU sed documents in-place editing as -i[SUFFIX] (sed --help), and running this exact command on GNU sed treats the replacement expression as a filename and exits with code 2.

Useful? React with 👍 / 👎.

git add go/expr.go
git commit -m "go: v$(V)"
git tag go/v$(V)
git push origin main go/v$(V)
if command -v gh >/dev/null 2>&1; then gh release create go/v$(V) --title "go/v$(V)" --notes "Go module release v$(V)"; fi

tidy-go:
cd go && go mod tidy

tags-go:
git tag -l 'go/v*' --sort=-version:refname

reset:
npm run reset
cd go && go clean -cache
cd go && go build ./...
cd go && go test -v ./...
42 changes: 33 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,20 +1,44 @@
# @jsonic/expr (JSONIC syntax plugin)
# @jsonic/expr

This plugin allows the [Jsonic](https://jsonic.senecajs.org) JSON
parser to support expression syntax.
An expression-syntax plugin for the [Jsonic](https://jsonic.senecajs.org)
parser, available in both TypeScript and Go.

Adds Pratt-parser expressions to Jsonic: infix, prefix, suffix, ternary,
and paren operators with configurable precedence. Expressions parse into
LISP-style S-expressions (arrays whose first element is the operator src),
which a user-supplied evaluator can reduce to values.

[![npm version](https://img.shields.io/npm/v/@jsonic/expr.svg)](https://npmjs.com/package/@jsonic/expr)
[![build](https://github.com/jsonicjs/expr/actions/workflows/build.yml/badge.svg)](https://github.com/jsonicjs/expr/actions/workflows/build.yml)
[![Coverage Status](https://coveralls.io/repos/github/jsonicjs/expr/badge.svg?branch=main)](https://coveralls.io/github/jsonicjs/expr?branch=main)
[![Known Vulnerabilities](https://snyk.io/test/github/jsonicjs/expr/badge.svg)](https://snyk.io/test/github/jsonicjs/expr)
[![DeepScan grade](https://deepscan.io/api/teams/5016/projects/22469/branches/663909/badge/grade.svg)](https://deepscan.io/dashboard#view=project&tid=5016&pid=22469&bid=663909)
[![Maintainability](https://api.codeclimate.com/v1/badges/d44aad76c5a355d01f30/maintainability)](https://codeclimate.com/github/jsonicjs/expr/maintainability)

| ![Voxgig](https://www.voxgig.com/res/img/vgt01r.png) | This open source module is sponsored and supported by [Voxgig](https://www.voxgig.com). |
| ---------------------------------------------------- | --------------------------------------------------------------------------------------- |
## Install

TypeScript:

```sh
npm install @jsonic/expr jsonic
```

Go:

```sh
go get github.com/jsonicjs/expr/go
```

## Documentation

Docs are organised following the [Diátaxis](https://diataxis.fr) framework:

- **[Tutorial](docs/tutorial.md)** — start here. Parse your first expression in
TS and Go.
- **[How-to guides](docs/how-to.md)** — focused recipes: add an operator,
plug in an evaluator, use paren-preval for function calls, restrict to
strict math.
- **[Reference](docs/reference.md)** — exported types and functions,
`OpDef` schema, default operator set, grammar group tags.
- **[Explanation](docs/explanation.md)** — design notes: Pratt algorithm,
S-expression AST, paren/ternary/preval semantics, why `g=expr` tagging.

## License

MIT. See [LICENSE](LICENSE).
96 changes: 96 additions & 0 deletions docs/explanation.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,96 @@
# Explanation

Background reading on how the plugin works. Useful before you customise
precedence tables or diagnose a surprising parse.

- [Pratt parsing in one page](#pratt-parsing-in-one-page)
- [Why S-expressions](#why-s-expressions)
- [Paren semantics and preval](#paren-semantics-and-preval)
- [Ternary as a two-token operator](#ternary-as-a-two-token-operator)
- [How the plugin plugs into Jsonic](#how-the-plugin-plugs-into-jsonic)
- [The `g=expr` tagging convention](#the-gexpr-tagging-convention)

---

## Pratt parsing in one page

Pratt parsing is a top-down operator-precedence algorithm where every
operator carries two numbers: a _left binding power_ (how strongly it
pulls in the term to its left) and a _right binding power_ (how strongly
it pulls to the right).

When the parser sits between two operators, it compares the left-side
operator's _right_ BP to the right-side operator's _left_ BP. Whichever
is higher wins the shared term.

That's enough to handle precedence (`*` beats `+`), associativity
(left-associative when `left < right`, right-associative when
`left > right`), and mixed pre/in/post-fix operators without separate
grammar productions per operator.

## Why S-expressions

The parse result is an array starting with the operator:
`['+', 1, ['*', 2, 3]]`. This has three properties we want:

- **Uniform shape**: infix, prefix, suffix, ternary, and paren all fit
the same `[op, ...terms]` template.
- **Easy to walk**: a single recursive function (your `evaluate`) can
fold the tree bottom-up.
- **Round-trippable**: `Simplify` converts to pure strings/arrays/maps,
which survives `JSON.stringify` and deep-equality comparisons.

## Paren semantics and preval

Parens are modelled as unary operators whose single term is whatever is
inside. So `(1+2)` parses as `['(', ['+', 1, 2]]` rather than collapsing
the parens away. Keeping the paren op in the tree lets evaluators
distinguish grouping from implicit structure (important when you add
paren kinds like `[...]` that mean "list" rather than "group").

`preval` extends this: a paren op can consume the token preceding `(`
as an extra operand, turning `foo(1,2)` into `['(', 'foo', 1, 2]`.
Combined with `preval.allow`, this gives you named function-call syntax
without needing a full function grammar.

## Ternary as a two-token operator

`a?b:c` is a single operator with two syntactic tokens (`?` and `:`) and
three operands. The plugin exposes this as a normal `OpDef` with
`src: ['?', ':']` and `ternary: true`. Nesting is governed by the same
BP comparison as binary operators — `1?2:3?4:5` chains right because the
ternary op is right-associative by convention.

## How the plugin plugs into Jsonic

On `use`, the plugin:

1. Registers each operator token via `jsonic.options({ fixed: ... })`
so the lexer recognises them.
2. Extends a few existing Jsonic rules (`val`, `list`, `map`, `pair`,
`elem`) with alts that backtrack when an operator token appears and
hand control to the plugin's own `expr` rule.
3. Defines three new rules — `expr`, `paren`, and (optional) `ternary`
— that implement the Pratt algorithm as ordered alternates.

The result is that expression syntax is additive: it enters via
backtracking from the existing value/list/map rules, and returns the
same kind of node (a value) back to them when it finishes.

## The `g=expr` tagging convention

Every alt the plugin adds carries `expr` in its grammar group (`g`)
field. Users of a shared Jsonic instance can then strip the expression
grammar cleanly:

```ts
jsonic.options({ rule: { exclude: 'expr' } })
```

Internally the plugin tags alts by snapshotting `rs.Open`/`rs.Close`
before each rule modifier and appending `expr` only to newly-added
alts. Pre-existing alts (the base Jsonic grammar) are left untouched.

This mirrors jsonic's own `GrammarSetting.Rule.Alt.G` mechanism, which
applies when rules are installed via `jsonic.grammar()` (the plugin
uses `jsonic.rule()` instead, so the tag is applied manually).
130 changes: 130 additions & 0 deletions docs/how-to.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,130 @@
# How-to guides

Focused recipes. Each guide assumes you've read the [Tutorial](tutorial.md).

- [Add a custom infix operator](#add-a-custom-infix-operator)
- [Add a prefix or suffix operator](#add-a-prefix-or-suffix-operator)
- [Group with parentheses](#group-with-parentheses)
- [Build function-call syntax with paren-preval](#build-function-call-syntax-with-paren-preval)
- [Add a ternary operator](#add-a-ternary-operator)
- [Disable a default operator](#disable-a-default-operator)
- [Strip expr alts with a group filter](#strip-expr-alts-with-a-group-filter)

---

## Add a custom infix operator

Define an operator by name under `op`. Give it an `src` token and a
`left`/`right` precedence pair (lower number = lower priority).
Left-associative operators have `left < right`; right-associative have
`left > right`.

TypeScript:

```ts
Jsonic.make().use(Expr, {
op: {
power: { infix: true, src: '^', left: 260, right: 250 }, // right-assoc
},
})('2^3^2') // ['^', 2, ['^', 3, 2]]
```

Go:

```go
expr.Parse("2^3^2", map[string]interface{}{
"op": map[string]interface{}{
"power": map[string]interface{}{
"infix": true, "src": "^", "left": 260, "right": 250,
},
},
})
```

## Add a prefix or suffix operator

Prefix operators use `right` only; suffix operators use `left` only.

```ts
Expr, {
op: {
bang: { prefix: true, src: '!', right: 200 }, // !x
factor: { suffix: true, src: '!', left: 210 }, // x!
},
}
```

Because both overloads share the `!` src, Jsonic disambiguates by
position: `!` appearing before a term is prefix, after a term is suffix.

## Group with parentheses

Parens are defined as operators with `paren: true`, `osrc`, `csrc`:

```ts
Expr, {
op: {
brace: { paren: true, osrc: '[', csrc: ']' },
},
}
```

The default operator set already includes `plain: { paren:true, osrc:'(',
csrc:')' }`. Multiple paren kinds can coexist.

## Build function-call syntax with paren-preval

`preval` turns `foo(1,2)` into a paren expression whose `preval` value
is `foo`. Useful for call-like syntax.

```ts
Expr, {
op: {
call: {
paren: true, osrc: '(', csrc: ')',
preval: { active: true, required: true },
},
},
}
```

With `preval.required`, a paren only matches when preceded by a value.
Use `preval.allow: ['foo','bar']` to restrict which preceding names are
valid.

## Add a ternary operator

Ternary is defined by two tokens in `src`:

```ts
Expr, {
op: {
cond: { ternary: true, src: ['?', ':'], left: 80, right: 90 },
},
}
```

Parses `a ? b : c` as `['?', a, b, c]`.

## Disable a default operator

Set the named op to `null` in your options:

```ts
Jsonic.make().use(Expr, { op: { remainder: null } })
```

`1%2` now fails to parse — no `%` operator is registered.

## Strip expr alts with a group filter

Every grammar alternate added by the plugin is tagged with `expr` in
its `g` (group) field. Use Jsonic's `rule.exclude` option to remove
the expression grammar entirely:

```ts
Jsonic.make().use(Expr).options({ rule: { exclude: 'expr' } })
```

This is how you temporarily reuse a shared Jsonic instance without
expression parsing.
Loading
Loading