Skip to content

feat(vm): inspect protocol for VM values via boundary display structs#218

Merged
davydog187 merged 4 commits into
mainfrom
dx/inspect-protocol
May 9, 2026
Merged

feat(vm): inspect protocol for VM values via boundary display structs#218
davydog187 merged 4 commits into
mainfrom
dx/inspect-protocol

Conversation

@davydog187
Copy link
Copy Markdown
Contributor

Inspect protocol for VM values

Plan: .agents/plans/A27-inspect-protocol.md

Goal

Implement Inspect for the four opaque VM value tags so iex shows
something useful instead of a tuple soup. Today, a {:tref, 7} in
the iex output tells the user nothing about what's in the table; a
{:lua_closure, _, _} tells them nothing about the function.

Targets:

  • {:tref, integer()} — table reference. Renders contents (or
    a short summary if large) and the table id.
  • {:lua_closure, _proto, _upvalues} — Lua function. Renders
    source location and arity (with +... for varargs).
  • {:native_func, fun} — Elixir-defined Lua callable. Renders
    module/function/arity via the function's own inspect output.
  • {:udref, integer()} — userdata reference. Renders the
    underlying Elixir term and the udref id.

Approach: boundary wrap

When a Lua value crosses out to Elixir (Lua.eval/2 / Lua.eval!/2
return path), wrap each VM tag in a thin display struct. The struct
exists only for outbound display; internally the VM still uses the
tagged-tuple form, and pattern matches against {:tref, _} etc.
inside the VM are unaffected.

Module layout:

  • Lua.VM.Display.Table[:id, :peek, :ref]
  • Lua.VM.Display.Closure[:source, :line, :arity, :vararg?, :ref]
  • Lua.VM.Display.NativeFunc[:fun, :ref]
  • Lua.VM.Display.Userdata[:id, :term, :ref]

The display structs each carry the original VM tag tuple on :ref, so
callers can round-trip the value back into the VM. Lua.set!/3,
Lua.encode!/2, Lua.decode!/2, and Lua.call_function/3 all
unwrap display structs at their entry points so wrapped values flow
back through the public API without ceremony. A new public
Lua.unwrap/1 helper exposes the underlying tuple for any caller
that needs it directly.

Decode-mode matrix

tag decode: true (default) decode: false
{:tref, _} unchanged (list of tuples) %Lua.VM.Display.Table{}
{:udref, _} unchanged ({:userdata, term}) %Lua.VM.Display.Userdata{}
{:lua_closure, _, _} %Lua.VM.Display.Closure{} %Lua.VM.Display.Closure{}
{:native_func, _} %Lua.VM.Display.NativeFunc{} %Lua.VM.Display.NativeFunc{}

Default-decode tables and userdata keep their existing
Elixir-friendly shapes (list of {k, v} tuples and
{:userdata, term}) so deflua flows and existing doctests are
unaffected. Closures and native funcs are wrapped in both modes
because their default decoded form has always been the raw tuple.
Two pre-existing tests pattern-matching on {:tref, _} from
decode: false were updated to match the new %Lua.VM.Display.Table{}
shape (and exercise Lua.unwrap/1).

Success criteria

  • Lua.eval!(Lua.new(), "return {1, 2, 3}", decode: false) shows
    #Lua.Table<id: 11, [1, 2, 3]> instead of {:tref, 11}. Verified
    manually and in test/lua/vm/display_test.exs.
  • Closures render as #Lua.Closure<source: "<eval>", line: 1, arity: 2>
    (and arity: N+... for variadics) in both decode modes.
  • Native funcs render as #Lua.NativeFunc<#Function<...>> in both
    decode modes.
  • Lua.eval!(lua, "return opaque", decode: false) for a userdata
    shows #Lua.Userdata<id: 0, term: %{secret: 42}>.
  • Inspect output respects Inspect.Opts — large table peeks
    truncate to :limit, and to_doc/2 is used for nested values.
  • No regression in default-decode shape: tables stay as
    list-of-tuples; {:userdata, term} is preserved.
  • mix test passes (1654 tests, 0 failures, +28 from
    test/lua/vm/display_test.exs).

Manual verification

iex> {[t], _} = Lua.eval!(Lua.new(), "return {1, 2, 3}", decode: false)
iex> t
#Lua.Table<id: 11, [1, 2, 3]>

iex> {[t], _} = Lua.eval!(Lua.new(), "return {a = 1, b = 2}", decode: false)
iex> t
#Lua.Table<id: 11, %{"a" => 1, "b" => 2}>

iex> {[c], _} = Lua.eval!(Lua.new(), "return function(a, b) return a + b end")
iex> c
#Lua.Closure<source: "<eval>", line: 1, arity: 2>

iex> {[c], _} = Lua.eval!(Lua.new(), "return function(a, ...) return a end")
iex> c
#Lua.Closure<source: "<eval>", line: 1, arity: 1+...>

iex> {[f], _} = Lua.eval!(Lua.new(), "return string.lower")
iex> f
#Lua.NativeFunc<#Function<1.12573577/2 in Lua.VM.Stdlib.String.string_lower>>

iex> lua = Lua.set!(Lua.new(), [:opaque], {:userdata, %{secret: 42}})
iex> {[u], _} = Lua.eval!(lua, "return opaque", decode: false)
iex> u
#Lua.Userdata<id: 0, term: %{secret: 42}>

Changes

 .agents/plans/A27-inspect-protocol.md | 124 ++++++++++++----
 lib/lua.ex                            |  50 ++++++-
 lib/lua/vm/display.ex                 | 159 ++++++++++++++++++++
 lib/lua/vm/display/closure.ex         |  57 +++++++
 lib/lua/vm/display/native_func.ex     |  41 ++++++
 lib/lua/vm/display/table.ex           |  47 ++++++
 lib/lua/vm/display/userdata.ex        |  44 ++++++
 test/lua/util_test.exs                |   6 +-
 test/lua/vm/display_test.exs          | 269 ++++++++++++++++++++++++++++++++++
 test/lua_test.exs                     |   8 +-
 10 files changed, 771 insertions(+), 34 deletions(-)

Verification

$ mix format
$ mix compile --warnings-as-errors
Generated lua app

$ mix test
55 doctests, 51 properties, 1654 tests, 0 failures, 30 skipped

$ mix test --only lua53
29 tests, 0 failures, 23 skipped (1731 excluded)

Discoveries

  • Lua.Table already exists as a public utility module (for casting
    decoded tables to lists/maps). Display structs live under
    Lua.VM.Display.* to make the namespace clear and avoid collisions.
    Inspect output still uses the short forms (#Lua.Table<...>,
    #Lua.Closure<...>, etc.) regardless of full module path.
  • Closures need source/line/arity for human display. The existing
    Lua.Compiler.Prototype struct carries :source, :lines (a
    {first, last} tuple), :param_count, and :is_vararg — the
    display wrap reads from those four fields, no proto changes needed.
  • Round-trip support (Lua.set!/3, Lua.encode!/2, etc.) had to be
    added to keep the wrap transparent for existing flows. The cleanest
    fix was a single Display.unwrap/1 call at each public-API entry
    point, plus an opportunistic Util.encoded?/1 shortcut in encode!
    to handle the case where the unwrapped value is already a raw VM tag.

Out of scope (intentional)

  • Changing the internal tuple representation. The wrap is a display
    affordance, not a data-model change.
  • Pretty-printing for very large tables beyond Inspect.Opts.limit
    truncation. The peek caps at :limit entries and shows for
    the rest.
  • Lua.dbg/2 helper — that's plan A28.
  • Wrapping userdata in default decode: true mode (would change the
    existing {:userdata, term} API; out of scope by design).

davydog187 added 4 commits May 9, 2026 06:42
Tighten Success criteria, Implementation notes, and Risks to remove
ambiguity about which value tags get wrapped in which decode mode.
The wrap layer leaves `decode: true` table/userdata shape unchanged
(preserving the existing list-of-tuples and `{:userdata, term}` API
that all doctests and `deflua` flows depend on), and only wraps
closures/native_funcs in default decode mode (where they already
leak as raw tuples today). Decode: false wraps all four kinds.

Module names move under `Lua.VM.Display.*` to avoid collision with
the existing `Lua.Table` public utility module.

Plan: A27
Wrap opaque VM tags returned from `Lua.eval/2` and `Lua.eval!/2` in
display structs so iex shows something legible instead of raw tuples
like `{:tref, 7}` or `{:lua_closure, _, _}`. Internal pattern matches
on those tags are unaffected because the wrap fires at the eval
boundary only.

Decode-mode matrix:

| tag                  | decode: true                | decode: false               |
|----------------------|-----------------------------|-----------------------------|
| {:tref, _}           | unchanged (list of tuples)  | %Lua.VM.Display.Table{}     |
| {:udref, _}          | unchanged ({:userdata, t})  | %Lua.VM.Display.Userdata{}  |
| {:lua_closure, _, _} | %Lua.VM.Display.Closure{}   | %Lua.VM.Display.Closure{}   |
| {:native_func, _}    | %Lua.VM.Display.NativeFunc{}| %Lua.VM.Display.NativeFunc{}|

Adds a public `Lua.unwrap/1` helper for callers that need the raw
VM tag (e.g. to round-trip back through internal helpers), and
teaches `Lua.set!/3`, `Lua.encode!/2`, `Lua.decode!/2`, and
`Lua.call_function/3` to unwrap display structs at their entry
points so the wrapped values round-trip seamlessly.

Default-decode tables and userdata keep their existing list-of-tuples
and `{:userdata, term}` shapes, so deflua flows and existing
doctests are unaffected. Two pre-existing tests that pattern-matched
on `{:tref, _}` from `decode: false` were updated to match the new
`%Lua.VM.Display.Table{}` shape (and exercise `Lua.unwrap/1`).

Plan: A27
@davydog187 davydog187 merged commit fba1355 into main May 9, 2026
4 checks passed
@davydog187 davydog187 deleted the dx/inspect-protocol branch May 9, 2026 19:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant