ipc: add NodeRpc interface for LND

[ViniciusCestarii]

Closes #24

Add new IPC interface NodeRpc implementing methods necessary for LND.

I tried to mirror the necessary JSON-RPC methods so it is friendly to use and easier to follow up with an IPC translator.

I have implemented a simple C++ cli client for testing this new interface: https://github.com/ViniciusCestarii/bitcoin-ipc-demo/tree/2140-lnd-ipc

AI diclaimer: I used Claude Opus 4.8 to help converting the JSON-RPC methods to IPC.

[rustaceanrob]

IWYU can be run locally with:

env -i HOME="$HOME" PATH="$PATH" USER="$USER" MAKEJOBS="-j1" FILE_ENV="./ci/test/00_setup_env_native_iwyu.sh" ./ci/test_run_all.sh

You may want to change the make jobs to j16 or whatever matches your physical cores.

I have this script locally to run clang tidy:

set -euo pipefail

REPO_ROOT="."
BUILD_DIR="${REPO_ROOT}/build_tidy"
JOBS="${MAKEJOBS:--j$(nproc)}"

# Resolve compilers and tools: prefer versioned names if TIDY_LLVM_V is set,
# otherwise fall back to whatever is on PATH.
if [[ -n "${TIDY_LLVM_V:-}" ]]; then
    CLANG="clang-${TIDY_LLVM_V}"
    CLANGXX="clang++-${TIDY_LLVM_V}"
    RUN_CLANG_TIDY="run-clang-tidy-${TIDY_LLVM_V}"
else
    CLANG="${CC:-clang}"
    CLANGXX="${CXX:-clang++}"
    RUN_CLANG_TIDY="run-clang-tidy"
fi

cmake -S "$REPO_ROOT" -B "$BUILD_DIR" \
    -DCMAKE_C_COMPILER="$CLANG" \
    -DCMAKE_CXX_COMPILER="$CLANGXX" \
    -DCMAKE_C_FLAGS_RELWITHDEBINFO='-O0 -g0' \
    -DCMAKE_CXX_FLAGS_RELWITHDEBINFO='-O0 -g0' \
    -DCMAKE_EXPORT_COMPILE_COMMANDS=ON \
    -DBUILD_TESTS=ON \
    -DBUILD_BENCH=ON \
    -DBUILD_FUZZ_BINARY=ON \
    -Wno-error=dev

cmake --build "$BUILD_DIR" "$JOBS"

"$RUN_CLANG_TIDY" \
    -config-file="${REPO_ROOT}/src/.clang-tidy" \
    -quiet \
    -p "$BUILD_DIR" \
    "$JOBS" \
    "$@"

[rustaceanrob]

Ideally this gets upstreamed in the libmultiprocess subtree

[ViniciusCestarii]

Opened bitcoin-core/libmultiprocess#292

[ViniciusCestarii]

Fuzz job was canceled, not an actual fuzz failure, so unrelated to this PR

[ismaelsadeeq]

This is nice.

It would be nice if you could also vibe code an e2e demonstration of usage of this with LND.

[GeorgeTsagk]

It would be nice if you could also vibe code an e2e demonstration of usage of this with LND.

a side note: middleware for the IPC<>RPC communication might be needed

[josibake]

As more of a conceptual question: is implementing a per method map what we want to do? I haven't looked at the code but just judging by the commit messages that seems to be the approach here.

What I had in my head (shower thought, so not at all sure if its reasonable/possible/a good idea/etc) is something where this node maintains generic IPC interfaces (there are already a few upstream PRs for an IPC chain interface, for example). Then these generic interfaces are consumed by a bridging process. In that process is where the code for constructing the RPC calls lives and it is turned into what the downstream consumer wishes to consume (REST, JSON-RPC, Capnproto, ZMQ, etc). Ideally these would even live in their own repo, i.e., a 2140-dev/bridges, or even better the consumer's repo!

As currently written, I'm not super stoked about this approach because it feels like "swap json rpc with capnproto." @ViniciusCestarii really appreciate you exploring the work, however! Would you be interested in exploring the design I just described? If so, I'd recommend starting with the Bitcoin Core IPC interfaces currently proposed to see if that sparks any ideas. This is not a suggestion to map them over to here, however, unless they make sense. Those interfaces were largely designed with legacy compatibility in mind, whereas we can take a fresh look here.

Once you have a proposal for a more generic interface, then I'd suggest writing a bridge (bonus, you can write it in whatever language you want), and if it all looks good we can look into adding it to a bridges repo and looking at ways to package the node + bridges.

[willcl-ark]

bit of a tangent but FWIW i have a branch here dropping libmultiprocess, and using native capnp for a few interfaces here in case it’s of interest: bitcoin/bitcoin@master...willcl-ark:bitcoin:drop-libmultiprocess i have not yet added the chain interface yet, which will require the most boilerplate code.

but whether you have libmiltiprocess or not, i think i agree with josie; the rough (clean) approach should be to expose clean, generic ipc methods. these are then consumed internally, and by things like zmq, rpc, rest, or directly via IPC connection/unix socket.

[ViniciusCestarii]

After thinking through the discussion, I get the "this is just json-rpc with capnp" critique now, and I agree with it.

My current approach maps the specific RPC methods LND needs directly onto the IPC interface. The problem is that this bakes one consumer's API shape into the node's wire interface permanently, and it doesn't actually prove anything about the architecture: of course it works, I designed the IPC methods to be exactly what LND wanted. It's circular.

The more justifiable approach is to expose clean, generic capabilities on the node (a Chain capability, etc., close to what interfaces::Chain already provides internally), and put the consumer-specific logic in a separate bridge process that translates those capabilities into whatever a consumer wants (for LND, the bitcoind JSON-RPC + ZMQ shape) as @josibake suggested.

Two reasons why I believe this is better:

1. The node stays lean and reusable. No huge consumer-specific RPC code that I mapped in the Bitcoin Core IPC interface. This will only exist on the bridge.
2. The LND bridge becomes a real proof. If a bridge built on top of the generic interface can fully reconstruct everything LND needs, then we've demonstrated the generic interface is complete enough to serve a demanding, production consumer.

This design seems more challenging but much more exciting! I appreciate all the feedback!

[josibake]

@ViniciusCestarii thanks for taking the feedback! Given this is more of an open ended design question (with LND as the practical example), it might be worth opening a discussion for this or continuing the discussion on the already open issue. Once you have a design you are happy with, we can come back to the implementation in the PR.