RPC/txoutproof: Support including (and verifying) proofs of wtxid by luke-jr · Pull Request #32844 · bitcoin/bitcoin

luke-jr · 2025-06-30T23:05:05Z

Feature requested by @SomberNight for Electrum's Lightning stuff. (Please confirm this does what you need)

Unfortunately WIP because the wtxid merkle root calculation doesn't match and I don't have time to figure out why not yet (maybe after concept ACK? or if anyone else wants to take a look...)

Mostly backward compatible, but a segwit-enabled proof will verify the generation tx in old versions.

DrahtBot · 2025-06-30T23:05:09Z

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Code Coverage & Benchmarks

For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/32844.

Reviews

See the guideline for information on the review process.

Type	Reviewers
Concept ACK	Sjors, ajtowns, fjahr

If your review is incorrectly listed, please copy-paste  into the comment that the bot should ignore.

Conflicts

Reviewers, this pull request conflicts with the following ones:

#33116 (refactor: Convert uint256 to Txid by marcofleon)
#33094 (refactor: unify container presence checks by l0rinc)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

LLM Linter (✨ experimental)

Possible typos and grammar issues:

In RPCResult description: "If verify_witness is true and the proof invalid" -> "If verify_witness is true and the proof is invalid" [missing “is” makes the sentence ungrammatical]

No other typos were found.

^{drahtbot_id_4_m}

SomberNight · 2025-07-01T00:25:34Z

Feature requested by @SomberNight for Electrum's Lightning stuff. (Please confirm this does what you need)

Wow, thank you for working on this! I was meaning to open an issue to ask for this, but did not even get to that yet.

The request I would like an electrum server, assuming txindex=1, to be able to serve is:

bc.tx.get_merkle_witness(txid), (so given just a txid,)

return a dict containing:

wtxid
block_height
block_hash
pos (index of tx in block)
merkle or wmerkle
- exactly one of the two
- merkle is the Satoshi-SPV proof, in case there is no witness commitment in the block
- wmerkle is the witness-SPV proof otherwise
cb_tx, the generation/coinbase tx
cb_proof, the Satoshi-SPV proof for the generation tx

Obviously the electrum server can act as middleware and transform the response, assuming the necessary fields are available or obtainable.

As far as I can tell, but I haven't tried yet, the RPC response from this PR can be transformed to what I describe above. So the RPC looks to be doing what I need. :)

I think it is non-trivial, so let me detail how an electrum/light client would then use bc.tx.get_merkle_witness(txid):

it is assumed client already has a full raw tx, and wants to verify whether it has been mined
client also has full header chain, proof-of-work verified
client calls bc.tx.get_merkle_witness(txid)
client calculates wtxid of full tx it has, compares it with received wtxid
using cb_tx, cb_proof, block_height, block_hash
- client Satoshi-SPVs the received coinbase tx, against block header merkle_root
client extracts witness_commitment from cb_tx
1. if there is no witness_commitment,
  - client Satoshi-SPVs the txid, using merkle, block_height, block_hash, pos, against block header merkle_root
  - now client has an SPV guarantee that block only contains non-segwit txs, and tx in particular is also non-segwit (wtxid==txid) and it was mined in this block
2. if there is a witness_commitment,
  - client must do witness-SPV, even if tx seems non-segwit (as e.g. malicious electrum server could have stripped away the witness)
  - client does witness-SPV using wmerkle and pos, against witness_commitment
  - client must check that length of wmerkle matches length of cb_proof. This ensures there are the same number of levels in the wtxid merkle tree and in the txid merkle tree. (the two trees should have the same number of leaves, but this seems hard to check as a light client) This is intended as some protection against merkle-tree-construction weaknesses (cf. banning of 64 byte transactions and similar).
now client has an SPV guarantee that wtxid was mined in this block
- or if some check failed, then the client can connect to another server

Sjors · 2025-07-01T17:37:54Z

Concept ACK. @SomberNight it's still useful if you open an issue to track the reason you need this, just in case this implementation PR doesn't make.

Sjors

Took an initial look at the code, but got a bit confused. Will check again later.

…saction (not a CBlock)

luke-jr · 2025-07-09T03:53:53Z

Reworked this to cover @SomberNight 's requirements and address feedback. Test is now passing.

DrahtBot · 2025-08-13T20:34:09Z

🐙 This pull request conflicts with the target branch and needs rebase.

ajtowns

Transaction proofs are an interoperability feature, I don't think it makes much sense to implement this without also documenting it and providing test vectors via a BIP.

Concept ACK

ajtowns · 2025-10-30T16:57:56Z

    /** Public only for unit testing */
    CBlockHeader header;
    CPartialMerkleTree txn;
+    /** Only used by subclass WtxidInBlockMerkleProof */


There is no such subclass?

I think all this logic should entirely be in a separate class though. For efficiency probably two paths:

if the generation tx has no witness commitment; provde all (w)txids and the generation tx via the block partial merkle tree.

if the generation tx does have a witness commitment; provide just the generation tx's merkle path back to the block header, and prove all wtxids via a partial merkle tree back to the witness commitment

ajtowns · 2025-10-30T17:02:47Z

            {"proof", RPCArg::Type::STR_HEX, RPCArg::Optional::NO, "The hex-encoded proof generated by gettxoutproof"},
+            {"options", RPCArg::Type::OBJ_NAMED_PARAMS, RPCArg::Optional::OMITTED, "",
+                {
+                    {"verify_witness", RPCArg::Type::BOOL, RPCArg::Default{false}, "If true, also verifies the associated wtxid/hash of the specified transactions (if included in proof)"},


I think this should probably be automatic, with the current "array of txids" result being stuck behind a -deprecatedrpc option.

I think that makes sense and we could even do the same for gettxoutproof I think (if that wasn't already intended with the comment as well).

ajtowns · 2025-10-30T17:06:21Z

+                    {RPCResult::Type::STR_HEX, "blockhash", "The block hash this proof links to"},
+                    {RPCResult::Type::NUM, "blockheight", "The height of the block this proof links to"},
+                    {RPCResult::Type::NUM, "confirmations", /*optional=*/true, "Number of blocks (including the one with the transactions) confirming these transactions"},
+                    {RPCResult::Type::NUM, "confirmations_assumed", /*optional=*/true, "The number of unverified blocks confirming these transactions (eg, in an assumed-valid UTXO set)"},


This information (height, confirmations, assumeutxo info) seems better obtained via getblockheader on the blockhash to me.

ajtowns · 2025-10-30T17:09:13Z

+                if (merkleBlock.m_gentx->GetHash() != vMatch[0]) return res;
+                if (!merkleBlock.m_gentx->IsCoinBase()) return res;
+                witness_commit_outidx = GetWitnessCommitmentIndex(*merkleBlock.m_gentx);
+                if (witness_commit_outidx == NO_WITNESS_COMMITMENT) {


I would have expected this logic to be in merkleblock.cpp as a standard part of verifying a proof (eg so that it could be included in kernel), not in the RPC code.

ajtowns · 2025-10-30T17:10:09Z

+
+                const CBlockIndex* pindex = chainman.m_blockman.LookupBlockIndex(merkleBlock.header.GetHash());
+                const auto& active_chain = chainman.ActiveChain();
+                if (!pindex || !active_chain.Contains(pindex) || pindex->nTx == 0) {


Being able to easily prove/verify that a tx was included in a reorged block seems fine?

fjahr

Concept ACK

Is there something special about the naming "generation tx" here? It seems to be just an older alias for the coinbase transaction without any other special meaning. I can not recall seeing it used anywhere in the code base or online discourse. If I am not overlooking something it would probably be better to just call it coinbase tx instead which should be less confusing to newer contributors that haven't heard this term before.

fjahr · 2025-10-30T17:57:34Z

+        expected_proven['confirmations'] = 1
+        del expected_proven['tx'][0]['txid']
+        del expected_proven['tx'][1]['txid']
+        assert_equal(self.nodes[0].verifytxoutproof(proofres['proof'], verify_witness=True), expected_proven)


Should also cover an invalid proof here

fjahr · 2025-10-30T18:09:52Z

            {"proof", RPCArg::Type::STR_HEX, RPCArg::Optional::NO, "The hex-encoded proof generated by gettxoutproof"},
+            {"options", RPCArg::Type::OBJ_NAMED_PARAMS, RPCArg::Optional::OMITTED, "",
+                {
+                    {"verify_witness", RPCArg::Type::BOOL, RPCArg::Default{false}, "If true, also verifies the associated wtxid/hash of the specified transactions (if included in proof)"},


I think that makes sense and we could even do the same for gettxoutproof I think (if that wasn't already intended with the comment as well).

DrahtBot · 2026-01-27T00:18:10Z

⌛ There hasn't been much activity lately and the patch still needs rebase. What is the status here?

Is it still relevant? ➡️ Please solve the conflicts to make it ready for review and to ensure the CI passes.
Is it no longer relevant? ➡️ Please close.
Did the author lose interest or time to work on this? ➡️ Please close it and mark it with one of the labels 'Up for grabs' or 'Insufficient Review', so that it can be picked up in the future.

sedited · 2026-03-03T13:15:37Z

There's been unaddressed review comments here for some time already. @luke-jr do you plan to respond to these? If not, maybe one of the reviewers could take charge of this change?

Sjors · 2026-05-08T15:45:43Z

I plan to take this over and address earlier feedback, unless someone else beats me to it.

Sjors · 2026-06-12T11:09:34Z

Here's an initial sketch: Sjors#116. I plan to open a PR on this repo after some polishing.

For the test vectors I used small and early blocks, so we can include them as fixtures for a roundtrip test. These could be turned into a BIP, but I prefer if someone else does that, @SomberNight? There might be a better set of test vectors, I can switch to those.

I limited my PR to proving a single transaction at a time. Is there a need to provide multiple at once? In that case we might need a BIP to define an encoding for that, as it's less trivial than just an array of hashes. The RPC command takes an array of wtxid's, so it's easy to add support for this later.

Sjors · 2026-06-12T17:14:52Z

@SomberNight wrote:

The request I would like an electrum server, assuming txindex=1, to be able to serve is:

bc.tx.get_merkle_witness(txid), (so given just a txid,)

I assume you can also make the request by wtxid? If not, why not?

DrahtBot added the CI failed label Jul 1, 2025

SomberNight reviewed Jul 1, 2025

View reviewed changes

Comment thread src/rpc/txoutproof.cpp Outdated

SomberNight reviewed Jul 1, 2025

View reviewed changes

Comment thread src/rpc/txoutproof.cpp Outdated

DrahtBot mentioned this pull request Jul 1, 2025

refactor: Txid type safety (parent PR) #32189

Closed

Sjors reviewed Jul 1, 2025

View reviewed changes

Comment thread src/consensus/validation.h Outdated

Comment thread src/consensus/validation.h Outdated

Comment thread src/consensus/validation.h Outdated

Comment thread src/merkleblock.cpp Outdated

Comment thread src/merkleblock.h Outdated

Docfix/RPC: txid is no longer the transaction hash

3004f26

luke-jr force-pushed the rpc_gettxoutproof_segwit branch from 3abfd6d to e826ffd Compare July 9, 2025 03:49

luke-jr marked this pull request as ready for review July 9, 2025 03:49

Allow calling GetWitnessCommitmentIndex with just the generation tran…

528a8c9

…saction (not a CBlock)

luke-jr force-pushed the rpc_gettxoutproof_segwit branch 2 times, most recently from e95981b to 4b3a33a Compare July 9, 2025 03:52

RPC/txoutproof: Support new proof format covering witness data

23edd3d

luke-jr force-pushed the rpc_gettxoutproof_segwit branch from 4b3a33a to 23edd3d Compare July 9, 2025 04:01

luke-jr requested review from Sjors and SomberNight July 9, 2025 04:03

DrahtBot removed the CI failed label Jul 9, 2025

This was referenced Jul 30, 2025

refactor: unify container presence checks #33094

Closed

refactor: Convert uint256 to Txid #33116

Merged

DrahtBot added the Needs rebase label Aug 13, 2025

achow101 requested review from ajtowns and fjahr and removed request for SomberNight October 22, 2025 15:29

ajtowns suggested changes Oct 30, 2025

View reviewed changes

fjahr reviewed Oct 30, 2025

View reviewed changes

achow101 self-requested a review May 8, 2026 14:57

fanquake added Up for grabs labels May 8, 2026

fanquake closed this May 8, 2026

Sjors mentioned this pull request Jun 12, 2026

rpc: add gettxproof and verifytxproof for transactions with witness Sjors/bitcoin#116

Open

1 task

This comment was marked as off-topic.

Sign in to view

Conversation

luke-jr commented Jun 30, 2025

Uh oh!

DrahtBot commented Jun 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Code Coverage & Benchmarks

Reviews

Conflicts

LLM Linter (✨ experimental)

Uh oh!

Uh oh!

SomberNight commented Jul 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Sjors commented Jul 1, 2025

Uh oh!

Sjors left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

luke-jr commented Jul 9, 2025

Uh oh!

DrahtBot commented Aug 13, 2025

Uh oh!

ajtowns left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

fjahr left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

DrahtBot commented Jan 27, 2026

Uh oh!

sedited commented Mar 3, 2026

Uh oh!

Sjors commented May 8, 2026

Uh oh!

Sjors commented Jun 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment was marked as off-topic.

Sjors commented Jun 12, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

9 participants

DrahtBot commented Jun 30, 2025 •

edited

Loading

SomberNight commented Jul 1, 2025 •

edited

Loading

ajtowns left a comment •

edited

Loading

Sjors commented Jun 12, 2026 •

edited

Loading