Skip to content

Fix silent key corruption above 125-bit intervals: widen distance field to 256 bits - #158

Open
Dileep-Kumar-5 wants to merge 7 commits into
JeanLucPons:masterfrom
Dileep-Kumar-5:feat/256bit-distance-field
Open

Fix silent key corruption above 125-bit intervals: widen distance field to 256 bits#158
Dileep-Kumar-5 wants to merge 7 commits into
JeanLucPons:masterfrom
Dileep-Kumar-5:feat/256bit-distance-field

Conversation

@Dileep-Kumar-5

@Dileep-Kumar-5 Dileep-Kumar-5 commented Aug 6, 2026

Copy link
Copy Markdown

The bug

ENTRY.d packs the travelled distance into 128 bits as b127=sign b126=kangaroo type, b125..b0 distance, leaving 126 bits of magnitude. Distances span the interval width, so anything wider than 125 bits overflows — hence the README's "This program is limited to a 125bit interval search."

The problem is what happens when you exceed it. Convert() masks the excess away with & 0x3FFFFFFFFFFFFFFF, stores a valid-looking entry, and on collision yields a wrong private key with no error, no warning and no crash.

Measured on puzzle #140's real range (2^139), current master vs this branch, same input, same machine:

DPs stored -wcheck
master 376,836 0.024% OK (376,747 corrupt)
this branch 408,118 100.000% OK

master reports normal progress throughout. Nothing indicates the run is worthless. Filed separately as #159.

The change

The wider field is opt-in, and the default build is byte-compatible with upstream.

DIST_WORDS selects the distance width: 2 by default, 4 with -DWIDE_DIST (or by uncommenting WIDE_DIST in Constants.h).

default -DWIDE_DIST
max interval 125 bits 253 bits
ENTRY 32 bytes 48 bytes
ITEM 56 bytes 72 bytes
KSIZE 10 words 12 words
DP packet 40 bytes 56 bytes
kangaroo transfer 16 bytes 32 bytes

The default column is exactly today's layout, including the original work-file and protocol magics, so a default build reads and writes upstream's files and talks to upstream peers unchanged.

Rather than #ifdef-ing every use site, a dist_t typedef plus DIST_WORDS lets one implementation serve both widths — the sign and type flags always sit in the top two bits of the top word, so Convert, CalcDistAndType, CreateEntry, sameDist and the four network checksum accumulations became loops. That also deleted the legacy 126-bit overloads and Widen(): at DIST_WORDS == 2, dist_t simply is int128_t. Only genuinely width-specific code stayed conditional.

Over-range distances now abort instead of truncating, and name the fix:

HashTable::Convert: travelled distance exceeds 126 bits.
Interval is too large for the DP entry format (max 125 bits).
Rebuild with -DWIDE_DIST for intervals up to 253 bits.
Aborting rather than storing a truncated distance.

Three other defects fixed along the way, all independent of the width work:

  • Latent out-of-bounds write: ModNeg256Order(dist[g]) writes r[0..3] into a 2-word dist[g], clobbering the next kangaroo. Only dormant because USE_SYMMETRY ships commented out.
  • CUDA 13 build break: cudaDeviceProp::computeMode was removed, so GPUEngine.cu no longer compiles at all against a current toolkit. Switched to cudaDeviceGetAttribute. This is why the branch builds today. Filed separately as GPUEngine.cu does not compile with CUDA 13: cudaDeviceProp::computeMode was removed #160.
  • Stale SM table: _ConvertSMVer2Cores() stops at sm_75, so Ampere and newer return 0. Not just the (16x0 cores) banner — GetGridSize() does *y = 2 * _ConvertSMVer2Cores(...), gets 0, and falls back to *y = 128, so the default grid Y on an sm_86 card is 128 instead of the intended 256. Added sm_80/86/87/89/90 and Blackwell. Measured back to back with -g forced: 32x128 = 338.9 MK/s, 32x256 = 345.5 MK/s (+2.0%).

Verification

RTX 3050 Laptop, sm_86, CUDA 13.3.73, MSVC 14.44. Both widths built and run. All DP validity via -wcheck:

default build -DWIDE_DIST build
56-bit sample key correct correct
#120, 2^119 34,906 DPs, 100.000% 100.000%
#140, 2^139 aborts, exit 255 34,762 DPs, 100.000%
work-file magic 0xFA6A8001 (upstream's) 0xFA6A8011

Each build rejects the other's work file with "Not a work file" — a width mismatch is refused, never misparsed.

Also verified on the wide build: 2^134 (#135, which master cannot attempt at all) at 100.000%, and client/server on localhost — 196,231 DPs over TCP at 100.000%, kangaroo backup uploaded (536,576 x 32 bytes), downloaded (2^19.03 kangaroos loaded, 0 created), and DPs generated from the downloaded herd validating at 100.000%.

tools/check_layout.py and tools/test_distfield.py verify both widths in one run — producer/consumer agreement on every offset, the AddDist carry chain, the checksum truncation, the magics, and the bit-packing round-trip — and assert the default reproduces upstream's sizes. Neither needs a compiler or a GPU.

Costs

Only when -DWIDE_DIST is enabled: DP tables +50% RAM, device kangaroo memory +20%, DP packets +40%, kangaroo transfers +100%, and 4-7% throughput. ptxas: same 98 registers, zero spills either way, stack frame 18560 -> 20608 bytes.

A default build costs nothing and changes no format.

Splitting

The CUDA 13 fix (#160) and the SM table fix are each one self-contained commit with no bearing on the distance width, and are worth taking on their own even if the rest is rejected — without the first, nobody can build the GPU version against a current toolkit at all. Happy to open them separately if you prefer.

🤖 Generated with Claude Code

Dileep-Kumar-5 and others added 5 commits August 5, 2026 22:56
…vals)

The 125-bit interval limit was a struct field width, not a mathematical
one. ENTRY.d packed the travelled distance into 128 bits as
"b127=sign b126=kangaroo type, b125..b0 distance", leaving 126 bits of
magnitude. Distances span the interval width, so anything wider than
125 bits overflowed.

It overflowed silently. Convert() masked the excess away with
& 0x3FFFFFFFFFFFFFFF, stored a valid-looking entry, and on collision
produced a wrong private key with no error, no warning and no crash.
Measured on the real puzzle JeanLucPons#140 range (2^139): stock JLP stores
376,747 corrupt DPs out of 376,836 (-wcheck reports 0.024% OK) while
reporting normal progress throughout.

Distance is now 256 bits, flags in the top two bits, 254 bits of
magnitude -> intervals up to 253 bits. Over-range distances abort
instead of truncating.

Storage and host side:
- int256_t; ENTRY 32 -> 48 bytes; ENTRY_SIZE replaces hard-coded 32/16
- sameDist() compares all four words; the old check compared two of
  four and would mis-flag a real collision as a duplicate
- work file magics bumped: old files are rejected, not misparsed

GPU:
- kernel dist[GPU_GRP_SIZE] 2 -> 4 words, KSIZE 10 -> 12 (11 -> 13 sym)
- new Add256 carries through all four words
- jD[NB_JUMP] 2 -> 4 words; SetParams uploads the full jump table
- ITEM_SIZE 56 -> 72; OutputDP writes 8 distance words, kIdx at +17
- fixes a latent out-of-bounds write: ModNeg256Order(dist[g]) wrote
  r[0..3] into a 2-word dist[g], clobbering the next kangaroo. Only
  dormant because USE_SYMMETRY ships commented out.

Network:
- DP packet 40 -> 56 bytes, all four distance words sent
- kangaroo blocks 16 -> 32 bytes, checksums cover all four words
- SERVER_HEADER bumped so old peers fail the handshake

The now-unreachable 126-bit Add/Convert/CalcDistAndType/Widen overloads
are removed.

Verified: CPU build (MSVC) solves the 56-bit sample key correctly, and
on the 2^139 puzzle JeanLucPons#140 range stores 408,118 DPs at 100.000% -wcheck.
NOT verified: the GPU path has never been through nvcc, and client/
server mode compiles but was not exercised at runtime.

Costs: DP tables +50% RAM, device kangaroo memory +20%, DP packets
+40%, kangaroo transfers +100%. Wider dist raises local-memory
pressure in the kernel; expect to lower -g.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two changes needed to actually compile the GPU path.

cudaDeviceProp::computeMode was removed in CUDA 13, so GPUEngine.cu no
longer compiled at all -- this is upstream breakage, unrelated to the
256-bit distance work. The cudaDevAttrComputeMode device attribute still
exists, so query that instead and keep the same startup line. Guard the
index since the attribute is not bounded by the sComputeMode table.

build_gpu.bat now prefers a toolkit extracted beside the repo before
falling back to the Program Files install. Installing CUDA properly
needs admin; NVIDIA's per-component redist archives (cuda_nvcc,
cuda_cudart, cuda_crt, libnvvm -- 89 MB total vs ~9 GB) extract to a
user-writable directory and are enough to build.

Verified on an RTX 3050 Laptop (sm_86, CUDA 13.3.73, MSVC 14.44):
- solves the 56-bit sample key correctly at 322 MK/s
- on the 2^139 puzzle JeanLucPons#140 range, 713,781 GPU-produced DPs at
  100.000% -wcheck
- ptxas: 98 registers, 0 spills, stack frame 18560 -> 20608 bytes
- throughput 382.7 -> 366.7 MK/s, about 4% slower (single run each)

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The kangaroo backup never completed: the server writes <file>.tmp and
only renames it into place when the client's checksum matches, so every
transfer silently produced a stale .tmp and no backup.

checkSum is an Int (5 x 64 bits) but only its low 32 bytes are put on the
wire, and the receiver compares with IsEqual, which tests all five words.
With the old 126-bit distances a herd summed to well under 2^256 and
bits64[4] stayed zero, so nobody noticed. The 256-bit distance puts the
kangaroo type at b254 and the sign at b255, so ~2^19 kangaroos overflow
past 2^256 and bits64[4] no longer matches the freshly-zeroed receiver.

Define the checksum as mod 2^256 -- what the wire actually carries -- and
clear bits64[4] on both sides before sending and before comparing. All
four sites: server send/compare and client send/compare.

check_layout.py now asserts all four sites truncate before use.

Verified on localhost, 2^139 range, GPU client:
- DP wire: 196,231 DPs arrive over TCP, 100.000% -wcheck
- upload:  kang written and renamed, 17,170,448 bytes (536,576 x 32)
- download: "2^19.03 kangaroos loaded, 0 created"
- DPs generated from downloaded kangaroos: 216,445, 100.000% -wcheck

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
These are what the previous two commits refer to; they lived outside the
repo until now.

tools/check_layout.py parses the sources and asserts producer/consumer
agreement on everything the widening touched by hand: each ITEM uint32
written exactly once and read at the same offset, device slot usage
bounded by KSIZE, the Add256 carry chain order, DP struct size against
its own runtime assertion, all four checksum sites truncating before
use, and both format magics bumped. It caught two real defects during
this work -- KBuff mallocs still cast to int128_t*, and an over-broad
rule of its own.

tools/test_distfield.py models Convert/CalcDistAndType mask-for-mask and
checks sign/type/magnitude round-trip at 139..253 bits, that a 254-bit
magnitude is rejected rather than truncated, and that the flag bits are
disjoint from the magnitude.

Neither needs a compiler or a GPU:
  python tools/check_layout.py
  python tools/test_distfield.py

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
tools/verify_target.py derives the P2PKH address from a target's
compressed pubkey and checks it against the published puzzle address,
plus that the range is the canonical [2^(n-1), 2^n-1]. A pubkey pasted
from a forum that does not belong to the address makes an entire run
worthless and nothing in the solver would ever say so. Self-check
derives puzzle JeanLucPons#105's address from its published private key.

tools/kangaroo_est.py estimates runtime, cost and DP storage. It now
flags puzzles that are already solved -- JeanLucPons#120 prints a 2.67x "profit"
that is pure fiction because the coins were swept years ago -- and
rejects a custom GPU profile that is missing --watts instead of
crashing on None.

Verified both builds against puzzle JeanLucPons#120 (2^119, inside the old cap):
  stock   17,088 DPs  100.000% OK
  patched 15,795 DPs  100.000% OK
versus puzzle JeanLucPons#140 (2^139, outside it):
  stock      0.024% OK
  patched  100.000% OK
So the widening fixes the >125-bit case without disturbing the case
stock already handled.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Dileep-Kumar-5 and others added 2 commits August 6, 2026 06:27
The table stops at sm_75, so every GPU from Ampere on falls through to
the `return 0` default. Two consequences, and only the first is cosmetic:

- the startup banner prints "(16x0 cores)" on an RTX 3050
- GetGridSize() computes `*y = 2 * _ConvertSMVer2Cores(...)`, gets 0, and
  lands on the `if(*y <= 0) *y = 128` fallback -- so the default grid Y is
  128 instead of the intended 2 x 128 = 256 on any sm_86 card

Added sm_80/86/87/89/90/a0/a1/c0 with NVIDIA's per-SM core counts.

Measured on an RTX 3050 Laptop, same binary, -g forced, back to back:

  grid 32x128 (old default)   338.9 MK/s
  grid 32x256 (new default)   345.5 MK/s   +2.0%

Modest. A short sample suggested +16%, but that did not survive a matched
window -- this laptop GPU throttles, so only back-to-back deltas mean
anything. The grid also doubles kangaroo memory, 57 -> 105 MB.

Nothing here depends on the 256-bit distance work; it is a
correctness fix to a lookup table that has simply gone stale.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The 256-bit distance is now opt-in. DIST_WORDS selects the width:
2 by default, 4 with -DWIDE_DIST (or by uncommenting WIDE_DIST in
Constants.h).

This addresses the main objection to the previous form. A default build
now reproduces upstream's layout exactly -- ENTRY 32 bytes, ITEM 56,
KSIZE 10, DP packet 40, kangaroo transfer 16, and the original work-file
and protocol magics -- so it reads and writes upstream's files and talks
to upstream peers. Only -DWIDE_DIST changes any of that.

Rather than #ifdef every use site, a dist_t typedef plus DIST_WORDS lets
one implementation serve both widths: the sign and kangaroo-type flags
always occupy the top two bits of the top word, so Convert,
CalcDistAndType, CreateEntry, sameDist and the four network checksum
accumulations became loops over DIST_WORDS. That also removed the legacy
126-bit Add/Convert/CalcDistAndType overloads and Widen() -- at
DIST_WORDS == 2 dist_t simply is int128_t.

Only genuinely width-specific code is conditional: the two extra device
slots in Load/StoreKangaroos, the four extra uint32 in OutputDP, the
Add128/Add256 selection behind AddDist, and the format magics.

Exceeding the compiled limit still aborts, and now names the fix:

  HashTable::Convert: travelled distance exceeds 126 bits.
  Interval is too large for the DP entry format (max 125 bits).
  Rebuild with -DWIDE_DIST for intervals up to 253 bits.
  Aborting rather than storing a truncated distance.

Verified, both builds, RTX 3050 Laptop / CUDA 13.3 / sm_86:
- default and wide both solve the 56-bit sample key correctly
- default on JeanLucPons#120 (2^119): 34,906 DPs, 100.000% -wcheck
- default on JeanLucPons#140 (2^139): aborts as above, exit 255
- wide on JeanLucPons#140 (2^139): 34,762 DPs, 100.000% -wcheck
- default writes magic 0xFA6A8001, i.e. upstream's HEADW
- each build rejects the other's work file ("Not a work file")

tools/check_layout.py and tools/test_distfield.py now verify BOTH widths
in one run, and assert the default reproduces upstream's sizes. Neither
needs a compiler or a GPU. build_gpu.bat takes an optional "wide" arg.

README documents the option, the costs and the incompatibility.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@Dileep-Kumar-5

Copy link
Copy Markdown
Author

Updated in af9c5aa: the wider distance field is now opt-in, and a default build is byte-compatible with upstream. The description above has been rewritten accordingly.

I originally offered this as an option and it was the right call — it removes the main reason to reject the PR.

DIST_WORDS selects the width: 2 by default, 4 with -DWIDE_DIST (or by uncommenting WIDE_DIST in Constants.h).

default -DWIDE_DIST
max interval 125 bits 253 bits
ENTRY 32 bytes 48 bytes
ITEM 56 bytes 72 bytes
KSIZE 10 words 12 words
DP packet 40 bytes 56 bytes
kangaroo transfer 16 bytes 32 bytes

The default column is exactly today's layout, original work-file and protocol magics included. I verified that end to end rather than by inspection: a default build writes a work file whose magic reads back as 0xFA6A8001, i.e. upstream's HEADW. It reads and writes upstream's files and talks to upstream peers unchanged.

What this means for review: the format change is no longer something you have to accept to take the fix. On a default build the only behavioural difference is that an over-range interval now stops with an error instead of silently producing a wrong key.

The implementation also got smaller. Instead of #ifdef-ing every use site, a dist_t typedef plus DIST_WORDS lets one implementation serve both widths — the sign and type flags always occupy the top two bits of the top word, so Convert, CalcDistAndType, CreateEntry, sameDist and the four network checksum accumulations became loops over DIST_WORDS. That deleted the legacy 126-bit Add/Convert/CalcDistAndType overloads and Widen() outright: at DIST_WORDS == 2, dist_t is int128_t. Only genuinely width-specific code stayed conditional — the two extra device slots in Load/StoreKangaroos, the four extra uint32 in OutputDP, the Add128/Add256 choice behind AddDist, and the magics.

Both configurations built and run on an RTX 3050 Laptop (sm_86, CUDA 13.3.73, MSVC 14.44):

default build -DWIDE_DIST build
56-bit sample key correct correct
#120, 2^119 34,906 DPs, 100.000% -wcheck 100.000%
#140, 2^139 aborts, exit 255 34,762 DPs, 100.000%

Each build rejects the other's work file with "Not a work file", so a width mismatch is refused rather than misparsed.

tools/check_layout.py and tools/test_distfield.py now check both widths in a single run and assert the default reproduces upstream's sizes, so a future edit that quietly breaks compatibility fails the check. Neither needs a compiler or a GPU:

python tools/check_layout.py
python tools/test_distfield.py

The CUDA 13 fix (#160) and the sm_86 table fix remain independent single commits, still worth taking on their own even if the width work isn't wanted.

@Dileep-Kumar-5

Copy link
Copy Markdown
Author

Additional verification on puzzle #135, which is the target that motivates the whole patch and the one case af9c5aa's commit message does not cover — I ran it after committing.

#135 is a 2^134 interval, so master cannot attempt it at all: it is above the 125-bit cap, and before this branch it would have run happily while corrupting every DP.

Target checked before spending GPU time on it, since a pubkey that does not belong to the address makes a run worthless and nothing in the solver would say so:

puzzle #135
  pubkey on curve   yes
  address expected  16RGFo6hjq9ym6Pj7N5H7L1NR1rVPJyw2v
  address derived   16RGFo6hjq9ym6Pj7N5H7L1NR1rVPJyw2v   MATCH
  range             2^134 .. 2^135-1   OK

Default build refuses it, which is the intended behaviour:

HashTable::Convert: travelled distance exceeds 126 bits.
Interval is too large for the DP entry format (max 125 bits).
Rebuild with -DWIDE_DIST for intervals up to 253 bits.
Aborting rather than storing a truncated distance.

-DWIDE_DIST build runs it correctly:

range 2^134
ops 10,808,848,384 in 28s = 386.0 MK/s
DPs 10,423
-wcheck 100.000% OK

Worth noting for the review: ptxas output is byte-identical to the hard-coded 256-bit version from before the refactor — 98 registers, 0 spill stores, 0 spill loads, 20608-byte stack frame. Making the width configurable via DIST_WORDS cost nothing in generated code; the #if blocks and loops all fold away at compile time.

So the full validated matrix across both builds is now:

interval default build -DWIDE_DIST
56-bit sample key solves correctly solves correctly
2^119 (#120) 100.000% 100.000%
2^134 (#135) refuses, exit 255 100.000%
2^139 (#140) refuses, exit 255 100.000%

For context on the scale rather than the correctness: #135 in one pass is ~3.07e20 operations, which on this RTX 3050 Laptop is roughly 25,200 GPU-years. The run above covered about 3.5e-9 percent of it. Nobody should read these numbers as progress on the puzzle — they only demonstrate that the DPs being produced are valid, which on master they would not be.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant