Fix silent key corruption above 125-bit intervals: widen distance field to 256 bits - #158
Fix silent key corruption above 125-bit intervals: widen distance field to 256 bits#158Dileep-Kumar-5 wants to merge 7 commits into
Conversation
…vals) The 125-bit interval limit was a struct field width, not a mathematical one. ENTRY.d packed the travelled distance into 128 bits as "b127=sign b126=kangaroo type, b125..b0 distance", leaving 126 bits of magnitude. Distances span the interval width, so anything wider than 125 bits overflowed. It overflowed silently. Convert() masked the excess away with & 0x3FFFFFFFFFFFFFFF, stored a valid-looking entry, and on collision produced a wrong private key with no error, no warning and no crash. Measured on the real puzzle JeanLucPons#140 range (2^139): stock JLP stores 376,747 corrupt DPs out of 376,836 (-wcheck reports 0.024% OK) while reporting normal progress throughout. Distance is now 256 bits, flags in the top two bits, 254 bits of magnitude -> intervals up to 253 bits. Over-range distances abort instead of truncating. Storage and host side: - int256_t; ENTRY 32 -> 48 bytes; ENTRY_SIZE replaces hard-coded 32/16 - sameDist() compares all four words; the old check compared two of four and would mis-flag a real collision as a duplicate - work file magics bumped: old files are rejected, not misparsed GPU: - kernel dist[GPU_GRP_SIZE] 2 -> 4 words, KSIZE 10 -> 12 (11 -> 13 sym) - new Add256 carries through all four words - jD[NB_JUMP] 2 -> 4 words; SetParams uploads the full jump table - ITEM_SIZE 56 -> 72; OutputDP writes 8 distance words, kIdx at +17 - fixes a latent out-of-bounds write: ModNeg256Order(dist[g]) wrote r[0..3] into a 2-word dist[g], clobbering the next kangaroo. Only dormant because USE_SYMMETRY ships commented out. Network: - DP packet 40 -> 56 bytes, all four distance words sent - kangaroo blocks 16 -> 32 bytes, checksums cover all four words - SERVER_HEADER bumped so old peers fail the handshake The now-unreachable 126-bit Add/Convert/CalcDistAndType/Widen overloads are removed. Verified: CPU build (MSVC) solves the 56-bit sample key correctly, and on the 2^139 puzzle JeanLucPons#140 range stores 408,118 DPs at 100.000% -wcheck. NOT verified: the GPU path has never been through nvcc, and client/ server mode compiles but was not exercised at runtime. Costs: DP tables +50% RAM, device kangaroo memory +20%, DP packets +40%, kangaroo transfers +100%. Wider dist raises local-memory pressure in the kernel; expect to lower -g. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two changes needed to actually compile the GPU path. cudaDeviceProp::computeMode was removed in CUDA 13, so GPUEngine.cu no longer compiled at all -- this is upstream breakage, unrelated to the 256-bit distance work. The cudaDevAttrComputeMode device attribute still exists, so query that instead and keep the same startup line. Guard the index since the attribute is not bounded by the sComputeMode table. build_gpu.bat now prefers a toolkit extracted beside the repo before falling back to the Program Files install. Installing CUDA properly needs admin; NVIDIA's per-component redist archives (cuda_nvcc, cuda_cudart, cuda_crt, libnvvm -- 89 MB total vs ~9 GB) extract to a user-writable directory and are enough to build. Verified on an RTX 3050 Laptop (sm_86, CUDA 13.3.73, MSVC 14.44): - solves the 56-bit sample key correctly at 322 MK/s - on the 2^139 puzzle JeanLucPons#140 range, 713,781 GPU-produced DPs at 100.000% -wcheck - ptxas: 98 registers, 0 spills, stack frame 18560 -> 20608 bytes - throughput 382.7 -> 366.7 MK/s, about 4% slower (single run each) Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The kangaroo backup never completed: the server writes <file>.tmp and only renames it into place when the client's checksum matches, so every transfer silently produced a stale .tmp and no backup. checkSum is an Int (5 x 64 bits) but only its low 32 bytes are put on the wire, and the receiver compares with IsEqual, which tests all five words. With the old 126-bit distances a herd summed to well under 2^256 and bits64[4] stayed zero, so nobody noticed. The 256-bit distance puts the kangaroo type at b254 and the sign at b255, so ~2^19 kangaroos overflow past 2^256 and bits64[4] no longer matches the freshly-zeroed receiver. Define the checksum as mod 2^256 -- what the wire actually carries -- and clear bits64[4] on both sides before sending and before comparing. All four sites: server send/compare and client send/compare. check_layout.py now asserts all four sites truncate before use. Verified on localhost, 2^139 range, GPU client: - DP wire: 196,231 DPs arrive over TCP, 100.000% -wcheck - upload: kang written and renamed, 17,170,448 bytes (536,576 x 32) - download: "2^19.03 kangaroos loaded, 0 created" - DPs generated from downloaded kangaroos: 216,445, 100.000% -wcheck Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
These are what the previous two commits refer to; they lived outside the repo until now. tools/check_layout.py parses the sources and asserts producer/consumer agreement on everything the widening touched by hand: each ITEM uint32 written exactly once and read at the same offset, device slot usage bounded by KSIZE, the Add256 carry chain order, DP struct size against its own runtime assertion, all four checksum sites truncating before use, and both format magics bumped. It caught two real defects during this work -- KBuff mallocs still cast to int128_t*, and an over-broad rule of its own. tools/test_distfield.py models Convert/CalcDistAndType mask-for-mask and checks sign/type/magnitude round-trip at 139..253 bits, that a 254-bit magnitude is rejected rather than truncated, and that the flag bits are disjoint from the magnitude. Neither needs a compiler or a GPU: python tools/check_layout.py python tools/test_distfield.py Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
tools/verify_target.py derives the P2PKH address from a target's compressed pubkey and checks it against the published puzzle address, plus that the range is the canonical [2^(n-1), 2^n-1]. A pubkey pasted from a forum that does not belong to the address makes an entire run worthless and nothing in the solver would ever say so. Self-check derives puzzle JeanLucPons#105's address from its published private key. tools/kangaroo_est.py estimates runtime, cost and DP storage. It now flags puzzles that are already solved -- JeanLucPons#120 prints a 2.67x "profit" that is pure fiction because the coins were swept years ago -- and rejects a custom GPU profile that is missing --watts instead of crashing on None. Verified both builds against puzzle JeanLucPons#120 (2^119, inside the old cap): stock 17,088 DPs 100.000% OK patched 15,795 DPs 100.000% OK versus puzzle JeanLucPons#140 (2^139, outside it): stock 0.024% OK patched 100.000% OK So the widening fixes the >125-bit case without disturbing the case stock already handled. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The table stops at sm_75, so every GPU from Ampere on falls through to the `return 0` default. Two consequences, and only the first is cosmetic: - the startup banner prints "(16x0 cores)" on an RTX 3050 - GetGridSize() computes `*y = 2 * _ConvertSMVer2Cores(...)`, gets 0, and lands on the `if(*y <= 0) *y = 128` fallback -- so the default grid Y is 128 instead of the intended 2 x 128 = 256 on any sm_86 card Added sm_80/86/87/89/90/a0/a1/c0 with NVIDIA's per-SM core counts. Measured on an RTX 3050 Laptop, same binary, -g forced, back to back: grid 32x128 (old default) 338.9 MK/s grid 32x256 (new default) 345.5 MK/s +2.0% Modest. A short sample suggested +16%, but that did not survive a matched window -- this laptop GPU throttles, so only back-to-back deltas mean anything. The grid also doubles kangaroo memory, 57 -> 105 MB. Nothing here depends on the 256-bit distance work; it is a correctness fix to a lookup table that has simply gone stale. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The 256-bit distance is now opt-in. DIST_WORDS selects the width: 2 by default, 4 with -DWIDE_DIST (or by uncommenting WIDE_DIST in Constants.h). This addresses the main objection to the previous form. A default build now reproduces upstream's layout exactly -- ENTRY 32 bytes, ITEM 56, KSIZE 10, DP packet 40, kangaroo transfer 16, and the original work-file and protocol magics -- so it reads and writes upstream's files and talks to upstream peers. Only -DWIDE_DIST changes any of that. Rather than #ifdef every use site, a dist_t typedef plus DIST_WORDS lets one implementation serve both widths: the sign and kangaroo-type flags always occupy the top two bits of the top word, so Convert, CalcDistAndType, CreateEntry, sameDist and the four network checksum accumulations became loops over DIST_WORDS. That also removed the legacy 126-bit Add/Convert/CalcDistAndType overloads and Widen() -- at DIST_WORDS == 2 dist_t simply is int128_t. Only genuinely width-specific code is conditional: the two extra device slots in Load/StoreKangaroos, the four extra uint32 in OutputDP, the Add128/Add256 selection behind AddDist, and the format magics. Exceeding the compiled limit still aborts, and now names the fix: HashTable::Convert: travelled distance exceeds 126 bits. Interval is too large for the DP entry format (max 125 bits). Rebuild with -DWIDE_DIST for intervals up to 253 bits. Aborting rather than storing a truncated distance. Verified, both builds, RTX 3050 Laptop / CUDA 13.3 / sm_86: - default and wide both solve the 56-bit sample key correctly - default on JeanLucPons#120 (2^119): 34,906 DPs, 100.000% -wcheck - default on JeanLucPons#140 (2^139): aborts as above, exit 255 - wide on JeanLucPons#140 (2^139): 34,762 DPs, 100.000% -wcheck - default writes magic 0xFA6A8001, i.e. upstream's HEADW - each build rejects the other's work file ("Not a work file") tools/check_layout.py and tools/test_distfield.py now verify BOTH widths in one run, and assert the default reproduces upstream's sizes. Neither needs a compiler or a GPU. build_gpu.bat takes an optional "wide" arg. README documents the option, the costs and the incompatibility. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
Updated in I originally offered this as an option and it was the right call — it removes the main reason to reject the PR.
The default column is exactly today's layout, original work-file and protocol magics included. I verified that end to end rather than by inspection: a default build writes a work file whose magic reads back as What this means for review: the format change is no longer something you have to accept to take the fix. On a default build the only behavioural difference is that an over-range interval now stops with an error instead of silently producing a wrong key. The implementation also got smaller. Instead of Both configurations built and run on an RTX 3050 Laptop (sm_86, CUDA 13.3.73, MSVC 14.44):
Each build rejects the other's work file with "Not a work file", so a width mismatch is refused rather than misparsed.
The CUDA 13 fix (#160) and the |
|
Additional verification on puzzle #135, which is the target that motivates the whole patch and the one case #135 is a 2^134 interval, so Target checked before spending GPU time on it, since a pubkey that does not belong to the address makes a run worthless and nothing in the solver would say so: Default build refuses it, which is the intended behaviour:
Worth noting for the review: ptxas output is byte-identical to the hard-coded 256-bit version from before the refactor — 98 registers, 0 spill stores, 0 spill loads, 20608-byte stack frame. Making the width configurable via So the full validated matrix across both builds is now:
For context on the scale rather than the correctness: #135 in one pass is ~3.07e20 operations, which on this RTX 3050 Laptop is roughly 25,200 GPU-years. The run above covered about 3.5e-9 percent of it. Nobody should read these numbers as progress on the puzzle — they only demonstrate that the DPs being produced are valid, which on |
The bug
ENTRY.dpacks the travelled distance into 128 bits asb127=sign b126=kangaroo type, b125..b0 distance, leaving 126 bits of magnitude. Distances span the interval width, so anything wider than 125 bits overflows — hence the README's "This program is limited to a 125bit interval search."The problem is what happens when you exceed it.
Convert()masks the excess away with& 0x3FFFFFFFFFFFFFFF, stores a valid-looking entry, and on collision yields a wrong private key with no error, no warning and no crash.Measured on puzzle #140's real range (2^139), current
mastervs this branch, same input, same machine:-wcheckmastermasterreports normal progress throughout. Nothing indicates the run is worthless. Filed separately as #159.The change
The wider field is opt-in, and the default build is byte-compatible with upstream.
DIST_WORDSselects the distance width: 2 by default, 4 with-DWIDE_DIST(or by uncommentingWIDE_DISTinConstants.h).-DWIDE_DISTThe default column is exactly today's layout, including the original work-file and protocol magics, so a default build reads and writes upstream's files and talks to upstream peers unchanged.
Rather than
#ifdef-ing every use site, adist_ttypedef plusDIST_WORDSlets one implementation serve both widths — the sign and type flags always sit in the top two bits of the top word, soConvert,CalcDistAndType,CreateEntry,sameDistand the four network checksum accumulations became loops. That also deleted the legacy 126-bit overloads andWiden(): atDIST_WORDS == 2,dist_tsimply isint128_t. Only genuinely width-specific code stayed conditional.Over-range distances now abort instead of truncating, and name the fix:
Three other defects fixed along the way, all independent of the width work:
ModNeg256Order(dist[g])writesr[0..3]into a 2-worddist[g], clobbering the next kangaroo. Only dormant becauseUSE_SYMMETRYships commented out.cudaDeviceProp::computeModewas removed, soGPUEngine.cuno longer compiles at all against a current toolkit. Switched tocudaDeviceGetAttribute. This is why the branch builds today. Filed separately as GPUEngine.cu does not compile with CUDA 13: cudaDeviceProp::computeMode was removed #160._ConvertSMVer2Cores()stops atsm_75, so Ampere and newer return 0. Not just the(16x0 cores)banner —GetGridSize()does*y = 2 * _ConvertSMVer2Cores(...), gets 0, and falls back to*y = 128, so the default grid Y on an sm_86 card is 128 instead of the intended 256. Addedsm_80/86/87/89/90and Blackwell. Measured back to back with-gforced: 32x128 = 338.9 MK/s, 32x256 = 345.5 MK/s (+2.0%).Verification
RTX 3050 Laptop, sm_86, CUDA 13.3.73, MSVC 14.44. Both widths built and run. All DP validity via
-wcheck:-DWIDE_DISTbuild0xFA6A8001(upstream's)0xFA6A8011Each build rejects the other's work file with "Not a work file" — a width mismatch is refused, never misparsed.
Also verified on the wide build: 2^134 (#135, which
mastercannot attempt at all) at 100.000%, and client/server on localhost — 196,231 DPs over TCP at 100.000%, kangaroo backup uploaded (536,576 x 32 bytes), downloaded (2^19.03 kangaroos loaded, 0 created), and DPs generated from the downloaded herd validating at 100.000%.tools/check_layout.pyandtools/test_distfield.pyverify both widths in one run — producer/consumer agreement on every offset, theAddDistcarry chain, the checksum truncation, the magics, and the bit-packing round-trip — and assert the default reproduces upstream's sizes. Neither needs a compiler or a GPU.Costs
Only when
-DWIDE_DISTis enabled: DP tables +50% RAM, device kangaroo memory +20%, DP packets +40%, kangaroo transfers +100%, and 4-7% throughput. ptxas: same 98 registers, zero spills either way, stack frame 18560 -> 20608 bytes.A default build costs nothing and changes no format.
Splitting
The CUDA 13 fix (#160) and the SM table fix are each one self-contained commit with no bearing on the distance width, and are worth taking on their own even if the rest is rejected — without the first, nobody can build the GPU version against a current toolkit at all. Happy to open them separately if you prefer.
🤖 Generated with Claude Code