feat: miner log download via Hypercore P2P streaming#85
Merged
tekwani merged 4 commits intoJun 1, 2026
Conversation
Introduces a LogDownloader class that fetches miner log files directly from wrk-miner via Hypercore/Hyperswarm, bypassing the HRPC pipeline. Wired up in the http worker after facilities are ready.
boris91
approved these changes
May 29, 2026
mukama
approved these changes
Jun 1, 2026
tekwani
approved these changes
Jun 1, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What are new additions?
Adds a three-step REST flow for downloading miner log files. Since log files
can be large and are generated async by wrk-miner, the client submits a job,
polls until ready, then streams the file directly.
Endpoints:
POST /auth/miners/:minerId/download-logs— submits adownloadLogsaction, returns{ jobId, statusUrl, fileUrl }GET /auth/miners/:minerId/download-logs/:jobId/status— polls completion; returnspending | ready | failed | expiredGET /auth/miners/:minerId/download-logs/:jobId/file— streams the binary logGET /auth/download-logs/:id— legacy single-step endpoint for existing clientsHow is it done ?
Introduces
LogDownloader(workers/lib/log-downloader.js), which fetches log bytes directly from thewrk-miner peer over Hypercore/Hyperswarm P2P — bypassing the HRPC action pipeline entirely. The action
result only carries metadata (
coreKey,byteLength,expiresAt); the actual bytes are transferredlazily as the HTTP client reads the response stream.
A single shared connection handler on
net_r0.swarmroutes incoming peers to the correct in-flightdownload by matching
peerInfo.topicsagainst activediscoveryKeys. After the stream ends (or errors),downloaded blocks are cleared from Corestore storage and the session is closed.