Skip to content

feat: miner log download via Hypercore P2P streaming#85

Merged
tekwani merged 4 commits into
tetherto:developfrom
paragmore:feat/support-download-miner-logs
Jun 1, 2026
Merged

feat: miner log download via Hypercore P2P streaming#85
tekwani merged 4 commits into
tetherto:developfrom
paragmore:feat/support-download-miner-logs

Conversation

@paragmore
Copy link
Copy Markdown

What are new additions?

Adds a three-step REST flow for downloading miner log files. Since log files
can be large and are generated async by wrk-miner, the client submits a job,
polls until ready, then streams the file directly.

Endpoints:

  • POST /auth/miners/:minerId/download-logs — submits a downloadLogs action, returns { jobId, statusUrl, fileUrl }
  • GET /auth/miners/:minerId/download-logs/:jobId/status — polls completion; returns pending | ready | failed | expired
  • GET /auth/miners/:minerId/download-logs/:jobId/file — streams the binary log
  • GET /auth/download-logs/:id — legacy single-step endpoint for existing clients

How is it done ?

Introduces LogDownloader (workers/lib/log-downloader.js), which fetches log bytes directly from the
wrk-miner peer over Hypercore/Hyperswarm P2P — bypassing the HRPC action pipeline entirely. The action
result only carries metadata (coreKey, byteLength, expiresAt); the actual bytes are transferred
lazily as the HTTP client reads the response stream.

A single shared connection handler on net_r0.swarm routes incoming peers to the correct in-flight
download by matching peerInfo.topics against active discoveryKeys. After the stream ends (or errors),
downloaded blocks are cleared from Corestore storage and the session is closed.

paragmore added 3 commits May 29, 2026 20:44
Introduces a LogDownloader class that fetches miner log files directly
from wrk-miner via Hypercore/Hyperswarm, bypassing the HRPC pipeline.
Wired up in the http worker after facilities are ready.
@paragmore paragmore marked this pull request as draft May 29, 2026 15:21
@paragmore paragmore marked this pull request as ready for review June 1, 2026 15:29
@tekwani tekwani merged commit 50616a9 into tetherto:develop Jun 1, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants