New update Closes #58 - #63
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Build Resumable Batch Payment / Airdrop Engine with Checkpointing and Fee-Bump Retry (starforge batch)
Closes #58
Summary
This PR adds a first-class
starforge batchcommand for safe, resumable CSV payouts (airdrops, contributor payments, faucet campaigns). It expandssrc/utils/tx_batch.rsinto a full batch engine with CSV parsing/validation, operation chunking (100 ops/tx), atomic checkpoint files, and fee-bump / sequence retry on submission failures.New commands:
starforge batch pay --file recipients.csv --wallet payer --network testnet [--dry-run]starforge batch status --file recipients.csvstarforge batch resume --file recipients.csv --wallet payer --network testnetKey behavior:
stellar-strkey, amount format, asset format)--dry-runreports total cost (fees + amounts) with read-only balance validation — no transactions submitted<file>.batch-state.jsonatomically after each confirmed transaction chunkbatch payauto-detects the checkpoint and resumes from the first pending row--max-retries, default 3) before rows are markedfailed; remaining chunks continueChanges
src/utils/tx_batch.rssrc/commands/batch.rspay/status/resumesubcommands with confirmation flowsrc/utils/horizon.rssubmit_payment_with_retry,fetch_transaction_by_hash,post_signed_transactionrefactorsrc/main.rs,commands/mod.rs,command_tree.rsCargo.tomlcsvdependencyREADME.mdProblem Statement
src/utils/tx_batch.rspreviously provided only a minimal helper for grouping operations into a transaction. There was no supported workflow for paying out hundreds or thousands of recipients from a CSV file (e.g., a token airdrop, contributor payout, or testnet faucet campaign). Doing this safely required hand-rolling sequence number management, chunking operations into valid transaction sizes, handling partial failures, and manually tracking which recipients were already paid — with a real risk of double-paying or silently skipping recipients if the process crashed midway.Proposed Solution
Build a first-class
starforge batch paycommand that turns a CSV ofdestination,amount,asset[,memo]rows into a safe, resumable, auditable payment run:starforge batch pay --file recipients.csv --wallet payer --network testnet [--dry-run]— validates every row (address checksum, asset existence, amount format) before submitting anything, and reports total cost (fees + amounts) up front.Operations are automatically chunked into the maximum operations-per-transaction Stellar allows, with sequence numbers reserved and incremented locally to avoid
tx_bad_seqraces.A checkpoint file (
<file>.batch-state.json) records the status of every row (pending,submitted,confirmed,failed) as the run progresses, written atomically after each transaction confirms.If the process is interrupted or crashes, re-running the same command detects the checkpoint file and resumes from the first pending row — already-confirmed rows are never resubmitted.
Failed transactions (e.g., insufficient fee, transient network error) are retried with fee-bump transactions, reusing the existing fee-bump/sequence-retry logic already shipped for single transactions, up to a configurable retry limit before being marked failed for manual review.
starforge batch status --file recipients.csv— prints a summary (counts by status, total paid, total failed) from the checkpoint file without submitting anything.starforge batch resume --file recipients.csv— explicit resume entry point in addition to auto-detection.Example Usage
CSV format (
destination,amount,asset[,memo]):Sample run:
If the process is killed mid-run, re-run the same
batch paycommand or usebatch resume. Already-confirmed rows are never resubmitted; the checkpoint file records each row aspending,submitted,confirmed, orfailed.Acceptance Criteria
--dry-runreports accurate total cost without submitting any transaction.Test Plan
cargo buildsucceedsstarforge batch pay --file recipients.csv --wallet payer --network testnet --dry-runvalidates all rows and prints cost without submittingrecipients.csv.batch-state.jsonstarforge batch resume— confirmed rows are not double-paid, pending rows completestarforge batch status --file recipients.csvshows correct counts by statustx_batch.rs: CSV edge cases, chunk boundaries (101 rows → 2 txs), checkpoint resume, retry-then-fail accounting