feat(deploy): offer poster — mint WBTC and post one exact-coin ZSwap offer per minute, re-offering released coins - #57
Merged
Conversation
…et mint, journal, quote sizing - pinned-wallet.ts: buildWalletFacade reproduced with CustomShieldedWallet + a coin selector that, when armed, returns exactly one (type, nonce) coin or nothing; the eight @midnightntwrk/wallet-sdk-* packages become root deps at the versions the lockfile already resolved (hoisting reshuffle only). - faucet-mint.ts: zswap-da faucet colour derivation (rawTokenType over the FNV domain separator), mint_shielded wrapper returning the coin + nullifier, known-tokens registration that disambiguates the 409 cases. - poster-journal.ts: atomic file-backed coin -> offers record with kernel status mapping and re-offer candidate selection gated on availableCoins. - poster-quote.ts: want-leg sizing from GET /v1/quote (suggested_to_amount, sponsored verdict). 117 unit tests, no network.
…alth
The long-running poster (spec 00007, FR-001…FR-016). Every POST_INTERVAL_MS it
posts one Offer File whose ONLY input is a coin it can name: a coin an earlier
offer released (terminal-but-unspent AND back in availableCoins), or one it
mints that tick from its own DUST.
Decisions live behind injected dependencies so they are unit-testable without a
wallet, a node or a kernel:
deploy/scripts/lib/poster-config.ts env -> PosterConfig; POSTER_SEED xor
POSTER_MNEMONIC (BIP-39 seed derived exactly as midnight-env does it),
seven refused seed collisions, blank-is-unset, offline colour derivation,
contract-address resolution (env -> share dir -> package dir, network
aware), and redaction so a seed can never reach a log line
deploy/scripts/lib/poster-tick.ts reconcile + tick + offerCoin +
mintCoin, the failure taxonomy, and the exact-coin assertion: the builder
returns the nullifiers, the tick decides, reverts the recipe on any
mismatch and posts nothing
deploy/scripts/lib/poster-scheduler.ts fixed-RATE, never-overlapping loop
with injected clock/timers; overruns warn, count and start the next tick
at once; rolling p50/p95
deploy/scripts/lib/poster-health.ts /health (503 only after
HEALTH_STALE_TICKS consecutive FAILED ticks), /metrics, /journal
deploy/scripts/offer-poster.ts wiring only: the pinned facade, the
sync that does not block on dust progress, DUST registration, the contract
join, token-name registration, the journal, DRY_RUN, signals, shutdown.
main() runs only under import.meta.main.
packages/solver-core/offer-files.ts gains an OPTIONAL contractAddress argument
on joinOfferFiles; getContractAddress() hard-codes the `undeployed` file name,
which is right for the compose stack and wrong for preprod. Every existing
one-argument caller is unaffected.
offer-poster.ts imports packages/solver-core/offer-files.ts DYNAMICALLY: that
module builds a CompiledContract at load time from Compact build output a fresh
clone does not have, so a static import would make the file unloadable in tests
and in CI. It also avoids rxjs, which does not resolve from deploy/.
Tests: 98 new (15 scheduler, 35 config, 36 tick, 12 health); the whole
deploy/scripts/lib suite is 215/215. Typecheck clean in all nine new files
under the repo's compiler options and under the stricter set. No change to the
full-suite failure set.
…`poster`)
Nothing changes with the profile off: `docker compose config --services` is
byte-identical, and no existing service, volume or port is touched.
- deploy/images/kernel/entrypoint-offer-poster.sh: sources entrypoint-common.sh,
requires ZSWAP_API + MIDNIGHT_NETWORK_ID, fails fast (78) when neither
POSTER_SEED nor POSTER_MNEMONIC is set (naming the OFFER_POSTER_* keys the
operator actually edits), unsets every blank poster knob, adopts the contract
address, waits for the kernel API, mkdir -p's the journal directory and
`exec`s bun so PID 1 receives SIGTERM. No marker file: this is a loop, not a
one-shot, and idempotence lives in the journal.
- deploy/images/kernel/Dockerfile: COPY the new entrypoint (after `COPY . .`, so
the ~20 min proving-key layer stays cacheable); header list NINE -> TEN.
- deploy/compose.yml: `offer-poster` behind profile `poster`, restart
unless-stopped, depends_on kernel healthy + offerfiles-deploy completed,
offerfiles-deploy read-only + a new offer-poster-state volume, bun healthcheck
on 9977 with a 15 m start_period, health port published on
${BIND_ADDR}:${HOST_OFFER_POSTER_HEALTH_PORT:-19977}. The Midnight endpoints
are spelled out instead of merging *midnight-endpoints, because that anchor
carries MIDNIGHT_WALLET_SEED and the poster refuses to share a seed.
- deploy/down.sh: --profile poster on both `docker compose down` lines, or the
container and its chain-keyed volume survive teardown.
- deploy/.env.example: OFFER_POSTER_* block (dedicated seed, funding, quote-sized
want leg, sponsorship must match on kernel and batcher) and the host port.
- deploy/README.md: "Offer poster" — exact-coin guarantee and how to verify it,
journal/re-offer semantics, funding, one facade per seed, DRY_RUN, the
/health /metrics /journal endpoints, and the preprod env block.
- .github/workflows/ci.yml: run `deploy/scripts/lib` in the unit job. Those
suites (215 tests) have never been visible to CI.
The 00005 price service (bc5c673) added `packages/price-feed` as a workspace member but never added it to the kernel Dockerfile's manifest-copy list, so the `bun install --frozen-lockfile` in that layer resolved a different workspace set from the one bun.lock records and refused every build from that merge onwards: error: lockfile had changes, but lockfile is frozen The image is normally reused from a cache, so nothing exercised the build until 00007's P5 needed one. Reproduced by replaying the Dockerfile's manifest-only context by hand: it fails without packages/price-feed/package.json and installs 2231 packages in 3.9 s with it. kernel-image-manifests.test.ts keeps the list honest — it expands the root package.json's workspace globs and asserts a COPY line names each manifest. It fails on the unfixed Dockerfile and passes on the fixed one.
…xample DUST regenerates continuously from registered NIGHT at a rate roughly three orders of magnitude faster than a single mint's fee at the funding level P5 tested (4x5e12 NIGHT); the wallet's DUST balance rose on every single before/after sample across 12 consecutive mints, so a positive per-mint cost could not be isolated and the code default (1, "any dust at all") is left as the recommended floor rather than replaced with a fabricated number.
The knob bounds initSwap's ttl -- the wallet's local deadline for an unconfirmed recipe, releasing the coin back to availableCoins if it never confirms. It does not shorten a posted shielded offer's kernel-advertised lifetime, which is min(ROOT_WINDOW_SECONDS, OFFER_TTL_SECONDS) on the kernel process (currently 1h on every network) and reads nothing per-offer from the client -- traced in packages/node/state-machine.ts and packages/node/env.ts, per plan Q8 (resolution: option A, docs only).
Both sides added services, entrypoints and docs at the same anchors: register-minted-tokens (main) and offer-poster (this branch) are kept side by side; the packages/price-feed manifest fix landed on main independently, so the duplicate comment is dropped and the COPY line kept once.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
A long-running Docker Compose service,
offer-poster, indeploy/, off by default (docker compose --profile poster up -d offer-poster). Given a DUST-funded wallet it mintsGIVE_AMOUNTWBTC everyPOST_INTERVAL_MS(default 60s, fees from its own DUST) and posts one ZSwap offer whose only input is that exact coin, sized from the kernel's own quote so the batcher sponsors it (payFees:false). Released coins (offer expired/cancelled, or orphaned by a crash between mint and post) are re-offered before minting a new one. Every coin and every offer built from it is recorded in a durable, atomically-written journal on its own volume.Spec:
spec/00007-offer-poster-service.mdin the organizer workspace. Plan and full research/measurement trail:plans/00007-offer-poster-service.md(same workspace) — every claim below has a path and line number there.Exact-coin mechanism
Coin selection is not addressable through the public wallet API (
initSwaptakes{colour: amount}only; the default selector is smallest-first). This service assembles its own wallet facade (deploy/scripts/lib/pinned-wallet.ts) aroundCustomShieldedWallet(...).withCoinSelection(...), arming a pinned selector with the just-minted coin's nonce before building each offer and disarming it in afinally. AfterfinalizeTransaction, the tick assertscollectNullifiers(tx)deep-equals[coin.nullifier]and the fallible offer has zero inputs — on any mismatch the recipe is reverted (wallet.revert) and nothing is posted. Verified live in P5: 10/10 posted offers'computed.inputNullifiers(read back from the kernel, not assumed) matched the journaled nullifier of the coin minted that tick; three pre-loaded "control" WBTC coins the poster never minted were never touched across a full day of testing (mint, re-offer, settlement, a mid-tick kill, and an enforce-mode run) — confirmed via a finalavailableCoinssnapshot, byte-identical to the pre-test one.Journal and re-offer semantics
A JSON file at
POSTER_JOURNAL_FILE(own named volume, atomic temp+rename writes) keyed by coin nonce:{type, value, nullifier, mintTx, state: minted|offered|spent|lost, offers: [...]}. Every tick reconciles non-terminal offers againstGET /v1/offers/:hash/status(the kernel emits exactlylive|consumed|cancelled|expired|not_found— neverrejected, which is poster-local for a 4xx-refused post) and only proposes a coin as a re-offer candidate when both its latest offer is kernel-terminal and the coin is back in the wallet'savailableCoins—cancelleddoes not by itself prove release (a partial/split settlement is also classifiedcancelled), soavailableCoinsis the actual proof, not the status. A crash between mint and post leaves the coin journaled asminted(written as the first action after the mint confirms, before quote/build/post) and it is picked up as a candidate on the next reconcile rather than leaked.Live-verified (P5, SC-003/SC-004): a coin whose offer expired carried a second offer after
mode=reofferfired with zero mints in that tick; adocker killissued right after themintlog line and beforepostproduced a clean orphan re-offered on the very next tick after restart (RestartCount=0, no crash loop, no duplicate offer).Want leg: sized from the kernel's own quote
Per build,
GET /v1/quote?from_token=<give>&to_token=<want>&from_amount=<coin.value>—suggested_to_amountalready lands exactly on the sponsorship threshold, so the offer is sponsored by construction unlessWANT_AMOUNTforces a fixed amount (then the poster only warns; the kernel is the one that actually refuses it underenforce). Verified underBATCHER_SPONSOR_POLICY=enforceon both kernel and batcher (SC-007): normal operation stayed accepted for 3 ticks (0NOT_SPONSORED/UNPRICED_TOKENin either service's logs); a forced, ~150x-over-thresholdWANT_AMOUNTwas refused by the kernel (422 NOT_SPONSORED), the recipe was reverted, and the offer id came back404— genuinely nothing was posted, not just "assumed refused."Fees
Mints are balanced and submitted from the poster's own DUST (not routed through the batcher). ZSwap offers stay
payFees:false— the taker balances, the batcher sponsors the Celestia fee.P5 measurements (live undeployed stack, project
op00007, 2026-09-03)live, 10/10computed.inputNullifiersmatched, 0/12 ticks failed, control coins untouched throughout.spenton the very next reconcile poll.mode=reofferlogged before any mint) — required aligning the kernel's ownOFFER_TTL_SECONDSalongside the poster's TTL knob (see the TTL clarification below).phase=mintandphase=post→ clean orphan, re-offered within 1 tick (bound was 2), no duplicate, no crash loop.enforcemode accepted normal operation, refused a forced over-threshold amount as above.p50TickMs=48407,p95TickMs=51978— comfortably under the 60s interval, 0 overruns across 12 ticks.provision-solver-fees.tsuses) DUST generation outpaced one mint's cost by roughly three orders of magnitude — the wallet's DUST balance rose on every sampled before/after mint, never fell, over 12 consecutive 1-mint/min ticks. A per-mint fee could not be isolated from a balance diff at this generation rate (filed asissues/00014— the mint path doesn't logtx.fees()/tx.cost()directly).POSTER_MIN_DUSTis left at the code default (1, "any DUST at all") — empirically correct at this funding level, reasoning recorded in.env.example../down.sh(no-v— an unknown flag would exit 2) left zeroop00007containers/volumes/networks, independently re-verified with freshdocker ps -a/volume ls/network lsfilters.New env knobs
All under
deploy/.env.example's new "offer poster (profile: poster)" block:OFFER_POSTER_SEED/_MNEMONIC(required, xor),_GIVE_TOKEN(WBTC),_GIVE_AMOUNT(1000),_WANT_TOKEN(WETH),_WANT_AMOUNT(blank = quoted),_INTERVAL_MS(60000),_TTL_MINUTES(60),_MIN_DUST(blank = code default1, P5-measured),_DRY_RUN,_JOURNAL_RESET,HOST_OFFER_POSTER_HEALTH_PORT(19977), plus a set of blank-means-default passthroughs (coin-visible timeout, reconcile interval, max re-offers/tick, shutdown grace, health-stale-ticks, sync/dust-wait timeouts, post/live retry knobs). Health endpoint onPOSTER_HEALTH_PORT(literal9977in the container):/health,/metrics(Prometheus text),/journal.TTL clarification (Q8)
OFFER_POSTER_TTL_MINUTESis thettlpassed toinitSwap— the wallet's own local deadline for an unconfirmed recipe, governing when the wallet gives up on it and releases the coin back toavailableCoins. It does not shorten a posted, live, shielded offer's kernel-advertised lifetime, which ismin(ROOT_WINDOW_SECONDS, OFFER_TTL_SECONDS)on the kernel process (currently 1h on every network) and reads nothing per-offer from any client — traced inpackages/node/state-machine.tsandpackages/node/env.ts. This surprised SC-003's first attempt (candidates stayed at 0 for 7+ ticks with only the poster-side knob shortened) and is corrected in this PR'sdeploy/.env.exampleanddeploy/README.md(previously they read as if the knob controlled both). Not a poster correctness bug — re-offer candidacy already fails closed on the kernel-side half, so a short poster TTL alone is safe, just produces no re-offers by itself. Full trace in plan Q8;OFFER_TTL_SECONDS/ROOT_WINDOW_SECONDSpromotion to a documented stack-wide knob is filed separately asissues/00013(kernel/batcher-wide concern, out of this PR's scope).Changes outside
deploy/packages/solver-core/offer-files.ts:joinOfferFiles(walletResult, address?)gains an optional second argument, defaulting to the existinggetContractAddress()behaviour. Additive — all 12 existing one-argument call sites (all inpackages/tests) are unaffected. Needed becausegetContractAddress()hard-codes theundeployedcompose stack's contract-share path, which is wrong for a preprod dry run (SC-006); the poster resolves its own address in priority order (MIDNIGHT_CONTRACT_ADDRESS→ the shared contract-share file → the repo-local fallback).package.json/bun.lock: eight@midnightntwrk/wallet-sdk-*packages promoted to root direct dependencies, at the versions the lockfile already resolved for@effectstream/midnight-contracts. Necessary becausedeploy/is not a workspace member and resolves bare specifiers from the root only. The lockfile update also reshuffled@polkadot/types' fallback-root hoisting; verified inert — a full symlink-graph diff against the pre-change tree showed exactly 9 changed links (the 8 new root links plus the one fallback-root@polkadot/typeslink used only by undeclared imports; every package that declares its own@polkadot/typesversion is unaffected), andbun install --frozen-lockfilepasses clean both before and after.deploy/images/kernel/Dockerfile(pre-existing bug, not introduced by this branch, but fixed here because P5 needed a buildable image): theCOPYblock never includedpackages/price-feed's manifest, so no kernel image could build frommainsince the price-feed package was added (feat(prices): seeded asset prices, price-feed service (CoinGecko), /v1/prices, quote sources #54). OneCOPYline added; unit-tested (kernel-image-manifests.test.ts). Calling this out prominently since it meansmainitself has been unbuildable — worth confirming on the maintainer's side independent of this PR..github/workflows/ci.yml:deploy/scripts/libadded to the "Run unit suites" step. That directory is not a workspace member so it was previously invisible to CI entirely — none of this service's own unit tests (or the pinned-wallet/faucet-mint/journal/quote modules') ran anywhere before this.Not a breaking change
With the
posterprofile off (the default),docker compose config --servicesis byte-identical to this branch's base — measured, not assumed.deploy/down.shgained--profile posteron both itsdocker compose downinvocations so the new service and itsoffer-poster-statevolume are actually removed by a plain./down.shonce the profile has been used (Compose only tears down what the selected profiles select).CI caveat
deploy/scripts/lib(and every other suite) now runs on CI's pinned bun 1.3.3, while every measurement and test run above was taken on bun 1.3.11. That pin is deliberate (see theci.ymlcomment) but a known WASM-segfault flake exists on 1.3.3 for wasm-heavy suites (this repo touches ledger wasm infaucet-mint.test.ts/pinned-wallet.test.tsandBun.serveon port 0 inposter-health.test.ts) — if the first CI run on this PR shows a segfault, rerun once before investigating further.Testing
bun test deploy/scripts/lib: 230/230 pass (9 files, 1229expect()calls) — pinned-selector, faucet-mint colour derivation, journal round-trip/candidacy, quote sizing, config parsing, tick/scheduler/health logic, and the kernel-image-manifest fix, all in the space-free scratch clone.check:pgtypessteps replicated locally): 1356 pass / 2 skip / 1 pre-existing failure (solver-offerfiles-real-image-secret-scan.test.ts's large-inventory case — reproduces on the pre-branch tree too; filed asissues/00018, not caused by this branch).typecheck:backend,typecheck:solver, and the aggregatetypecheck(addstypecheck:price-feed): 0 diagnostics each, with CI's declaration-only Compact stub. (deploy/itself has no repo-level typecheck gate at all — pre-existing, filed asissues/00016; the new files were separately typechecked by hand throughout P1–P3 with 0 diagnostics.)op00007): SC-001, SC-002, SC-003, SC-004, SC-007 all verified against a real compose stack, as detailed above; SC-006 (preprod dry run) verified read-only against preprod in this PR's own handoff — see the plan for the exact result and the.envblock for whoever deploys preprod next.Related issues filed (out of this PR's scope)
issues/00012–00020in the organizer workspace, covering: compose image tags not project-scoped (a real cross-session collision hit during P5, worked around with a session-unique tag — not fixed here since it's a pre-existing, stack-wide hazard); the undocumented kernel TTL knobs; missing DUST-fee instrumentation on the mint path; a stale api-example;deploy/'s missing typecheck gate; several smallAPI.md/README.mddrifts; the flaky secret-scan test; one transient, self-recovered RPC error; and the lack of a reusable "settle one poster offer" smoke-test helper.