feat(batcher): sponsorship gate — pay the Celestia fee only for offers priced below reference (re-land of #55 onto main) - #56
Merged
Conversation
…rence The batcher holds the wallet that pays for every published offer, and until now it sponsored anything structurally valid: checkForCelestiaSponsorship was a no-op stub. It now applies the same rule the node's quote already shows the maker — evaluateSponsorship() from @zswap-da/offer-guard — so the UI cannot promise sponsorship the batcher then refuses, or vice versa. The batcher has no database, so it polls the node's GET /v1/prices (PriceCache, every 10 minutes by default) instead of resolving prices itself. A failed refresh keeps the last good snapshot and lets it age, which is what makes "the node answered ten minutes ago" distinguishable from "the node has been down for two days". Policy, all validated at startup so a typo throws instead of silently sponsoring everything: BATCHER_SPONSOR_POLICY enforce | warn | off (default warn) BATCHER_SPONSOR_UNPRICED allow | reject (default allow) BATCHER_NODE_API_URL, BATCHER_PRICE_REFRESH_MS, BATCHER_PRICE_MAX_AGE_MS SPONSOR_DISCOUNT_BPS bootstrap only, until the node answers once Defaults keep every current deployment behaving as it does today: warn logs what enforce would have refused, and unpriced test tokens keep flowing. validateInput is now async. The SDK core already awaited it, so nothing on the production path changes; direct callers (the tests) await it now.
…'s refusals
A maker whose offer is not worth a Celestia fee learned it, until now, as a
500 INTERNAL after the blob had already crossed the network — a status that
tells them "server problem, retry unchanged", which is the opposite of what to
do. POST /v1/offers now asks the question itself, before liveness and proof
verification, and answers 422 with the numbers:
{ "error": "NOT_SPONSORED", "reason": "wants 1.0% below reference,
sponsorship needs >= 2.5% below", "give_usd": ..., "want_usd": ...,
"implied_discount": 0.01, "sponsor_discount": 0.025 }
It is a mirror of the batcher's gate, not a second opinion: the same
evaluateSponsorship over the same resolution order the quote and /v1/prices
use, and the same two policy variables the batcher reads
(BATCHER_SPONSOR_POLICY, BATCHER_SPONSOR_UNPRICED). Under the default `warn`
the node logs what `enforce` would have refused and forwards, so this release
changes no deployment's behaviour until the policy is switched.
The pre-check is READ-ONLY: unlike the quote it never writes a demo price row,
so a refused submission cannot create the row that makes the next one look
priced.
When the batcher refuses after the node forwarded (a node in `warn` with a
batcher in `enforce`, or a stale batcher snapshot), NOT_SPONSORED /
UNPRICED_TOKEN / PRICE_UNAVAILABLE now surface as 422 carrying the batcher's
own message, instead of 500.
Also: policy parsing moved into @zswap-da/offer-guard so the two processes
cannot disagree about what a value means; compose and the three env examples
carry the new variables; API.md's "step 5 is not implemented" paragraph is
replaced by what was actually built, including why this is NOT enforced at STM
ingestion (the namespace is permissionless — an unsponsored offer posted
directly is still indexed).
Found by running a real batcher against a real node (Brief B, B4), not by
reading code. The batcher SDK wraps an adapter refusal:
{"success":false,"error":"Validation failed",
"message":"NOT_SPONSORED: wants 0.0% below reference, …"}
`error` is the generic wrapper text, so the mapping added in the previous
commit matched nothing and the maker still got 500 INTERNAL — the exact defect
it was written to remove. Both fields are now scanned, and the test fixture is
the verbatim body a live batcher returned.
Follow-on from the Q-10 correction on the base branch: `PriceSource` no longer has a `fixed` member, so listing it in MARKET_SOURCES stopped typechecking. Every asset price is now a fetched USD price and nothing is pinned to a dollar, so the market sources are `feed | seed | manual`. A `fixed` row from an older node is not silently trusted after this: it falls into the existing "unknown source" path and is downgraded to `fallback`, i.e. the offer is unpriced and BATCHER_SPONSOR_UNPRICED decides. That is the safe direction — no fee is paid against a price this build cannot vouch for.
…e table
Q-11: `PriceCache` polled the node's whole `GET /v1/prices` every ten minutes
and held a snapshot. That is fine for six tokens and wrong for thousands —
the payload grows with the registry, the batcher downloads prices for colours
no offer will ever mention, and a token minted since the last poll is
invisible until the next one.
`PriceLookup` replaces it. `validateInput` collects the offer's leg colours
and asks `GET /v1/prices?tokens=` for exactly those, in ONE request; answers
are cached per colour for `BATCHER_PRICE_TTL_MS` (default 10 min), so a busy
pair costs one request per TTL rather than one per offer, and an unfamiliar
colour is answered now instead of at the next poll. `sponsor_discount` and
the node's feed status are refreshed from every response.
A colour the node returns nothing for is cached as an ANSWER ("no price
here"), not as a gap — otherwise every offer on a test token would re-ask.
That keeps the two questions the policy already separated genuinely
separate: "the node has no market price for this" is
BATCHER_SPONSOR_UNPRICED's, "I could not ask" is BATCHER_SPONSOR_POLICY's.
Failure behaviour is unchanged in spirit and sharper in detail. A failed
re-ask does not erase what is cached: entries keep their original timestamp,
are still served while they are inside `BATCHER_PRICE_MAX_AGE_MS`, and only
then does the colour become unavailable (enforce → PRICE_UNAVAILABLE with
the colours named, warn → sponsor + log once a minute, off → skip). So a
brief node restart no longer makes every offer unavailable.
`BATCHER_PRICE_REFRESH_MS` is dropped with the poll it configured — setting
it is inert rather than an error, so a stale compose file does not block a
deploy. `BATCHER_PRICE_MAX_AGE_MS` keeps its name and gains a startup check
that it is >= the TTL: the reverse would strand colours that have a
perfectly recent answer in hand.
One optional request survives at startup — the NIGHT colour, seeded on every
network — so an operator can still read "am I price-aware?" off the startup
log. It is fire-and-forget and non-fatal.
Verified live against the real node API and the real batcher binary on
random free ports (Celestia deliberately unreachable, torn down after): one
probe at startup and nothing else; at reference the node answers 422 and the
batcher refuses with zero Celestia submissions, having asked only for the
want leg (the give leg was already cached); the same offer again asks the
node nothing; 2.5% below reference the offer queues.
…batcher-gate feat(sponsorship): pay the Celestia fee only for offers priced below reference
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
The batcher pays the Celestia fee only for offers whose wanted value is at least
the sponsorship discount below the given value at the node's reference prices —
and the node says the same thing earlier, and more clearly.
Until now
checkForCelestiaSponsorshipwas a no-op stub: every structurallyvalid offer was sponsored, including one priced exactly at reference or above
it. And when the batcher did refuse for any reason,
POST /v1/offersanswered500 INTERNAL— a status that tells the maker "server problem, retryunchanged", which is the opposite of what to do.
Builds on #54 (base is that branch; retarget to
mainonce #54 merges).Not breaking, by construction
The rollout defaults are
BATCHER_SPONSOR_POLICY=warnandBATCHER_SPONSOR_UNPRICED=allow, so on any deployment that sets nothing:warnonly logs whatenforcewould have refused;The intended rollout is: deploy with the defaults, read a day of
would refuse (policy=warn) — NOT_SPONSORED: …lines from both processes, thenset
enforce.The one caller-visible change:
ZswapCelestiaAdapter.validateInputis nowasync. The SDK core already awaited it (batcher-sdk core/batcher.ts:513), sonothing on the production path changes; direct callers (three test files) await
it now.
How
One rule, three readers, no drift.
evaluateSponsorship()in@zswap-da/offer-guard— added in #54 and already behind/v1/quote'ssponsoredflag — is now also what the batcher and the node's pre-check ask.packages/batcher/price-lookup.ts(89cfc3e; wasprice-cache.ts) — thebatcher has no database and does not mirror the price table: for each
offer it collects the leg colours and fetches the missing or expired ones in
ONE
GET /v1/prices?tokens=…request, cached per colour forBATCHER_PRICE_TTL_MS(10 min default).sponsor_discountand the feedstatus are refreshed from every response. A failed refresh keeps the cached
entry and lets it age; past
BATCHER_PRICE_MAX_AGE_MSit stops countingas an answer (a colour with no usable entry is unavailable; a colour the
node does not price is unpriced). One startup probe (
?tokens=<NIGHT>)answers "am I price-aware?" — the only pre-offer request.
ZswapCelestiaAdapter.validateInput— the gate runs LAST, after proofverification. It is the cheapest check in the ladder, but it needs
gives/wants, and refusing an offer with values read out of an unverifiedtransaction would be refusing it with numbers that were never real.
POST /v1/offers— a mirror, not a second opinion: same function, sameresolution order as the quote and
/v1/prices, same two env variables. Itruns after the byte-identical dedup probe and before liveness/crypto, and is
read-only — unlike the quote it never writes a demo price row, so a
refused submission cannot create the row that makes the next one look priced.
batcher-client.ts—NOT_SPONSORED/UNPRICED_TOKEN/PRICE_UNAVAILABLEfrom the batcher now surface as422carrying thebatcher's own message. Every other batcher failure is still a 500; it is not
the maker's fault.
New
422body onPOST /v1/offers:{ "error": "NOT_SPONSORED", "reason": "wants 0.0% below reference, sponsorship needs ≥ 2.5% below", "give_usd": 19181.81, "want_usd": 19181.81, "implied_discount": 0, "sponsor_discount": 0.025 }This is NOT enforced at STM ingestion, deliberately
The MIP-0006 namespace is permissionless: an unsponsored offer posted straight
to Celestia at the maker's own expense is still indexed. The gate decides who
rides the batcher's wallet for free, not what is a valid offer.
API.md's"step 5 — not implemented" paragraph is rewritten to say exactly this, so a UI
does not start assuming every indexed offer was sponsored.
Policy
BATCHER_SPONSOR_POLICYenforce|warn|offwarnBATCHER_SPONSOR_UNPRICEDallow|rejectallowBATCHER_NODE_API_URLhttp://127.0.0.1:9999BATCHER_PRICE_TTL_MS600000BATCHER_PRICE_MAX_AGE_MS172800000(48 h); must be ≥ the TTL (startup error otherwise)BATCHER_PRICE_REFRESH_MS89cfc3e(inert if still set)SPONSOR_DISCOUNT_BPS250The two processes read the same names, and the parsing lives in
@zswap-da/offer-guardso they cannot disagree about what a value means. Aninvalid value throws at startup: an operator who typed
enfrocewantsoffers refused, and silently sponsoring everything is what they were
preventing. On the batcher
SPONSOR_DISCOUNT_BPSis only a bootstrap — oncethe node answers, the node's
sponsor_discountwins, so the batcher cannotrefuse at a threshold the UI never showed the maker.
compose.ymlpasses the knobs to bothkernelandbatcher. There isdeliberately no
depends_onfrom batcher to kernel — the kernel alreadypoints back at the batcher and the pair would deadlock; the startup probe is
non-fatal by design.
Tests
bun test packages/database packages/node packages/offer-guard packages/price-feed packages/batcher packages/validator→ 541 pass / 2 skip / 0 fail, 3798 assertions, 54 files
(
mainbaseline 378, after #54 474 — +67 here).bun run typecheck0 diagnostics on all three gates;bun run check:pgtypesgreen.New suites:
packages/batcher/price-cache.test.ts(25),packages/batcher/sponsorship-gate.test.ts(21),packages/node/offers-sponsorship.test.ts(21).Everything is driven with the real proven offer fixture
(
packages/validator/fixtures/valid-offer.bech32, gives 1 000 000 NIGHT / wants5 000 000 of a test colour), so the price of the wanted leg is the only dial and
give_usd = 19181.81can be checked by hand.Red before green: on the base branch that fixture is sponsored
(
validateInput -> {"valid":true}) andPOST /v1/offersforwards it to thebatcher even with
BATCHER_SPONSOR_POLICY=enforceset, because nothing readsit. Both assertions now flip.
Verified against real processes
The real node and the real batcher binary were run as two processes over
real sockets on random free ports, with only Celestia absent (this host could
not take a second full devnet — another agent's Midnight stack was up). Real
HTTP between them throughout; torn down after.
89cfc3e): atstartup only the probe
GET /v1/prices?tokens=0000…0000; validating theat-reference offer produced exactly
GET /v1/prices?tokens=ffff…ffff(thewant leg — the give leg was already cached); the repeat produced no request
(TTL).
GET /v1/pricesnever appears without?tokens=. (The first run one80fe37used the table mirror:prices: 4 tokens, discount 2.50%.)422 NOT_SPONSORED; the same blob straight to/send-input→400 … "NOT_SPONSORED: wants 0.0% below reference …", andzero Celestia submissions — the SDK threw
InputValidationErroratcore/batcher.ts:513.(
Processing 1 inputs for target: celestia); the only failure left is thedeliberately unreachable Celestia RPC.
warn+ batcherenforce: the node logs, forwards, and answers422with the batcher's own sentence.
200 queued, plussponsoring an unpriced offer (… BATCHER_SPONSOR_UNPRICED=allow).That run found a real bug (fixed in
e80fe37): the SDK wraps an adapterrefusal as
{"error":"Validation failed","message":"NOT_SPONSORED: …"}, and thefirst version of the mapping read only
error— so it matched"Validation failed"and still answered 500. Both fields are scanned now, andthe test fixture is the verbatim body a live batcher returned.
Not done here
bun run test:grand— needs a fresh full Midnight + Celestia devnet(~88 minutes exclusive). This machine is shared and its Docker was hosting
another stack. This branch adds no tables and no SQL, so the suite's three
static schema lists are untouched, and under the shipped defaults neither
process refuses anything.
exit criterion on preprod, together with the
enforceswitch and thebrowser walkthrough.
Found in passing, not fixed
packages/batcheris in no typecheck gate and does not currently typecheck(4 pre-existing errors in
celestia.tsandmidnight-balancing.ts). One realerror in this branch's new code was caught only by an ad-hoc
tscrun. Writtenup separately; adding the gate means fixing those four first, which is not this
change's scope.