A queue service with an SQS-shaped API, Postgres-backed to start, storage swappable later

[aaylward]

Follows #1389, which moves the one_d4 index worker to C++ and keeps the Postgres table as the queue. That is the right call for #1389 — the table is the queue since #1279, and rewriting the worker is enough work on its own. This issue is the next question: put a small SQS-shaped API in front of the queueing behavior so the physical storage stops being load-bearing, and keep Postgres underneath for now (and maybe forever).

The goal is a seam, not a new database. Postgres stays the implementation; what changes is that nothing outside the queue service knows that.

What we have today

indexing_requests is worked through IndexingRequestStore, and it is already a real queue — the operations map almost one-for-one onto SQS:

IndexingRequestStore	SQS
createOrAdopt(...) — atomic dedupe on a content key, returns whether you created it	SendMessage + FIFO MessageDeduplicationId
claimNext(ownerId, lease, now) — oldest live unheld row	ReceiveMessage with a visibility timeout
renewLease(id, ownerId, lease, now)	ChangeMessageVisibility
handBack(id, ownerId, now)	ChangeMessageVisibility(0)
terminal updateStatusOwned(...)	DeleteMessage
reclaimStale(staleAfter, now) + MAX_ATTEMPTS = 3	visibility expiry + maxReceiveCount → DLQ
IndexWorkerLifecycle's 5 s IDLE_POLL + in-process nudge	long polling (WaitTimeSeconds)

So the vocabulary already exists. The work is not inventing queue semantics — it is separating them from everything else the table does.

The design challenge

1. The table is not only a queue. It plays five roles at once:

1. work queue (claim / lease / retry / poison-retire),
2. job status record — GET /v1/index/{id} reads status, gamesIndexed, errorMessage back out of it, through IndexRequestService,
3. idempotency registry — createOrAdopt returns the existing request so a duplicate submit attaches to the run in flight and the caller gets the same id back,
4. progress channel — updateStatusOwned writes gamesIndexed as the run proceeds,
5. retention target (deleteOlderThan).

SQS gives you (1) and a weak version of (3). It gives you nothing of (2), (4), or (5) — a message is an opaque body that is deleted on success, so there is nowhere to read "how did job X end?" afterwards. Any design that says "replace the table with a queue API" has to answer where roles 2–5 live, and the answer must not be "the queue service also stores chess indexing status", because that is the domain leaking into the generic thing and it would defeat the swap.

The split this forces: the queue carries message identity, delivery, and retry; one_d4 keeps a jobs table for status/progress/dedupe-of-user-intent. The message body becomes little more than a job id. Roughly: indexing_requests loses owner_id, lease_expires_at, and attempts, and keeps everything else.

2. Fencing stops being free. Today every write in a run is fenced by holdsLease(id, ownerId, now) against the same Postgres the write goes to (#1278). Split the queue out and the lease lives in one service while the writes go to another database — the classic distributed-lock hole: a worker paused past its visibility timeout wakes up and writes, while a second worker is already running the same message. SQS cannot help here: receipt handles are opaque and unordered. So this design deliberately deviates: every delivery carries a monotonically increasing fencing token, the job row stores the highest token it has seen, and a fenced write is UPDATE ... WHERE fencing_token <= :token. Without this the API is not safe to adopt, and it is the single most important thing SQS's shape does not give us.

3. Enqueue becomes a dual write. Today creating the request is enqueueing it — one row, one transaction, no window. With a queue service, submit must write a job row (one_d4's DB) and send a message (queue service), and either half can fail. Options: a transactional outbox in one_d4, or — simpler and probably enough here — make SendMessage idempotent on the job id and have a reconciler re-send for jobs sitting in PENDING with no live delivery. Worth deciding explicitly rather than discovering.

4. At-least-once meets a non-idempotent write. Lease expiry already means a message can run twice, so this is not new — but it is worth writing down that the occurrences flush (delete + re-insert, ConcurrentFlushTest) is the part that doubles under concurrent runs, and the fencing token from (2) is what protects it.

5. Our dedupe window is not SQS's. FIFO dedupe is a fixed 5-minute window on a caller-supplied id. Ours is "at most one live job per content key, for however long that job runs" — enforced by a partial unique index on non-terminal rows, and hours long for a big backfill. That is a property of the jobs table, not the queue, and it should stay there. The queue's own dedupe (if any) is a much weaker safety net.

6. Ordering and fairness. claimNext is oldest-first (ORDER BY created_at). SQS standard queues promise neither ordering nor fairness. Since a future adapter might not offer oldest-first either, the contract should say what it actually guarantees — I'd propose "approximately oldest-first, best effort, no ordering guarantee" so a Redis or in-memory adapter isn't born non-conforming.

7. The inline path exists. IndexRequestService.submitHybrid runs single-month requests to completion on the calling thread, never touching the queue. Any redesign has to keep that path (it is a latency feature, not an accident) or explicitly retire it.

Initial design

Shape

A small standalone service, domains/platform/apis/<name> (alongside prom_proxy), C++ on the existing stack: smithy-cpp for the API, //domains/platform/libs/pg for storage, //domains/platform/libs/aura for the serving chain, linux_amd64_oci_binary for the image. Name suggestions: hopper, sluice, chute — bikeshed freely.

Protocol: alloy#simpleRestJson, the same binding portrait uses, so the C++ worker in #1389 gets a generated client (see #1390) and the model is the contract. Long polling is just a slow response; no streaming needed in v1. If push delivery ever beats polling, smithy-cpp's event-stream support is there, but polling is the right v1.

Operations

service Hopper {
    version: "2026-08-17"
    operations: [SendMessage, ReceiveMessages, ExtendVisibility,
                 DeleteMessage, ReleaseMessage, GetQueueStats]
}

/// Enqueue. `dedupeKey` collapses re-sends onto the live message, and the
/// response says whether this call created it — the property IndexRequestService
/// needs to decide whether to dispatch.
operation SendMessage {
    input := {
        @required queue: QueueName
        @required body: MessageBody        // opaque to the service
        dedupeKey: String
        delaySeconds: Integer
    }
    output := { @required messageId: String, @required created: Boolean }
    errors: [QueueFull, QueueNotFound]
}

/// Long-polling receive. Each delivery carries a receipt handle and a fencing
/// token strictly greater than any previous delivery of the same message.
operation ReceiveMessages {
    input := {
        @required queue: QueueName
        maxMessages: Integer               // 1..10, default 1
        visibilityTimeoutSeconds: Integer  // default per-queue
        waitTimeSeconds: Integer           // 0..20, long poll
    }
    output := { @required messages: DeliveryList }
}

structure Delivery {
    @required messageId: String
    @required receiptHandle: String
    @required body: MessageBody
    @required fencingToken: Long      // monotonic per message
    @required receiveCount: Integer
    @required enqueuedAt: Timestamp   // @timestampFormat("epoch-seconds")
}

plus ExtendVisibility(queue, receiptHandle, seconds) (the heartbeat, returning the unchanged token), DeleteMessage(queue, receiptHandle) (ack), ReleaseMessage(queue, receiptHandle) (nack → immediately visible), and GetQueueStats(queue) → {visible, inFlight, oldestVisibleAgeSeconds} for dashboards and alerting. Stale receipt handles get a typed ReceiptExpired error rather than a silent no-op — a heartbeat that quietly fails is how you get two live runs.

Deliberately not in v1: PurgeQueue (destructive, admin-gated at best), queue creation over the API (queues are configuration, declared at deploy), and message attributes/filters.

Storage seam

QueueStore (pure virtual)          <- the seam
├── PostgresQueueStore             <- v1, //domains/platform/libs/pg
├── InMemoryQueueStore             <- tests, and the proof the seam is real
└── ...                            <- Redis / SQS / NATS, later

One contract test suite runs against every adapter — the Beyoncé-rule pattern golf_hub already uses for its smithy contract tests. The design is not proven by argument; it is proven when a second adapter passes the same suite unmodified. That is the acceptance criterion for this issue, and InMemoryQueueStore is cheap enough that it can land in the same PR as the Postgres one.

Postgres schema is essentially what indexing_requests already carries, minus the domain:

CREATE TABLE queue_messages (
  id             UUID PRIMARY KEY,
  queue          VARCHAR(64)  NOT NULL,
  body           BYTEA        NOT NULL,
  dedupe_key     VARCHAR(600),          -- partial unique index over live rows
  visible_at     TIMESTAMP    NOT NULL, -- delay + visibility timeout
  receipt        UUID,                  -- current delivery, NULL when visible
  fencing_token  BIGINT       NOT NULL DEFAULT 0,
  receive_count  INT          NOT NULL DEFAULT 0,
  enqueued_at    TIMESTAMP    NOT NULL
);

ReceiveMessages is SELECT ... FOR UPDATE SKIP LOCKED over visible_at <= now() ordered by enqueued_at — the standard Postgres-as-queue pattern, and a better fit than today's claim because SKIP LOCKED lets concurrent receivers avoid each other without lock conflicts. Long polling rides LISTEN/NOTIFY on //domains/platform/libs/pg:listener, which already exists with reconnect-and-re-LISTEN healing and was built for exactly this fan-out — a notify is a wake-up, every wake re-reads state, so a dropped notification costs latency and not work. That is precisely the guarantee a long-poll needs.

Safety

• Not publicly routed. Internal network only, like /v1/analyze. Plus @httpBearerAuth in the model — smithy-cpp's ClientConfig has bearer_token as a per-request callback, so this costs a config line on the client.
• Bounded everything: message body ≤ 64 KB, maxMessages ≤ 10, waitTimeSeconds ≤ 20, visibility timeout ≤ 1 h, and a per-queue depth ceiling that makes SendMessage return a typed QueueFull instead of growing without limit. Most of these are Smithy @range traits, i.e. validated by generated code rather than by hand.
• Poison messages: maxReceiveCount per queue → dead-letter queue, replacing today's MAX_ATTEMPTS = 3 retire. A DLQ is just another queue, so draining it needs no new API.
• Retention: per-queue TTL sweeper, same shape as the existing RetentionPolicy.
• Rate limiting through the aura middleware chain, already there.
• The API is a fixed set of operations over opaque bodies — no queue creation, no arbitrary predicates, nothing that turns it into a general database proxy.

Migration

1. Service + model + PostgresQueueStore + InMemoryQueueStore + the shared contract suite. No consumers yet.
2. The C++ index worker from one_d4: rewrite the index worker in C++ on Disservin/chess-library #1389 consumes it from day one — new code, no migration, and it is the natural first client. IndexRequestService sends the message; the job row keeps status.
3. Retire the queue columns (owner_id, lease_expires_at, attempts) from indexing_requests once the Java worker is gone. Cannot happen before the fencing token is in place and the jobs table honors it.
4. A third adapter when there is a reason. If none ever appears, Postgres stays and the seam still paid for itself in testability.

Deviations from SQS, on purpose

SQS	Here	Why
opaque unordered receipt handles	receipt handle + monotonic fencing token	fenced writes across a service boundary; the whole safety story
FIFO dedupe, fixed 5-minute window	dedupe on the jobs table, for the life of the job	ours is a business key, not a retry artifact
no ordering, no fairness	"approximately oldest-first, best effort"	matches claimNext without over-promising for future adapters
256 KB bodies	64 KB	bodies are a job id and a few parameters; smaller ceiling, fewer surprises
PurgeQueue in the API	not in v1	destructive and rarely what you meant

Open questions

• Does the fencing token belong in the queue API at all, or is it a jobs-table concern? I think the queue must issue it — only the queue knows the delivery order — but it does put a domain-flavored field in a generic API, and I would like that challenged.
• Who else wants this? golf_hub and r3dr have no queue today. If the answer is "nobody yet", v1 should stay deliberately small rather than growing knobs for hypothetical users.
• Does submitHybrid's inline path survive? It bypasses the queue entirely for single-month requests. Keeping it means two code paths forever; retiring it means a latency regression on the most common request shape.
• One service or a library? Everything above works as a C++ library linked into each consumer, with no network hop and no new container — the seam would still be real. A service buys language-independence (the Java one_d4 API also enqueues) and one place to swap storage; a library buys simplicity and no dual-write. The Java submit path is probably what decides it.

[aaylward]

Broker selection: none of RabbitMQ / ZeroMQ / Kafka. Keep Postgres.

The obvious next question on this issue is whether the storage seam should be pointed at a real broker on day one. Working through it: no — and the shortlist that usually gets named is the wrong shortlist.

The host settles it before semantics do

deploy/consolidated/compose.yaml declares 6.5 CPUs and 6.1 GB of memory limits across 15 containers, on a host with 2 vCPUs and not a lot of RAM. Those are ceilings rather than reservations, so it works only because nearly everything is idle nearly all the time. shared_postgres — the database three services depend on — runs at 0.25 CPU / 256 MB.

A broker also replaces nothing. The jobs table stays regardless (roles 2–5 above), so a broker is strictly added infrastructure: a second durable store, a second volume, a second backup story, a second crash-loop mode, on one box with no replication. One durable store is a feature here.

Each candidate fails on shape, not just size

Worth separating, since "too heavy" is the weaker argument:

Kafka — wrong model. A partitioned log with consumer-group offsets: no visibility timeout, no per-message ack, no selective redelivery. A run holding a message for 20 minutes head-of-line-blocks its partition, and max.poll.interval.ms rebalances it out from under itself. ONE_D4_INDEX_SLOTS=4 would need 4+ partitions and still block. DLQ, dedupe-on-business-key and {visible, inFlight} all become machinery we write anyway — and it is a JVM broker competing with the Java one_d4 service for two cores.

RabbitMQ — right category, wrong primitives. AMQP 0-9-1 has no visibility-extension operation, so ExtendVisibility — the heartbeat this design's safety story rests on — has no counterpart. A delivery is held on an open channel for the life of the run, and the broker's delivery-acknowledgement timeout (30 min by default) fights the "visibility timeout ≤ 1 h" bound. Acks are channel-scoped, so a dropped connection requeues immediately rather than after a lease we control. No fencing token; it would have to be synthesized from x-death on republish. No official C++ client either — vendoring AMQP-CPP or rabbitmq-c into Bazel is its own project. We would pay for a stateful Erlang service and still hand-build deviations 1 and 2 from the table above.

ZeroMQ — not a broker. No persistence, no acks, no durability; messages live in process memory and a restart loses the queue. Cheap precisely because it does none of what durable jobs need.

NATS JetStream — not on the original list, and the only off-the-shelf option that natively provides every non-negotiable here. Single static Go binary, tens of MB RSS, no JVM or BEAM. AckWait is a visibility timeout, InProgress() is ExtendVisibility, Nak is ReleaseMessage, MaxDeliver gives the DLQ, Nats-Msg-Id gives a dedupe window, pull-with-expires is long polling. And every delivery carries Metadata().Sequence.Consumer, monotonically increasing across redeliveries — that is the fencing token, issued by the broker, exactly as open question 1 argues it must be. Official C and Java clients. Still not worth deploying today, but it is the one to reach for if the day comes.

Why Postgres is right, not merely affordable

The dual write disappears. This is the strongest argument, and it inverts design challenge 3. Any external broker turns submit into a write to one system plus a send to another, forcing an outbox or a reconciler. Postgres-backed, SendMessage can enqueue in the same transaction as the job row. The problem does not get solved; it stops existing. Trading that away for a second daemon on a 2-vCPU host is a bad deal at any volume this repo will see.

Most of PostgresQueueStore already ships. domains/games/apis/one_d4_worker/pg_queue.cc already does FOR UPDATE SKIP LOCKED claim, Heartbeat, and lease-fenced terminal writes, in C++, in production, from #1389. The v1 adapter is largely that file with the domain columns dropped and fencing_token added. //domains/platform/libs/pg:listener already exists with reconnect-and-re-LISTEN healing, which is the long-poll wake-up. This is the path where we extend working code instead of adding a dependency.

Headroom is enormous. Postgres-as-queue with SKIP LOCKED sustains thousands of messages/sec. The producer here is humans submitting index requests against 4 worker slots, where a run spends minutes waiting on chess.com. That is three or four orders of magnitude of margin — and on 2 vCPUs a broker's own poll loops and heartbeats would cost more CPU than the message volume does.

Ordering comes free. ORDER BY enqueued_at gives real oldest-first, matching claimNext. The "approximately oldest-first, best effort" contract wording is still right — don't over-promise for future adapters — but the v1 adapter will happen to be strictly better than its contract, which is the correct direction.

Trigger conditions, written down instead of revisited

• Sustained enqueue rates where SKIP LOCKED contention or table bloat shows up in metrics. Realistically thousands/sec.
• Fan-out: multiple independent consumers each needing every message. The one thing the table genuinely cannot do well, and the case that would justify JetStream on merit rather than on volume.
• Workers on hosts that should not hold database credentials at all.

None hold today, and the acceptance criterion already makes deferring free: QueueStore + PostgresQueueStore + InMemoryQueueStore against one contract suite means a broker, if it ever arrives, is a third adapter and a compose entry.

Knock-on for open question 4 (service or library)

This sharpens it, and partly dissolves the stated tradeoff: with Postgres underneath, neither option has a dual write, so that argument for the library goes away. What remains is that a library means implementing the store twice — C++ for the worker, Java for the submit path — which is two chances to drift and a contract suite that has to run in two languages. A service means one implementation, and on a host already running 15 containers, one more small C++ binary at 128–256 MB is a categorically different proposition from a stateful broker: no volume, no backups, restartable at will. Service, sized small.

---

Generated by Claude Code

[aaylward]

Prior art survey: OSS implementations of Google's pull task queues

Follow-up to the broker comment above. If we're building an SQS-shaped lease API, the closer lineage is Google's pull task queues, so I went looking for what already implements those semantics and what can be borrowed or adopted outright.

The lineage, and its status

Three generations, worth separating because they get conflated:

1. App Engine Task Queue, pull mode (2010) — lease_tasks(lease_seconds, max_tasks), delete_tasks, modify_task_lease, plus tag grouping (lease_tasks_by_tag).
2. Cloud Tasks v2beta2 — the same model as first-class RPCs: LeaseTasks, AcknowledgeTask, RenewLease, CancelLease. Rate-limited to 10 LeaseTasks/sec per queue; past that it returns OK with zero results.
3. Cloud Tasks v2 GA — pull queues removed. Google steers to Pub/Sub.

So the shape this issue models outlived the product. That matters for reading the survey: the thin ecosystem around this API reflects the API being retired, not the design being bad.

One detail from generation 2 is directly relevant to design challenge 2. RenewLease, AcknowledgeTask and CancelLease all require the task's current schedule_time, taken from the previous LeaseTasks/RenewLease response, and fail if it doesn't match (RenewLeaseRequest). That is CAS used as a lease check — the same instinct as our typed ReceiptExpired. It is still not a fencing token: it proves ownership to the queue, not to a second database. Even Google's successor (Pub/Sub modifyAckDeadline over an opaque ackId) doesn't close that gap.

Category A: implementations of the actual Google API

Two exist. Both are test doubles.

DevPullQueue.java`` — Google's own, open sourced in appengine-java-standard (Apache 2.0). Reference semantics, and small: a synchronized HashMap keyed by task name, `queryAndOwnTasks` sorting by ETA ascending, a binary search (`availableTaskCount`) cutting the list at `now`, leases applied by rewriting ETA to `now + leaseSeconds`. `modifyTaskLease` validates the passed ETA against the stored one and that the lease hasn't expired before extending.

Two things stand out. No retry count at all — no counterpart to maxReceiveCount/DLQ, so it never modeled poison messages. And modifyTaskLease's ETA check is exactly our ReceiptExpired semantics, implemented as CAS on a timestamp. It's a couple hundred lines, which is independent support for the claim that InMemoryQueueStore is cheap enough to land in the same PR.

aertje/cloud-tasks-emulator — Go, gRPC, actively maintained, targets the push API (HTTP targets, OIDC minting). No lease/ack surface, because v2 has none. "Just run an emulator" is not a path to pull semantics.

Net: nobody maintains a production-grade clone of the pull API. Adoption isn't on the table.

Category B: the same semantics under different names

System	Lease primitive	Extend	DLQ	Long poll	Footprint	Fits here?
beanstalkd	reserve + TTR	touch ✓	bury/kick	reserve-with-timeout	tiny C daemon	No — all jobs in RAM
pgmq	read(vt, qty)	set_vt ✓	archive table only	read_with_poll (busy)	zero, it's Postgres	Closest
Faktory	FETCH + reserve_for	not in protocol	retries + dead set	yes	Go + Redis	Partly
River	SKIP LOCKED	✓	✓	LISTEN/NOTIFY	library, no daemon	Go-only
ElasticMQ	literal SQS	ChangeMessageVisibility ✓	✓	✓	JVM/Pekko	No — JVM test double

beanstalkd is the purest semantic match, and predates SQS: reserve/TTR/touch/release/bury/delete maps one-for-one onto lease/visibility/extend/nack/DLQ/ack. But all data is kept in memory; the binlog is recovery-only, read solely at startup. RAM scales with queue depth — wrong property for this host, and it reintroduces the dual write. Its bury/kick design is still worth weighing against our "a DLQ is just another queue" line: burying preserves the original job id, so a kicked job resumes its identity instead of becoming a new message in a second queue.

pgmq is the one that matters. It is essentially this issue's design, already built: send / read(queue, vt, qty) / pop / delete / archive / set_vt (functions), over tables q_<name> and a_<name> with msg_id, read_ct, enqueued_at, last_read_at, vt, message, headers. Three findings:

• It installs SQL-only. Since pgmq#379 you can psql -f pgmq.sql into a stock Postgres — no compiled extension, no custom image (INSTALLATION.md). It drops into the existing one_d4_migrate one-shot against postgres:18 unchanged. This is the thing I expected to disqualify it, and it doesn't.
• Its long polling is worse than what we already have. read_with_poll(queue, vt, qty, max_poll_seconds, poll_interval_ms) busy-polls inside the function, holding a Postgres backend for the whole wait — pgmq#342 is exactly that, 100% CPU per connection at short intervals. //domains/platform/libs/pg:listener on LISTEN/NOTIFY is strictly better, and the two compose fine: pgmq for the store, pg:listener for the wake-up.
• read_ct is already our receiveCount — monotonic per message, incremented on every delivery.

Faktory is interesting for the polyglot problem: a language-agnostic wire protocol (FETCH/ACK/FAIL) rather than a library, which is the property a C++ worker plus a Java submit path needs. reserve_for defaults to 1800s with a 60s minimum. I found no lease-extension command in the protocol spec — worth confirming before relying on it, but if so it has RabbitMQ's problem: long runs hold the reservation rather than renewing it.

River and ElasticMQ are mostly evidence rather than options: River's headline guarantee is transactional enqueue, which is the dual-write elimination argued above; ElasticMQ shows that a full SQS-compatible server already exists if we wanted literal SQS — it's just a JVM test double this host can't spare a core for.

The fencing gap: nobody has it

Searched specifically for this. Essentially no production queue issues a fencing token with delivery — not SQS, Pub/Sub, beanstalkd, pgmq, Faktory, or Cloud Tasks. The single hit was pivovarit/fencepost, a Java Postgres library minting monotonic tokens from per-lock sequences (fencepost_st_<name> via nextval, non-transactional so tokens are visible immediately) exposed as runLocked(token -> {...}). It has a queue side too (fencepost_queue with visible_at, picked_by, attempts, dead_at), but the token lives on the lock path, not on delivery. At 3 stars and no releases it's a proof of concept, not a dependency.

Two readings, both true:

• Open question 1 asks whether the fencing token belongs in the queue API. Nobody puts it there — weak evidence against, and it deserves stating plainly rather than burying.
• But none of these systems fences writes against a different database. They fence against themselves, where CAS on a receipt or a schedule_time suffices. The split here — lease in the queue service, writes in one_d4's DB — is the case their designs don't cover. The absence is explained by their scope, not by the idea being wrong.

And the cost is near zero on the adapter we're actually building: pgmq's read_ct shows a per-message monotonic delivery counter is one column and one increment.

That surfaces a real question about Delivery. It requires both fencingToken: Long and receiveCount: Integer. In a Postgres adapter derived from a delivery counter those are the same number. They should still stay separate in the contract — a JetStream adapter's Sequence.Consumer is a stream position, not a receive count — but the contract suite should assert the properties independently (token strictly increases across deliveries; receiveCount counts deliveries) so an adapter that aliases them still passes honestly.

What I'd take into v1

1. Evaluate pgmq as PostgresQueueStore's implementation, not just as inspiration. SQL-only install fits the migration one-shot; the smithy service would sit over pgmq.send/read/delete/set_vt instead of hand-rolled SQL, keep pg:listener for long polling, and add fencing_token as the one extension. Tradeoffs to weigh: a vendored SQL blob in migrations/, and its archive table isn't a DLQ.
2. Steal the CAS-on-lease check. Google requires schedule_time on all three lease-holding ops. The receipt handle should carry the same property so ReceiptExpired is structurally impossible to miss, not merely typed.
3. Add a floor to the visibility timeout. We bound the ceiling at 1h; Faktory's 60s minimum exists because sub-minute reservations produce redelivery storms under any GC pause or slow query.
4. Keep the fencing token, and note in the model docs that it is a deliberate deviation with no upstream precedent. The survey backs the "SQS's shape does not give us this" line — it isn't just SQS, it's the whole field.
5. Read DevPullQueue.java before writing InMemoryQueueStore — same job, same size, and its gaps (no retry count, coarse locking) are a checklist of what to do differently.

---

Generated by Claude Code