nat: lock-free SNAT port allocation (#2852 Phase 1)

[psaab]

Addresses #2852 (Phase-1 lock-free allocation; Phase-2 map-sharding + the loss-cluster conn/sec ceiling remain deferred).

What

Replaces the single Mutex<PortAllocatorLiveState> hot-path serialization of pool-mode SNAT port allocation with a per-pool-address atomic occupancy bitmap, so the port claim is lock-free. The #4619 microbench proved the mutex is a measurable bottleneck (current shape negative-scales 2.87M→0.62M allocs/sec M=1→8, p99 tail +100×; the new lock-free bitmap is 1.4–1.6× at M=6/8) — no PLAN-KILL.

Design (converged v3 plan: Claude-SMR + Codex + AGY)

• Lock-free claim: AddressOccupancy (Vec<AtomicU64> occupancy bitmap + atomic fresh-port cursor). A fetch_or CAS on the bit IS the ownership token (a set bit cannot be re-claimed), replacing the mutex-guarded owner_by_translated / addr_index_by_translated maps and next_port_offset_by_addr. The non-persistent new-flow hot path claims lock-free and takes the retained mutex only for a tiny reuse-check + exact-cap-check + live_by_flow insert.
• F1 conditional bit-clear on release (guarded by the live_by_flow record).
• F2 FIFO recycle (userspace-dp NAT: SNAT port recycling is LIFO (Vec push/pop at back) — favors 2MSL/TIME_WAIT port-reuse collisions; prefer FIFO (VecDeque) #3011) preserved exactly — a VecDeque behind a per-ADDRESS mutex (not the global one). Lock order is always global → recycle (innermost), so no deadlock and F5 is sidestepped (Phase 1 has no two-map-shard path).
• F7 addr_index stored in LiveAllocation (O(1) release).
• F4 global cap kept exact as live_by_flow.len() under the tiny insert mutex — no fetch_add-reserve, so the microbench's M-in-flight overshoot cannot falsely exhaust a tiny pool near capacity. Strictly better than release-before-reserve, and available because Phase 1 keeps the maps under one mutex.

Persistent NAT keeps its lease-decision + claim atomic under the mutex (allocate_translation_locked, cold path). Deterministic-v4 (#4559), reserve_flow HA reservation (#4388), and lease reuse (#2397/#4643) all preserved on the bitmap.

Validation

• Full cargo test: 3743 passed / 0 failed. NAT suite 209/0.
• 4 new RED-on-revert concurrency tests: concurrent fill is exact + collision-free (no double-alloc / no false exhaustion / no over-cap), concurrent churn has no double-alloc and no leaked bit, release frees the bit (reusable), tiny pool fills to exact capacity then exhausts.
• Verified RED-on-revert: breaking the bit-clear turns release/churn red; breaking the CAS arbiter turns the userspace-dp SNAT allocator: claim_free_port_locked tries only ONE sequential port per call (no retry on collision) and discards collided recycled ports #3047 forward-probe/recycle-retain and userspace-dp HA/NAT: peer-synced session's NAT pool port not reserved in the secondary's allocator -> post-failover port collision (ps-review-007 P1) #4388 reserve tests red.
• Preserved: userspace-dp NAT: SNAT port recycling is LIFO (Vec push/pop at back) — favors 2MSL/TIME_WAIT port-reuse collisions; prefer FIFO (VecDeque) #3011 FIFO, userspace-dp SNAT allocator: claim_free_port_locked tries only ONE sequential port per call (no retry on collision) and discards collided recycled ports #3047 collision, userspace-dp HA/NAT: peer-synced session's NAT pool port not reserved in the secondary's allocator -> post-failover port collision (ps-review-007 P1) #4388 reserve, nat: deterministic NAT (CGNAT port block-size) validated+committed but silently unenforced on userspace dataplane (ps-034 M-01, med→low-med) #4559 deterministic tests.
• Cluster smoke (nice-to-have, not run here): make test-failover (exercises the reserve_flow HA path) and the loss-cluster new-flow-ceiling measurement (needs a connection-rate generator, distinct from the bulk-throughput perf-test skill) remain lab-deferred gates.

Deferred

Phase 2 (hash-shard live_by_flow + the persistent-lease maps) and the loss-cluster conn/sec ceiling measurement. The residual tiny map mutex is now the serialization point; Phase 2 is warranted only if the lab shows it as the next bottleneck. Design is in the converged v3 plan on the research/2852-portalloc branch.

🤖 Generated with Claude Code

https://claude.ai/code/session_015oARShYtiJJ2H4UB4nXGqi

[Copilot]

Pull request overview

This PR implements #2852 Phase 1 by replacing the pool-mode SNAT allocator’s single hot-path Mutex<PortAllocatorLiveState> port-ownership tracking with a per-address lock-free atomic occupancy bitmap (AddressOccupancy). Port claim becomes lock-free (bitmap fetch_or), while the remaining mutex is kept for the smaller critical section around live_by_flow / persistent leases / exact cap checks.

Changes:

• Introduces AddressOccupancy (atomic bitmap + atomic cursor + per-address Mutex<VecDeque<u16>> FIFO recycle) and removes owner_by_translated, addr_index_by_translated, and next_port_offset_by_addr.
• Reworks allocation/release/reserve paths to use bitmap ownership tokens, and updates white-box tests to use new debug accessors.
• Adds new concurrency “fail-on-revert” tests and updates documentation/research notes + _Log.md.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
userspace-dp/src/nat/allocator.rs	Implements lock-free per-address port ownership via atomic bitmap + adjusts allocation/release/reserve logic and snapshot accounting.
userspace-dp/src/nat/tests_pool.rs	Updates white-box assertions to new debug accessors and adds new concurrency correctness tests for the lock-free claim path.
userspace-dp/README.md	Documents the new lock-free allocation design and references the new fail-on-revert tests.
docs/research/2852-portalloc/microbench-results.md	Records that Phase 1 shipped and explains the design + merge-gate status.
_Log.md	Logs the write/edit action and summarizes the Phase 1 change per repo convention.