pof: fix conditional delay buffer stalling under sustained reordered traffic

[howhangliu]

Summary

Fixes three defects in the POF (Packet Ordering Function) worker that
cause the conditional delay buffer to fill permanently and drop all
subsequent packets under sustained, reordered traffic.

Adds a test/pof/pof.py case that fails on main and passes here, plus
the network-namespace testbed the bug was originally found on.

The bug

Observed with a FRER stream at 100 pkt/s replicated over two member
paths with independent delay (reordering between the members). After a
short time DNT logs

buffer is full, dropping new packet

and never recovers — every subsequent packet on the stream is dropped.

1. Expired deadline re-arms to take_any_time instead of firing

In pof_thread(), the poll timeout was computed as:

if (next_deadline && timespeccmp(next_deadline, &now, >))
    timespecsub(next_deadline, &now, &timeout);
else
    timeout = pof->pof_take_any_time;

The else branch conflates two different cases: no deadline at all
and deadline already in the past. When the earliest forward deadline
had already passed, the thread armed take_any_time (seconds) rather
than firing immediately. Under continuous traffic each new arrival
re-armed it again, so held packets were never released, the buffer
filled, and everything after that was dropped.

Split the cases: NULL deadline keeps take_any_time, a deadline in
the past arms a zero timeout.

2. eventfd counter values tested as bit flags

eventfd accumulates: two concurrent POF_IN_ORDER_PKT (1) writes read
back as 2, which is POF_OUT_OF_ORDER_PKT. Testing bits of the sum
therefore misclassifies events and drops in-order wakeups.

Attempt a forward on any positive event; pof_try_forward() already
only forwards when the head is next in order, so this is harmless.

3. get_next_deadline() did not select the earliest deadline

The scan used timespeccmp(..., !=) against a moving target and started
at q_head, so it ended up tracking the last differing element rather
than the minimum, and left next_to_forward unset in the single-element
case. Compare with < and seed the scan properly.

Additionally, pof_try_forward() dereferenced pof->q_head
unconditionally; it can be NULL when an event is read after a timeout
already drained the queue. Guarded.

Reproduction

test/pof/pof.py gains a case that fails on main and passes with this
branch:

# main                          # this branch
... 6/7 successfully            ... 7/7 successfully
Test POF expired deadline ... X Test POF expired deadline ... OK
[POF] [WARNING] buffer is full, dropping new packet.   (x6)

Why the existing cases miss it

	existing cases	new case
MaxDelay	520 ms	16 ms
TakeAnyTime	1000 ms	5000 ms
BufferSize	200	32
packets	100	300

Two independent reasons the current suite cannot reach the bug: each case
sends 100 packets into a BufferSize = 200 queue, so buffer is full is
arithmetically unreachable; and MaxDelay = 520 ms sits far above the
30 ms path delay, so a forward deadline essentially never expires while
packets are held.

dntbr1_stall.ini inverts both, and the case drives a single member path
and briefly downs it mid-ping, leaving a sequence gap that can never be
filled - so POF must release the held head on its deadline.

The assertion is on DNT's own buffer is full, dropping new packet
warning, not on ping delivery. The deliberate link-down costs packets in
both the healthy and the stalled case, so delivered counts land only a few
packets apart (287 vs 293 observed) - not a usable signal. The warning is
logged at the default level, so no extra verbosity is needed.

The original scenario

dnt_testbed/ is where this was first hit: a FRER stream replicated over
two member paths with independent exponential delay, recovered and
reordered by a second DNT instance. It is self-contained (network
namespaces, veth, tc netem, no external hardware);
dnt_testbed/README.md has the walkthrough:

cd dnt_testbed
sudo bash -c 'source env_frer_aoi.sh && setup'
sudo ./verify_setup.sh
sudo ./run_point.sh

Notes for review

• The testbed ships scripts and docs only; generated result files are
  gitignored. dnt_testbed/README.md is written around a specific
  study (a delay/history-length sweep), so parts of it are more
  detailed than a pure regression harness would need — happy to trim it
  down, split it into a separate PR, or move it under test/ if you
  would rather keep it out of the tree root.
• The testbed pulls in python3-scapy and python3-regex in addition
  to what .gitlab-ci.yml already installs.
• The new test covers the expired-deadline stall only. The eventfd
  counter fix and the q_head NULL guard have no direct coverage here.
  The NULL deref in particular is only reachable with POF:PACKET
  logging enabled, since log_packet() guards evaluation of its own
  arguments - at the default level the offending dereference is never
  executed.