Incremental EntityCache refresh instead of full rebuild each tick

[bburda]

Summary

Each discovery refresh rebuilds the whole entity cache.

GatewayNode::refresh_entity_cache() (src/ros2_medkit_gateway/src/gateway_node.cpp, around lines 1885-2081) runs the discovery and merge pipeline to build fresh area, component, app and function vectors, then calls ThreadSafeEntityCache::update_all(...). update_all replaces all primary vectors and calls rebuild_all_indexes(), which rebuilds all seven hash maps on every tick. There is no diff or change detection.

The refresh interval is the main CPU lever. A 200 ms refresh costs about 4x the CPU of the 1000 ms default, while memory differences across configs stay within a few MiB. Measured with a benchmark harness in selfpatch_demos (sweep lane).

Proposed solution

• Compare the new entity set against the cached one and apply only the changes (add, remove, update), instead of rebuilding and swapping every refresh.
• Skip the work when nothing changed. On a stable graph most refresh ticks produce the same entities.

Additional context

Full rebuild every tick is wasted work on a stable graph. The cost scales with graph size and refresh frequency, which is why a fast refresh is expensive.

[bburda]

Adding a memory finding that changes the scope here.

On a stable graph the cost is CPU, as written. But under graph churn (nodes and topics repeatedly created and destroyed, with a bounded total count) the full rebuild on every graph event causes unbounded process memory growth, not "a few MiB".

Measured on the gateway_node process, runtime_only, no HTTP load, 20 min:

• static graph (constant nodes): USS plateaus, slope 95% CI includes zero.
• churning graph (~5 node-changes/s, bounded total): USS rises about linearly at ~46 KB/s, no plateau.

It is not a forgot-to-free leak. heaptrack shows the allocations are freed (tiny net-leak at exit, no attributable leaked call-site). The growth is allocator-retained churn from the allocate-new + free-old of all entity containers and all index maps on every graph event (the rebuild path above). Allocator tuning only mitigates it: MALLOC_ARENA_MAX=2 cuts about 62%, jemalloc does not help. So the incremental refresh proposed here is the real fix, and it bounds memory under churn, not only CPU on a stable graph.

Related: bounding the rclcpp executor and HTTP server thread pools (separate issue) lowers the arena count and reduces the retained memory as a side effect.

Minimal self-contained reproducer, run next to the gateway:

# churn.py: recycle a bounded pool of nodes so the graph keeps changing
import itertools, time, rclpy
from rclpy.node import Node
from rclpy.executors import SingleThreadedExecutor
from std_msgs.msg import String

rclpy.init()
ex = SingleThreadedExecutor()
ids, pool, nxt = itertools.count(), [], time.monotonic() + 2.0
while True:
    ex.spin_once(timeout_sec=0.2)
    if time.monotonic() < nxt:
        continue
    for _ in range(min(5, len(pool))):      # remove 5
        n = pool.pop(0); ex.remove_node(n); n.destroy_node()
    for _ in range(5):                        # add 5 fresh, new names each cycle
        i = next(ids); n = Node(f"churn_{i}")
        n.create_publisher(String, f"/churn{i}/t0", 10)
        ex.add_node(n); pool.append(n)
    nxt = time.monotonic() + 2.0

Sample the gateway USS over time:

awk '/^Private/{s+=$2} END{print s}' /proc/<gateway_pid>/smaps_rollup

Regression check with the benchmark:
The synthetic graph in the selfpatch_demos benchmark can churn nodes (recycle a bounded pool, BENCH_CHURN_SEC / BENCH_CHURN_COUNT). Sample the gateway USS WITHOUT heaptrack, once with a static graph and once with churn, then compare the late-window slope. heaptrack itself inflates USS, so the no-heaptrack USS is the signal to gate on. Pass criterion for this issue: before the fix the churn slope is strongly positive while static is flat; after the fix the churn slope drops to the static (flat) level.

[bburda]

Update on verification. The selfpatch_demos benchmark now has a one-command gate for this, so the fix can be checked without manual USS sampling:

python -m benchmark.benchmark churn --entities 50 --duration 600

It runs the synthetic gateway against a static graph and a churning graph (bounded node count), without heaptrack, and compares the steady-window USS slope. The check is relative to the static control, so a noisy control does not cause false flags. PASS means the churn slope is at the static level; LEAK means it is well above static (exit code 1).

On the current gateway it reports LEAK (churn slope ~45000 B/s vs static, exit 1). The fix is complete when this gate reports PASS.