Add brokered timerfd support to ulitebox by willportnoy · Pull Request #1125 · microsoft/litebox

Will Portnoy (willportnoy) · 2026-08-05T02:31:57Z

Adds Linux timerfd support to the ulitebox broker. Timer expiry is delegated to the host through the broker — a dedicated epoll reactor owns real host timerfds and publishes readiness through the notification ring, mirroring the brokered TCP and eventfd model. The guest gets timerfd_create/timerfd_settime/timerfd_gettime plus read, close, and epoll through a new shim subsystem.

Introduce a broker-owned timerfd that delegates timer expiry to the host platform, mirroring the brokered TCP/eventfd model: the dedicated epoll reactor in litebox_broker_platform_linux_userland owns a real Linux timerfd and publishes READ readiness via the notification ring; the broker core exposes a provider-driven TimerfdObject; broker_local gains a create/set/get/read client; and the wire protocol carries the new Timerfd request/response family. Details: - protocol: TimerfdSpec + Create/Set/Get/Read messages and wire codec; BrokerOperation::Timerfd / BrokerResult::Timerfd; bump MAX_ENCODED_ACTIVE_MESSAGE_SIZE 38->54 (timerfd set is the new largest active message, still within the 112-byte control-ring slot). - core: TimerfdProvider/PlatformTimerfd traits, TimerfdObject bounded by max_references, ObjectEntry::Timerfd, UnsupportedTimerfdProvider default plus a with_timerfd_provider builder to avoid a BrokerCore::new ripple. - platform: LinuxTimerfdProvider + reactor owning host timerfd fds in an edge-triggered epoll set; readiness snapshot updated on expiry/drain. - userland: wire LinuxTimerfdProvider into the broker binary. - host/local: dispatch + client plumbing; exhaustive Timerfd match arms. The userland_broker integration test drives the real broker binary end to end: it arms a 5ms one-shot host timer, observes READ readiness, and drains exactly one expiration. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: ea6a6c0d-6bb9-4eed-b8f6-5059a53e326d

Add litebox::event::timer::Timer, a local-core timer backed by a broker-owned host timerfd, mirroring EventCounter: new/set_time/get_time/ read plus IOPollable and Drop, with a TimerError enum (non_exhaustive, matching the EventCounterError sibling convention) and the corresponding BrokerObjectError conversions. Extend the BrokerControl trait with create/set/get/read_timerfd and implement them on BrokerLocalControl. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: ea6a6c0d-6bb9-4eed-b8f6-5059a53e326d

Add the guest syscall surface so a sandboxed program can use timerfds: timerfd_create/timerfd_settime/timerfd_gettime, plus read/close/epoll routing, wrapping litebox::event::timer::Timer in a TimerfdSubsystem that mirrors the eventfd subsystem. - common_linux: Itimerspec ABI struct, TfdFlags/TfdTimerFlags, the three SyscallRequest variants and their decode arms, and From<TimerError> for Errno (TryOpError<TimerError> converts via the existing generic impl). - shim: TimerfdSubsystem/TimerFile; sys_timerfd_* with itimerspec marshalling (EINVAL on negative guest seconds, EOVERFLOW on host->guest overflow, NULL old_value tolerated); an extra run_on_raw_fd closure and matching arms in do_read (8-byte expiration count), write (EINVAL), do_close, epoll (EpollDescriptor/DescriptorRef), fcntl and ioctl. clockid is restricted to CLOCK_REALTIME/CLOCK_MONOTONIC. New owned-enum variants trip E0004 at every dispatch site by design; each is resolved with an explicit Timerfd arm, no wildcards. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: ea6a6c0d-6bb9-4eed-b8f6-5059a53e326d

Drop the fd suffix from the broker-facing timer object so it matches the Event/Socket/Pipe convention (the fd suffix belongs to the shim subsystem, not the broker object). No behavior change. - protocol: timerfd.rs -> timer.rs, wire/timerfd.rs -> wire/timer.rs; TimerfdRequest/Response -> TimerRequest/Response, TimerfdSpec -> TimerSpec, BrokerOperation::Timerfd/BrokerResult::Timerfd -> ::Timer, encode/decode_timerfd_* -> _timer_*. - core: timerfd.rs -> timer.rs; TimerfdProvider/PlatformTimerfd/ TimerfdObject/TimerfdResource/UnsupportedTimerfdProvider -> Timer*; ObjectEntry::Timerfd -> ::Timer; with_timerfd_provider -> with_timer_provider. - local/host/platform/userland: create/set/get/read_timerfd -> *_timer; LinuxTimerfdProvider/LinuxTimerfd -> LinuxTimer*; handle_timerfd_request -> handle_timer_request. - litebox: BrokerControl timer methods and the guest Timer follow suit. The shim keeps TimerfdSubsystem / sys_timerfd_* / TfdFlags and rustix keeps TimerfdClockId/TimerfdFlags, matching how eventfd names its shim subsystem while the broker object is Event. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: ea6a6c0d-6bb9-4eed-b8f6-5059a53e326d

Two timerfd read/settime outcomes previously collapsed to EIO now reach the guest with the errno Linux uses. - EINVAL: the shim validates the itimerspec before issuing the broker request, rejecting a tv_nsec outside [0, 1e9) (negative seconds were already rejected). This is where Linux validates, so a bad timerfd_settime now returns EINVAL instead of a host round-trip that surfaced as EIO. Covered by unit tests. - ECANCELED: a CANCEL_ON_SET timer whose backing clock is set discontinuously reports ECANCELED once and disarms. The reactor now detects the host ECANCELED read and carries a cancelled outcome through the timerfd read path (TimerRead + ReadTimerResponse.cancelled -> BrokerControl -> guest Timer), which the guest maps to ECANCELED via a new TimerError::Cancelled. This is kept entirely within timerfd-owned code so the shared broker ErrorCode/BrokerError wire enums are untouched; the guest-visible behavior is read() -> ECANCELED. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: ea6a6c0d-6bb9-4eed-b8f6-5059a53e326d

Add tests/timerfd.c, a self-validating guest that exercises the timerfd syscall surface end to end: one-shot and interval timers, gettime, nonblocking and blocking reads, poll and epoll wakeups, dup sharing, CLOEXEC/NONBLOCK flags, an unknown clock, and an out-of-range itimerspec. Its assertions encode native Linux semantics. test_runner_broker_timerfd_with_rewriter runs the same binary twice: once on the native baseline (the gold standard, proving the assertions match real Linux) and once under Litebox with a broker-owned host timer, and asserts the broker released one object per created timer. spawn_test_broker now installs LinuxTimerProvider so broker-backed guests can create timers. This covers the blocking-read and epoll wakeup paths the broker-only integration test does not, and the EINVAL fidelity fix (a tv_nsec >= 1e9 timerfd_settime returns EINVAL under Litebox exactly as it does natively). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: ea6a6c0d-6bb9-4eed-b8f6-5059a53e326d

Bring the timerfd work up to date with ulitebox (df4eb7b -> 9efa6cf). The only conflict was in litebox_runner_linux_userland/tests/run.rs, where both sides appended to BROKER_ONLY_C_TESTS: upstream added tcp_broker_server.c and udp_broker.c, this branch added timerfd.c. Resolved as the union of all entries. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: ea6a6c0d-6bb9-4eed-b8f6-5059a53e326d

The merge from ulitebox brought in a zero-argument init_platform(); the timerfd broker-control test, added on this branch against the old one-argument signature, still passed None. Drop the argument to match every other call site. This is test-only and unblocks the clippy --all-features and nextest CI jobs. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: ea6a6c0d-6bb9-4eed-b8f6-5059a53e326d

github-actions · 2026-08-11T22:23:31Z

🤖 SemverChecks 🤖 ⚠️ Potential breaking API changes detected ⚠️

Click for details

--- failure enum_variant_added: enum variant added on exhaustive enum ---

Description:
A publicly-visible enum without #[non_exhaustive] has a new variant.
        ref: https://doc.rust-lang.org/cargo/reference/semver.html#enum-variant-new
       impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.50.0/src/lints/enum_variant_added.ron

Failed in:
  variant BrokerOperation:Timer in /home/runner/work/litebox/litebox/litebox_broker_protocol/src/message.rs:50
  variant BrokerResult:Timer in /home/runner/work/litebox/litebox/litebox_broker_protocol/src/message.rs:166

--- failure enum_no_repr_variant_discriminant_changed: enum variant had its discriminant change value ---

Description:
The enum's variant had its discriminant value change. This breaks downstream code that used its value via a numeric cast like `as isize`.
        ref: https://doc.rust-lang.org/reference/items/enumerations.html#assigning-discriminant-values
       impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.50.0/src/lints/enum_no_repr_variant_discriminant_changed.ron

Failed in:
  variant SyscallRequest::Pipe2 68 -> 71 in /home/runner/work/litebox/litebox/litebox_common_linux/src/lib.rs:2331
  variant SyscallRequest::Clone 69 -> 72 in /home/runner/work/litebox/litebox/litebox_common_linux/src/lib.rs:2335
  variant SyscallRequest::Clone3 70 -> 73 in /home/runner/work/litebox/litebox/litebox_common_linux/src/lib.rs:2338
  variant SyscallRequest::SetThreadArea 71 -> 74 in /home/runner/work/litebox/litebox/litebox_common_linux/src/lib.rs:2343
  variant SyscallRequest::ClockGettime 72 -> 75 in /home/runner/work/litebox/litebox/litebox_common_linux/src/lib.rs:2346
  variant SyscallRequest::ClockGetres 73 -> 76 in /home/runner/work/litebox/litebox/litebox_common_linux/src/lib.rs:2350
  variant SyscallRequest::ClockNanosleep 74 -> 77 in /home/runner/work/litebox/litebox/litebox_common_linux/src/lib.rs:2354
  variant SyscallRequest::Gettimeofday 75 -> 78 in /home/runner/work/litebox/litebox/litebox_common_linux/src/lib.rs:2360
  variant SyscallRequest::Time 76 -> 79 in /home/runner/work/litebox/litebox/litebox_common_linux/src/lib.rs:2364
  variant SyscallRequest::Getrlimit 77 -> 80 in /home/runner/work/litebox/litebox/litebox_common_linux/src/lib.rs:2367
  variant SyscallRequest::Setrlimit 78 -> 81 in /home/runner/work/litebox/litebox/litebox_common_linux/src/lib.rs:2371
  variant SyscallRequest::Prlimit 79 -> 82 in /home/runner/work/litebox/litebox/litebox_common_linux/src/lib.rs:2375
  variant SyscallRequest::SetTidAddress 80 -> 83 in /home/runner/work/litebox/litebox/litebox_common_linux/src/lib.rs:2386
  variant SyscallRequest::Gettid 81 -> 84 in /home/runner/work/litebox/litebox/litebox_common_linux/src/lib.rs:2389
  variant SyscallRequest::SetRobustList 82 -> 85 in /home/runner/work/litebox/litebox/litebox_common_linux/src/lib.rs:2390
  variant SyscallRequest::GetRobustList 83 -> 86 in /home/runner/work/litebox/litebox/litebox_common_linux/src/lib.rs:2393
  variant SyscallRequest::GetRandom 84 -> 87 in /home/runner/work/litebox/litebox/litebox_common_linux/src/lib.rs:2398
  variant SyscallRequest::Getpid 85 -> 88 in /home/runner/work/litebox/litebox/litebox_common_linux/src/lib.rs:2403
  variant SyscallRequest::Getppid 86 -> 89 in /home/runner/work/litebox/litebox/litebox_common_linux/src/lib.rs:2404
  variant SyscallRequest::Getuid 87 -> 90 in /home/runner/work/litebox/litebox/litebox_common_linux/src/lib.rs:2405
  variant SyscallRequest::Geteuid 88 -> 91 in /home/runner/work/litebox/litebox/litebox_common_linux/src/lib.rs:2406
  variant SyscallRequest::Getgid 89 -> 92 in /home/runner/work/litebox/litebox/litebox_common_linux/src/lib.rs:2407
  variant SyscallRequest::Getegid 90 -> 93 in /home/runner/work/litebox/litebox/litebox_common_linux/src/lib.rs:2408
  variant SyscallRequest::Sysinfo 91 -> 94 in /home/runner/work/litebox/litebox/litebox_common_linux/src/lib.rs:2409
  variant SyscallRequest::CapGet 92 -> 95 in /home/runner/work/litebox/litebox/litebox_common_linux/src/lib.rs:2412
  variant SyscallRequest::GetDirent64 93 -> 96 in /home/runner/work/litebox/litebox/litebox_common_linux/src/lib.rs:2416
  variant SyscallRequest::SchedGetAffinity 94 -> 97 in /home/runner/work/litebox/litebox/litebox_common_linux/src/lib.rs:2421
  variant SyscallRequest::SchedYield 95 -> 98 in /home/runner/work/litebox/litebox/litebox_common_linux/src/lib.rs:2426
  variant SyscallRequest::Futex 96 -> 99 in /home/runner/work/litebox/litebox/litebox_common_linux/src/lib.rs:2427
  variant SyscallRequest::Execve 97 -> 100 in /home/runner/work/litebox/litebox/litebox_common_linux/src/lib.rs:2430
  variant SyscallRequest::Umask 98 -> 101 in /home/runner/work/litebox/litebox/litebox_common_linux/src/lib.rs:2435
  variant SyscallRequest::Prctl 99 -> 102 in /home/runner/work/litebox/litebox/litebox_common_linux/src/lib.rs:2438
  variant SyscallRequest::Alarm 100 -> 103 in /home/runner/work/litebox/litebox/litebox_common_linux/src/lib.rs:2441
  variant SyscallRequest::Pause 101 -> 104 in /home/runner/work/litebox/litebox/litebox_common_linux/src/lib.rs:2444
  variant SyscallRequest::SetITimer 102 -> 105 in /home/runner/work/litebox/litebox/litebox_common_linux/src/lib.rs:2445
  variant SyscallRequest::GetITimer 103 -> 106 in /home/runner/work/litebox/litebox/litebox_common_linux/src/lib.rs:2450
  variant SyscallRequest::Statx 104 -> 107 in /home/runner/work/litebox/litebox/litebox_common_linux/src/lib.rs:2454

Weidong Cui (wdcui) · 2026-08-14T00:33:59Z

Will Portnoy (@willportnoy): I ran multiple agents and got the following findings.

All 8 reviewers completed within the five-minute budgets.

Highest-confidence findings:

High — Timer readiness is lost after the first drain or rearm
litebox_broker_core/src/timer.rs:135-158 , litebox_broker_platform_linux_userland/src/timer.rs:501-542
Independently confirmed by three reviewers. Empty readiness transitions are not published, allowing the next READ publication to be coalesced and blocking reads to hang indefinitely.
High — One session’s publication failure can destroy all sessions’ timers
litebox_broker_platform_linux_userland/src/timer.rs in TimerReactor::run / fail_all_timers
A per-timer readiness publication error terminates the shared reactor, clears every session’s timers, and permanently disconnects subsequent operations.
Medium — Resource exhaustion is incorrectly exposed as EAGAIN
litebox_common_linux/src/errno/mod.rs:590-593
Confirmed by both shim reviewers. Blocking reads, timerfd_create , and control operations can return an errno Linux callers do not expect, potentially causing busy retries or hard failures.
Medium — Valid Linux timerfd clocks are rejected
litebox_shim_linux/src/syscalls/timerfd.rs:82-87
Both shim reviewers identified rejection of CLOCK_BOOTTIME and alarm clock IDs.
Medium — Platform readiness is queried while holding the object lock
litebox_broker_core/src/session.rs:327
This differs from the socket path and can block close/teardown behind the timer snapshot mutex.
Medium — Timer count can exceed readiness-sink capacity
litebox_broker_core/src/timer.rs:99-120
More than 4,096 registrations can trigger the shared-reactor failure above, producing a broker-wide timer denial of service.
Medium — Queue backpressure leaks into syscall behavior
litebox_broker_platform_linux_userland/src/timer.rs , TimerReactorClient::request
A transiently full 64-entry command queue makes settime / gettime fail rather than applying internal backpressure.

Lower-confidence or disputed findings include strict timer-response boolean validation, protocol-version compatibility, epoll behavior after closing one duplicated descriptor, and a scheduler-sensitive 20 ms native test. The two protocol-version reviewers and two epoll reviewers reached opposing conclusions on those items.

check_readiness held the object-store read lock while calling a timer's readiness(), which acquires the platform snapshot mutex. Event and pipe readiness are cheap atomic loads and stay under the lock, but socket readiness already deferred outside it; make timer match by cloning the resource under the lock and reading readiness after releasing it. Addresses review finding #5. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: ea6a6c0d-6bb9-4eed-b8f6-5059a53e326d

#1: the timer reactor published READ on each expiry edge, but after a reader drained the previous expiration the level stayed READ, so the notification ring coalesced the unchanged value and never woke a parked blocking reader again — an indefinite hang on the second and later expirations of an interval timer (or a re-armed one-shot). Republish the edge instead, matching how the socket reactor forces a wakeup when a readable count grows without a level change. A new timer_reactor integration test drives the real reactor through a coalescing-model sink and reproduces the hang; timerfd.c gains consecutive blocking reads. #4: CLOCK_BOOTTIME is a valid Linux timerfd clock but was rejected with EINVAL. Accept it in the shim and the broker clock mapping, with unit and end-to-end coverage. The *_ALARM clocks stay rejected: they require CAP_WAKE_ALARM and can wake a suspended system, outside sandbox authority. Addresses review findings #1 and #4. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: ea6a6c0d-6bb9-4eed-b8f6-5059a53e326d

Will Portnoy (willportnoy) · 2026-08-14T17:18:00Z

AI-authored, human-reviewed.

Thanks Weidong Cui (@wdcui) — the multi-agent pass was genuinely useful. Pushed fixes for the three timer-specific findings; the rest look shared with the socket reactor (details below).

Fixed

Support convenient global platforms for shims #1 (readiness lost → blocking-read hang): the reactor now republishes the READ edge instead of publish, so a parked reader is woken on every expiration, not just the first — the coalesced unchanged-READ was the hang. Mirrors how the socket reactor forces a wakeup on a readable-count change. Added a timer_reactor integration test that drives the real reactor through a coalescing-model sink and reproduces the hang, plus consecutive blocking reads in timerfd.c. (c4a4498)
Implement a tar backed FileSystem #4 (clocks): accept CLOCK_BOOTTIME in the shim and the broker clock mapping, with unit + e2e coverage. I left the *_ALARM clocks rejected on purpose — they need CAP_WAKE_ALARM and can wake a suspended system, which seems outside the sandbox’s authority. Happy to add them if you’d prefer parity with native. (c4a4498)
Support a parametric layered FileSystem #5 (readiness under the object lock): timer now clones the resource under the lock and reads readiness after releasing it, matching the socket path. There’s no lock inversion today, so this is contention hygiene rather than a live deadlock. (715a193)

Shared with the socket reactor — proposing to keep as-is or fix holistically

Begin initial implementation of the POSIX shim #2 / Where should we check untrusted pointers into memory? #6: a publish failure ends the whole reactor via fail_all_* — socket.rs does the same today. Per-object isolation would be better handled for both providers together.
User-pointers support #7: queue-full → ResourceExhausted is the same try_send pattern the socket reactor uses.
Fix CI for semver checks by fetching main #3: ResourceExhausted → EAGAIN mirrors the eventfd mapping; I kept it consistent rather than diverging a single provider.

Glad to take any of the shared ones in this PR if you’d like them addressed here.

Will Portnoy (willportnoy) and others added 6 commits August 4, 2026 16:50

Will Portnoy (willportnoy) marked this pull request as ready for review August 11, 2026 20:30

Will Portnoy (willportnoy) and others added 2 commits August 11, 2026 21:52

Will Portnoy (willportnoy) and others added 2 commits August 14, 2026 14:16

Navigation Menu

Uh oh!

Add brokered timerfd support to ulitebox - #1125

Add brokered timerfd support to ulitebox#1125
Will Portnoy (willportnoy) wants to merge 10 commits into
uliteboxfrom
wportnoy/ulitebox-timerfd

Will Portnoy (willportnoy) commented Aug 5, 2026

Uh oh!

github-actions Bot commented Aug 11, 2026

Uh oh!

Weidong Cui (wdcui) commented Aug 14, 2026

Uh oh!

Will Portnoy (willportnoy) commented Aug 14, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

Will Portnoy (willportnoy) commented Aug 5, 2026

Uh oh!

github-actions Bot commented Aug 11, 2026

Uh oh!

Weidong Cui (wdcui) commented Aug 14, 2026

Uh oh!

Will Portnoy (willportnoy) commented Aug 14, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants