Headless backend fails on macOS: ftruncate returns EINVAL on POSIX shared memory

[jserv]

Summary

The headless backend crashes on macOS because ftruncate() on a POSIX shared memory object fails with EINVAL for the 2.4 MB double-buffered framebuffer.

Reproduction

make defconfig
# switch to headless backend in .config
make
./demo-headless

Output:

Failed to resize shared memory to 2473984 bytes: Invalid argument
ERROR src/api.c:38: Backend initialization failed (640x480)

This happens with or without CONFIG_MEM_TLSF -- it is a backend-specific issue.

Root cause

backend/headless.c:316 calls ftruncate(tx->shm_fd, tx->shm_size) on an fd from shm_open(). On macOS, ftruncate on POSIX shared memory objects has stricter constraints than Linux:

• macOS requires shm_unlink of any stale object before shm_open with O_CREAT, or the existing object's size conflicts with the new truncation.
• macOS may also reject ftruncate if the requested size exceeds per-process POSIX shm limits (kern.sysv.shmmax).

The current code at backend/headless.c:305 opens with O_CREAT | O_RDWR but does not shm_unlink before opening, so a leftover object from a previous crashed run causes ftruncate to fail.

Suggested fix

Call shm_unlink(TWIN_HEADLESS_SHM_NAME) before shm_open() to remove any stale object, matching the cleanup already done in _twin_headless_exit() at line 372. Alternatively, open with O_CREAT | O_EXCL and fall back to unlink + retry if EEXIST.