devfs proxy does an unbounded, guest-controlled vzalloc(mmap_length) (fails on 32-bit; allocation-DoS)

[lacraig2]

Summary

When a guest mmap()s a devfs pseudofile, the devfs proxy in src/portal/portal_devfs.c
allocates a single contiguous (k)vzalloc buffer sized to the entire guest-supplied mmap
length — once to seed the shmem backing on first mmap, and again to flush it back on file
release. The length comes straight from vma->vm_end - vma->vm_start (then from the shmem
file's i_size) with no bound or validation. A large or garbage length triggers a vmalloc
of that size, which on a 32-bit donor kernel (vmalloc address space is only ~128–240 MB)
fails:

spr_voip: vmalloc: allocation failure: 821952512 bytes, mode:0x14080c2(GFP_KERNEL|__GFP_HIGHMEM|__GFP_ZERO)
CPU: 1 PID: 3342 Comm: spr_voip Tainted: G        W  O    4.10.0 #1
 [<...>] (vzalloc) from [<...>] (igloo_devfs_proxy_release+0xac [igloo])
 [<...>] (igloo_devfs_proxy_release [igloo]) from [<...>] (__fput)
 [<...>] (__fput) from [<...>] (task_work_run) ...

Found while rehosting a Cisco IP Phone 6821 under Penguin (rehosting/penguin:latest, armel,
donor kernel 4.10): the phone's DSP library libdspg.so mmap()s /dev/sharedmem with a
length derived from an ioctl() query. When that query isn't answered with a real size, the
length is garbage (here ~784 MB / 821952512) and the proxy tries to vzalloc it.

Root cause

src/portal/portal_devfs.c

1. mmap seed path — igloo_devfs_proxy_mmap():

size_t size = vma->vm_end - vma->vm_start;                 // guest-controlled
...
shm_file = shmem_kernel_file_setup(pe->name, size, ...);   // sparse — OK
void *buffer = kvzalloc(size, GFP_KERNEL);                 // >= 4.12
void *buffer = vzalloc(size);                              // < 4.12 (e.g. 4.10 donor)
ssize_t bytes = igloo_fetch_mmap_page(file, buffer, 0, size);

2. release flush path — igloo_devfs_flush_shm_to_hypervisor() (from igloo_devfs_proxy_release()):

size = i_size_read(file_inode(pe->shm_file));              // == the mmap length
buffer = kvzalloc(size, GFP_KERNEL);                       // >= 4.12
buffer = vzalloc(size);                                    // < 4.12
bytes = kernel_read(pe->shm_file, buffer, size, &pos);
... file->f_op->write(file, buffer, bytes, &pos);          // push back to hypervisor

Both allocate size bytes contiguously in one shot. The shmem file itself is created sparse
(VM_NORESERVE) and is fine; only the staging buffer is the problem. On the seed path a
NULL from a failed vzalloc is tolerated (the content fetch is skipped), but on the release
path the failure just drops the writeback — and on both paths the huge allocation attempt spams
the kernel log and, once the sparse shmem is touched, balloons guest RAM.

Impact

• Correctness: any pseudofile a guest maps with a large length is silently left un-seeded
  and its writeback is dropped. Legitimately large device mmaps (DSP / framebuffer / shared-mem
  regions, common on media SoCs) simply don't work on 32-bit guests.
• Robustness / DoS: the allocation size is fully guest-controlled and unbounded — a guest
  can repeatedly mmap()+close() a proxied pseudofile to force large kernel vmalloc
  allocations on demand (vmalloc-space exhaustion / memory pressure on the emulated kernel).
• Diagnosability: the failure surfaces only as a kernel vmalloc warning on an unrelated
  process during __fput, far from the offending mmap() — easy to misattribute.

Reproduce

1. Model a devfs pseudofile whose size/ioctl handshake yields a large mmap length. (In our
   case /dev/sharedmem: its ioctl(0xc0045300) size query was answered with a constant 0
   that never filled the caller's struct, so the length read back was garbage 821952512.)
2. In the guest: mmap(fd, length=821952512, ...) then close(fd).
3. dmesg shows vmalloc: allocation failure: 821952512 bytes ... igloo_devfs_proxy_release
   (and, on the first mmap, the same from igloo_devfs_proxy_mmap).

Suggested fix

Stage the fetch/flush in bounded chunks rather than one (k)vzalloc of the full length:

• Use a fixed staging buffer (one page, or a small #define, e.g. 64 KiB) and loop
  igloo_fetch_mmap_page() / kernel_read() + writeback over [off, off+stage) windows. The
  shmem file is already sparse, so chunked copy is sufficient and uses O(1) kernel memory
  regardless of mmap size.
• Independently, bound/validate the mmap length against a sane per-device maximum and reject
  (-EINVAL/-ENOMEM) or clamp oversize requests, so a guest cannot drive unbounded kernel
  allocation.
• Prefer kvzalloc on all supported kernels (it can fall back to vmalloc), though chunking
  makes the allocator choice moot.

Environment

• igloo_driver from rehosting/igloo-dev-source:v3.0.13; reproduced on rehosting/penguin:latest
  (wrapper 3.0.18.dev17).
• Guest arch armel, donor kernel 4.10.