iCloud files download on demand, then go back to the cloud

[TN019]

Reported from a run over a folder of lecture recordings in iCloud Drive: every file failed with extract: ... looks like an iCloud placeholder that is not downloaded locally. In Finder, right-click it and choose "Download Now".

What was actually happening

check_readable() refused any file whose st_blocks was 0 and told the user to go download it by hand. But macOS materializes a dataless file on read — the kernel asks the file provider for the bytes the moment anything opens it. Nothing was stopping these files except our own pre-flight check.

Verified on a real iCloud file: a one-byte read took 2.43s and the dataless flag was gone afterwards.

Download, use, put back

• access.materialize() triggers the download with a single read. That read blocks for the whole transfer (minutes for a 138 MB recording), so it runs on a thread while the caller polls the file growing, stays responsive to the stop button, and gives up on a size-scaled deadline (120s floor + 12s/MB) instead of hanging forever with no network.
• access.evict() returns the file to cloud-only afterwards — Finder's "Remove Download". brctl evict and fileproviderctl no longer have that subcommand on current macOS, so the only supported route is NSFileManager.evictUbiquitousItemAtURL:, reached through the Objective-C runtime with ctypes rather than by taking a PyObjC dependency for one selector. It returns False rather than raising: failing to tidy up is never worth failing a job.
• Only files this run downloaded are ever evicted. Files already on disk are untouched, and the folder is left exactly as it was found. The eviction sits in a finally, so a failed or errored job doesn't leave a download behind.

On the reported folder: 82 videos, 81 of them in the cloud, 12.8 GB total. Peak disk cost is now one file (~150 MB) instead of 12.8 GB.

Full round trip against a real iCloud file: dataless=True → download (1.46s) → readable → evict → dataless=True.

A false positive worth fixing

Those files stat as flags=compressed,dataless — two flags. A fully-downloaded but APFS-compressed file also reports st_blocks == 0, so the old heuristic would have rejected it as "not downloaded" too. Detection now reads the UF_DATALESS flag, which is the authoritative signal; the blocks heuristic survives only as a fallback for platforms with no st_flags.

Localized failures

The screenshot showed English error text in a Chinese UI. ScriptoError has carried an i18n key and params since it was written — errors.py says so explicitly (R7) — but _fail() flattened it to str(exc) and dropped both, so every catalog entry under errors.* was unreachable from the run page. The key and params now travel on StatusEvent to the row, which renders them through t(); errors with no key (a raw ffmpeg or engine message) still show through in English. A key whose params don't match falls back to the English text rather than displaying a half-substituted template.

errors.icloud_placeholder is deleted — nothing raises it any more.

Also

• New downloading job status, shown as "downloading from iCloud" / "从 iCloud 下载中" with a progress bar. Progress is best-effort: it is read from the block count, and a provider that only publishes blocks at the end will sit at 0 until the file lands.
• Settings toggle "Return iCloud files to the cloud after processing" (default on) if you would rather keep downloads.

Tests

199 passed / 5 skipped, 12 new: dataless detection, the compressed-file false positive, download-by-reading, skipping a local file, timeout, stop, failed download, evict not raising on ordinary files; pipeline download-then-evict, local files never evicted, the toggle being respected, eviction after a failed job; and error localization including both fallback paths.

Note

The rows in the report looked duplicated but are not — they are different files with identical names across Week N/<session>/ folders, and the row shows only the basename (full path is in the tooltip). Not addressed here; say the word if the row should carry its folder.