Skip to content

feat(media): transcode HEIC/HEIF to JPEG on desktop upload#1257

Open
tellaho wants to merge 1 commit into
mainfrom
tho/heic-transcode-desktop
Open

feat(media): transcode HEIC/HEIF to JPEG on desktop upload#1257
tellaho wants to merge 1 commit into
mainfrom
tho/heic-transcode-desktop

Conversation

@tellaho

@tellaho tellaho commented Jun 24, 2026

Copy link
Copy Markdown
Collaborator

Category: fix

User Impact: iPhone HEIC/HEIF photos now preview and upload correctly in the desktop chat composer instead of showing a blank preview and shipping an unviewable file.

Problem: HEIC/HEIF (Apple's default iPhone photo format) didn't render in the desktop chat composer — the preview went blank and the uploaded file was unviewable for everyone. Shend and tho both hit it; tho had to manually convert to JPEG. Slack handles HEIC fine, so this was a visible gap. Mobile already transcodes HEIC→JPEG before upload; desktop did not.

Solution: Transcode HEIC/HEIF to JPEG in Rust before upload, mirroring the existing video-transcode branch. Because the composer previews the server-returned blob URL of the uploaded bytes (not a local File), a single upload-side fix covers both upload and preview — no JS decode lib or canvas pass needed. Detection mirrors mobile's full ftyp brand set so the two platforms stay consistent.

File changes

desktop/src-tauri/src/commands/media_transcode.rs
Adds `is_heic_file(buf)` (ftyp box at offset 4 + mobile's full brand set `{heic,heix,hevc,hevx,heim,heis,mif1,msf1}`), `has_heic_extension(path)` for the picker path, and `transcode_heic_to_jpeg` (`ffmpeg -y -loglevel error -i -frames:v 1 -q:v 2 <out.jpg>`, 60s timeout, same anti-deadlock pattern as the video branch). 11 unit tests covering the brand table (including variants the `infer` crate misses), JPEG-not-HEIC, empty input, plus an ffmpeg-gated round-trip.

desktop/src-tauri/src/commands/media.rs
Wires HEIC detection + transcode into both upload paths (`process_picked_path` and `upload_media_bytes`) ahead of the existing video/passthrough branch. Resulting JPEG is re-detected by `detect_and_validate_mime`. Missing ffmpeg is a hard error, consistent with the video path — no silent fallback.

Reproduction Steps

  1. Run the desktop app.
  2. In the chat composer, paste or pick an iPhone HEIC photo (both clipboard and file-picker paths are covered).
  3. The preview should render correctly, and the uploaded image should be viewable by everyone (it ships as JPEG).

Notes

  • Detection uses magic bytes (ftyp brands) as the source of truth on the paste/drag byte path; the picker path also checks the file extension.
  • ffmpeg is required for HEIC, same as for video.
Screen Recording 2026-06-24 at 4 53 03 PM

iPhone HEIC/HEIF photos rendered blank in the desktop composer and were
unviewable for everyone because Chromium/the Tauri webview cannot decode
HEIC. Mobile already transcodes to JPEG before upload; desktop did not.

Add HEIC detection mirroring mobile (full ftyp brand set, broader than the
infer crate) and an ffmpeg HEIC->JPEG transcode, wired into both upload
paths: process_picked_path (extension OR magic-bytes) and upload_media_bytes
(magic-bytes only — no filename on the paste/drag path). Because the composer
preview renders the server-returned blob URL, this single Rust fix covers
both upload and preview. Missing ffmpeg errors like the video path (no silent
blank-preview fallback).

Co-authored-by: Taylor Ho <taylorkmho@gmail.com>
Signed-off-by: Taylor Ho <taylorkmho@gmail.com>
@tellaho tellaho enabled auto-merge (squash) June 25, 2026 01:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant