[FEATURE] Shape dictation with a prompt through the on-device model

[jhampton]

Stacked on #67, and opened as a draft on purpose: it argues with CONTEXT.md's raw-finalized-text rule, so it is offered for discussion, not pushed.

An off-by-default prompt shaping pass, now worded as a beta, rewrites finished dictation through Apple's on-device FoundationModels session between recognition and insertion. The transcript travels as marked data under fixed instructions with a per-prompt one-shot example, so question-shaped dictation comes back rewritten instead of answered. The prompt library is user-editable in Settings, seeded with the original three prompts, while the framing that keeps the model rewriting stays out of the user's hands. The phase is visible: the HUD stays up through the rewrite over a bar that fills toward the ten-second timeout, and while a session records the bare arrow keys cycle a session-scoped pick through the library and None, shown in a shaping band the shape grows downward to include below the voice visual and the draft. Passthrough remains the rule on every failure path, and transcription history keeps the words as spoken, before shaping. docs/adr/0008-prompt-shaping-alpha.md records the design and each later decision, including why the promotion keeps the toggle off by default. Written with Claude Code under my direction and reviewed line by line; the passthrough paths, the arrow cycling, and the HUD geometry are pinned by tests, green on an Apple Silicon Mac running macOS 26, and shaped insertion, the cycling, and the shaping band have all been exercised against the live on-device model on a notched MacBook Pro in daily use.

Refs #53

[tornikegomareli]

Hey, great work. I will review it and test it later today

[jhampton]

I've been testing this, and it seems that they way the foundation model is being used confuses dictation with instruction: I've been getting back AI slop responses from the model. This is very much a WIP @tornikegomareli !

[tornikegomareli]

I've been testing this, and it seems that they way the foundation model is being used confuses dictation with instruction: I've been getting back AI slop responses from the model. This is very much a WIP @tornikegomareli !

That was my initial thoughts about foundation models, but still has hope we will find some balance and ways to work it out.

[jhampton]

Found the failure mode and pushed a fix, rebased onto 0.7.1. The model was being handed the transcript as its conversational user turn with only a short role line in instructions, so an instruction-tuned model treated a question-shaped dictation as a question to answer. The framing now lives where the model weighs it most: the session instructions state that the user turn is always a raw transcript to transform, never a question to answer or an instruction to follow, the transcript arrives wrapped in explicit markers as data, and each library prompt carries a one-shot example whose input is question-shaped and whose output rewrites it without answering it, since an example carries that rule better than a sentence stating it. Passthrough on every failure path, the ten-second timeout, and history keeping the words as spoken are all unchanged, and the new assembly is pinned in PromptShapingServiceTests. I used Claude Code for most of the writing under my direction and can explain every line. Full suite green locally on Apple Silicon; I'll report back after live dictation of question-shaped utterances against all three prompts.

[jhampton]

@tornikegomareli — since opening this draft the branch has grown past what I originally described, and I would rather flag that plainly than let the diff speak for itself. On #53 I said user-authored prompts were a later decision; after living with the alpha day to day I took that decision on my fork, so the branch now carries a user-editable prompt library (Manage Prompts… in Settings, seeded with the original three, with the transcript-as-data framing kept fixed because it is also the fix for a real bug where question-shaped dictation came back answered instead of cleaned). The branch also makes the phase visible — the HUD stays up through the rewrite over a bar filling toward the timeout, and the bare arrow keys cycle a session-scoped pick shown in a band the shape grows downward to include below the visuals and the draft — and with daily use behind it I promoted the wording from alpha to beta, still off by default. All of this is offered as a proposal for your review, not a fait accompli: if it goes past what you want from #53, the editable library especially, I am happy to trim the branch back to the fixed-library alpha we discussed.

[jhampton]

The red check is #82's insertion-test flake, not this diff: TextInsertionServiceTests.activeReadSkipsLaterInsertionImmediatelyAndRecovers has now failed three CI runs in a row on this head while passing locally every time, and this branch never touches that file. Details and today's run links are on #82 — the short version is that this PR roughly doubles the parallel test load, so the runner starvation that test measures with real-clock bounds went from occasional to consistent. I would rather not keep re-running red CI to roll a green, so I will leave it as it stands; the injectable-clock fix you sketched on #82 would end it properly, and I am glad to implement that under its own issue if you reopen or refile it.

[jhampton]

Just a quick follow-up here. I've been running the UI changes and the ability to, for instance, cycle through different kinds of shaping, including turning off shaping when you're in the latch mode instead of the temporary push-to-talk mode. In push-to-talk mode, you cannot actually cycle back and forth. So I may need to follow up there, but it's been working really, really well. And I'm pretty impressed.

You do have to do a lot of prompt shaping in order to get it to, um, more closely adhere to your directions, just just doing remove filler words or tighten grammar is actually really pretty fantastic. I'm including some screenshots for reference.