Add speaker diarization (speaker attribution) for multi-speaker conversations

[akirill0v]

Hi, and thanks a lot for FluidVoice – it’s an awesome project!

Hi! I'd like to request a speaker diarization feature for FluidVoice. Currently, all transcribed text is attributed to a single speaker, which makes it difficult to read or use transcripts from meetings, interviews, and podcasts. It would be great if the app could automatically detect different voices and label them (e.g., "Speaker 1", "Speaker 2") in the live preview and history. This would significantly improve the utility of FluidVoice for recording multi-person conversations without requiring manual attribution.

Thanks for the great tool!

[altic-dev]

I will add it to the ask list and work on it soon for you and others!! Thanks for trying FV :)

[benja0x40]

Hey, would be please to help and keen to start learning Swift to do so.

I did some successful tests on Mac with pyannote/speaker-diarization-3.1 and nvidia/parakeet-tdt-0.6b-v2, though exclusively in Python. Also found an alternative model for native Mac diarization (FluidInference/speaker-diarization-coreml) but didn't make any tests with it.

Below are other improvements I was considering to include in my Python pipeline.

Pre Diarization-Transcription

• Voice Activity Detection, based on pipecat-ai/smart-turn-v2 model
• Sound Quality Improvement (e.g. noise reduction), postponed

Post Diarization-Transcription

• Multilingual Sentence Detection, based on segment-any-text/sat-3l-sm or better
• Multilingual Punctuation, based on oliverguhr/fullstop-punctuation-multilang-large
• Multilingual Spelling Correction, based on oliverguhr/spelling-correction-multilingual-base

The rationale for post Diarization-Transcription stages was that accurate diarization is a pretty complex task, for which un-mixing and splitting distinct voice signals is only a starting point. Also the main missing component in my planned pipeline was proper handling of domaine specific dictionaries (jargon and names) per conversation.

[altic-dev]

Thanks for the interest! Yeah its not a straight forward task like you said. fluidinference is already part of FVoice stack actually! So whatever you get from it is essentially very easy to integrate. It should also have some of the other components you nentioned. Converting models to run on mac natively is tricky but doable. Play w it and update me how it goes :)

[postoso]

Took a crack at this — #378 adds speaker labeling to File Transcription using the offline diarizer from the FluidAudio fork that's already in the dependency tree (no new deps). Toggle + optional speaker-count hint, "Speaker N" labels with timestamps in the results, labeled text in history/exports, graceful fallback if diarization fails. Tested on a real 3-person meeting (auto-detected all 3) and a synthetic 2-speaker file.

Scoped to file transcription (not live dictation) to keep it reviewable. @altic-dev happy to adjust UI placement or anything else.

[asmit203]

I am interested in like if we want to add this feature to the live transcription how will that be handled like will that be more of another mode like podcast mode or something or something completely different like in the history we can just show but live it doesn't have any effect also can it remember over different pieces of convo so voices can be attributed name?

[altic-dev]

@postoso you think we can close this? I merged a PR of yours. We can create a new Issue / PR to track meeting notes mode.

Also, if you hve anything specific you'd like to work on - lmk and we can have a discussion for you to build something you like :)

[postoso]

Yeah, happy to close this one. Diarization shipped in #378, and meeting notes mode is different enough that it deserves its own issue. I can open one with what I had in mind if you want it tracked.

On the offer, yes, I would like to build something bigger. I want to think about what actually fits rather than throw out an idea now, so let me come back to you on that.

In the meantime the thing I keep running into is contributor-side friction, and most of it is small enough that I can just pick it up:

• The PR archive comment workflow fails on every fork PR, so outside contributors never get the build download link. The artifact builds fine, it is only the comment that never posts. Filed fix(ci): post the archive link on pull requests from forks #857.
• DictationE2ETests is about one test away from SwiftLint's type_body_length limit, so adding a test there fails CI for reasons unrelated to the change.

There is a bigger one behind those: the Debug build shares com.FluidApp.app with the release app, which is why hosted tests run against real user defaults and why you have to quit FluidVoice to run them. That one touches the project file though, so it is more invasive and entirely your call.

None of this is visible if you build from a branch in the repo, which is probably why it has stuck around.