Who is speaking: speaker labels in meeting recordings

A meeting transcript is hard to read when every line looks the same. We now mark who is speaking: the other people in the call become Speaker 1, Speaker 2 and so on. This is still marked experimental in the app.

Why we changed the method

Our first attempt cut the call audio into pieces and compared voice prints. On a real recording it failed. The microphone and the other participants arrive mixed in one channel, and the voices of different people landed in the same group. Names are not a way out either: a name picked up from the text often goes to the wrong person.

The new method is a model called Nemotron 3 Diarization, made by NVIDIA. “Diarization” means sorting a recording into who spoke when. It handles up to eight voices in one meeting. Its training data does not include Russian or Ukrainian, so we checked it ourselves.

How we checked it

We took one real meeting recording, about 72 minutes with four people. We cut 60 short pieces of about two and a half seconds from random places and listened to each one. In 51 of them we could say with confidence who was speaking, and we kept those 51.

For each piece we looked at which number the program gave it. The new model put all 51 with the right person: 51 of 51, and it used exactly four numbers for four people. The previous approach got 27 of 51, about 53%, and spread the voices over seven numbers, each a mix of people.

Read this with care. It is one meeting, a short list of pieces, checked by one person, in the spoken languages of that call. On whole transcript lines, which are longer than our pieces and sometimes hold two speakers, the new model agreed with our ear on 47 of 51. We did not test other recordings yet.

What stays on your Mac

The model is a download of about 100 MB from huggingface.co. It downloads only when you press the button in the Downloads tab. The meeting itself never touches the network: the labelling runs on your Mac after the recording.

The app does not remember voices. The numbers exist only inside one meeting, and nobody is identified by them. Speaker 2 in today’s call has no link to Speaker 2 in tomorrow’s. There is no voice profile to store, so there is nothing to leak.

More on what goes out and when is on the page what leaves your Mac.