We tried cleaning the audio first. It did not help

It sounds obvious: remove the noise, even out the volume, and the recognizer will hear better. We tested that on our own recordings. The result was the opposite of what we expected.

What we tried

We took recordings from four voices, 1,434 phrases in total, and ran each phrase through thirteen variants of processing before recognition. Then we compared the number of mistakes with the untouched recording.

• Volume normalization, which brings every recording to the same level. It made things worse: about two points more mistakes per hundred words. One English phrase after another came out written in Cyrillic letters.
• Noise suppression with RNNoise. On a voice that was already clean, it made things worse.
• Noise suppression with GTCRN. It only shuffled the mistakes around: some went away, others appeared, and Russian got slightly worse.
• A high-pass filter at 90 Hz, which cuts low rumble. No change at all.

Why it does not work

Cleaning the sound does not simply give the recognizer a better recording. It changes the sound, and the recognizer reacts to that change in ways we did not expect.

The volume result surprised us most. The recognizer decides which language it is hearing, and the loudness of the sound pushes that decision. Louder audio pulls it towards Russian, so an English phrase starts to appear in Cyrillic. Quieter audio lets it go back to the language of the phrase. A small push at the input flipped the alphabet in about 150 of 1,434 phrases.

We also looked for one ideal volume level and did not find it. Boosting the sound hurt, and each voice had its own best level, more than 15 dB apart from the others.

What did help a little

Only one thing gave a steady gain: making the sound three decibels quieter. That removed about half a point of mistakes per hundred words, and no voice got worse. Much quieter audio (6 dB and below) brought a new problem: Russian began to slide into Ukrainian spelling.

The gain is small, so we shipped it only as an experiment. In the dictation settings there is a switch, “Improve recognition (experimental)”, and it is off by default. It does not touch live subtitles.

So we do not put a noise filter in front of the recognizer. If we want better results on Russian and Ukrainian, we will look at how the model chooses the language, not at the sound.