How accurate is VocalCode?

On 200 utterances of LibriSpeech test-clean, VocalCode's default English model (NVIDIA Parakeet TDT v3) scores 2.33% word error rate with a median latency of 154 ms. On 200 utterances of the MAGICDATA Mandarin test set, its Chinese model (Paraformer) scores 5.54% character error rate with a median latency of 43 ms. Both run on CPU with no GPU.

How does that compare to a base Whisper model?

On the same audio, the base Whisper model scored 4.13% word error rate in English and 31.12% character error rate in Chinese. The Chinese gap is not a matter of degree: base Whisper frequently emits English words for Chinese speech, for example returning 'My My My My' for the Mandarin word for buying and selling.

Is this a comparison of products or of models?

Of models. It measures the speech recognition model each tool ships, on identical audio, and does not account for application-level post-processing that another product might add.

Benchmark

Almost every dictation tool claims high accuracy and none of them show a number. Here are ours, the method that produced them, and the places where we lose.

Results

	VocalCode default	Whisper base
English — word error rate	2.33%	4.13%
English — median latency	154 ms	426 ms
Chinese — character error rate	5.54%	31.12%
Chinese — median latency	43 ms	178 ms

200 utterances per language. English scored over 4,675 words, Chinese over 2,092 characters. All on CPU, no GPU. Measured 4 August 2026 on VocalCode 0.4.19.

Method

English: the first 200 utterances of LibriSpeech test-clean, a standard public corpus with published transcripts. FLAC converted to 16 kHz mono.
Chinese: 200 utterances sampled across speakers from the MAGICDATA Mandarin test set (OpenSLR 68), using its own TRANS.txt as ground truth.
Scoring: word error rate for English (lowercased, punctuation stripped) and character error rate for Chinese (punctuation and whitespace stripped) — the standard measures for each.
Latency is the model's own decode time per utterance, as reported by the app, not wall clock including model load.
Both models were run through the same application, on the same machine, over the same files, with the language explicitly declared to each — so Whisper was not handicapped by having to guess.

What the Chinese gap actually looks like

A 31% character error rate does not mean "one character in three is wrong". It means a large share of the output is not Chinese at all. Base Whisper hallucinates English on Mandarin audio:

reference   买卖
whisper     My My My My My My My My My My My My

reference   嗨
whisper     ( 回家 )

reference   喂喂喂喂在不在
whisper     Wait wait wait 差不多

Telling it the language is Chinese does not fix this — we checked, and the output was identical either way. It is a property of the model, not of language detection. This is why VocalCode ships a dedicated Chinese model rather than relying on a general multilingual one.

Where we lose

Parakeet returned nothing at all on 1 of the 200 English utterances — a very short one. Whisper base returned something every time. If you dictate a lot of two-word fragments, that is a real difference and it is not in our favour.
Proper nouns are hard for both. "Stephanos Dedalos" came back as "Stefanos Dedlos" from ours and "Steffanol's dead loss" from Whisper. Ours is closer, but the replacement dictionary exists precisely because neither is reliable on names.
This is a model comparison, not a product comparison. Another application shipping the same Whisper model may post-process its output in ways this does not measure.

Which Whisper, and why it matters

The Whisper model measured here is the ~150 MB class. That is the tier of model that free dictation tools typically ship — Superwhisper's own documentation rates its free models 5, 3 and 1 out of 10 and puts everything it rates 8 or above behind Pro, including Parakeet and its dedicated Chinese model.

So the honest summary is not "we beat Superwhisper". It is: the model VocalCode gives you by default is the class of model other tools charge a subscription for — and against the class they give away, these are the numbers.

We did not test the 500 MB model in their free tier; we do not have it. If someone runs that comparison we will link it here, including if it goes against us.

Reproducing this

Both corpora are public and free. VocalCode ships the harness that produced these numbers:

vocalcode-app transcribe file1.wav file2.wav …

It prints one tab-separated row per file — path, decode milliseconds, raw text, cleaned text — and the model is whichever one your configuration selects. Point it at LibriSpeech, score against the published transcripts, and you will get our English column. If you get something different, please tell us.

Try it — 30 days free Windows & macOS · runs locally · $4.99 once if you keep it