Skip to main content

89 audio models, six tasks

Compare AI audio models

89 audio models from 30 providers, sorted into six tasks. Most popular for speech to text: Granite Speech 5.0 470M TurboCTC NC (IBM). For speech output: Kokoro-82M (hexgrad).

providers
30
with open weights
22
with a published price
38

Which audio model for which task?

The three most popular models per task. Each task name leads to its section below.
Speech to text32 modelsAudio becomes text: transcription, subtitles, dictation.
Speech output23 modelsText or speech becomes a voice: read-aloud, voice-over, voice conversion.
Real-time voice dialog12 modelsSpeech in, speech out: voice agents and phone bots.
Speech translation8 modelsSpoken language into another language, as text or voice.
Music and sound11 modelsSongs, loops and sound effects from a description.
Audio analysis3 modelsWho speaks when, and what happens in the audio.

What is gaining popularity right now, across models, vendors, harnesses and AI subscriptions, is collected in All AI trends.

GraduallyBench Audio

Two tests of our own: how many words does a speech recognizer get wrong, and how completely does a text come through in speech output?
Speech recognition
20models with a German and English value, 8 more in the table
Speech output
20models with a German and English value, 3 more in the table
Technical tests only
24music, dialogue, translation and more
Data as of
11/10/2026evaluated automatically

Speech recognition

Up to 36 read recordings in German and English; not every model covers both languages. The word error rate (WER) counts wrong, missing and extra words per 100 words of the reference, the character error rate (CER) does the same for characters.

Speech recognition word error rate

Misrecognized words in percent (German and English), lower is better

20 of 20 models

8 more models have no complete German and English value and are not in the chart.

As of 11/10/2026. Source: GraduallyBench Audio

Sorted by the German and English word error rate; models without this value come last.

ModelWord error rateGerman and English (%) German EnglishCharacter error rateText on silenceScope
NVIDIA Nemotron 3.5 TranscribeNVIDIA2.526 / 1,040––1.056 / 5,8424 of 4in the final transcript36 / 36OpenRouter
Scribe v2 MedicalElevenLabs2.526 / 1,040––0.845 / 5,8420 of 4in the final transcript36 / 36OpenRouter
Scribe v2ElevenLabs2.728 / 1,0403.620 / 5501.68 / 4900.848 / 5,8420 of 4in the final transcript36 / 36fal
Voxtral Mini Transcribe 3Mistral AI3.233 / 1,040––1.378 / 5,8420 of 4in the final transcript36 / 36OpenRouter
Voxtral Small Transcribe 3Mistral AI3.233 / 1,040––1.272 / 5,8420 of 4in the final transcript36 / 36OpenRouter
MAI-Transcribe-1.5Microsoft3.334 / 1,040––1.375 / 5,8420 of 4in the final transcript36 / 36OpenRouter
MAI-Transcribe-2Microsoft3.435 / 1,040––1.375 / 5,8420 of 4in the final transcript36 / 36OpenRouter
MAI-Transcribe-2-Streaming Partially testedMicrosoft3.636 / 1,0114.525 / 5502.411 / 461–0 of 4in the stream35 / 36Gateway
Gemini 3.5 TranscribeGoogle3.738 / 1,040––1.691 / 5,8420 of 4in the final transcript36 / 36OpenRouter
GPT-4o Mini TranscribeOpenAI4.143 / 1,040––1.586 / 5,8421 of 4in the final transcript36 / 36OpenRouter
GPT-TranscribeOpenAI4.345 / 1,0406.737 / 5501.68 / 4901.7100 / 5,8420 of 4in the final transcript36 / 36Provider directly
Scribe v2 RealtimeElevenLabs4.648 / 1,0407.139 / 5501.89 / 490–0 of 4in the stream36 / 36Gateway
Universal-3.6 Pro Realtime Partially testedAssemblyAI4.6145 / 3,1207.3121 / 1,6501.624 / 1,4701.6285 / 17,526not assessed96 / 108LiveKit
GPT-Realtime-WhisperOpenAI4.951 / 1,0407.642 / 5501.89 / 4901.8103 / 5,8420 of 4in the final transcript36 / 36Provider directly
Voxtral Mini Transcribe 2Mistral AI4.951 / 1,040––1.799 / 5,8422 of 4in the final transcript36 / 36OpenRouter
Granite Speech 5.0 470M TurboCTCIBM5.355 / 1,0408.949 / 5501.26 / 4902.6151 / 5,8424 of 4in the final transcript36 / 36Local
Gemini 3.5 Transcribe Live Partially testedGoogle5.453 / 9767.843 / 5502.310 / 426–0 of 4in the stream35 / 36Gateway
Nova-3Deepgram5.557 / 1,0408.547 / 5502.010 / 4901.7102 / 5,8420 of 4in the final transcript36 / 36Cloudflare
GPT-Live-TranscribeOpenAI5.557 / 1,0408.547 / 5502.010 / 4902.1125 / 5,8420 of 4in the final transcript36 / 36Provider directly
OrukeetOruk5.759 / 1,0408.949 / 5502.010 / 4901.691 / 5,8420 of 4in the final transcript36 / 36Local
Voxtral Mini Transcribe RealtimeMistral AI6.568 / 1,04010.457 / 5502.211 / 4903.0173 / 5,8420 of 4in the final transcript36 / 36Local
LFM3-Audio-2BLiquid AI6.871 / 1,04010.457 / 5502.914 / 4902.7159 / 5,8424 of 4in the final transcript36 / 36Local
Confucius4-R2T2 Partially testedNetEase Youdao6.968 / 98911.258 / 5182.110 / 4712.6145 / 5,555not assessed30 / 36Public demo
Muse Voice TranscribeMeta Superintelligence Labs9.195 / 1,040––5.4313 / 5,8420 of 4in the final transcript36 / 36OpenRouter
FluxDeepgram––4.924 / 490–0 of 2in the stream18 / 18Cloudflare
Audio8-ASR-Infinite Partially testedEdge0 AI–69.776 / 1092.914 / 49029.8196 / 657, German0 of 4in the final transcript20 / 36 · 4 / 4Local
Cartesia Ink-2 Partially testedCartesia––2.712 / 4471.024 / 2,350, Englishnot assessed15 / 20Public demo
Phonon-2 Partially testedFermion Research–31.1171 / 5502.412 / 4900.922 / 2,580, English0 of 4in the final transcript20 / 36 · 16 / 16Local

Speech output

Up to 32 texts in German and English. The generated speech is transcribed back to text with GPT Transcribe and compared with the source. This measures whether the text comes through completely, not how natural the voice sounds.

Speech output text comparison

Differing words after re-transcription in percent (German and English), lower is better

20 of 20 models

3 more models have no complete German and English value and are not in the chart. Smaller test scope: Fish Audio S2 Pro, Fish Audio S2.1 Pro, MAI-Voice-2, MAI-Voice-2-Flash, Sonic 3.6, Voxtral Mini TTS, GPT-4o Mini TTS.

As of 11/10/2026. Source: GraduallyBench Audio

Sorted by the German and English word error rate; models without this value come last.

ModelWord error rateGerman and English (%) German EnglishCharacter error rateScope
MiniMax Speech 2.8 (Turbo)MiniMax1.912 / 6420.62 / 3223.110 / 3201.242 / 3,37032 / 32OpenRouter
Eleven v3 ConversationalElevenLabs2.315 / 6421.65 / 3223.110 / 3201.861 / 3,37032 / 32OpenRouter
Fish Audio S2 ProFish Audio2.58 / 3211.93 / 1613.15 / 1601.627 / 1,68516 / 16OpenRouter
MiniMax Speech 2.8 (HD)MiniMax2.516 / 6421.96 / 3223.110 / 3201.549 / 3,37032 / 32OpenRouter
Eleven v3ElevenLabs2.617 / 6421.65 / 3223.812 / 3201.862 / 3,37032 / 32OpenRouter
Fish Audio S2.1 ProFish Audio2.89 / 3211.93 / 1613.86 / 1601.830 / 1,68516 / 16OpenRouter
MAI-Voice-2.1-FlashMicrosoft3.019 / 6422.27 / 3223.812 / 3202.172 / 3,37032 / 32OpenRouter
Gemini 3.1 Flash TTSGoogle3.321 / 6422.27 / 3224.414 / 3202.379 / 3,37032 / 32OpenRouter
MAI-Voice-2Microsoft3.411 / 3212.54 / 1614.47 / 1602.033 / 1,68516 / 16OpenRouter
MAI-Voice-2.1Microsoft3.422 / 6423.712 / 3223.110 / 3202.379 / 3,37032 / 32OpenRouter
Eleven v4 TurboElevenLabs3.623 / 6421.65 / 3225.618 / 3202.067 / 3,37032 / 32OpenRouter
Gemini 3.8 Flash-Lite TTSGoogle3.623 / 6422.89 / 3224.414 / 3202.171 / 3,37032 / 32OpenRouter
Gemini 3.8 Flash TTSGoogle3.623 / 6421.65 / 3225.618 / 3201.963 / 3,37032 / 32OpenRouter
MAI-Voice-2-FlashMicrosoft3.712 / 3214.37 / 1613.15 / 1603.153 / 1,68516 / 16OpenRouter
Eleven v4ElevenLabs4.026 / 6421.96 / 3226.320 / 3202.482 / 3,37032 / 32fal
Eleven Flash v2.5ElevenLabs4.227 / 6422.58 / 3225.919 / 3202.481 / 3,37032 / 32OpenRouter
Sonic 3.6Cartesia4.715 / 3215.08 / 1614.47 / 1603.458 / 1,68516 / 16Public demo
Voxtral Mini TTSMistral AI4.715 / 3216.210 / 1613.15 / 1602.847 / 1,68516 / 16OpenRouter
GPT-4o Mini TTSOpenAI5.016 / 3213.15 / 1616.911 / 1604.067 / 1,68516 / 16Provider directly
Voxtral Mini TTS 3Mistral AI5.133 / 6424.715 / 3225.618 / 3202.896 / 3,37032 / 32OpenRouter
Aura-2Deepgram––4.113 / 3203.149 / 1,588, English16 / 16Cloudflare
Kokoro-82M Technical testhexgrad––3.15 / 1602.520 / 794, English8 / 8Local
Maya 2 Native Technical testMaya Research––8.814 / 1604.536 / 794, English8 / 8Local

Technical tests only

These models have no value comparable with the tables above. The tests mainly checked whether access works and a valid output is produced.

ModelStatusValid outputs per runWhat is available
Eleven Music v2.5ElevenLabsTechnical test4 / 4Only a technical test is available. The selection of test inputs is limited.
Eleven Realtime v1 MaxElevenLabsPartially tested5 / 6 · 1 / 1The test coverage is incomplete. Only a technical test is available.
Eleven Realtime v1 MiniElevenLabsTechnical test6 / 6 · 1 / 1Only a technical test is available. The selection of test inputs is limited.
Eleven Text to Sound v2ElevenLabsTechnical test8 / 8Only a technical test is available. The selection of test inputs is limited.
Falcon-ASRTechnology Innovation InstituteTechnical test6 / 6 · 2 / 2Only a technical test is available. The selection of test inputs is limited.
Gemini 3.5 Live TranslateGoogleTechnical test2 / 2Only a technical test is available. The selection of test inputs is limited.
Gemini 3.8 LiveGooglePartially tested1 / 1 · 1 / 1The test coverage is incomplete. Only a short functional test is available.
Gemini 3.8 Live Extended ThinkingGoogleShort functional test1 / 1Only a short functional test is available. The functional test does not assess content quality.
GPT-Live 1OpenAITechnical test8 / 8Dialogue tasks after exclusions: 50.0 % (3 / 6 assessable dialogue cases). Only a technical test is available. The selection of test inputs is limited.
GPT-Realtime 2.1OpenAITechnical test8 / 8Dialogue tasks after exclusions: 83.3 % (5 / 6 assessable dialogue cases). Only a technical test is available. The selection of test inputs is limited.
GPT-Realtime 2.1 MiniOpenAIPartially tested7 / 8 · 1 / 1 · 1 / 1Dialogue tasks after exclusions: 100.0 % (5 / 5 assessable dialogue cases). The test coverage is incomplete. Only a technical test is available.
GPT-Realtime TranslateOpenAITechnical test8 / 8Only a technical test is available. The selection of test inputs is limited.
Index-Echo-S2ST-2BBilibili Index TeamPartially tested1 / 1 · 2 / 3The test coverage is incomplete. Only a short functional test is available.
Index-Echo-S2ST-9BBilibili Index TeamPartially tested1 / 1 · 2 / 3The test coverage is incomplete. Only a short functional test is available.
Index-Echo-S2TT-2BBilibili Index TeamTechnical test9 / 9Word distance from the translation reference: 72.5 % (37 / 51 reference words). Only a technical test is available. The selection of test inputs is limited.
Index-Echo-S2TT-9BBilibili Index TeamTechnical test1 / 1 · 6 / 6Only a short functional test is available. The functional test does not assess content quality.
Lyria 3 ClipGoogleTechnical test4 / 4Only a technical test is available. The selection of test inputs is limited.
Lyria 3.5GoogleTechnical test4 / 4Only a technical test is available. The selection of test inputs is limited.
MiniMax-Music3MiniMaxTechnical test4 / 4Only a technical test is available. The selection of test inputs is limited.
MOSS-Audio-4B-ThinkingOpenMOSS TeamPartially tested8 / 8The test coverage is incomplete. Only a short functional test is available.
Nemotron-3-DiarizationNVIDIATechnical test8 / 8Only a technical test is available. The selection of test inputs is limited.
pyannote Community-1pyannoteTechnical test8 / 8Diarization, technical pretest: 15.8 % (67.32 / 426.42 seconds reference speaker time). Only a technical test is available. The selection of test inputs is limited.
SFX v3ElevenLabsTechnical test8 / 8Only a technical test is available. The selection of test inputs is limited.
YuE2-3Bm-a-pTechnical test2 / 2Only a technical test is available. The selection of test inputs is limited.
All test details by model

74 models in alphabetical order, with every test run, limitation and measurement.

Audio8-ASR-Infinite

Partially tested
  • The test coverage is incomplete.
  • The model’s language coverage was not fully checked.
  • Only the indicated language subset was tested.
Show test details

Speech recognition

Local · English

20 valid outputs from 20 attempted and 36 planned test slots.

  • The test process stopped unexpectedly.
  • Some cases were answered after a retry.
Word error rate (WER)
2.86 %(14 / 490 reference words)English
Silence cases with words in the final transcript
0 %(0 / 4 assessable silence cases)EnglishAfter Unicode normalization, word segments containing letters or digits count. Punctuation-only output does not count.

Speech recognition

Local · German

4 valid outputs from 4 attempted and 4 planned test slots.

  • The English language token was forced for these German recordings. German is outside the documented Chinese and English coverage.
Word error rate (WER)
69.72 %(76 / 109 reference words)German
Character error rate (CER)
29.83 %(196 / 657 reference characters)German

Aura-2

Benchmark run
  • The text comparison uses a second speech recognizer as a proxy.
  • Naturalness and voice quality were not evaluated.
  • The voice actually used is not confirmed.
  • The route does not confirm the exact model version.
  • Only the indicated language subset was tested.
Show test details

Speech output with text comparison

Cloudflare · English

16 valid outputs from 16 attempted and 16 planned test slots.

Speech output text comparison, WER
4.06 %(13 / 320 reference words)English
Speech output text comparison, CER
3.09 %(49 / 1,588 reference characters)English

Cartesia Ink-2

Partially tested
  • The test coverage is incomplete.
  • The route does not confirm the exact model version.
  • Silence was not reliably evaluated.
  • The number of actual provider requests is not attested.
  • Only the indicated language subset was tested.
Show test details

Speech recognition

Public demo · English

15 valid outputs from 20 attempted and 20 planned test slots.

Word error rate (WER)
2.68 %(12 / 447 reference words)English
Character error rate (CER)
1.02 %(24 / 2,350 reference characters)English
Silence assessment
Not availableEnglish

Technical measurement

Public demo · English

1 valid output from 1 attempted and 1 planned test slots.

  • Only a short functional test is available.
  • The functional test does not assess content quality.
  • Only a technical test is available.
  • The selection of test inputs is limited.

No further scored metric is available for this test.

Confucius4-R2T2

Partially tested
  • The test coverage is incomplete.
  • Only a technical test is available.
  • The selection of test inputs is limited.
  • The route does not confirm the exact model version.
  • Silence was not reliably evaluated.
  • Completion in the user interface was not confirmed.
Show test details

Speech recognition

Public demo · German, English

30 valid outputs from 35 attempted and 36 planned test slots.

Word error rate (WER)
11.2 %(58 / 518 reference words)German
Character error rate (CER)
3.77 %(116 / 3,074 reference characters)German
Word error rate (WER)
2.12 %(10 / 471 reference words)English
Character error rate (CER)
1.17 %(29 / 2,481 reference characters)English
Word error rate (WER)
6.88 %(68 / 989 reference words)German, English
Character error rate (CER)
2.61 %(145 / 5,555 reference characters)German, English
Silence assessment
Not availableGerman, English

Technical measurement

Public demo · English

1 valid output from 1 attempted and 1 planned test slots.

  • Only a short functional test is available.
  • The functional test does not assess content quality.
  • Only the indicated language subset was tested.

No further scored metric is available for this test.

Eleven Flash v2.5

Benchmark run
  • The text comparison uses a second speech recognizer as a proxy.
  • Naturalness and voice quality were not evaluated.
  • The voice actually used is not confirmed.
  • The route does not confirm the exact model version.
Show test details

Speech output with text comparison

OpenRouter · German, English

32 valid outputs from 32 attempted and 32 planned test slots.

Speech output text comparison, WER
2.48 %(8 / 322 reference words)German
Speech output text comparison, CER
1.35 %(24 / 1,782 reference characters)German
Speech output text comparison, WER
5.94 %(19 / 320 reference words)English
Speech output text comparison, CER
3.59 %(57 / 1,588 reference characters)English
Speech output text comparison, WER
4.21 %(27 / 642 reference words)German, English
Speech output text comparison, CER
2.4 %(81 / 3,370 reference characters)German, English

Eleven Music v2.5

Technical test
  • Only a technical test is available.
  • The selection of test inputs is limited.
  • The route does not confirm the exact model version.
  • The functional test does not assess content quality.
Show test details

Technical measurement

fal · Language not documented

4 valid outputs from 4 attempted and 4 planned test slots.

No further scored metric is available for this test.

Eleven Realtime v1 Max

Partially tested
  • The test coverage is incomplete.
  • Only a technical test is available.
  • The selection of test inputs is limited.
  • The route does not confirm the exact model version.
Show test details

Technical measurement

Gateway · German, English

5 valid outputs from 6 attempted and 6 planned test slots.

No further scored metric is available for this test.

Technical measurement

Gateway · English

1 valid output from 1 attempted and 1 planned test slots.

  • Only a short functional test is available.
  • The functional test does not assess content quality.
  • Only the indicated language subset was tested.

No further scored metric is available for this test.

  • Only a technical test is available.
  • The selection of test inputs is limited.
  • The route does not confirm the exact model version.
Show test details

Technical measurement

Gateway · German, English

6 valid outputs from 6 attempted and 6 planned test slots.

No further scored metric is available for this test.

Technical measurement

Gateway · English

1 valid output from 1 attempted and 1 planned test slots.

  • Only a short functional test is available.
  • The functional test does not assess content quality.
  • Only the indicated language subset was tested.

No further scored metric is available for this test.

  • Only a technical test is available.
  • The selection of test inputs is limited.
  • The route does not confirm the exact model version.
  • The functional test does not assess content quality.
Show test details

Technical measurement

fal · Language not documented

8 valid outputs from 8 attempted and 8 planned test slots.

No further scored metric is available for this test.

Eleven v3

Benchmark run
  • The text comparison uses a second speech recognizer as a proxy.
  • Naturalness and voice quality were not evaluated.
  • The voice actually used is not confirmed.
  • The route does not confirm the exact model version.
Show test details

Speech output with text comparison

OpenRouter · German, English

32 valid outputs from 32 attempted and 32 planned test slots.

Speech output text comparison, WER
1.55 %(5 / 322 reference words)German
Speech output text comparison, CER
1.01 %(18 / 1,782 reference characters)German
Speech output text comparison, WER
3.75 %(12 / 320 reference words)English
Speech output text comparison, CER
2.77 %(44 / 1,588 reference characters)English
Speech output text comparison, WER
2.65 %(17 / 642 reference words)German, English
Speech output text comparison, CER
1.84 %(62 / 3,370 reference characters)German, English
  • The text comparison uses a second speech recognizer as a proxy.
  • Naturalness and voice quality were not evaluated.
  • The voice actually used is not confirmed.
  • The route does not confirm the exact model version.
Show test details

Speech output with text comparison

OpenRouter · German, English

32 valid outputs from 32 attempted and 32 planned test slots.

Speech output text comparison, WER
1.55 %(5 / 322 reference words)German
Speech output text comparison, CER
1.18 %(21 / 1,782 reference characters)German
Speech output text comparison, WER
3.13 %(10 / 320 reference words)English
Speech output text comparison, CER
2.52 %(40 / 1,588 reference characters)English
Speech output text comparison, WER
2.34 %(15 / 642 reference words)German, English
Speech output text comparison, CER
1.81 %(61 / 3,370 reference characters)German, English

Eleven v4

Benchmark run
  • The text comparison uses a second speech recognizer as a proxy.
  • Naturalness and voice quality were not evaluated.
  • The voice actually used is not confirmed.
  • The route does not confirm the exact model version.
Show test details

Speech output with text comparison

fal · German, English

32 valid outputs from 32 attempted and 32 planned test slots.

Speech output text comparison, WER
1.86 %(6 / 322 reference words)German
Speech output text comparison, CER
1.8 %(32 / 1,782 reference characters)German
Speech output text comparison, WER
6.25 %(20 / 320 reference words)English
Speech output text comparison, CER
3.15 %(50 / 1,588 reference characters)English
Speech output text comparison, WER
4.05 %(26 / 642 reference words)German, English
Speech output text comparison, CER
2.43 %(82 / 3,370 reference characters)German, English

Eleven v4 Turbo

Benchmark run
  • The text comparison uses a second speech recognizer as a proxy.
  • Naturalness and voice quality were not evaluated.
  • The voice actually used is not confirmed.
  • The route does not confirm the exact model version.
Show test details

Speech output with text comparison

OpenRouter · German, English

32 valid outputs from 32 attempted and 32 planned test slots.

Speech output text comparison, WER
1.55 %(5 / 322 reference words)German
Speech output text comparison, CER
1.18 %(21 / 1,782 reference characters)German
Speech output text comparison, WER
5.63 %(18 / 320 reference words)English
Speech output text comparison, CER
2.9 %(46 / 1,588 reference characters)English
Speech output text comparison, WER
3.58 %(23 / 642 reference words)German, English
Speech output text comparison, CER
1.99 %(67 / 3,370 reference characters)German, English

Falcon-ASR

Technical test
  • Only a technical test is available.
  • The selection of test inputs is limited.
  • The route does not confirm the exact model version.
  • The functional test does not assess content quality.
Show test details

Technical measurement

Public demo · English

6 valid outputs from 6 attempted and 6 planned test slots.

  • Only the indicated language subset was tested.

No further scored metric is available for this test.

Technical measurement

Public demo · Arabic, English

2 valid outputs from 2 attempted and 2 planned test slots.

  • Only a short functional test is available.

No further scored metric is available for this test.

Fish Audio S2 Pro

Benchmark run
  • The text comparison uses a second speech recognizer as a proxy.
  • Naturalness and voice quality were not evaluated.
  • The voice actually used is not confirmed.
  • The route does not confirm the exact model version.
Show test details

Speech output with text comparison

OpenRouter · German, English

16 valid outputs from 16 attempted and 16 planned test slots.

Speech output text comparison, WER
1.86 %(3 / 161 reference words)German
Speech output text comparison, CER
0.79 %(7 / 891 reference characters)German
Speech output text comparison, WER
3.13 %(5 / 160 reference words)English
Speech output text comparison, CER
2.52 %(20 / 794 reference characters)English
Speech output text comparison, WER
2.49 %(8 / 321 reference words)German, English
Speech output text comparison, CER
1.6 %(27 / 1,685 reference characters)German, English

Fish Audio S2.1 Pro

Benchmark run
  • The text comparison uses a second speech recognizer as a proxy.
  • Naturalness and voice quality were not evaluated.
  • The voice actually used is not confirmed.
  • The route does not confirm the exact model version.
Show test details

Speech output with text comparison

OpenRouter · German, English

16 valid outputs from 16 attempted and 16 planned test slots.

Speech output text comparison, WER
1.86 %(3 / 161 reference words)German
Speech output text comparison, CER
0.79 %(7 / 891 reference characters)German
Speech output text comparison, WER
3.75 %(6 / 160 reference words)English
Speech output text comparison, CER
2.9 %(23 / 794 reference characters)English
Speech output text comparison, WER
2.8 %(9 / 321 reference words)German, English
Speech output text comparison, CER
1.78 %(30 / 1,685 reference characters)German, English

Flux

Benchmark run
  • The route does not confirm the exact model version.
  • Only the indicated language subset was tested.
Show test details

Speech recognition

Cloudflare · English

18 valid outputs from 18 attempted and 18 planned test slots.

Word error rate (WER)
4.9 %(24 / 490 reference words)English
Silence cases with text during the stream
0 %(0 / 2 assessable silence cases)EnglishIntermediate text fragments and punctuation count even if they are absent from the final transcript.
  • The text comparison uses a second speech recognizer as a proxy.
  • Naturalness and voice quality were not evaluated.
  • The voice actually used is not confirmed.
  • The route does not confirm the exact model version.
Show test details

Speech output with text comparison

OpenRouter · German, English

32 valid outputs from 32 attempted and 32 planned test slots.

Speech output text comparison, WER
2.17 %(7 / 322 reference words)German
Speech output text comparison, CER
2.02 %(36 / 1,782 reference characters)German
Speech output text comparison, WER
4.38 %(14 / 320 reference words)English
Speech output text comparison, CER
2.71 %(43 / 1,588 reference characters)English
Speech output text comparison, WER
3.27 %(21 / 642 reference words)German, English
Speech output text comparison, CER
2.34 %(79 / 3,370 reference characters)German, English
  • Only a technical test is available.
  • The selection of test inputs is limited.
  • The route does not confirm the exact model version.
  • The functional test does not assess content quality.
Show test details

Technical measurement

Vertex AI · German, English

2 valid outputs from 2 attempted and 2 planned test slots.

No further scored metric is available for this test.

  • The route does not confirm the exact model version.
Show test details

Speech recognition

OpenRouter · German, English

36 valid outputs from 36 attempted and 36 planned test slots.

Word error rate (WER)
3.65 %(38 / 1,040 reference words)German, English
Character error rate (CER)
1.56 %(91 / 5,842 reference characters)German, English
Silence cases with words in the final transcript
0 %(0 / 4 assessable silence cases)German, EnglishAfter Unicode normalization, word segments containing letters or digits count. Punctuation-only output does not count.
  • The test coverage is incomplete.
  • The route does not confirm the exact model version.
  • The streaming test has limited coverage.
Show test details

Speech recognition

Gateway · German, English

35 valid outputs from 35 attempted and 36 planned test slots.

Word error rate (WER)
7.82 %(43 / 550 reference words)German
Word error rate (WER)
2.35 %(10 / 426 reference words)English
Word error rate (WER)
5.43 %(53 / 976 reference words)German, English
Silence cases with text during the stream
0 %(0 / 4 assessable silence cases)German, EnglishIntermediate text fragments and punctuation count even if they are absent from the final transcript.
  • The text comparison uses a second speech recognizer as a proxy.
  • Naturalness and voice quality were not evaluated.
  • The voice actually used is not confirmed.
  • The route does not confirm the exact model version.
Show test details

Speech output with text comparison

OpenRouter · German, English

32 valid outputs from 32 attempted and 32 planned test slots.

Speech output text comparison, WER
1.55 %(5 / 322 reference words)German
Speech output text comparison, CER
0.67 %(12 / 1,782 reference characters)German
Speech output text comparison, WER
5.63 %(18 / 320 reference words)English
Speech output text comparison, CER
3.21 %(51 / 1,588 reference characters)English
Speech output text comparison, WER
3.58 %(23 / 642 reference words)German, English
Speech output text comparison, CER
1.87 %(63 / 3,370 reference characters)German, English
  • The text comparison uses a second speech recognizer as a proxy.
  • Naturalness and voice quality were not evaluated.
  • The voice actually used is not confirmed.
  • The route does not confirm the exact model version.
Show test details

Speech output with text comparison

OpenRouter · German, English

32 valid outputs from 32 attempted and 32 planned test slots.

Speech output text comparison, WER
2.8 %(9 / 322 reference words)German
Speech output text comparison, CER
1.57 %(28 / 1,782 reference characters)German
Speech output text comparison, WER
4.38 %(14 / 320 reference words)English
Speech output text comparison, CER
2.71 %(43 / 1,588 reference characters)English
Speech output text comparison, WER
3.58 %(23 / 642 reference words)German, English
Speech output text comparison, CER
2.11 %(71 / 3,370 reference characters)German, English

Gemini 3.8 Live

Partially tested
  • The test coverage is incomplete.
  • Only a technical test is available.
  • The selection of test inputs is limited.
  • The route does not confirm the exact model version.
  • A regular connection close was not confirmed.
  • Only the indicated language subset was tested.
Show test details

Technical measurement

Gateway · English

1 valid output from 1 attempted and 1 planned test slots.

  • Only a short functional test is available.
  • The functional test does not assess content quality.
  • Only text input was tested.

No further scored metric is available for this test.

Technical measurement

Vertex AI · English

1 valid output from 1 attempted and 1 planned test slots.

No further scored metric is available for this test.

  • Only a short functional test is available.
  • The functional test does not assess content quality.
  • Only a technical test is available.
  • The selection of test inputs is limited.
  • The route does not confirm the exact model version.
  • Only text input was tested.
  • Only the indicated language subset was tested.
Show test details

Technical measurement

Gateway · German

1 valid output from 1 attempted and 1 planned test slots.

No further scored metric is available for this test.

  • The route does not confirm the exact model version.
Show test details

Speech recognition

OpenRouter · German, English

36 valid outputs from 36 attempted and 36 planned test slots.

Word error rate (WER)
4.13 %(43 / 1,040 reference words)German, English
Character error rate (CER)
1.47 %(86 / 5,842 reference characters)German, English
Silence cases with words in the final transcript
25 %(1 / 4 assessable silence cases)German, EnglishAfter Unicode normalization, word segments containing letters or digits count. Punctuation-only output does not count.

GPT-4o Mini TTS

Benchmark run
  • The text comparison uses a second speech recognizer as a proxy.
  • Naturalness and voice quality were not evaluated.
  • The voice actually used is not confirmed.
  • The route does not confirm the exact model version.
Show test details

Speech output with text comparison

Provider directly · German, English

16 valid outputs from 16 attempted and 16 planned test slots.

Speech output text comparison, WER
3.11 %(5 / 161 reference words)German
Speech output text comparison, CER
2.36 %(21 / 891 reference characters)German
Speech output text comparison, WER
6.88 %(11 / 160 reference words)English
Speech output text comparison, CER
5.79 %(46 / 794 reference characters)English
Speech output text comparison, WER
4.98 %(16 / 321 reference words)German, English
Speech output text comparison, CER
3.98 %(67 / 1,685 reference characters)German, English

GPT-Live 1

Technical test
  • Only a technical test is available.
  • The selection of test inputs is limited.
  • The route does not confirm the exact model version.
  • The speech test uses synthetic speech.
  • Naturalness and voice quality were not evaluated.
  • Ambiguous tasks were manually excluded after the run.
  • The comparison uses literal translation references and does not assess semantic translation quality.
  • The dialogue assessment uses the provider-supplied transcript of the speech output. The actual audio content was not independently checked.
Show test details

Technical measurement

Provider directly · German, English

8 valid outputs from 8 attempted and 8 planned test slots.

Dialogue tasks after exclusions
50 %(3 / 6 assessable dialogue cases)German, English

GPT-Live-Transcribe

Benchmark run
  • The route does not confirm the exact model version.
Show test details

Speech recognition

Provider directly · German, English

36 valid outputs from 36 attempted and 36 planned test slots.

Word error rate (WER)
8.55 %(47 / 550 reference words)German
Character error rate (CER)
3.31 %(108 / 3,262 reference characters)German
Word error rate (WER)
2.04 %(10 / 490 reference words)English
Character error rate (CER)
0.66 %(17 / 2,580 reference characters)English
Word error rate (WER)
5.48 %(57 / 1,040 reference words)German, English
Character error rate (CER)
2.14 %(125 / 5,842 reference characters)German, English
Silence cases with words in the final transcript
0 %(0 / 4 assessable silence cases)German, EnglishAfter Unicode normalization, word segments containing letters or digits count. Punctuation-only output does not count.

GPT-Realtime 2.1

Technical test
  • Only a technical test is available.
  • The selection of test inputs is limited.
  • The route does not confirm the exact model version.
  • The speech test uses synthetic speech.
  • Naturalness and voice quality were not evaluated.
  • Ambiguous tasks were manually excluded after the run.
  • The comparison uses literal translation references and does not assess semantic translation quality.
  • The dialogue assessment uses the provider-supplied transcript of the speech output. The actual audio content was not independently checked.
Show test details

Technical measurement

Provider directly · German, English

8 valid outputs from 8 attempted and 8 planned test slots.

Dialogue tasks after exclusions
83.33 %(5 / 6 assessable dialogue cases)German, English

GPT-Realtime 2.1 Mini

Partially tested
  • The test coverage is incomplete.
  • Only a technical test is available.
  • The selection of test inputs is limited.
  • The route does not confirm the exact model version.
  • A regular connection close was not confirmed.
  • The speech test uses synthetic speech.
  • Naturalness and voice quality were not evaluated.
Show test details

Technical measurement

Provider directly · German, English

7 valid outputs from 8 attempted and 8 planned test slots.

  • Ambiguous tasks were manually excluded after the run.
  • The comparison uses literal translation references and does not assess semantic translation quality.
  • The dialogue assessment uses the provider-supplied transcript of the speech output. The actual audio content was not independently checked.
Dialogue tasks after exclusions
100 %(5 / 5 assessable dialogue cases)German, English

Technical measurement

Provider directly · Language not documented

1 valid output from 1 attempted and 1 planned test slots.

  • Only a short functional test is available.
  • The functional test does not assess content quality.

No further scored metric is available for this test.

Technical measurement

Provider directly · German

1 valid output from 1 attempted and 1 planned test slots.

  • Only a short functional test is available.
  • The functional test does not assess content quality.
  • Some inputs were used more than once.
  • Only the indicated language subset was tested.

No further scored metric is available for this test.

  • Only a technical test is available.
  • The selection of test inputs is limited.
  • The route does not confirm the exact model version.
  • The speech test uses synthetic speech.
  • Naturalness and voice quality were not evaluated.
  • Ambiguous tasks were manually excluded after the run.
  • The comparison uses literal translation references and does not assess semantic translation quality.
  • The dialogue assessment uses the provider-supplied transcript of the speech output. The actual audio content was not independently checked.
Show test details

Technical measurement

Provider directly · German, English

8 valid outputs from 8 attempted and 8 planned test slots.

Dialogue tasks after exclusions
Not availableGerman, English
  • The route does not confirm the exact model version.
Show test details

Speech recognition

Provider directly · German, English

36 valid outputs from 36 attempted and 36 planned test slots.

Word error rate (WER)
7.64 %(42 / 550 reference words)German
Character error rate (CER)
2.64 %(86 / 3,262 reference characters)German
Word error rate (WER)
1.84 %(9 / 490 reference words)English
Character error rate (CER)
0.66 %(17 / 2,580 reference characters)English
Word error rate (WER)
4.9 %(51 / 1,040 reference words)German, English
Character error rate (CER)
1.76 %(103 / 5,842 reference characters)German, English
Silence cases with words in the final transcript
0 %(0 / 4 assessable silence cases)German, EnglishAfter Unicode normalization, word segments containing letters or digits count. Punctuation-only output does not count.

GPT-Transcribe

Benchmark run
  • The route does not confirm the exact model version.
Show test details

Speech recognition

Provider directly · German, English

36 valid outputs from 36 attempted and 36 planned test slots.

Measured duration p50 / p95 1,154 / 1,974 ms (36 cases). Hosted request including network time. Averaged median, p95 by nearest rank.

Word error rate (WER)
6.73 %(37 / 550 reference words)German
Character error rate (CER)
2.42 %(79 / 3,262 reference characters)German
Word error rate (WER)
1.63 %(8 / 490 reference words)English
Character error rate (CER)
0.81 %(21 / 2,580 reference characters)English
Word error rate (WER)
4.33 %(45 / 1,040 reference words)German, English
Character error rate (CER)
1.71 %(100 / 5,842 reference characters)German, English
Silence cases with words in the final transcript
0 %(0 / 4 assessable silence cases)German, EnglishAfter Unicode normalization, word segments containing letters or digits count. Punctuation-only output does not count.
Show test details

Speech recognition

Local · German, English

36 valid outputs from 36 attempted and 36 planned test slots.

Word error rate (WER)
8.91 %(49 / 550 reference words)German
Character error rate (CER)
4.26 %(139 / 3,262 reference characters)German
Word error rate (WER)
1.22 %(6 / 490 reference words)English
Character error rate (CER)
0.47 %(12 / 2,580 reference characters)English
Word error rate (WER)
5.29 %(55 / 1,040 reference words)German, English
Character error rate (CER)
2.58 %(151 / 5,842 reference characters)German, English
Silence cases with words in the final transcript
100 %(4 / 4 assessable silence cases)German, EnglishAfter Unicode normalization, word segments containing letters or digits count. Punctuation-only output does not count.

Index-Echo-S2ST-2B

Partially tested
  • The test coverage is incomplete.
  • The functional test does not assess content quality.
  • Only a technical test is available.
  • The selection of test inputs is limited.
  • The route does not confirm the exact model version.
Show test details

Technical measurement

Public demo · Language not documented

1 valid output from 1 attempted and 1 planned test slots.

  • Only a short functional test is available.

No further scored metric is available for this test.

Technical measurement

Public demo · Chinese, English, Spanish

2 valid outputs from 3 attempted and 3 planned test slots.

  • The target language of the output was not confirmed.
  • Chinese identifies the submitted input. English and Spanish are requested target languages. The input audio identity and the actual language of the output audio were not independently confirmed.

No further scored metric is available for this test.

Index-Echo-S2ST-9B

Partially tested
  • The test coverage is incomplete.
  • The functional test does not assess content quality.
  • Only a technical test is available.
  • The selection of test inputs is limited.
  • The route does not confirm the exact model version.
Show test details

Technical measurement

Public demo · Language not documented

1 valid output from 1 attempted and 1 planned test slots.

  • Only a short functional test is available.

No further scored metric is available for this test.

Technical measurement

Public demo · Chinese, English, Spanish

2 valid outputs from 3 attempted and 3 planned test slots.

  • The target language of the output was not confirmed.
  • Chinese identifies the submitted input. English and Spanish are requested target languages. The input audio identity and the actual language of the output audio were not independently confirmed.

No further scored metric is available for this test.

Index-Echo-S2TT-2B

Technical test
  • Only a technical test is available.
  • The selection of test inputs is limited.
  • The comparison uses literal translation references and does not assess semantic translation quality.
Show test details

Speech translation

Local · Chinese, English

9 valid outputs from 9 attempted and 9 planned test slots.

Word distance from the translation reference
72.55 %(37 / 51 reference words)Chinese to English

Index-Echo-S2TT-9B

Technical test
  • The functional test does not assess content quality.
  • Only a technical test is available.
  • The selection of test inputs is limited.
  • The route does not confirm the exact model version.
  • The target language of the output was not confirmed.
Show test details

Technical measurement

Public demo · Language not documented

1 valid output from 1 attempted and 1 planned test slots.

  • Only a short functional test is available.

No further scored metric is available for this test.

Technical measurement

Public demo · Chinese, English, Japanese, Spanish

6 valid outputs from 6 attempted and 6 planned test slots.

No further scored metric is available for this test.

Kokoro-82M

Technical test
  • Only a technical test is available.
  • The selection of test inputs is limited.
  • The text comparison uses a second speech recognizer as a proxy.
  • Naturalness and voice quality were not evaluated.
  • The voice actually used is not confirmed.
  • Only the indicated language subset was tested.
Show test details

Speech output with text comparison

Local · English

8 valid outputs from 8 attempted and 8 planned test slots.

Speech output text comparison, WER
3.13 %(5 / 160 reference words)English
Speech output text comparison, CER
2.52 %(20 / 794 reference characters)English

LFM3-Audio-2B

Benchmark run
Show test details

Speech recognition

Local · German, English

Execution MLX. Precision BF16.

36 valid outputs from 36 attempted and 36 planned test slots.

Word error rate (WER)
10.36 %(57 / 550 reference words)German
Character error rate (CER)
4.11 %(134 / 3,262 reference characters)German
Word error rate (WER)
2.86 %(14 / 490 reference words)English
Character error rate (CER)
0.97 %(25 / 2,580 reference characters)English
Word error rate (WER)
6.83 %(71 / 1,040 reference words)German, English
Character error rate (CER)
2.72 %(159 / 5,842 reference characters)German, English
Silence cases with words in the final transcript
100 %(4 / 4 assessable silence cases)German, EnglishAfter Unicode normalization, word segments containing letters or digits count. Punctuation-only output does not count.

Lyria 3 Clip

Technical test
  • Only a technical test is available.
  • The selection of test inputs is limited.
  • The route does not confirm the exact model version.
  • The functional test does not assess content quality.
Show test details

Technical measurement

OpenRouter · Language not documented

4 valid outputs from 4 attempted and 4 planned test slots.

No further scored metric is available for this test.

Lyria 3.5

Technical test
  • Only a technical test is available.
  • The selection of test inputs is limited.
  • The route does not confirm the exact model version.
  • The functional test does not assess content quality.
Show test details

Technical measurement

fal · Language not documented

4 valid outputs from 4 attempted and 4 planned test slots.

No further scored metric is available for this test.

MAI-Transcribe-1.5

Benchmark run
  • The route does not confirm the exact model version.
Show test details

Speech recognition

OpenRouter · German, English

36 valid outputs from 36 attempted and 36 planned test slots.

Word error rate (WER)
3.27 %(34 / 1,040 reference words)German, English
Character error rate (CER)
1.28 %(75 / 5,842 reference characters)German, English
Silence cases with words in the final transcript
0 %(0 / 4 assessable silence cases)German, EnglishAfter Unicode normalization, word segments containing letters or digits count. Punctuation-only output does not count.

MAI-Transcribe-2

Benchmark run
  • The route does not confirm the exact model version.
Show test details

Speech recognition

OpenRouter · German, English

36 valid outputs from 36 attempted and 36 planned test slots.

Word error rate (WER)
3.37 %(35 / 1,040 reference words)German, English
Character error rate (CER)
1.28 %(75 / 5,842 reference characters)German, English
Silence cases with words in the final transcript
0 %(0 / 4 assessable silence cases)German, EnglishAfter Unicode normalization, word segments containing letters or digits count. Punctuation-only output does not count.
  • The test coverage is incomplete.
  • The route does not confirm the exact model version.
  • A regular connection close was not confirmed.
  • The streaming test has limited coverage.
  • Some cases were answered after a retry.
Show test details

Speech recognition

Gateway · German, English

35 valid outputs from 36 attempted and 36 planned test slots.

Word error rate (WER)
4.55 %(25 / 550 reference words)German
Word error rate (WER)
2.39 %(11 / 461 reference words)English
Word error rate (WER)
3.56 %(36 / 1,011 reference words)German, English
Silence cases with text during the stream
0 %(0 / 4 assessable silence cases)German, EnglishIntermediate text fragments and punctuation count even if they are absent from the final transcript.

MAI-Voice-2

Benchmark run
  • The text comparison uses a second speech recognizer as a proxy.
  • Naturalness and voice quality were not evaluated.
  • The voice actually used is not confirmed.
  • The route does not confirm the exact model version.
Show test details

Speech output with text comparison

OpenRouter · German, English

16 valid outputs from 16 attempted and 16 planned test slots.

Speech output text comparison, WER
2.48 %(4 / 161 reference words)German
Speech output text comparison, CER
1.01 %(9 / 891 reference characters)German
Speech output text comparison, WER
4.38 %(7 / 160 reference words)English
Speech output text comparison, CER
3.02 %(24 / 794 reference characters)English
Speech output text comparison, WER
3.43 %(11 / 321 reference words)German, English
Speech output text comparison, CER
1.96 %(33 / 1,685 reference characters)German, English

MAI-Voice-2-Flash

Benchmark run
  • The text comparison uses a second speech recognizer as a proxy.
  • Naturalness and voice quality were not evaluated.
  • The voice actually used is not confirmed.
  • The route does not confirm the exact model version.
Show test details

Speech output with text comparison

OpenRouter · German, English

16 valid outputs from 16 attempted and 16 planned test slots.

Speech output text comparison, WER
4.35 %(7 / 161 reference words)German
Speech output text comparison, CER
3.7 %(33 / 891 reference characters)German
Speech output text comparison, WER
3.13 %(5 / 160 reference words)English
Speech output text comparison, CER
2.52 %(20 / 794 reference characters)English
Speech output text comparison, WER
3.74 %(12 / 321 reference words)German, English
Speech output text comparison, CER
3.15 %(53 / 1,685 reference characters)German, English

MAI-Voice-2.1

Benchmark run
  • The text comparison uses a second speech recognizer as a proxy.
  • Naturalness and voice quality were not evaluated.
  • The voice actually used is not confirmed.
  • The route does not confirm the exact model version.
Show test details

Speech output with text comparison

OpenRouter · German, English

32 valid outputs from 32 attempted and 32 planned test slots.

Speech output text comparison, WER
3.73 %(12 / 322 reference words)German
Speech output text comparison, CER
2.19 %(39 / 1,782 reference characters)German
Speech output text comparison, WER
3.13 %(10 / 320 reference words)English
Speech output text comparison, CER
2.52 %(40 / 1,588 reference characters)English
Speech output text comparison, WER
3.43 %(22 / 642 reference words)German, English
Speech output text comparison, CER
2.34 %(79 / 3,370 reference characters)German, English

MAI-Voice-2.1-Flash

Benchmark run
  • The text comparison uses a second speech recognizer as a proxy.
  • Naturalness and voice quality were not evaluated.
  • The voice actually used is not confirmed.
  • The route does not confirm the exact model version.
Show test details

Speech output with text comparison

OpenRouter · German, English

32 valid outputs from 32 attempted and 32 planned test slots.

Speech output text comparison, WER
2.17 %(7 / 322 reference words)German
Speech output text comparison, CER
1.57 %(28 / 1,782 reference characters)German
Speech output text comparison, WER
3.75 %(12 / 320 reference words)English
Speech output text comparison, CER
2.77 %(44 / 1,588 reference characters)English
Speech output text comparison, WER
2.96 %(19 / 642 reference words)German, English
Speech output text comparison, CER
2.14 %(72 / 3,370 reference characters)German, English

Maya 2 Native

Technical test
  • Only a technical test is available.
  • The selection of test inputs is limited.
  • The text comparison uses a second speech recognizer as a proxy.
  • Naturalness and voice quality were not evaluated.
  • The voice actually used is not confirmed.
  • Only the indicated language subset was tested.
Show test details

Speech output with text comparison

Local · English

8 valid outputs from 8 attempted and 8 planned test slots.

Speech output text comparison, WER
8.75 %(14 / 160 reference words)English
Speech output text comparison, CER
4.53 %(36 / 794 reference characters)English

MiniMax Speech 2.8

Benchmark run
  • The text comparison uses a second speech recognizer as a proxy.
  • Naturalness and voice quality were not evaluated.
  • The voice actually used is not confirmed.
  • The route does not confirm the exact model version.
Show test details

Speech output with text comparison, HD

OpenRouter · German, English

32 valid outputs from 32 attempted and 32 planned test slots.

Speech output text comparison, WER
1.86 %(6 / 322 reference words)German
Speech output text comparison, CER
0.51 %(9 / 1,782 reference characters)German
Speech output text comparison, WER
3.13 %(10 / 320 reference words)English
Speech output text comparison, CER
2.52 %(40 / 1,588 reference characters)English
Speech output text comparison, WER
2.49 %(16 / 642 reference words)German, English
Speech output text comparison, CER
1.45 %(49 / 3,370 reference characters)German, English

Speech output with text comparison, Turbo

OpenRouter · German, English

32 valid outputs from 32 attempted and 32 planned test slots.

Speech output text comparison, WER
0.62 %(2 / 322 reference words)German
Speech output text comparison, CER
0.11 %(2 / 1,782 reference characters)German
Speech output text comparison, WER
3.13 %(10 / 320 reference words)English
Speech output text comparison, CER
2.52 %(40 / 1,588 reference characters)English
Speech output text comparison, WER
1.87 %(12 / 642 reference words)German, English
Speech output text comparison, CER
1.25 %(42 / 3,370 reference characters)German, English

MiniMax-Music3

Technical test
  • Only a technical test is available.
  • The selection of test inputs is limited.
  • The route does not confirm the exact model version.
  • The functional test does not assess content quality.
Show test details

Technical measurement

fal · Language not documented

4 valid outputs from 4 attempted and 4 planned test slots.

No further scored metric is available for this test.

MOSS-Audio-4B-Thinking

Partially tested
  • The test coverage is incomplete.
  • Only a short functional test is available.
  • The functional test does not assess content quality.
  • Only a technical test is available.
  • The selection of test inputs is limited.
  • Only the presence of an output was checked.
  • Five of the eight outputs in this MOSS test reached the token limit.
Show test details

Technical measurement

Local · German, English

8 valid outputs from 8 attempted and 8 planned test slots.

No further scored metric is available for this test.

  • The route does not confirm the exact model version.
Show test details

Speech recognition

OpenRouter · German, English

36 valid outputs from 36 attempted and 36 planned test slots.

Word error rate (WER)
9.13 %(95 / 1,040 reference words)German, English
Character error rate (CER)
5.36 %(313 / 5,842 reference characters)German, English
Silence cases with words in the final transcript
0 %(0 / 4 assessable silence cases)German, EnglishAfter Unicode normalization, word segments containing letters or digits count. Punctuation-only output does not count.
  • Only a technical test is available.
  • The selection of test inputs is limited.
  • The functional test does not assess content quality.
Show test details

Technical measurement

Local · German, English

8 valid outputs from 8 attempted and 8 planned test slots.

No further scored metric is available for this test.

Nova-3

Benchmark run
  • The route does not confirm the exact model version.
Show test details

Speech recognition

Cloudflare · German, English

36 valid outputs from 36 attempted and 36 planned test slots.

Measured duration p50 / p95 1,180.5 / 2,849 ms (36 cases). Hosted request including network time. Averaged median, p95 by nearest rank.

Word error rate (WER)
8.55 %(47 / 550 reference words)German
Character error rate (CER)
2.61 %(85 / 3,262 reference characters)German
Word error rate (WER)
2.04 %(10 / 490 reference words)English
Character error rate (CER)
0.66 %(17 / 2,580 reference characters)English
Word error rate (WER)
5.48 %(57 / 1,040 reference words)German, English
Character error rate (CER)
1.75 %(102 / 5,842 reference characters)German, English
Silence cases with words in the final transcript
0 %(0 / 4 assessable silence cases)German, EnglishAfter Unicode normalization, word segments containing letters or digits count. Punctuation-only output does not count.
  • The route does not confirm the exact model version.
Show test details

Speech recognition

OpenRouter · German, English

36 valid outputs from 36 attempted and 36 planned test slots.

Word error rate (WER)
2.5 %(26 / 1,040 reference words)German, English
Character error rate (CER)
0.96 %(56 / 5,842 reference characters)German, English
Silence cases with words in the final transcript
100 %(4 / 4 assessable silence cases)German, EnglishAfter Unicode normalization, word segments containing letters or digits count. Punctuation-only output does not count.

Orukeet

Benchmark run
Show test details

Speech recognition

Local · German, English

36 valid outputs from 36 attempted and 36 planned test slots.

Word error rate (WER)
8.91 %(49 / 550 reference words)German
Character error rate (CER)
2.15 %(70 / 3,262 reference characters)German
Word error rate (WER)
2.04 %(10 / 490 reference words)English
Character error rate (CER)
0.81 %(21 / 2,580 reference characters)English
Word error rate (WER)
5.67 %(59 / 1,040 reference words)German, English
Character error rate (CER)
1.56 %(91 / 5,842 reference characters)German, English
Silence cases with words in the final transcript
0 %(0 / 4 assessable silence cases)German, EnglishAfter Unicode normalization, word segments containing letters or digits count. Punctuation-only output does not count.

Phonon-2

Partially tested
  • The test coverage is incomplete.
  • The model’s language coverage was not fully checked.
  • A regular process exit is not attested.
  • Only the indicated language subset was tested.
Show test details

Speech recognition

Local · English

20 valid outputs from 20 attempted and 36 planned test slots.

Word error rate (WER)
2.45 %(12 / 490 reference words)English
Character error rate (CER)
0.85 %(22 / 2,580 reference characters)English
Silence cases with words in the final transcript
0 %(0 / 4 assessable silence cases)EnglishAfter Unicode normalization, word segments containing letters or digits count. Punctuation-only output does not count.

Speech recognition

Local · German

16 valid outputs from 16 attempted and 16 planned test slots.

Word error rate (WER)
31.09 %(171 / 550 reference words)German
Character error rate (CER)
10.36 %(338 / 3,262 reference characters)German

pyannote Community-1

Technical test
  • Only a technical test is available.
  • The selection of test inputs is limited.
  • The technical diarization pretest uses synthetic whole-clip speaker boundaries and is not a comparable DER benchmark.
Show test details

Technical measurement

Local · German, English

Execution CPU. Precision not documented.

8 valid outputs from 8 attempted and 8 planned test slots.

Diarization, technical pretest
15.79 %(67.32 / 426.42 seconds reference speaker time)German, English

Scribe v2

Benchmark run
  • The route does not confirm the exact model version.
Show test details

Speech recognition

fal · German, English

36 valid outputs from 36 attempted and 36 planned test slots.

Measured duration p50 / p95 2,745 / 6,215 ms (36 cases). Hosted request including network time. Averaged median, p95 by nearest rank.

Word error rate (WER)
3.64 %(20 / 550 reference words)German
Character error rate (CER)
0.98 %(32 / 3,262 reference characters)German
Word error rate (WER)
1.63 %(8 / 490 reference words)English
Character error rate (CER)
0.62 %(16 / 2,580 reference characters)English
Word error rate (WER)
2.69 %(28 / 1,040 reference words)German, English
Character error rate (CER)
0.82 %(48 / 5,842 reference characters)German, English
Silence cases with words in the final transcript
0 %(0 / 4 assessable silence cases)German, EnglishAfter Unicode normalization, word segments containing letters or digits count. Punctuation-only output does not count.

Scribe v2 Medical

Benchmark run
  • The route does not confirm the exact model version.
Show test details

Speech recognition

OpenRouter · German, English

36 valid outputs from 36 attempted and 36 planned test slots.

Word error rate (WER)
2.5 %(26 / 1,040 reference words)German, English
Character error rate (CER)
0.77 %(45 / 5,842 reference characters)German, English
Silence cases with words in the final transcript
0 %(0 / 4 assessable silence cases)German, EnglishAfter Unicode normalization, word segments containing letters or digits count. Punctuation-only output does not count.

Scribe v2 Realtime

Benchmark run
  • The route does not confirm the exact model version.
  • A regular connection close was not confirmed.
  • The streaming test has limited coverage.
Show test details

Speech recognition

Gateway · German, English

36 valid outputs from 36 attempted and 36 planned test slots.

Word error rate (WER)
7.09 %(39 / 550 reference words)German
Word error rate (WER)
1.84 %(9 / 490 reference words)English
Word error rate (WER)
4.62 %(48 / 1,040 reference words)German, English
Silence cases with text during the stream
0 %(0 / 4 assessable silence cases)German, EnglishIntermediate text fragments and punctuation count even if they are absent from the final transcript.

SFX v3

Technical test
  • Only a technical test is available.
  • The selection of test inputs is limited.
  • The route does not confirm the exact model version.
  • The functional test does not assess content quality.
Show test details

Technical measurement

fal · Language not documented

8 valid outputs from 8 attempted and 8 planned test slots.

No further scored metric is available for this test.

Sonic 3.6

Benchmark run
  • The route does not confirm the exact model version.
Show test details

Speech output with text comparison

Public demo · German, English

16 valid outputs from 16 attempted and 16 planned test slots.

  • The text comparison uses a second speech recognizer as a proxy.
  • Naturalness and voice quality were not evaluated.
  • The voice actually used is not confirmed.
Speech output text comparison, WER
4.97 %(8 / 161 reference words)German
Speech output text comparison, CER
3.82 %(34 / 891 reference characters)German
Speech output text comparison, WER
4.38 %(7 / 160 reference words)English
Speech output text comparison, CER
3.02 %(24 / 794 reference characters)English
Speech output text comparison, WER
4.67 %(15 / 321 reference words)German, English
Speech output text comparison, CER
3.44 %(58 / 1,685 reference characters)German, English

Technical measurement

Public demo · German, English

2 valid outputs from 2 attempted and 2 planned test slots.

  • Only a short functional test is available.
  • The functional test does not assess content quality.
  • Only a technical test is available.
  • The selection of test inputs is limited.

No further scored metric is available for this test.

  • The test coverage is incomplete.
  • The route does not confirm the exact model version.
  • Silence was not reliably evaluated.
  • The streaming test has limited coverage.
  • Some inputs were used more than once.
  • The speech test uses read speech.
Show test details

Speech recognition

LiveKit · German, English

96 valid outputs from 108 attempted and 108 planned test slots.

Word error rate (WER)
7.33 %(121 / 1,650 reference words)German
Character error rate (CER)
2.27 %(222 / 9,786 reference characters)German
Word error rate (WER)
1.63 %(24 / 1,470 reference words)English
Character error rate (CER)
0.81 %(63 / 7,740 reference characters)English
Word error rate (WER)
4.65 %(145 / 3,120 reference words)German, English
Character error rate (CER)
1.63 %(285 / 17,526 reference characters)German, English
Silence assessment
Not availableGerman, English

Technical measurement

LiveKit · English

1 valid output from 1 attempted and 1 planned test slots.

  • Only a short functional test is available.
  • The functional test does not assess content quality.
  • Only a technical test is available.
  • The selection of test inputs is limited.
  • Only the indicated language subset was tested.

No further scored metric is available for this test.

  • The route does not confirm the exact model version.
Show test details

Speech recognition

OpenRouter · German, English

36 valid outputs from 36 attempted and 36 planned test slots.

Word error rate (WER)
4.9 %(51 / 1,040 reference words)German, English
Character error rate (CER)
1.69 %(99 / 5,842 reference characters)German, English
Silence cases with words in the final transcript
50 %(2 / 4 assessable silence cases)German, EnglishAfter Unicode normalization, word segments containing letters or digits count. Punctuation-only output does not count.
  • The route does not confirm the exact model version.
Show test details

Speech recognition

OpenRouter · German, English

36 valid outputs from 36 attempted and 36 planned test slots.

Word error rate (WER)
3.17 %(33 / 1,040 reference words)German, English
Character error rate (CER)
1.34 %(78 / 5,842 reference characters)German, English
Silence cases with words in the final transcript
0 %(0 / 4 assessable silence cases)German, EnglishAfter Unicode normalization, word segments containing letters or digits count. Punctuation-only output does not count.
Show test details

Speech recognition

Local · German, English

36 valid outputs from 36 attempted and 36 planned test slots.

Word error rate (WER)
10.36 %(57 / 550 reference words)German
Character error rate (CER)
4.48 %(146 / 3,262 reference characters)German
Word error rate (WER)
2.24 %(11 / 490 reference words)English
Character error rate (CER)
1.05 %(27 / 2,580 reference characters)English
Word error rate (WER)
6.54 %(68 / 1,040 reference words)German, English
Character error rate (CER)
2.96 %(173 / 5,842 reference characters)German, English
Silence cases with words in the final transcript
0 %(0 / 4 assessable silence cases)German, EnglishAfter Unicode normalization, word segments containing letters or digits count. Punctuation-only output does not count.

Voxtral Mini TTS

Benchmark run
  • The text comparison uses a second speech recognizer as a proxy.
  • Naturalness and voice quality were not evaluated.
  • The voice actually used is not confirmed.
  • The route does not confirm the exact model version.
Show test details

Speech output with text comparison

OpenRouter · German, English

16 valid outputs from 16 attempted and 16 planned test slots.

Speech output text comparison, WER
6.21 %(10 / 161 reference words)German
Speech output text comparison, CER
3.03 %(27 / 891 reference characters)German
Speech output text comparison, WER
3.13 %(5 / 160 reference words)English
Speech output text comparison, CER
2.52 %(20 / 794 reference characters)English
Speech output text comparison, WER
4.67 %(15 / 321 reference words)German, English
Speech output text comparison, CER
2.79 %(47 / 1,685 reference characters)German, English

Voxtral Mini TTS 3

Benchmark run
  • The text comparison uses a second speech recognizer as a proxy.
  • Naturalness and voice quality were not evaluated.
  • The voice actually used is not confirmed.
  • The route does not confirm the exact model version.
Show test details

Speech output with text comparison

OpenRouter · German, English

32 valid outputs from 32 attempted and 32 planned test slots.

Speech output text comparison, WER
4.66 %(15 / 322 reference words)German
Speech output text comparison, CER
2.81 %(50 / 1,782 reference characters)German
Speech output text comparison, WER
5.63 %(18 / 320 reference words)English
Speech output text comparison, CER
2.9 %(46 / 1,588 reference characters)English
Speech output text comparison, WER
5.14 %(33 / 642 reference words)German, English
Speech output text comparison, CER
2.85 %(96 / 3,370 reference characters)German, English
  • The route does not confirm the exact model version.
Show test details

Speech recognition

OpenRouter · German, English

36 valid outputs from 36 attempted and 36 planned test slots.

Word error rate (WER)
3.17 %(33 / 1,040 reference words)German, English
Character error rate (CER)
1.23 %(72 / 5,842 reference characters)German, English
Silence cases with words in the final transcript
0 %(0 / 4 assessable silence cases)German, EnglishAfter Unicode normalization, word segments containing letters or digits count. Punctuation-only output does not count.

YuE2-3B

Technical test
  • Only a technical test is available.
  • The selection of test inputs is limited.
  • The test used quantized model weights.
  • The tested weights have noncommercial usage terms.
  • The functional test does not assess content quality.
Show test details

Technical measurement

Local · Language not documented

Execution Apple Metal. Precision Q4_0 · Decoder precision FP16.

2 valid outputs from 2 attempted and 2 planned test slots.

No further scored metric is available for this test.

WER compares words and CER compares characters with the reference. The speech output was re-transcribed with GPT Transcribe. Its exact backend revision is unconfirmed. Read speech and digital silence do not cover spontaneous speech or real background noise. Error rates can exceed 100 %. Where used, the test references were not independently reviewed by humans. Test coverage and methods differ by model. Each metric stands on its own; there is no combined overall score across tests. 12 additional models across Decision and Audio have no usable output and are omitted. Data compiled on 11/10/2026. How we test · Download all Decision and Audio results as JSON

First published cohorts

GraduallyBench STT v1

Speech recognition compared automatically.

The pilot uses the same 32 read-speech recordings in German and English, plus four digital-silence clips. The comparison uses source transcripts without human verification. Each language contains four speakers. Spontaneous speech and background noise are outside this cohort.

Word error rate (WER) counts replaced, omitted and inserted words relative to the reference words. Character error rate (CER) compares characters. Lower values mean fewer differences from the reference. WER can exceed 100 %. Counts below each rate show errors / reference units. Silence is evaluated separately.

Protocol audio-stt-read-speech-automated-v1. Actual provider billing is unknown.

GPT Transcribe

OpenAI · gpt-transcribe · https://api.openai.com/v1/audio/transcriptions · Tested Nov 5, 2026

Successful requests 36 / 36. The exact backend version is not confirmed by the requested route.

Read-speech transcription
LanguageSuccessful responsesWERCERWER, speaker mean
German16 / 166.73 %37 / 5502.42 %79 / 32626.59 %n = 4 / 4
English16 / 161.63 %8 / 4900.81 %21 / 25801.77 %n = 4 / 4
Digital silence
Language settingClips with invented speechInvented words
DE0 / 20
EN0 / 20

Request duration, median / p95 1,154 / 1,974 ms (n = 36). Includes network time and client processing.

Deepgram Nova 3

Cloudflare Workers AI · @cf/deepgram/nova-3 · Tested Nov 5, 2026

Successful requests 36 / 36. The exact backend version is not confirmed by the requested route.

Read-speech transcription
LanguageSuccessful responsesWERCERWER, speaker mean
German16 / 168.55 %47 / 5502.61 %85 / 32628.16 %n = 4 / 4
English16 / 162.04 %10 / 4900.66 %17 / 25802.02 %n = 4 / 4
Digital silence
Language settingClips with invented speechInvented words
DE0 / 20
EN0 / 20

Request duration, median / p95 1,180.5 / 2,849 ms (n = 36). Includes network time and client processing.

Scribe v2

fal.ai · fal-ai/elevenlabs/speech-to-text/scribe-v2 · Tested Nov 5, 2026

Successful requests 36 / 36. The exact backend version is not confirmed by the requested route.

Read-speech transcription
LanguageSuccessful responsesWERCERWER, speaker mean
German16 / 163.64 %20 / 5500.98 %32 / 32623.27 %n = 4 / 4
English16 / 161.63 %8 / 4900.62 %16 / 25801.68 %n = 4 / 4
Digital silence
Language settingClips with invented speechInvented words
DE0 / 20
EN0 / 20

Request duration, median / p95 2,745 / 6,215 ms (n = 36). Includes network time and client processing.

GraduallyBench TTS v1

Automatic text comparison after speech generation.

Each first audio output is fully decoded and transcribed once by gpt-transcribe. Word error rate (WER) and character error rate (CER) compare this transcript with the frozen input script. Lower values mean fewer text differences. Errors can also come from the transcription model.

This model-dependent text proxy does not measure naturalness, voice preference or human intelligibility. Numbers and names depend on the fixed normalization. The requested voice IDs are recorded; the provider does not independently confirm the delivered voice identity.

Protocol audio-tts-asr-content-proxy-v1. Actual provider billing is unknown.

Analysis date Nov 5, 2026

Eleven v4

fal.ai · elevenlabs/tts/eleven-v4

Generated
32
Fully decoded
32 / 32
Transcribed once
32 / 32

One already generated original was secured through a separate download.

ASR text comparison with the input script
LanguageOutputsWERCER
German16 / 161.86 %6 / 3221.8 %32 / 1782
English16 / 166.25 %20 / 3203.15 %50 / 1588

Requested voices jqcCZkN6Knx8BJ5TBdYR · VCgLBmBjldJmfphyB8sZ

Deepgram Aura 2

Cloudflare Workers AI · @cf/deepgram/aura-2-en

Generated
16
Fully decoded
16 / 16
Transcribed once
16 / 16
ASR text comparison with the input script
LanguageOutputsWERCER
GermanNot tested
English16 / 164.06 %13 / 3203.09 %49 / 1588

Requested voices asteria · orion

The most popular audio models

Popularity index from 0 to 100, higher is more popular. 19 of 89 models have a popularity value and appear here.

19 of 19 models

Speech to text compared

Audio becomes text: transcription, subtitles, dictation. 32 models.

The most popular models

Popularity index from 0 to 100, higher is more popular. 7 of 32 models have a popularity value and appear here.

7 of 7 models

Speech output compared

Text or speech becomes a voice: read-aloud, voice-over, voice conversion. 23 models.

The most popular models

Popularity index from 0 to 100, higher is more popular. 4 of 23 models have a popularity value and appear here.

4 of 4 models

Real-time voice dialog compared

Speech in, speech out: voice agents and phone bots. 12 models.

Speech translation compared

Spoken language into another language, as text or voice. 8 models.

The most popular models

Popularity index from 0 to 100, higher is more popular. 4 of 8 models have a popularity value and appear here.

4 of 4 models

Music and sound compared

Songs, loops and sound effects from a description. 11 models.

Audio analysis compared

Who speaks when, and what happens in the audio. 3 models.

The most popular models

Popularity index from 0 to 100, higher is more popular.

3 of 3 models

All audio models

Search, filter by task and sort. Prices as each profile states them; units differ between providers.
AI model directory
89 of 89 models
TaskWeightsPrice
Kokoro-82MhexgradSpeech outputOpenn/a68
Nemotron-3-DiarizationNVIDIAAudio analysisOpenn/a58
MiniMax-Music3MiniMaxMusic and soundOpenn/a55
Fish Audio S2 ProFish AudioSpeech outputOpenn/a52
pyannote Community-1pyannoteAudio analysisOpenn/a48
Granite Speech 5.0 470M TurboCTC NCIBMSpeech to textOpenn/a46
NVIDIA Nemotron 3.5 TranscribeNVIDIASpeech to textOpenn/a44
Confucius4-R2T2NetEase YoudaoSpeech to textOpenn/a43
OrukeetOrukSpeech to textOpenn/a41
Maya 2 NativeMaya ResearchSpeech outputOpenn/a34

Frequently asked questions about audio models

No single model covers every task, so the hub ranks each task separately. By popularity the leaders are currently: Speech to text: Granite Speech 5.0 470M TurboCTC NC; Speech output: Kokoro-82M; Real-time voice dialog: Amazon Nova 2 Sonic; Speech translation: Index-Echo-S2ST-9B; Music and sound: MiniMax-Music3; Audio analysis: Nemotron-3-Diarization. Popularity measures attention and use, not audio quality. A shared quality benchmark across all models of a task is not available yet, so test two or three candidates with your own recordings.

4 speech output models publish open weights, among them Kokoro-82M, Fish Audio S2 Pro and Maya 2 Native. You can run open models yourself without API fees, but you need suitable hardware, and the license decides commercial use. Hosted voices usually charge per character, token or minute.

12 models in the hub take speech in and answer with speech, which voice agents and phone bots need. The most popular are Amazon Nova 2 Sonic, Eleven Realtime v1 Max and Eleven Realtime v1 Mini. Response time and turn-taking decide how natural a conversation feels; compare them in your own setup before you build on one model.

22 of 89 audio models list local deployment on their profile, for example Kokoro-82M, Nemotron-3-Diarization and MiniMax-Music3. Use the "Runs locally" filter in the table for the full list. Check model size and license on each profile.

38 of 89 models have a published price on their profile. Units differ: per minute or hour of audio, per million characters, per million tokens or in credits, so the table shows each price as the provider states it. Examples: Voxtral Mini TTS: $16 per 1M characters; Amazon Nova 2 Sonic: Separate token rates for audio and text input and audio and text output; text rates also apply to transcription, tool calls, grounding, and history. Standard, Priority, and Flex tiers must be checked separately..

The hub sorts every model into one of six tasks: Speech to text (32), Speech output (23), Real-time voice dialog (12), Speech translation (8), Music and sound (11), Audio analysis (3). Each model has exactly one task, chosen from its documented input and output.

Changelog

The latest updates and improvements to our AI audio model comparison
November 4, 2026v1.0

Initial release

  • Published 89 audio models, each assigned to one of six tasks
  • Added popularity charts per task, a searchable table with filters and an FAQ