Best Open Source Speech-to-Text Models to Self-Host in 2026

Speech-to-text now sits underneath a lot of everyday software: meeting notes, voice agents, subtitles, call analytics, the dictation button in your editor. The easy route is a cloud API billed per minute, which also means every recording leaves your network.

The open models have closed most of that gap. On the Hugging Face Open ASR Leaderboard’s October 9, 2026 update, Qwen3-ASR-1.7B averages 5.40% word error rate on the English track, ahead of OpenAI’s API-only GPT-Transcribe (5.96%) and 0.70 points behind the best proprietary entry, Zoom’s Scribe v2 Pro (4.70%). It’s Apache 2.0. Whisper large-v3, still the default in most self-hosted stacks, sits 36th at 7.16%.

Below are the open-weight models worth running yourself, their caveats, and how to serve them. Numbers were checked against the leaderboard and model cards on October 15, 2026; the whisper.cpp and Python examples were run on an M3 Pro MacBook.

How these models are ranked

Rankings come from the Open ASR Leaderboard. It reports average word error rate (WER, the share of words a model gets wrong) and RTFx, seconds of audio processed per second of compute on an NVIDIA H200. The average now includes private held-out sets from Appen and DataoceanAI plus URGENT24, added on October 9, so older model-card numbers run lower: Cohere’s card says 5.42, the board now says 5.77. Don’t mix the two.

Artificial Analysis runs a second board, weighted 50% toward agent-style conversations. Voxtral Small is the best open model there at 2.8%, and the same Whisper large-v3 weights score 4.1% on fal.ai and 10.1% on Replicate. How a model is served can matter as much as which model it is.

Then there’s benchmark fitting. In an August 2026 study, Hume AI researchers played a parliament clip where the speaker says “Thank you” but the VoxPopuli reference transcript omits it. Six of 11 models omitted it too, including Cohere Transcribe, Canary-Qwen-2.5B, Granite Speech 4.1 and Parakeet TDT v2. Qwen3-ASR, Voxtral Mini, Whisper large-v3 and Moonshine wrote what was said. A low public-benchmark score partly measures how well a model learned that benchmark.

Grey dots are the other open models in the table below. Data: Open ASR Leaderboard, October 9, 2026.

Qwen3-ASR-1.7B: the most accurate open model

Qwen3-ASR came out of Alibaba’s Qwen team on January 29, 2026, in 1.7B and 0.6B sizes built on Qwen3-Omni. One model does language identification and transcription for 30 languages and 22 Chinese dialects, plus singing. The 1.7B leads the English board at 5.40% (RTFx 819) and is the second-best open model on the multilingual track at 6.56%, behind Voxtral Small. The 0.6B scores 6.24%.

Both run on vLLM, which gives you the OpenAI-style /v1/audio/transcriptions route; streaming works only through the vLLM backend. Timestamps need a separate model, Qwen3-ForcedAligner-0.6B, which covers 11 languages. Qwen publishes no VRAM figure. Weights and code are Apache 2.0.

Cohere Transcribe: built for long English recordings

Cohere Transcribe (March 2026) is a 2B Conformer encoder with a small Transformer decoder, trained from scratch on 14 languages. It’s second among open models on the short-form board (5.77%) and first on the long-form track: 12.23% across Earnings21, Earnings22 and CORAAL, beaten there only by proprietary APIs.

The card lists what’s missing: no timestamps, no diarization, no automatic language detection, and it is “eager to transcribe, even non-speech sounds,” so put a voice activity detector in front of it. The license is Apache 2.0, but the download is gated behind a contact-details form.

Granite Speech 4.1 and 5.0: IBM’s accuracy and speed pair

Granite Speech 4.1 2B (April 29, 2026) is a speech-language model for English, French, German, Spanish, Portuguese and Japanese, with speech translation built in. It scores 5.96% at RTFx 545, and a -plus variant adds speaker labels and word timestamps.

Granite Speech 5.0 470M TurboCTC (August 2026) goes the other way: an English-only CTC model with non-autoregressive decoding, aimed at edge devices. Its RTFx of 21,174 is the highest on the whole board, at 6.38% WER. Both are Apache 2.0; the 470M’s -nc sibling is CC-BY-NC-SA.

Parakeet TDT 0.6B v2 and v3: throughput on NVIDIA hardware

Parakeet TDT pairs a FastConformer encoder with a Token-and-Duration Transducer decoder, which predicts how many audio frames each token covers and jumps ahead instead of stepping through every frame. That’s where the speed comes from. v2 (English, May 2025) scores 6.02% at RTFx 5,988, and v3 (August 2025) covers 25 European languages with automatic language detection at 6.18%. Both take up to 24 minutes per pass, with word and segment timestamps.

They’re CC-BY-4.0, so commercial use is fine with attribution. The catch is serving: vLLM doesn’t support them. You run them through NVIDIA NeMo, sherpa-onnx, parakeet-mlx on Apple Silicon, or the parakeet-cli binary that now ships with whisper.cpp.

Voxtral and Omnilingual ASR: the multilingual picks

Mistral’s Voxtral Small (July 2025) is the best open model on the multilingual track (6.39% across German, French, Italian, Spanish, Portuguese, Hindi and Dutch) and on Artificial Analysis’s board. It’s a 24B audio LLM built on Mistral Small 3, so it needs about 55GB of GPU memory in bf16 and runs at RTFx 101. Voxtral Mini 3B needs about 9.5GB and scores 8.06% multilingual. Both are Apache 2.0. Mistral’s newer Voxtral Mini Transcribe V2 is API-only.

For languages nobody else covers, Meta’s Omnilingual ASR (November 2025, Apache 2.0) handles more than 1,600, from 300M to 7B parameters. The 7B LLM variant scores 7.00% on the multilingual track; standard variants cap clips at 40 seconds.

Whisper large-v3 and turbo: still the default for a reason

OpenAI hasn’t released an open-weight speech model since large-v3-turbo in October 2024; GPT-Transcribe, launched in July 2026, is API-only. Turbo still pulled 6.3 million downloads last month, more than any other model in this guide.

The reason is reach. Whisper covers 99 languages, it’s MIT licensed, and every runtime supports it: whisper.cpp on CPUs and Apple GPUs, faster-whisper on CUDA, WhisperX for diarization. Turbo cuts the decoder from 32 layers to 4 and needs about 6GB of VRAM in PyTorch; its q5_0 GGML build is a 574MB file. It wasn’t trained for translation, so use large-v3 for that.

VibeVoice-ASR: hour-long meetings with speakers attached

Microsoft’s VibeVoice-ASR (January 2026, MIT) takes up to 60 minutes of audio in a single pass and returns who spoke, when, and what, as one structured transcript. Most pipelines assemble that from three models: ASR, a diarizer like pyannote, and a forced aligner. It accepts hotwords for names and jargon, and covers 50+ languages.

The tradeoff is accuracy and size: 8.7B parameters and 8.16% on the English board, almost three points behind Qwen3-ASR. Use it when speaker labels are the product, as in meeting notes.

Real-time streaming: Nemotron 3.5 ASR and Voxtral Realtime

Batch models wait for the whole clip; voice agents can’t. NVIDIA’s Nemotron 3.5 ASR Streaming (June 2026) is a 600M cache-aware FastConformer-RNNT: it caches encoder state between chunks, so each new 80ms to 1.12s chunk doesn’t recompute the past. It covers 40 language-locales, and NVIDIA reports 240 concurrent streams on one H100 at 80ms chunks, against 14 for Parakeet RNNT 1.1B. It’s licensed OpenMDW-1.1.

Voxtral Mini 4B Realtime (February 2026, Apache 2.0, 13 languages) streams over vLLM’s /v1/realtime WebSocket with a delay you set between 240ms and 2.4s. At the recommended 480ms it scores 8.72% on FLEURS, against 5.90% for Mistral’s offline model. That gap, like Nemotron’s 9.75% on the English board, is the price of not waiting for the sentence to end.

On-device: Moonshine and Whistle

For phones, wearables and Raspberry Pis, Moonshine’s English streaming models (MIT, 34M to 245M parameters) run on a CPU and transcribe as you speak. Some older non-English Moonshine checkpoints use a non-commercial license, so check the one you download.

The newest entry is Cactus Compute’s Whistle, released October 2, 2026: one 16.9MB file covering seven European languages, Apache 2.0, no dependencies. On an M4 Pro CPU, Cactus measured 11.1ms to first token on ten seconds of audio, against 73.2ms for Whisper base and 22.8ms for Moonshine tiny v2. Its WER is published only as charts, where it trades wins with Whisper base. There’s no published parameter count, and it’s two weeks old, so test it on your own audio.

Serving: the API is standard, the runtime isn’t

Most servers now speak OpenAI’s POST /v1/audio/transcriptions, so client code doesn’t care which model sits behind it. The runtimes are still split by model family:

One thing bit me while testing: a fresh pip install faster-whisper now pulls PyAV 19, which removed an argument faster-whisper 1.2.1 still passes, so every transcription fails with TypeError: open() got an unexpected keyword argument 'metadata_errors'. The fix is merged upstream but unreleased. Until then, install with pip install faster-whisper "av<19".

Self-host Whisper and reach it from anywhere

Say you want a private transcription endpoint at home that a phone app, a script or an agent can call. whisper.cpp with large-v3-turbo is the shortest path:

brew install whisper.cpp
curl -L -o ggml-large-v3-turbo-q5_0.bin \
  https://huggingface.co/ggerganov/whisper.cpp/resolve/main/ggml-large-v3-turbo-q5_0.bin
whisper-server -m ggml-large-v3-turbo-q5_0.bin --port 8080 \
  --inference-path /v1/audio/transcriptions

On Linux, build from the repo with CMake. In my tests the server decoded WAV and FLAC but rejected M4A with Invalid request; --convert adds ffmpeg conversion. Test it with the 11-second samples/jfk.wav clip from the repo:

curl http://localhost:8080/v1/audio/transcriptions \
  -F file=@jfk.wav -F response_format=json

{"text":" And so, my fellow Americans, ask not what your country can do for you, ask what you can do for your country.\n"}

On an M3 Pro, warm requests took about 0.85 seconds for the 11-second clip on the Metal GPU.

The server listens on localhost only. To reach it from outside without opening a router port, start a Pinggy tunnel with key authentication so strangers can’t use your GPU:

ssh -p 443 -R0:localhost:8080 -t free.pinggy.io k:stt-demo-key

It prints an https://<random>.run.pinggy-free.link URL. Pinggy blocks any request without Authorization: Bearer stt-demo-key before it reaches your machine, and that’s exactly the header the OpenAI SDK sends from api_key, so the standard client works unchanged:

from openai import OpenAI

client = OpenAI(base_url="https://<your-url>.run.pinggy-free.link/v1", api_key="stt-demo-key")

with open("jfk.wav", "rb") as f:
    print(client.audio.transcriptions.create(model="whisper-1", file=f).text)

whisper-server ignores the model field, so any value works, and its README warns against running it with admin privileges. Free tunnels close after 60 minutes and get a new URL on restart; a Pro token keeps one URL. On an NVIDIA box, swap whisper-server for vLLM serving Qwen3-ASR or Cohere Transcribe and keep the client as is. For the other half of a voice loop, see our self-hosted text-to-speech guide.

How to choose

Conclusion

Before you commit to a model, run 50 clips of your own audio through two candidates on the same runtime and compute WER yourself. The leaderboards disagree, and the benchmark-fitting study shows why your audio is the only test set that settles it.