Murmel v2 available now: 30% fewer errors in Dutch speech-to-text

Murmel v2 is out, and it makes about 30% fewer errors than the first version.

A few months ago I introduced Murmel, a Dutch speech-to-text model I built because the existing options weren't good enough at Dutch. The response was overwhelming: municipalities, universities, ministries and companies lined up to test it on their own audio.

All of that feedback helped improving the interface. But also I have been working on collecting more data, enhance the architecture, and invest more in compute.

On several common public benchmarks, v2 now ranks third overall against the leading Dutch ASR systems, ahead of everything from OpenAI, Mistral, NVIDIA, Cohere and Qwen. The only two models still ahead are very well-funded proprietary frontier systems, and they're what I'm aiming at next.

A public benchmark, measured the same way for everyone

When I launched v1, I published my own benchmark on parliamentary audio. It was honest work, but it was still my benchmark, run by me. The fair question from readers was: how does this hold up when the model is measured on a shared, public yardstick?

These results answer exactly that: every model is run through the same three standardized public Dutch test sets, FLEURS, VoxPopuli, and Multilingual LibriSpeech, with the same evaluation pipeline, so the comparison is like-for-like on common data. One honest caveat applies to any benchmark like this: some models may be tuned on data that overlaps the test sets, so the numbers are best read as indicative rather than definitive, and I'd always encourage you to verify any figure that matters to you on your own audio.

Where v2 ranks

Average word error rate across the three benchmark datasets. Word error rate is the percentage of words the model gets wrong, so lower is better.

I'm proud to sit just behind the two models ahead of Murmel v2, Resonate-1 and ElevenLabs scribe_v2, both from well-funded, well-respected companies. Murmel ranks third, ahead of Mistral's Voxtral, Cohere, NVIDIA's Parakeet, Qwen, and OpenAI's Whisper-large-v3, which lands all the way down in tenth at nearly double Murmel's error rate.

How far v2 has come over v1

Against my own previous release, the improvement is consistent across every dataset. I evaluated v1 and v2 on the same three benchmarks with the same setup, so this is a clean apples-to-apples comparison.

That's an average reduction of roughly 3 percentage points, about a third fewer errors, with the biggest single gain on VoxPopuli, the hardest of the three. And none of it costs speed: v2's real-time factor is around 0.006, meaning it transcribes an hour of audio in roughly twenty seconds of compute, comfortably faster than several of the proprietary systems above it on the leaderboard.

The data sources Murmel is tested on

A benchmark is only as trustworthy as the data behind it. The three test sets used here, FLEURS, VoxPopuli, and Multilingual LibriSpeech, are established, publicly available datasets, each containing a Dutch portion, and each stresses the model in a different way. That spread is what makes the average meaningful.

FLEURS (Few-shot Learning Evaluation of Universal Representations of Speech) is Google's multilingual benchmark, built on read sentences from the FLoRes translation set. The Dutch portion is relatively clean, well-articulated speech from a range of speakers. It's the closest thing to a best-case condition, which is why error rates here are the lowest, and it tells you how good a model is when the audio cooperates. Murmel v2 scores 4.67% here, in striking distance of the proprietary leaders.

VoxPopuli is a large corpus drawn from European Parliament recordings: real plenary debates and speeches. This is spontaneous, accented, sometimes overlapping political speech recorded in a noisy chamber, frequently by non-native speakers. It is genuinely hard, which is exactly why it's the most relevant test for the public-sector use cases Murmel is built for. The high absolute numbers across every model reflect the difficulty of the material, not a Murmel-specific weakness, and v2's jump from 15.44% to 10.59% over v1 is the largest improvement anywhere in the comparison.

Multilingual LibriSpeech (MLS) is the Dutch slice of a large corpus derived from LibriVox audiobooks people reading published books aloud. It sits between the other two: clearer than parliamentary debate, but with longer, more varied sentences and a wide range of voices and recording conditions. It's a good measure of how a model handles sustained, narrative-style speech. Murmel v2 scores 6.25% here.

Together these three cover a useful spread: clean read speech (FLEURS), messy real-world institutional speech (VoxPopuli), and long-form narration (MLS). A model that improves on all three at once is improving in a real, generalisable way, not overfitting to one style.

One honest caveat, the same one I gave with v1: these are all relatively prepared or read forms of speech. Fully spontaneous audio, a medical consultation, a field interview, a call-centre recording, remains an active area of work, and rigorous evaluation on those domains is something I'm keen to do with the right partners.

Built in Europe, for Europe

The result reinforces the thesis behind Murmel from day one: a purpose-built Dutch model beats general-purpose alternatives, and the gap grows with every training run. On a public benchmark, the only two models still ahead of Murmel are very well-funded proprietary frontier systems, and v2 closed roughly a third of its remaining error rate in a single iteration.

Murmel runs on infrastructure based in the Netherlands, designed for high-volume transcription pipelines (increasingly a requirement rather than a preference in the public sector). Since v1 launched, more than 500 users have signed up to transcribe their Dutch audio with Murmel, and that feedback is what made v2 better.

You can try Murmel v2 free for two weeks. Start your free trial at murmel.nl.