Kurzweil Scorecard: He Won the Speech War. The HMM Generals Lost.

In 2005, Ray Kurzweil used Chapter Nine of The Singularity Is Near —
“Response to Critics” — to defend his predictions about machine speech.
The four claims in this batch are all defensive in posture: voice
recognition has commercial value; my own company shipped the first
ten-thousand-word system in 1987; Hidden Markov models are the standard
practical method; cars are starting to talk back. Read in 2026, every
single claim verifies on the historical record.

The footnote, the one Kurzweil’s chapter never wrote, is that the
machinery he was defending — Hidden Markov models, hand-tuned acoustic
features, finite-state grammars — is now almost extinct. The speech
recognition layer of the modern stack is built out of self-supervised
transformers trained on hundreds of thousands of hours of unlabeled
audio. The destination Kurzweil promised arrived early. The vehicle that
got there is one his book does not name.

The predictions

Kurzweil wrote the four claims in this batch as evidence that critics
had underestimated practical AI. He cited Voice Recognition: It Pays to
Talk and Finally, a Car That Talks Back as trade-press signals that
the market had already validated the technology. He paired these with
Lawrence Rabiner’s foundational IEEE tutorial on Hidden Markov Models,
which he positioned as the standard tool of the field. And in the
“About the Author” section of The Singularity Is Nearer (2024), he
restated the historical claim verbatim: he was “the principal inventor
of… commercially marketed large-vocabulary speech recognition
software.”

Each statement was true when made. The interesting question — the only
question worth asking twenty-one years later — is what happened next.

Where we actually are

The HMM era ended around 2017. Counting publications in our mirror
of the OpenAlex literature index, papers combining “hidden Markov”
with “speech” peaked at 386 in 2008 and stayed above 300 through 2014.
By 2017 that figure had collapsed to 222. By 2024 it was 113 — roughly
where it sat in 1991. The patent record tells a sharper version of the
same story. Speech-recognition patents that explicitly invoke Hidden
Markov methods went from 60 grants before 2010 to 22 in the first
half of the 2010s, then 12, then 6 in the most recent five-year window.
Patents in the same speech corpus that invoke neural networks, deep
learning, or transformers went the other direction: 91, 37, 125, 226.

The replacement is documented in three landmark papers our literature
search surfaces by citation count. Speech-Transformer (Dong et al.,
ICASSP 2018) showed a non-recurrent sequence-to-sequence model could
match RNN-based recognizers, with 1,053 citations. wav2vec 2.0 (Baevski
et al., 2020, 2,435 citations) made the breakthrough that defined the
era: “learning powerful representations from speech audio alone followed
by fine-tuning on transcribed speech can outperform the best
semi-supervised methods while being conceptually simpler.” Then
Conformer (Gulati et al., 2020, 379 citations) added local convolution
inside the attention block and pushed accuracy further. By the time
OpenAI released Whisper in 2022, trained on 680,000 hours of weakly
labeled multilingual audio, the baseline architecture for speech
recognition was a Mel spectrogram fed into a sequence-to-sequence
transformer. Hidden Markov models were no longer in the recipe.

The patent office is recording the transition in real time. Three
recent grants put faces to the trend. US 12,380,880, granted to
Deepgram in August 2025, describes “an end-to-end automatic speech
recognition (ASR) system… constructed by fusing a first ASR model
with a transformer,” then trained as a single model with teacher-student
distillation. The claims read like a statement of architectural finality:
the speech model and the language model are no longer separable
components stitched together by an HMM decoder, but a single computational
graph trained jointly. US 11,978,435, granted to Mitsubishi Electric
Research Laboratories in May 2024, claims “a context-expanded
transformer network” that processes long audio recordings — “lecture and
conversational speeches” — in a sliding window, using each utterance to
inform recognition of the next. And US 11,978,433, also granted to
Microsoft in May 2024, claims a multi-encoder transformer that picks
between close-talk and far-talk inputs based on signal characteristics —
the kind of noise-handling work that used to be done by feature engineers,
now folded inside the network.

The assignee list for end-to-end ASR patents from 2018 onward is
revealing. Google leads with 42 grants. Microsoft, which closed its
\$19.7-billion acquisition of Nuance in March 2022, sits in second with
15. Mitsubishi Electric Research, IBM, Tencent, Samsung, Amazon, and
Baidu round out the top tier. Deepgram, founded in 2015 specifically to
build transformer-based ASR rather than maintain an HMM pipeline, has
already reached the top fifteen.

The cars finally talked back, twenty years late. Kurzweil’s 2004
data point — Finally, a Car That Talks Back in Wired News — was
generous. Real automotive voice recognition that worked under road noise
took another decade. But by 2024, the technology landed all at once.
Mercedes-Benz announced a ChatGPT-backed update to MBUX rolled to “more
than three million vehicles globally,” and in January 2025 confirmed
that the new CLA would ship with Google Cloud’s Automotive
AI Agent for “agentic conversations” with multimodal reasoning and
multilingual support. Automotive voice patents in our index, which sat
at 4–10 per year through the 2000s, broke 30 a year by 2019 and ran
above 25 a year through 2025. The functionality Kurzweil promised
arrived. It just waited until the underlying recognition stack stopped
being HMM-based to actually work.

Industry consolidation closed the loop. The corporate genealogy is
nearly poetic. Kurzweil Computer Products spun out into ScanSoft, which
merged with Nuance in 2005, which absorbed the Dragon NaturallySpeaking
codebase from the wreckage of Lernout & Hauspie. Microsoft acquired
Nuance in 2022 for \$19.7 billion. The accounting line that began with
the Kurzweil Voice Report in 1987 — a ten-thousand-word HMM
recognizer — terminates in a Microsoft balance-sheet entry written one
year before the company began re-engineering its speech stack on top of
transformer-based foundation models. The brand survived. The mathematics
inside it did not.

The scorecard

What Kurzweil missed (and what he nailed)

The pattern in this batch is not “he was wrong.” It is “he was right
for a different reason than the one he gave.”

Kurzweil’s defense in 2005 rested on a specific architectural claim:
HMMs were proof that statistical pattern recognition could industrialize
speech, and speech was a beachhead for everything else AI would
eventually do. The first half of that argument has aged badly. HMMs did
not industrialize speech. They industrialized speech enough that
buyers stayed in the market while a different mathematical framework —
attention-based sequence-to-sequence models trained on raw audio at
scale — caught up and eventually overtook them. Whisper Large-v3 reaches
2.7% word error rate on clean audio, where commercial HMM systems in
2005 sat at roughly 10–15% under similar conditions. The MLPerf Inference
benchmark introduced in 2025 reported that Whisper-class models reduced
the WER of the prior MLPerf ASR baseline (an RNN-T model) by more than
72 percent.

The second half of his argument — that speech would be the visible
proof of practical AI — has aged extraordinarily well. Mercedes is
shipping a chat agent in three million vehicles. Microsoft owns the
descendant of the Kurzweil Voice Report. The end-to-end models that
replaced HMMs are now used to transcribe podcasts, caption livestreams,
and operate the dictation features inside operating systems used by
billions of people every day. Speech recognition is, finally, the
ambient infrastructure his 2005 chapter promised.

The lesson for forecasters is narrow but useful: when you are right
about the destination, the mechanism you anchor your defense on can
still get replaced under you. Kurzweil was defending HMMs as proof
that critics had underestimated AI’s trajectory. The trajectory is now
steeper than he predicted, and the proof is bigger than he predicted,
and the architecture inside both is not the one he was defending. There
is a worse outcome a forecaster can have. There are not many.

Method note

Counts come from the patents corpus on our research server (9.3 million
US patent grants and pre-grants) and from the OpenAlex literature
mirror (357 million scholarly works), filtered by full-text search and
publication date. Patent claims and abstracts were read in full for the
three end-to-end ASR grants discussed; high-citation papers were pulled
from the literature index and read at the abstract level. Recent
automotive and corporate developments came from web searches conducted
this session against Mercedes-Benz press materials, Microsoft and
OpenAI public filings, and the MLPerf 2025 inference benchmark
announcements.