Open-source speech recognition guide
What Is Whisper? OpenAI Speech Recognition Explained
Whisper is OpenAI’s open-source automatic speech recognition model. It can turn spoken audio into text, identify many languages, add timestamps, and translate supported speech into English.
What is OpenAI Whisper?
OpenAI released Whisper in September 2022 as a general-purpose speech recognition model. It was trained on about 680,000 hours of multilingual and multitask audio collected from the web. That broad training helps it cope with different accents, everyday background noise, and varied subject matter more reliably than many older speech-to-text systems.
Whisper can transcribe speech in 99 languages represented by its tokenizer, detect the spoken language, preserve segment-level timing, and translate supported non-English speech into English. The model is open source under the MIT license, so developers can download its code and weights, run it locally, and build private transcription workflows without sending recordings to a hosted service.
Who made Whisper?
Whisper was created by OpenAI, the research and deployment company behind GPT and DALL·E. Unlike a closed hosted product, the original Whisper repository includes public source code, model descriptions, and downloadable model weights. The official OpenAI Whisper repository is the primary source for installation instructions and technical details.
How does Whisper work?
Whisper uses a Transformer encoder-decoder architecture. It resamples audio, divides it into 30-second windows, and converts each window into a log-Mel spectrogram. The encoder reads that audio representation, while the decoder predicts text tokens and special tokens for tasks such as language identification, timestamps, transcription, and translation.
Its training method is often described as weak supervision: instead of learning only from a small, carefully labeled laboratory dataset, Whisper learned from a very large collection of naturally occurring audio and associated transcripts. This scale helps the model generalize to real recordings such as meetings, podcasts, lectures, and interviews.
Whisper capabilities and supported formats
| Capability | Detail |
|---|---|
| Transcription | Speech to text across 99 languages represented by the model |
| Translation | Supported non-English speech translated into English text |
| Timestamps | Segment timestamps, with word timing available in compatible implementations |
| Typical inputs | MP3, M4A, WAV, MP4, and WebM when the chosen runtime can decode them |
| Model sizes | tiny, base, small, medium, large, and turbo variants |
Ways to use Whisper
| Method | Best for | Setup effort |
|---|---|---|
| Local Python or CLI | Developers and repeatable batch jobs | High |
| Hosted speech-to-text API | Teams integrating transcription into an app | Medium |
| Whisper Web | Anyone who wants free, private browser transcription | None — open and go |
You can try Whisper now for free in your browser. There is no installation, and your audio stays on your device.
Open Whisper Web →Whisper limitations and real-world accuracy
Whisper is capable, but it is not perfect. It can invent plausible words during long silences, repeat phrases, or miss rare names and specialized vocabulary. Overlapping speakers, very noisy recordings, distant microphones, and heavily compressed audio can reduce accuracy. The original model is also designed around 30-second windows rather than low-latency streaming, although developers can build streaming-like pipelines around it.
For everyday transcription—meetings, podcasts, lectures, interviews, and subtitle drafts—Whisper is a strong starting point. Important transcripts still need human review, especially for names, numbers, quotations, medical or legal language, and any content used to make high-stakes decisions.
Whisper FAQ
What is Whisper used for?
Whisper is used for automatic speech recognition: transcribing audio and video into text in many languages and translating supported speech into English.
Is Whisper free?
Yes. OpenAI released the Whisper code and model weights under the MIT license, so you can run and self-host the model without a per-minute license fee.
Can I use Whisper without installing anything?
Yes. Whisper Web runs a browser-compatible Whisper model directly on your device, so there is no desktop app to install and your media is not uploaded to a transcription server.