# Whisper transcription

Canonical URL: https://getvidiyo.app/glossary/whisper-transcription
Last updated: 2026-10-08

> Definition: Whisper is an open-source automatic speech recognition model released by OpenAI and trained on 680,000 hours of multilingual audio. Whisper transcription means converting speech to text, with timestamps, using one of its models.

Whisper is the speech recognition model behind many modern transcription and captioning tools. It turns spoken audio into text with timestamps, handles accents and background noise well, and works across many languages. Because it's open source and small enough to run on a laptop, it made private, local transcription practical.

## Where does Whisper come from?

Whisper comes from OpenAI, whose 2022 paper (Radford et al.) describes models trained on 680,000 hours of multilingual and multitask audio ([source](https://arxiv.org/abs/2212.04356)). OpenAI released the code and weights under the MIT licence ([source](https://github.com/openai/whisper)). Open-source ports followed, notably whisper.cpp, a C/C++ implementation optimised for Apple Silicon that runs fully offline ([source](https://github.com/ggml-org/whisper.cpp)).

## Which Whisper models exist?

Whisper comes in several sizes that trade speed for accuracy ([source](https://github.com/openai/whisper)):

| Model | Notes |
|---|---|
| tiny, base, small, medium | Smaller and faster; each has an English-only `.en` variant that tends to do better on English |
| large | The most accurate multilingual model, slowest to run |
| turbo (large-v3-turbo) | Much faster than large with minimal accuracy loss; not trained for translation |

## How is Whisper used for video editing and captions?

Whisper is used in video editing to produce the transcript that everything else builds on: [word-level captions](https://getvidiyo.app/glossary/word-level-captions), [SRT files](https://getvidiyo.app/glossary/srt-file), searching footage by what was said, filler word and pause removal, and [transcript-based editing](https://getvidiyo.app/glossary/transcript-based-editing). With word timing added, each caption word and each cut is tied to the exact moment it's spoken.

## What are Whisper's limitations?

Whisper's main limitations are timing precision and occasional confident mistakes. It can mis-hear names and jargon, sometimes invent text during long silences or music, and its word timings are estimates that can drift by a few frames. Mixed-language speech is harder: Hindi, for example, comes out in Devanagari script, so Hinglish captions need some words fixed by hand.

## How does Vidiyo use Whisper?

Vidiyo transcribes on your Mac with whisper.cpp, so audio never leaves your computer. English uses the English small model (about 490 MB); every other language, and automatic detection, uses a language pack based on large-v3-turbo (about 575 MB). Each downloads once, the first time it's needed, and is checked against a pinned checksum. Word timing uses the method matched to the model, and words the aligner couldn't place confidently are marked as estimated so you can check them.

## Frequently asked questions

### Is Whisper free to use?

Yes. OpenAI released Whisper's code and model weights under the MIT licence, so you can run it on your own computer for free. OpenAI also sells hosted transcription through its API.

### Can Whisper run offline on a Mac?

Yes. Ports like whisper.cpp run Whisper models locally with no internet connection, and they're optimised for Apple Silicon. Audio never has to leave the machine.

### Does Whisper give word-level timestamps?

Yes, with an extra step. Whisper's basic output is timed by segment; word-level timing comes from aligning the model's attention to the audio, an option in OpenAI's package and in whisper.cpp. That's what makes word-by-word captions possible.

## Related terms

- [Transcript-based editing](https://getvidiyo.app/glossary/transcript-based-editing)
- [Word-level captions](https://getvidiyo.app/glossary/word-level-captions)
- [SRT file](https://getvidiyo.app/glossary/srt-file)
- [AI video editor](https://getvidiyo.app/glossary/ai-video-editor)

## Where Vidiyo does this

- [Private, local video editing with AI](https://getvidiyo.app/features/private-local-editing)
- [Auto captions that are transcribed on your Mac](https://getvidiyo.app/features/auto-captions)
- [Private video editing with local AI: what stays on your Mac](https://getvidiyo.app/blog/private-video-editing-local-ai)

## Sources

- [Whisper paper, Radford et al. 2022 (arXiv:2212.04356)](https://arxiv.org/abs/2212.04356) (accessed 2026-10-08)
- [openai/whisper (GitHub)](https://github.com/openai/whisper) (accessed 2026-10-08)
- [ggml-org/whisper.cpp (GitHub)](https://github.com/ggml-org/whisper.cpp) (accessed 2026-10-08)