Install the tools and prepare a run folder

This guide uses macOS on Apple silicon, Python 3, Homebrew, FFmpeg, and mlx-whisper. The commands were run with mlx-whisper 0.4.3, FFmpeg 8.1, and mlx-community/whisper-base-mlx. Open Terminal in a writable project folder. The tested source was an MP4 named input-video.mp4; for a MOV, MKV, or different filename, change the video= line to its actual path. Keep using the same Terminal window for the following commands. The MLX Whisper setup guide covers installation and model selection.

Set up on an Apple-silicon Mac
python3 -m venv .venv
source .venv/bin/activate
pip install 'mlx-whisper==0.4.3'
brew install ffmpeg

mlx_whisper --help
ffmpeg -version

video="$PWD/input-video.mp4"
ls -lh "$video"
run_dir="$(mktemp -d "$PWD/mlx-video-run-XXXXXXXX")"
printf 'Run folder: %s\n' "$run_dir"

If ls cannot find the video, fix the video= path before continuing. The unique run folder prevents a second attempt from overwriting the first transcript. Create a new one for every retry; if you close Terminal, reactivate .venv and set run_dir to the printed folder path. If mlx_whisper is not found, check that the environment is active. uname -m should print arm64 for this Apple-silicon workflow.

The first use of a Hugging Face model may download files. If the recording must be processed without a network connection, obtain a compatible MLX model directory in advance, replace the Hub identifier in --model with that directory's path, and test the command while disconnected before using sensitive material. The test reported here used a downloaded, cached model; it did not test a network-free first installation.

Find the speech track in the video

Inspect audio streams before extraction. The ffprobe documentation explains the stream fields and tags in this command. The stream index is the number to use with -map 0:index; a language tag can be missing, so listen when the correct track is uncertain.

List audio streams
ffprobe -v error -select_streams a \
  -show_entries stream=index,channels,start_time:stream_tags=language \
  -of compact "$video"

In our original test video, stream 1 was stereo commentary and stream 2 was the mono voice we wanted. The language tags were und, so they did not identify the track. Without -map, FFmpeg chose the stereo commentary: its automatic audio selection favors the stream with the most channels. An explicit -map 0:2 selected the intended voice. Substitute the stream index shown for your own video. If there is only one audio stream, -map 0:a:0 selects that first audio stream. If ffprobe shows no audio stream, there is no soundtrack for MLX Whisper to transcribe. See FFmpeg's stream-selection rules.

If several streams look plausible, preview each before processing the full file. This example previews stream 2; change 0:2 and the output filename for the next candidate. Listen for the intended speaker or language, not just a language tag or the number of channels.

Preview one candidate track
ffmpeg -hide_banner -loglevel error -n \
  -i "$video" -map 0:2 -t 10 -vn \
  -ar 16000 -ac 1 -c:a pcm_s16le \
  "$run_dir/track-2-preview.wav"
afplay "$run_dir/track-2-preview.wav"

Extract and check the audio

The command below uses stream 2 from the test video. It writes 16 kHz, mono, 16-bit PCM WAV, matching the normalization used in our run. The -n flag stops if that output already exists; create a new run folder for a fresh attempt.

Extract the chosen stream
ffmpeg -hide_banner -loglevel error -n \
  -i "$video" -map 0:2 -vn \
  -ar 16000 -ac 1 -c:a pcm_s16le \
  "$run_dir/normalized-audio.wav"

Before inference, confirm that the WAV has the expected duration, sample rate, and channel count. Listen to a few seconds with afplay "$run_dir/normalized-audio.wav". A correct codec and nonempty file do not prove that the selected stream contains the right speaker.

Inspect the normalized WAV
ffprobe -v error \
  -show_entries stream=sample_rate,channels:format=duration \
  -of default=noprint_wrappers=1 "$run_dir/normalized-audio.wav"

Our selected stream produced a 5.85-second WAV at 16,000 samples per second with one channel. If the WAV is silent, unexpectedly short, or contains the wrong voice, return to the stream list before trying another speech model. A video's audio can also start later than its picture; note the original stream's start_time when you need captions aligned to the full video timeline.

Transcribe once and inspect all five outputs

The MLX Whisper CLI accepts the model, spoken language, output name, output format, and word-timestamp options used here. Keep --language en for a known English track. For another known language, use its supported code; if the language is unknown, omit the flag to let the model detect it, then check the detected result against the audio. Choosing a language cannot fix a wrong audio track.

Create the transcript files
mlx_whisper "$run_dir/normalized-audio.wav" \
  --model mlx-community/whisper-base-mlx \
  --language en \
  --output-dir "$run_dir/transcript" \
  --output-name demo \
  --output-format all \
  --word-timestamps True \
  --verbose False

Check the files after the process finishes, especially if the terminal printed an error. Look for demo.txt, demo.srt, demo.vtt, demo.tsv, and demo.json inside this run's transcript folder. TXT is the readable transcript; SRT and VTT are caption formats; TSV supplies timing rows; JSON retains segments and, with this option, word timestamps. The CLI can write over files in an existing output folder, which is why each attempt gets a new run folder.

Check the outputs
ls -lh "$run_dir"/transcript/demo.*
python -m json.tool "$run_dir/transcript/demo.json" > /dev/null
sed -n '1,12p' "$run_dir/transcript/demo.srt"

What the video test actually produced

We made a 12-second MP4 with two original synthetic speech tracks. Its stereo commentary described itself as a production commentary track. The mono target introduced an Atlas briefing at nine thirty, named Mina Chen, and said “Do not approve.” Automatic FFmpeg selection transcribed the commentary; explicit selection of stream 2 transcribed the target. Both outputs read coherently, so listen to the selected WAV before accepting either one.

With the target stream and the base checkpoint, the TXT began “The Atlas briefing starts at 9.30” and kept both “do not approve” and “Mina Chen.” The recording said “nine thirty”; if that means a meeting time, change 9.30 to the format your viewers expect, such as 9:30. The SRT had two cues ending at 00:00:05,540, within the 5.85-second selected audio. Its first cue was long, so check whether a viewer can read it at playback speed.

We also tried --max-words-per-line 8; it split “do” and “not” across two cues. A word limit cannot choose a safe sentence break. Review negations, names, numbers, and clause boundaries after automatic caption wrapping. The test record includes the commands, transcripts, hashes, and the earlier clean, noise, and silence fixtures. All speech in these tests was synthetic; the results are not a measured accuracy claim for interviews, accents, or long recordings.

Check captions against the video

  1. Read the TXT while listening to the original video. Correct the places that can change meaning: names, times, money, negation, and technical terms. Keep the raw automatic TXT and put corrections in a separate file.
  2. Load the SRT or VTT with the video in a player or editor. Check the first spoken word, a point in the middle, and the final spoken word. A WAV extracted from a video may start at zero even when its audio stream begins after the first picture. Measure that delay against the original video before changing caption times.
  3. Read each caption at playback speed. Repair cues that are too long or split a phrase badly. The test's eight-word wrap split “do not”; a syntactically valid SRT can still mislead a viewer.

If the speech starts 2.4 seconds into the original video but the first SRT cue starts at zero, delay a copy of the SRT by 2.4 seconds. FFmpeg's -itsoffset option adds that offset to the subtitle timestamps; our command check moved the first cue from 00:00:00,000 to 00:00:02,400. Our synthetic video's target speech started at zero, so this shifted copy is an illustration, not the correct caption file for that video. Replace 2.4 with a measured delay and check playback again. A constant shift will not fix captions that drift farther out of sync over time.

Delay a copy of the SRT by 2.4 seconds
ffmpeg -hide_banner -loglevel error -n \
  -itsoffset 2.4 -i "$run_dir/transcript/demo.srt" \
  -c:s srt "$run_dir/transcript/demo-plus-2.4s.srt"
sed -n '1,4p' "$run_dir/transcript/demo-plus-2.4s.srt"

JSON word probabilities and segment metadata can help locate a suspicious stretch, but they do not certify a name, amount, or speaker. For reference-based word error rate and publication decisions, continue with the transcript accuracy workflow.

Adjust the workflow to the recording

SituationWhat to check or change
Stream map '0:2' matches no streamsThe example index does not exist in your file. Re-run the ffprobe command, use an index it actually lists, and preview that track before extracting the full WAV.
Empty or wrong transcriptListen to normalized-audio.wav. If it contains commentary, choose another stream; if it is silent or much shorter than expected, inspect the source audio and its start time before changing models. If ffprobe lists no audio stream, stop: this video has no soundtrack to transcribe.
Known non-English or mixed-language speechFor known Spanish speech, replace --language en with --language es; for an unknown language, omit the flag and review what was detected. If the language changes within the video, check short samples on both sides of the change instead of trusting one language choice for the whole file. Our video test covered English only.
Music, low speech, jargon, or namesExtract the same representative 30-second section and compare important words with the audio. Correct names and numbers manually; if trying another checkpoint, hold the audio and review criteria constant. Our single-checkpoint fixtures do not establish which model is best for real recordings.
A long video or memory pressurePilot sections near the start, middle, and end. If the full run fails, first confirm a short section completes, then try a smaller compatible checkpoint or process manageable sections while keeping their source offsets. Record the Mac, model, elapsed time, and memory use before making speed claims.
Captions shift or driftIf every cue is early or late by roughly the same amount, measure the difference and shift a copy as shown above. If the error grows over the video, a single offset is the wrong fix; compare the source duration, extracted audio, and caption timing in your editor.
Several or overlapping speakersWord timestamps are not speaker labels. Review speaker changes against the audio and use a separate speaker-attribution workflow when the transcript needs identities.

For a long recording, make a small diagnostic extract with FFmpeg's -ss and -t options before committing to the full run. Change both the stream index and sample time to match your video. Captions generated from a sample beginning at minute 20 start near zero for that sample; about 1,200 seconds must be added when placing them on the original video timeline, followed by a playback check. This is an example of the method, not a long-file performance result from our 12-second fixture.

Try 30 seconds beginning at minute 20
ffmpeg -hide_banner -loglevel error -n \
  -ss 00:20:00 -t 30 -i "$video" \
  -map 0:2 -vn -ar 16000 -ac 1 -c:a pcm_s16le \
  "$run_dir/sample-20m.wav"

Keep the run reproducible

Save the input filename, selected stream index, tool versions, checkpoint, language option, commands, output filenames, and any manual corrections alongside the transcript. Hash the source and normalized WAV if files may be transferred or reviewed later. That record lets someone distinguish a corrected caption from raw model output without repeating the inference.

Record the two input hashes
shasum -a 256 "$video" "$run_dir/normalized-audio.wav"

Process recordings you have permission to use and protect the resulting text as you would the video. Local inference removes the hosted transcription upload from this workflow; it does not resolve consent, copyright, or workplace retention requirements.