A language model cannot watch your footage. It can read every word you said, and know exactly when you said it. So we stopped trying to edit the pixels and edited the transcript instead. Transcribe the audio to word-level timestamps, make the cut decisions on the words, and let the video follow. The first thing it does is delete the silences: a raw test clip went from 12.5 seconds to 7.3 on its own, no timeline, no scrubbing.
What we did
The tool is about 350 lines of Python wrapped around two things that were
already on the box: faster-whisper for speech to text, and
ffmpeg for the pixels. Whisper returns a word with a start
and end time. That timing is the whole trick. If there is a gap longer
than 0.6 seconds between one word ending and the next beginning, that is
dead air, and dead air gets cut. Everything downstream is a recipe on top
of that: reframe to 9:16 or 1:1, burn captions sized for silent autoplay,
normalise the audio to broadcast loudness, prepend a branded title card.
Nothing about it is a graphical editor. You hand it a raw file and a one-line brief and it hands back a finished clip. Run it again on the same input and you get the same cut, because it is code, not taste. On a laptop-class CPU with no GPU, the whole pipeline (transcribe, tighten, re-transcribe for caption timing, reframe, caption, normalise, brand) ran in about eight seconds on a short clip.
Why it was worth doing
The interesting part is not the ffmpeg. Anyone can join two clips. The interesting part is that the edit decision layer is the transcript, and a transcript is something a model is genuinely good at reasoning over. Once the cut is a text operation, everything a language model already does to writing becomes a video edit: find the strong sentence, drop the rambling middle, pull the best line to the front as a hook, keep the clean take when the words differ. The captions fall out of the same transcript for free, which matters because most feeds autoplay silent, and an uncaptioned talking head is a muted stranger.
What's still off
Reading the words is not hearing the delivery. If you say the same sentence three times with identical words, the tool sees three identical transcripts and cannot tell which take you nailed. Energy, eye contact, the take that just lands better, that is still a human call, and the honest workflow is: the tool does the content and mechanical edit, you flag the takes you liked. It is also CPU-bound here, so a long 4K export is slow, and the small English model trades a little accuracy for speed on names and jargon. Building it turned up one good ffmpeg trap worth knowing: the concat demuxer trusts container timestamps and quietly ballooned a nine-second export to seventeen when the inputs had been filtered upstream. The fix was to join with the concat filter, which decodes and re-stamps, and to normalise every input to one spec first.
What's now in the stack
vidkit pipeline raw.mp4 -o out.mp4 --aspect 9:16 --title "Workloft": raw to platform-ready in one pass.vidkit tighten,captions,reframeandtranscribeeach run on their own.- Word-level silence detection as the cut engine, tunable with
--max-silence. - No cloud calls and no GPU:
faster-whisperint8 on CPU, ffmpeg for everything visual. - A standalone, dependency-light copy on GitHub. Steal what you like.