The Challenge
Short-form clips are how long-form content gets discovered — but producing them is tedious, skilled work. For every hour of source video, someone has to watch it all, find the moments worth clipping, reframe them to vertical, transcribe and time the captions, pick music, and export each platform-ready cut. Creators and brands either burn hours on this or pay an editor for every video.
The Problem
Each step is easy to automate in isolation; the hard part is making the whole chain run unattended with taste. "Find the best moments" is an editorial judgment, not a mechanical one — and a pipeline that clips the wrong 40 seconds produces perfectly formatted content nobody watches.
The Solution
Clipbot chains the full workflow into one pipeline, with an LLM making the editorial calls — scoring transcript segments for hook strength, emotional peaks, and self-contained narrative — and deterministic media code doing the cutting:
→ Transcribe (speech-to-text with word-level timing)
→ Highlight detection (LLM scores the transcript for clip-worthy moments)
→ Clip + reframe (cut to 9:16 vertical)
→ Animated captions (word-timed, styled)
→ Music + mix (AI recommends tracks from a described library)
→ Ready-to-post clips, thumbnails + a compilation cut
- LLM-scored highlight detection — the pipeline picks moments, not just timestamps
- Word-timed animated captions rendered onto every clip
- AI music recommendation against a curated, rights-clean library
- Translation support for multilingual output
- Speed presets to trade quality against turnaround time
- Modular pipeline (each stage is its own component) with a test suite over the critical stages
The Stack
Python end to end: yt-dlp for acquisition, Whisper-class transcription, LLM highlight scoring (provider-agnostic — OpenRouter, Gemini, or local Ollama), and FFmpeg for all cutting, reframing, captioning, and audio mixing. A web dashboard fronts the pipeline, deployed as a single Docker container.
The Outcome
What used to be an afternoon per video is now a URL paste: the pipeline returns a set of platform-ready vertical clips with captions, music, and thumbnails, plus a compilation cut. It's the same automation architecture we apply to client content pipelines — see the marketing agent crew on our projects page.