Render Vertical With Word Pop Captions

Build the 9:16 render with animated word level captions, then run the caption correction loop. This is the longest script in the pipeline and the one you will tune most.

The renderer does four jobs at once: it trims and concatenates the plan's segments, it composes the vertical frame, it generates animated captions from the word timings, and it burns everything together in one ffmpeg pass.

Two layouts, one canvas

The canvas is 1080 by 1920. Which layout you use depends on whether anything on screen carries the point.

Layout	Composition	Use when
split	Speaker camera across the top, caption band in the middle, the full screen share fit to width and anchored to the bottom	A demo, a dashboard, or anything visual carries the idea
face	Speaker camera cropped to 9:16 filling the whole frame, captions on the lower third	The person talking is the whole point

Fit the screen share, never crop it. Cropping a shared screen to fill a vertical frame cuts off exactly the thing you are pointing at. Fit it to width and let its height follow the source aspect, anchored to the bottom edge.

The caption style that works

Two or three word chunks, never full sentences
All caps, heavy font, thick outline so it survives any background
The chunk stays white while the active word pops in an accent color and scales up slightly
A small pop in scale animation when each chunk appears
Filler words stripped from captions automatically, so the text reads cleanly even though the audio keeps them

Paste into Claude Code

Write pipeline/shorts/render_vertical.py. This is the big one.

Canvas 1080x1920. Two layouts, chosen by a --layout flag:

  split: speaker cam across the top (1080x608), caption band in the middle,
         the FULL screen share region fit to width and anchored to the bottom
         edge. Its height follows the source aspect, so nothing is cropped.
  face:  the cam region cropped to 9:16 and scaled to fill the whole frame,
         captions on the lower third over the video.

Inputs:
- src video, out path
- --seg a:b, repeatable, in SOURCE video seconds, played in the order given
- --cam x:y:w:h  the speaker window inside the recording
- --screen x:y:w:h  the full screen share region
- --font, --hl (active word highlight as RRGGBB), --keep-fillers

Captions:
1. Load words from <src stem>.words.json, keep only words whose midpoint falls
   inside a segment, and REMAP each word's time onto the concatenated timeline.
2. Strip filler words from the captions unless --keep-fillers.
3. Chunk into 2 to 3 word groups, breaking early after . ? ! or ,
4. Generate an ASS subtitle file. For each chunk emit one Dialogue line PER
   WORD, re-rendering the whole chunk with the active word in the highlight
   color and scaled to 112 percent, the rest white at 100 percent. Add a pop in
   scale animation on the first word of each chunk.
5. Also dump the chunk text to <out>.chunks.txt so I can proofread it.

Render everything in ONE ffmpeg filter_complex: trim and concat the segments,
compose the layout, burn the ASS subtitles. libx264 crf 19 preset veryfast,
aac audio, faststart, strip metadata.

Usage example:
python pipeline/shorts/render_vertical.py work/ep-test.mp4 out/S1.mp4 \
  --seg 2988.4:3013.2 --seg 228.1:253.6 --layout split

The caption correction loop

Transcription will get names, products and numbers wrong. Fixing them by editing the transcript is the wrong move, because you lose the fix on the next run. Instead the renderer reads a per clip edits file and applies corrections every time.

Render once. The renderer writes <out>.chunks.txt, which is the caption text with timestamps.
Read that file, not the video. It takes thirty seconds and catches everything.
Put corrections in <out>.edits.json as a flat map of {"what it heard": "what was said"}, matched case insensitively across the word stream.
Re render. The corrections apply automatically and survive every future run.

An edits file

{
  "medi care": "Medicare",
  "web em cp": "WebMCP",
  "fifteen citations": "15 citations",
  "um so basically": ""
}

A replacement of empty string drops those words from the captions entirely. Use it for verbal tics the filler list does not catch. The timing of the removed words is redistributed across the replacement, so the captions stay in sync.

Check it worked

Play the clip with the sound off and read only the captions. If the idea still lands, the captions are doing their job. Then play it with sound and confirm the captions match the audio word for word. Captions that say something the speaker did not say are worse than no captions.

Where this bites

Corrections are for what the transcript misheard, not for what you wish had been said. If the speaker genuinely misspoke, fix it by cutting the segment, not by rewriting the caption. Captions that contradict the audio are jarring and they read as dishonest.

Questions

Why burn the captions in instead of uploading a caption file?

Because the animated word pop is the format that holds attention on a vertical feed, and no platform caption track can do it. Burned in also means the clip looks identical everywhere you post it.

Why chunks of two or three words?

Longer chunks make the eye read ahead and stop listening. Shorter chunks flicker. Two to three words with the active word highlighted keeps the eye moving at roughly the speed of speech.

Render Vertical With Word Pop Captions

Two layouts, one canvas

The caption style that works

The caption correction loop

Questions

Stuck on this step?