skip to content
Jobs

Narrated screencasts, synced to the run

Anchored scripts, aligned narration audio, and a recorder that keeps the voice in step with the replay.

Gantry milestones
A pipeline for scripted, narrated screencasts with synced timing.

Gantry can already record a run as a screencast: a fast-forwarded video of the build playing back on screen. What it could not do was talk over one. A narrated episode needs a written script, spoken audio, and a way to keep the two in step with the video — so that when the narrator says "here the tests go red," the video is showing exactly that moment. This job built that pipeline.

An episode starts as a Markdown script whose sentences carry hidden markers naming real events in the run. A validator proves those markers point at events that actually happened, and refuses scripts that dress up guessed timings or costs as measured fact. A by-hand text-to-speech step turns the script into audio, and a compiler reads where each spoken paragraph falls to build a timing plan that makes the video wait on the right moment for as long as the narration needs. A final step records the synced video, mixes in the narration, and checks the result is a genuine narrated cut rather than an old silent one. The paid voice step stays a manual, human-run action; everything the machine checks runs against fixtures, with no network call.

Build

The build splits into three milestones — script, sync, recording — each handing the next a finished contract, and within each the data model is defined before anything reasons over it. The run never went back: no gate rejected a sprint and no review forced a re-plan.

The organizing decision is dependency order made strict. The script format and its validator come first because everything downstream points at anchored events. Audio alignment comes before any sync decision because every timing choice depends on a stable, provider-independent record of when each paragraph is spoken. The recording milestone comes last because its media check is the skip-test that tells the fan-out which episodes still need rebuilding.

The same split repeats one level down, and it is the split that made each piece something a fresh, memoryless agent could finish alone. Anchor resolution — proving a marker names a real event — is one sprint; the honesty policy that rejects out-of-order anchors or invented duration claims is a separate one, and its brief states plainly that it can only run once resolution exists to say what each anchor resolves to. Timing pins are parsed and validated as a data model in one sprint, with no timeline work attempted; the shaper that obeys them is the next, consuming a typed, validated profile rather than inventing its own. Splitting those meant no agent had to design a parse grammar and a replay timeline engine in a single context.

The sync compiler is where the chain pays off. Its brief requires it to emit an ordinary timing profile — the very format the existing replay parser and shaper already read — rather than a parallel schedule format, so the compiler is a translation onto machinery that milestone two had already proven. The run stayed green from first sprint to last: no gate sent a sprint back, no review redrew a boundary. For a build cut this way that reads less like luck than like the decomposition spending the difficulty in advance — each fresh agent opened its context onto a predecessor that had already been tested, so what would have fought back inside a sprint was settled by the ordering instead.

Feature

Gantry's screencast tooling could only produce silent replays. This job adds the whole narrated path — anchored scripts, a validator that guards provenance honesty, aligned narration audio, a pinned-timing sync compiler, and a recording fan-out — and it still stands at HEAD, its engine modules and shell scripts intact.

Before this job, an episode of Gantry's screencast series was silent. The tooling curated a manifest of runs, recorded a fast-forwarded replay of each, and wrote metadata, but nothing spoke. There was also no defense against a narrator overclaiming: a script could assert that a run took a measured amount of time even when that run's timings were reconstructed or invented.

The design answers both. The anchored-script format ties narration to run events by hidden markers, and the validator in `src/engine/script.rs`, invoked through `scripts/check-script.sh`, checks structure, chronological order, and enough anchor coverage to sync — then refuses measured time-or-cost language for a timeline the run does not actually carry. Narration audio and a normalized paragraph-level alignment come from a by-hand step in `src/engine/tts.rs` and `scripts/tts-narrate.sh`. Timing pins live in `src/engine/timing.rs`, the pinned shaper in `src/engine/shape.rs`, and the compiler that turns script-plus-alignment into an ordinary timing profile in `src/engine/sync_profile.rs`. The final narrated video is recorded and muxed, validated by the narrated media-check script, and `scripts/gen-narrated-screencast-map.sh` fans the sequence out one unit per episode.

Standing in the tree today, all of that is present, and `docs/screencasts/README.md` still documents the narrated series end to end, including the explicit boundary that the paid narration step and live-registry filming sit outside the automated fixture gate. The visible drift is in the tests: two top-level integration files the job added, one for the narrated media check and one for the screencast map spec, are no longer at HEAD — the behaviour they exercised survives in the shipped scripts and engine modules, but that specific test scaffolding was later removed or folded elsewhere. The capability the job set out to build is intact, and newer narrated-recording scripts now sit alongside it, reusing the same approach rather than replacing it.

26 sessions
peak 153,829 · median execute 111,992 · heaviest 153,829 (02-audio-sync-and-pinned-timing / 04) context
unavailable tokens
unavailable cost
3 x 12 milestones x sprints
109 edits
720 commands
2h 24m duration
9 execute · 13 review · 4 plan · 3 gate-build roles
0 x 0 fixes x replans
codex harness