Building Dialogue from Operations

How dialogue audio is built from small, reusable operations you chain into recipes -- synthesise, record, voice-change, assemble, and trim.

Dialogue audio is built from a handful of small operations -- each does one thing -- that you chain together into a recipe. You can run the whole recipe with one press, or open any step to see and adjust it. The platform builds a sensible recipe for you from the screenplay; you only step in when you want to.

This is the same building-block idea used everywhere else in the media gallery, applied to voices. For the one-press surface over these operations -- generate-all, take review, and episode preview -- see Audio & Voice Pipeline. A few capabilities are still on the way; they're listed at the end.

Media carries its character

Every piece of dialogue audio is labelled with the character it belongs to. You see the character on the media tile, so you can tell whose line a clip is without playing it. That label travels with the audio as it moves through a recipe -- when an operation produces a separate file for each character, each file keeps its character.

This is what lets the platform wire steps together for you: a later step can simply ask for "the audio for VENN from the previous step," and the right file connects automatically.

The operations

Line to audio (text-to-speech)

Turns one written line -- VENN (angry): What do you mean! -- into an audio file in that character's voice. The emotion is optional. You pick which of the character's configured text-to-speech voices to use; it defaults to the character's draft voice.

Generated audio is cached for the whole scene, keyed by the line itself. If you rewrite the screenplay and a line moves to a different shot -- or shots get reordered, inserted, or deleted -- the unchanged line is reused from the cache instead of being synthesised again. (Deterministic engines such as Google reuse cached audio; other engines re-synthesise.)

Assemble a shot's timing

A shot often has a single line, but some have a back-and-forth between characters. The assemble operation takes the lines for a shot and lays them out with the right gaps -- pause, line, pause, line, pause -- and produces one audio file per character, with each character silent while the others speak. One file per character is exactly what a video model needs to lip-sync. Assemble currently handles up to four lines per shot.

The timing comes ready-made from the screenplay, so assemble usually just runs.

Record a performance

Prefer a real voice? Record provides a recorded performance as the source for a voice changer. You can record one line at a time (with a countdown and silence trim), or bring a whole-shot recording into the shot's gallery, labelled with its character.

Voice changer

Run a recorded performance through a character's voice changer to convert it into that character's voice, keeping the performance and timing of the original. You pick which configured voice changer to use -- ElevenLabs speech-to-speech or a ComfyUI workflow. The output stays labelled with the same character.

Trim

Trim shaves seconds off the start or end of an audio clip, keeping its character label. Enter the numbers, or use the waveform to set them.

Recipes the platform builds for you

From the screenplay, the platform assembles a good default recipe for a shot:

Text-to-speech: one line to audio step per spoken line, feeding one assemble step -- wired up by character.
Recorded performance: record line into a voice changer, per character.

These are starting points. Open any recipe to see the steps, swap a voice, adjust timing, or add a recording where it matters.

Choosing the keeper

An operation can leave you with more than one option -- a quick first pass and a higher-quality alternate, or several takes of a recording. Star the one you want to keep; archive the ones you don't. The starred take is the one downstream steps and the merge use. Takes are tracked per line.

What's not built yet

The rest of this page is live today. These pieces are designed but not yet runnable:

Split -- recording a whole conversation in one take and marking spans per character to get a labelled file each. (And the automatic version: lining a recording up against the screenplay with speech recognition, chopping a full-scene take into shots and characters, and flagging duplicate or flubbed takes.)
Full shot takes -- choosing between whole-shot performance takes. Take selection today is per line only; a whole-shot recording is a voice-changer source, not a pickable shot take.
The assemble timeline editor -- dragging lines around to nudge timing (assemble uses the screenplay's timing for now).
Trim speed and position, video trim, and reuse-an-edit -- trim today is start/end, on audio only.
In-place whole-shot capture -- recording a whole shot directly on the page (per-line capture exists; whole-shot recordings come in through the gallery for now).