Building Dialogue from Operations

How dialogue audio is built from small, reusable operations you chain into recipes -- synthesise, record, voice-change, assemble, and trim.

Dialogue audio is built from a handful of small operations -- each does one thing -- that you chain together into a recipe. You can run the whole recipe with one press, or open any step to see and adjust it. The platform builds a sensible recipe for you from the screenplay; you only step in when you want to.

This is the same building-block idea used everywhere else in the media gallery, applied to voices. For the one-press surface over these operations -- generate-all, take review, and episode preview -- see Audio & Voice Pipeline. A few capabilities are still on the way; they're listed at the end.

Media carries its character

Every piece of dialogue audio is labelled with the character it belongs to. You see the character on the media tile, so you can tell whose line a clip is without playing it. That label travels with the audio as it moves through a recipe -- when an operation produces a separate file for each character, each file keeps its character.

This is what lets the platform wire steps together for you: a later step can simply ask for "the audio for VENN from the previous step," and the right file connects automatically.

The operations

Line to audio (text-to-speech)

Turns one written line -- VENN (angry): What do you mean! -- into an audio file in that character's voice. The emotion is optional. You pick which of the character's configured text-to-speech voices to use; it defaults to the character's draft voice.

Generated audio is cached for the whole scene, keyed by the line itself. If you rewrite the screenplay and a line moves to a different shot -- or shots get reordered, inserted, or deleted -- the unchanged line is reused from the cache instead of being synthesised again. (Deterministic engines such as Google reuse cached audio; other engines re-synthesise.)

Assemble a shot's timing

A shot often has a single line, but some have a back-and-forth between characters. The assemble operation takes the lines for a shot and lays them out with the right gaps -- pause, line, pause, line, pause -- and produces one audio file per character, with each character silent while the others speak. One file per character is exactly what a video model needs to lip-sync. Assemble currently handles up to four lines per shot.

The timing comes ready-made from the screenplay, so assemble usually just runs.

Record a performance

Prefer a real voice? Record provides a recorded performance as the source for a voice changer. You record it on the step itself: press Record and a recorder opens with a countdown, showing you the lines to read from.

There are two shapes. Record line audio captures one line at a time and trims the silence off each end for you. Record performance captures the whole shot in one go -- one performance covering every line that character has in the shot -- which is what a voice changer wants as its source. Either way the clip lands in the shot's gallery labelled with its character and becomes that step's output, ready for the next step. A performance you recorded elsewhere can still be brought into the gallery instead.

Voice changer

Run a recorded performance through a character's voice changer to convert it into that character's voice, keeping the performance and timing of the original. You pick which configured voice changer to use -- ElevenLabs speech-to-speech or a ComfyUI workflow. The output stays labelled with the same character.

Trim

Trim shaves seconds off the start or end of an audio clip, keeping its character label. Enter how many seconds to remove from each end; leave the end at zero to keep the clip running to its finish.

Recipes the platform builds for you

From the screenplay, the platform assembles a good default recipe for a shot:

Text-to-speech: one line to audio step per spoken line, feeding one assemble step -- wired up by character.
Recorded performance: record line into a voice changer, per character.

These are starting points. Open any recipe to see the steps, swap a voice, adjust timing, or add a recording where it matters.

Choosing the keeper

An operation can leave you with more than one option -- a quick first pass and a higher-quality alternate, or several takes of a recording. Star the one you want to keep; archive the ones you don't. The starred take is the one downstream steps and the merge use. Takes are tracked per line.

What's not built yet

The rest of this page is live today. These pieces are designed but not yet runnable:

Split -- recording a whole conversation in one take and marking spans per character to get a labelled file each. (And the automatic version: lining a recording up against the screenplay with speech recognition, chopping a full-scene take into shots and characters, and flagging duplicate or flubbed takes.)
Full shot takes -- choosing between whole-shot performance takes. Take selection today is per line only; a whole-shot recording is a voice-changer source, not a pickable shot take.
The assemble sequencer editor -- dragging lines around to nudge timing (assemble uses the screenplay's timing for now).
Trim speed and position, video trim, and reuse-an-edit -- trim today is start/end, on audio only, set as numbers.