Building Dialogue from Operations
How dialogue audio is built from small, reusable operations -- synthesise, record, voice-change, assemble, and trim.
Dialogue audio is built from a handful of small operations -- each does one thing -- that you run from the gallery's Recipe panel, or that the one-press buttons run for you.
This is the same building-block idea used everywhere else in the media gallery, applied to voices. For the one-press surface over these operations -- generate-all, take review, and episode preview -- see Audio & Voice Pipeline. A few capabilities are still on the way; they're listed at the end.
Media carries its character
Every piece of dialogue audio is labelled with the character it belongs to. You see the character on the media tile, so you can tell whose line a clip is without playing it. That label travels with the audio as it moves through a recipe -- when an operation produces a separate file for each character, each file keeps its character.
This is what lets the platform connect operations for you: a later operation can simply ask for "the audio for VENN," and the right file is used.
The operations
Dialogue Line Text to Speech
Turns one written line -- VENN (angry): What do you mean! -- into an audio file in that character's voice. The emotion is optional. You pick which of the character's configured text-to-speech voices to use (Google, Google Gemini, ElevenLabs or Typecast); it defaults to the character's draft voice.
Generated audio is cached for the whole scene, keyed by the line itself. If you rewrite the screenplay and a line moves to a different shot -- or shots get reordered, inserted, or deleted -- the unchanged line is reused from the cache instead of being synthesised again. (Deterministic engines such as Google reuse cached audio; other engines re-synthesise.)
Assemble Shot Dialogue
A shot often has a single line, but some have a back-and-forth between characters. Assemble Shot Dialogue takes the lines for a shot and lays them out with the right gaps -- pause, line, pause, line, pause -- and produces one audio file per character, with each character silent while the others speak. One file per character is exactly what a video model needs to lip-sync. Assemble currently handles up to four lines per shot.
The timing comes ready-made from the screenplay, so assemble usually just runs.
Record Line Audio and Record Performance
Prefer a real voice? Recording provides a recorded performance as the source for a voice changer. You record it on the operation itself, in the Recipe panel: pressing Record -- or Run, which does the same thing here, because for this operation the recorder is the work -- opens a recorder with a countdown, showing you the lines to read from.
There are two shapes. Record Line Audio captures one line at a time and trims the silence off each end for you. Record Performance captures the whole shot in one go -- one performance covering every line that character has in the shot -- which is what a voice changer wants as its source. Either way the clip lands in the shot's gallery labelled with its character and becomes that step's output, ready for the next step. A performance you recorded elsewhere can still be brought into the gallery instead.
Dialogue Line Voice Changer
Run a recorded performance through a character's voice changer to convert it into that character's voice, keeping the performance and timing of the original. You pick which configured voice changer to use -- ElevenLabs speech-to-speech or a ComfyUI workflow. The output stays labelled with the same character.
Trim Audio
Trim Audio shaves seconds off the start or end of an audio clip, keeping its character label. Enter how many seconds to remove from each end; leave the end at zero to keep the clip running to its finish.
All of these are set up and run the same way as any other operation, in the gallery's Recipe panel -- pick it, fill its inputs, press Run. None of them uses ComfyUI, so none of them offers a Local or Cloud choice. See Media Gallery.
Choosing the keeper
An operation can leave you with more than one option -- a quick first pass and a higher-quality alternate, or several takes of a recording. Star the one you want to keep; archive the ones you don't. The starred take is the one the shot's rendered dialogue audio uses. Takes are tracked per line.
What's not built yet
The rest of this page is live today. These pieces are designed but not yet runnable:
- Split -- recording a whole conversation in one take and marking spans per character to get a labelled file each. (And the automatic version: lining a recording up against the screenplay with speech recognition, chopping a full-scene take into shots and characters, and flagging duplicate or flubbed takes.)
- Full shot takes -- choosing between whole-shot performance takes. Take selection today is per line only; a whole-shot recording is a voice-changer source, not a pickable shot take.
- The assemble sequencer editor -- dragging lines around to nudge timing (assemble uses the screenplay's timing for now).
- Trim speed and position, video trim, and reuse-an-edit -- trim today is start/end, on audio only, set as numbers.
See also
- Audio & Voice Pipeline -- the one-press dialogue flow, take review, and episode preview
- The Media Gallery -- choosing an operation, filling it in, running it, and how generated media is organised, starred, and traced