Models, Workflows & Recipes

The three building blocks that turn an entity into media -- what each is, how they relate, and how you create them.

Ordinary Animator turns a description into a finished image or clip using three building blocks, from smallest to largest:

  • A model is a single AI model -- the thing that actually generates a picture, a video, or a voice. Examples: Flux, Qwen-Image, or SDXL for images; Wan for video; and LoRAs that add a particular style or character on top of a base model. Each model has its own look and its own hardware appetite.
  • A workflow is one ComfyUI workflow -- a node graph that performs a single generation operation, such as "make a background image" or "animate a still". A workflow uses one or more models.
  • A recipe is an ordered series of operations that reaches a goal -- for example "make a background, place the character on it, then fix the lighting". Running a ComfyUI workflow is the most common kind of operation; other steps are built-in operations (crop an image, resize, merge audio). A recipe also remembers how to feed one step's output into the next, so the whole thing can be reused.

Put simply:

A recipe is a series of operations. A workflow is the most common operation. A workflow uses models.

Three more words you will meet

  • AI Models -- a project view holding the short-list of models you have chosen for this project (see the next section). This was previously labelled "My Models".
  • Cookbook -- where recipes are kept for reuse. See Recipes & the Cookbook.
  • Template -- a read-only starting point you copy. A workflow template is a workflow kept in a library ready to be copied and used; a recipe template is the same idea for recipes. The word always means the same thing: you copy it, then your copy is yours to change. (Mnemonic: the last word says what it is -- a workflow template is a template, like an email template.)

Choosing your models is mostly a hardware decision

The main reason to choose models per project is what your hardware can run. When you generate locally, picking your models -- and their quantization (a compressed build of a model that uses less graphics memory in exchange for a little quality) -- is one of the biggest decisions, driven by your GPU's memory. A model that wants 24 GB will not load on a 12 GB card, but a quantized build of it might. Comfy Cloud keeps a large library of models available, so this hardware question matters most for local generation.

The second reason is a consistent look: reusing the same models across a project keeps characters and style recognisable from shot to shot.

Either way, choosing your AI Models early narrows everything after it -- the recipes and workflows offered later are filtered to the models you picked. Until you choose a set, everything is available and the galleries gently suggest picking one; once you have chosen, the set is a firm filter -- adding a model to it is how you unlock more.

How each building block gets made

You rarely start from scratch, and whatever you make becomes reusable. From most common to rarest:

  1. Pick your AI Models. On the AI Models view you choose the models for the project. The assistant can help: tell it what you are making and whether you generate in the cloud or on your own computer, and it suggests models that fit.
  2. Find a recipe that already fits. Look in the Cookbook for a saved recipe that reaches your goal.
  3. If none fits, build a recipe from operations. Assemble the steps yourself -- ComfyUI workflows (filtered to your chosen models) and built-in operations -- wiring each step's output into the next. When it works, save it to the Cookbook so it is a "find" next time.
  4. If an operation has no workflow, create the workflow. When no existing ComfyUI workflow performs an operation you need, a new one is authored, added to your library, and slotted into the recipe.

Each step falls back to the next only when nothing suitable exists, and everything you create is saved -- so your library and Cookbook grow richer the more you work.

Assistant-guided assembly -- partly built. Everything above works by hand: choose models on the AI Models view, browse the Cookbook, add steps from the operation picker, and use Save as Recipe to capture what worked. The assistant can now help you find what fits: the Suggest button on a media gallery's guide bar asks it to look at what the gallery already has, your chosen models, and your recipes, then recommend the best next step -- with the real settings each model wants. Having it compose a new recipe or author a missing workflow for you is still being built.

Where the prompt comes from

When a workflow step needs a text prompt, you do not have to hand-write one for every model. From what you already captured about the shot -- who is in it, the location, the camera, the mood -- a prompt can be generated for you, phrased the way that step's model prefers (some models want short comma-separated tags, others want a flowing description).

You also never pick the model twice. In a recipe, the model belongs to the generating step: that step's workflow -- chosen from your AI Models -- determines which model runs. A prompt step that feeds it reads the model back through the recipe's wiring and defaults to the same one automatically, so the prompt is phrased for the model that will actually generate. You can still override the prompt step's model if you want a different phrasing.

Model-tailored prompts -- partly built. A built-in prompt generator already produces prompt text you can feed into a step. Tailoring that prompt to the specific model's preferred style, using the model knowledge described below, is the next step.

The model knowledge behind the advice

The assistant's model suggestions and prompts are only as good as what it knows about each model: what look it produces, how much graphics memory it needs, and how it likes to be prompted. That knowledge is gathered per model, so a new model arrives as a new entry rather than a rebuild. See From Storyboard to Video for how this feeds per-shot advice.

Model knowledge base -- being gathered. The look, hardware, and prompting notes for each model are being collected now; wiring them into live suggestions comes next.

Where to go next