Models, Workflows & Recipes

The three building blocks that turn an entity into media -- what each is, how they relate, and how you create them.

Ordinary Animator turns a description into a finished image or clip using three building blocks, from smallest to largest:

  • A model is a single AI model -- the thing that actually generates a picture, a video, or a voice. Examples: Flux, Qwen-Image, or SDXL for images; Wan for video; and LoRAs that add a particular style or character on top of a base model. Each model has its own look and its own hardware appetite.
  • A workflow is one ComfyUI workflow -- a node graph that performs a single generation operation, such as "make a background image" or "animate a still". A workflow uses one or more models.
  • A recipe is an ordered series of operations that reaches a goal -- for example "make a background, place the character on it, then fix the lighting". Running a ComfyUI workflow is the most common kind of operation; other steps are built-in operations (crop an image, resize, merge audio). A recipe also remembers how to feed one step's output into the next, so the whole thing can be reused.

Put simply:

A recipe is a series of operations. A workflow is the most common operation. A workflow uses models.

Three more words you will meet

  • AI Models -- a project view holding the short-list of models you have chosen for this project (see the next section). This was previously labelled "My Models".
  • Cookbook -- your project's own collection of recipes. See Recipes & the Cookbook.
  • Library -- a collection of workflows you can copy into a project. Ordinary Animator provides one; your local ComfyUI and Comfy Cloud are others. Copying a workflow in is what makes it available to your recipes -- much like turning on a model.

Everything in a library already runs

There is no such thing as a half-finished workflow waiting to be filled in. Every workflow in every library is complete and ready to run, which is why you can copy one in and try it immediately. When a workflow is nearly what you want -- the right idea but the wrong style model, say -- it is adapted into another working workflow rather than left with blanks.

Workflows are also kept deliberately small and single-purpose, one stage of the job each. That is what lets you see every intermediate result, keep the good ones, and re-run just the stage you are unhappy with instead of starting over.

Choosing your models is mostly a hardware decision

The main reason to choose models per project is what your hardware can run. When you generate locally, picking your models -- and their quantization (a compressed build of a model that uses less graphics memory in exchange for a little quality) -- is one of the biggest decisions, driven by your GPU's memory. A model that wants 24 GB will not load on a 12 GB card, but a quantized build of it might. Comfy Cloud keeps a large library of models available, so this hardware question matters most for local generation.

The second reason is a consistent look: reusing the same models across a project keeps characters and style recognisable from shot to shot.

Either way, choosing your AI Models early narrows everything after it -- the recipes and workflows offered later are filtered to the models you picked. Until you choose a set, everything is available and the galleries gently suggest picking one; once you have chosen, the set is a firm filter -- adding a model to it is how you unlock more.

How each building block gets made

You rarely start from scratch, and whatever you make becomes reusable. From most common to rarest:

  1. Pick your AI Models. On the AI Models view you choose the models for the project. The assistant can help: tell it what you are making and whether you generate in the cloud or on your own computer, and it suggests models that fit.
  2. Find a recipe that already fits. A gallery suggests recipes that suit what you are looking at, and your project's own recipes are there alongside the built-in ones.
  3. If none fits, build a recipe from operations. Assemble the steps yourself -- ComfyUI workflows (filtered to your chosen models) and built-in operations -- wiring each step's output into the next.
  4. If an operation has no workflow, bring one in or adapt one. Copy a workflow from a library into your project, or start from one that is close and have it adapted to what you need.

Each step falls back to the next only when nothing suitable exists, so your project grows richer the more you work.

You can also start from the other end. Browsing workflows first is perfectly reasonable: find one that does what you want, and turn on the models it needs. Turning a model on works the other way too -- it brings the workflows that use it to the surface.

Assistant-guided assembly -- partly built. Everything above works by hand: choose models on the AI Models view, browse your recipes, and add steps from the operation picker. The assistant can now help you find what fits: the Suggest button on a media gallery's guide bar asks it to look at what the gallery already has, your chosen models, and your recipes, then recommend the best next step -- with the real settings each model wants. Having it compose a new recipe or adapt a workflow for you is still being built.

Where the prompt comes from

When a workflow step needs a text prompt, you do not have to hand-write one for every model. From what you already captured about the shot -- who is in it, the location, the camera, the mood -- a prompt can be generated for you, phrased the way that step's model prefers (some models want short comma-separated tags, others want a flowing description).

You also never pick the model twice. In a recipe, the model belongs to the generating step: that step's workflow -- chosen from your AI Models -- determines which model runs. A prompt step that feeds it reads the model back through the recipe's wiring and defaults to the same one automatically, so the prompt is phrased for the model that will actually generate. You can still override the prompt step's model if you want a different phrasing.

Model-tailored prompts -- partly built. A built-in prompt generator already produces prompt text you can feed into a step. Tailoring that prompt to the specific model's preferred style, using the model knowledge described below, is the next step.

The model knowledge behind the advice

The assistant's model suggestions and prompts are only as good as what it knows about each model: what look it produces, how much graphics memory it needs, and how it likes to be prompted. That knowledge is gathered per model, so a new model arrives as a new entry rather than a rebuild. See From Storyboard to Video for how this feeds per-shot advice.

Model knowledge base -- being gathered. The look, hardware, and prompting notes for each model are being collected now; wiring them into live suggestions comes next.

Where to go next