Models, Workflows & Recipes

The three building blocks that turn an entity into media -- what each is, how they relate, and how you create them.

Ordinary Animator turns a description into a finished image or clip using three building blocks, from smallest to largest:

  • A model is a single AI model -- the thing that actually generates a picture, a video, or a voice. Examples: Flux, Qwen-Image, or SDXL for images; Wan for video; and LoRAs that add a particular style or character on top of a base model. Each model has its own look and its own hardware appetite.
  • A workflow is one ComfyUI workflow -- a node graph that performs a single generation operation, such as "make a background image" or "animate a still". A workflow uses one or more models.
  • A recipe is an ordered series of operations that reaches a goal -- for example "make a background, place the character on it, then fix the lighting". Running a ComfyUI workflow is the most common kind of operation; other steps are built-in operations (crop an image, resize, merge audio). A recipe also remembers how to feed one step's output into the next, so the whole thing can be reused.

Put simply:

A recipe is a series of operations. A workflow is the most common operation. A workflow uses models.

Three more words you will meet

  • AI Models -- a project view holding the short-list of models you have chosen for this project (see the next section). This was previously labelled "My Models".
  • Cookbook -- your project's own collection of recipes. See Recipes & the Cookbook.
  • Library -- a collection of workflows you can copy into a project. Ordinary Animator provides one; your local ComfyUI and Comfy Cloud are others. Copying a workflow in is what makes it available to your recipes -- much like turning on a model.

Everything in a library already runs

There is no such thing as a half-finished workflow waiting to be filled in. Every workflow in every library is complete and ready to run, which is why you can copy one in and try it immediately. When a workflow is nearly what you want -- the right idea but the wrong style model, say -- it is adapted into another working workflow rather than left with blanks.

Workflows are also kept deliberately small and single-purpose, one stage of the job each. That is what lets you see every intermediate result, keep the good ones, and re-run just the stage you are unhappy with instead of starting over.

Choosing your models is mostly a hardware decision

The main reason to choose models per project is what your hardware can run. When you generate locally, picking your models -- and their quantization (a compressed build of a model that uses less graphics memory in exchange for a little quality) -- is one of the biggest decisions, driven by your GPU's memory. A model that wants 24 GB will not load on a 12 GB card, but a quantized build of it might. Comfy Cloud keeps a large library of models available, so this hardware question matters most for local generation.

The second reason is a consistent look: reusing the same models across a project keeps characters and style recognisable from shot to shot.

Either way, choosing your AI Models early narrows everything after it -- the recipes and workflows offered later are filtered to the models you picked. Until you choose a set, everything is available and the galleries gently suggest picking one; once you have chosen, the set is a firm filter -- adding a model to it is how you unlock more.

How each building block gets made

You rarely start from scratch, and whatever you make becomes reusable. From most common to rarest:

  1. Pick your AI Models. On the AI Models view you choose the models for the project. The assistant can help: tell it what you are making and whether you generate in the cloud or on your own computer, and it suggests models that fit.
  2. Find a recipe that already fits. A gallery suggests recipes that suit what you are looking at, and your project's own recipes are there alongside the built-in ones.
  3. If none fits, build a recipe from operations. Assemble the steps yourself -- ComfyUI workflows (filtered to your chosen models) and built-in operations -- wiring each step's output into the next.
  4. If an operation has no workflow, bring one in or adapt one. Copy a workflow from a library into your project, or start from one that is close and have it adapted to what you need.

Each step falls back to the next only when nothing suitable exists, so your project grows richer the more you work.

You can also start from the other end. Browsing workflows first is perfectly reasonable: find one that does what you want, and turn on the models it needs. Turning a model on works the other way too -- it brings the workflows that use it to the surface.

Every one of those routes works by hand, and the assistant can do each of them for you. It will find what fits -- the Suggest button on a media gallery's guide bar asks it to look at what the gallery already has, your chosen models, and your recipes, then recommend the best next step, with the real settings each model wants. It will compose a recipe: describe the result you want and it builds the steps on the gallery you are looking at, wires each step to the one before, and fills in what it can (see Asking the Assistant to build one). And it will adapt a workflow to run where you need it -- swapping a model for a smaller build, and the loader nodes that build needs, so a workflow that was too big for your graphics card can run locally, or one built on your machine can run on Comfy Cloud. It proposes the adapted workflow; the platform checks it before it is allowed to run.

Where the prompt comes from

When a workflow step needs a text prompt, you do not have to hand-write one for every model. From what you already captured about the shot -- who is in it, the location, the camera, the mood -- a prompt can be generated for you, phrased the way that step's model prefers (some models want short comma-separated tags, others want a flowing description).

You also never pick the model twice. In a recipe, the model belongs to the generating step: that step's workflow -- chosen from your AI Models -- determines which model runs. A prompt step that feeds it reads the model back through the recipe's wiring and defaults to the same one automatically, so the prompt is phrased for the model that will actually generate. You can still override the prompt step's model if you want a different phrasing.

The tailoring is not cosmetic. Where a step asks for its prompt to be rewritten, the text is put through the prompt conventions recorded for the model that step will actually run, so the same shot yields a short tag list for one model and a flowing description for another. When no model can be worked out from what the step is bound to, the prompt is left exactly as you wrote it rather than phrased for a guess.

The model knowledge behind the advice

The assistant's model suggestions and prompts are only as good as what it knows about each model: what look it produces, how much graphics memory it needs, and how it likes to be prompted. That knowledge is gathered per model, so a new model arrives as a new entry rather than a rebuild. See From Storyboard to Video for how this feeds per-shot advice.

Those notes ship with the platform and the assistant reads them when it recommends -- along with a verdict on whether each model fits the graphics card you told it about, so it can say "this one needs the cloud" instead of recommending something that will not load.

Where to go next