Experiments
A question with an answer -- try models on your own artwork without filling up a working gallery.
An Experiment is a question with an answer.
"Which background remover keeps the ink lines on George?" is a question you can only answer by trying two or three on your own artwork. Doing that in George's own gallery works once -- and then his gallery is a pile of trial outputs nobody wants to keep, and the good picture is somewhere in the middle of it.
An experiment is the place that work belongs. It has its own Media Gallery, it belongs to the project rather than to any one character or shot, and it is meant to be deleted once you know the answer.
Subjects -- what it is about
An experiment is about one or more subjects: George, shot 12, the mug. George is not its parent -- he is its subject. That is why the same experiment can be about a character and a shot at once, and why the answer it produces is useful everywhere, not just on the page you started from.
The first subject is the primary one, and it supplies the context a trial runs with: a trial about George reads George's description and his look, so what comes out is representative rather than generic.
An experiment can also be about nothing in particular -- "which upscaler do I like" is a fair question with no subject at all.
New experiment from selection -- how one starts
An experiment starts from the pictures you want to work on. In any gallery -- a character, a location, a prop, a scene, a shot, an episode or the project's own -- select the tiles you care about and choose New experiment from selection. You are asked for a title, the question it answers, and what it is about (the gallery's own entity is filled in for you), and then the experiment opens holding every tile you had selected.
Those tiles are links, not copies: the originals stay exactly where they are, and deleting the experiment later takes nothing away from them.
The Links menu's New experiment is the same gesture from the other direction, and it brings your selection along too. Either way you land in a workspace holding your own artwork, ready to try things on it.
Try -- inside the experiment
Try lives in the experiment, and only there. Inside one, select the input you want to work on, choose an operation, and every model listed under it offers Try beside it: it runs that model on your selected input, with the operation's own default settings, into this experiment.
Try the next model and it joins the same experiment, on the same input. That is what makes two tries a comparison rather than two separate piles. Try a different operation and that joins it too -- an experiment is a workspace, not one operation's trial set. (Choosing Start an experiment in a model's build choice, described below, also opens a workspace of this kind.)
Try changes nothing. It is not the same gesture as picking the model row, which runs the operation here AND remembers that model as this project's choice for the operation. Try remembers nothing. Trying is not choosing.
In an ordinary gallery the picker offers only the model rows themselves: picking one runs the operation there and remembers it. There is no Try, because a trial belongs in a workspace you opened for it rather than in the gallery you are trying to keep tidy.
If nothing is selected, Try says so: a trial runs on your own artwork, so it needs a picture.
Or just ask for the job, and let the Assistant offer one
You do not have to know that an experiment is the answer. Ask the Assistant for the work itself -- "remove the background from this" -- and it looks up which model your project uses for that operation. If there is one, it just uses it and says which: "using BiRefNet, your project default for background removal." Nothing is re-opened and nothing is asked.
If there is no model for that operation yet, it does not guess one and it does not hand you a list of names to judge blind. It proposes an experiment in a couple of lines -- the question, the picture, and two to four models it can actually run for you, each with where it would run and what it would cost. Then it waits. Nothing runs until you say yes.
Say yes and it does the rest: it creates the experiment from what you had selected, opens it, and runs every candidate on those pictures, one result per model. Then it asks you which one wins. It never decides that for you, and your character's own gallery is untouched throughout.
Once you are standing in an experiment you can keep asking there. "Try every remove background workflow you can find" lists the ones it can actually run, says where each runs, runs all of them on your selected inputs, and ends by asking which won. It never starts a second experiment while you are in one.
See the Assistant for the whole of that conversation.
Try the recommended set
Above an operation's model list, Try the recommended set runs the top few recommended models that can actually run for you -- up to four -- on the same input, all into this experiment. It asks once first, listing each model and where it will run, with the price beside any that has one.
Where a trial runs, and why
A trial goes to Comfy Cloud whenever the model is available there and you have Cloud switched on, even if you normally work on your own machine. It runs on your PC only when that model's files are already installed.
That is deliberate. Trying a model should not cost you a fourteen-gigabyte download to find out whether you like the result -- open-source trials in the cloud cost pennies, and if a model wins you can install it afterwards. Try never offers you an install.
When a model can run in neither place -- Cloud is switched off, the workflow needs something Comfy Cloud does not have, or the files are not installed here or do not fit your card -- Try is still listed, greyed out, and its hint says why, in the same words the model's row uses (for example Cloud: switch Cloud on in Settings).
The assistant follows the same rule. A trial is a trial whoever starts it, so when you ask the assistant to try something in an experiment it goes to the same place Try would have sent it -- the Local/Cloud switch on the Perform operation panel is about your ordinary production runs and does not change where a trial goes. The assistant says where each trial ran when it reports back, and if a model can run in neither place it tells you that instead of running it somewhere else.
The Try button never shows a price. When you try several models at once, the confirmation lists each paid provider model's measured price; open-source cloud trials show no figure, because a fraction of a penny is not a decision.
Choosing a default -- when the experiment was started to pick one
Some experiments exist to answer one question: which workflow should this project use for this operation with this model? You get one when you pick a model that this project has not run before and choose Start an experiment instead of using it straight away. The experiment opens titled "Choose the Remove Background (BiRefNet) default", and the Experiment button holds the card that makes the choice.
That card names one specific workflow -- the current recommendation. It starts as the one you had selected when you began, and it shows what the build is, where it runs for you, and how many results it has made in this experiment. Underneath it says who recommended it: Recommended by you, or Recommended by the assistant with the reason.
Beside it is a list of every workflow that does that same operation with that same model -- each one, whether it came with OA, you imported it, or you saved a snapshot of it. Each entry starts with where it lives -- OA Catalog, Project Library, or Project snapshot and the day it was taken -- and adds the file name only when two would otherwise read the same. Choose a different one and it becomes the recommendation. Nothing is decided by choosing: the project is untouched until you approve.
Workflows you added to the project before OA started recording what each one is for join that list the first time you open the Workflows page: OA reads each saved workflow's own description of itself and files it under the operation and model it names. Nothing is changed about the workflow itself -- it simply starts appearing beside the other ways of doing the same job.
- Try runs the recommended workflow on your selected picture, into this experiment. With more than one, Try all runs every one that can run for you.
- Make <workflow> the default is the approval. It names the workflow it will set, so what you are agreeing to is on the button -- and it may not be the one you started with. Approving sets that workflow as this project's default for the pair and records the experiment as Decided.
While that choice is open the approve button takes Decide's place: Decide is the general way an experiment ends, and this experiment already knows what its decision is about.
The options are always for the same operation and model -- the pair in the title. A different model is a different question, and a different experiment.
The Assistant can move the recommendation
Ask it to try the builds and it can run them all here, look at the results, and recommend one with a reason -- "the alpha matte one is the only one that keeps the ink lines". That changes the recommendation, so the button then offers its choice. It still cannot approve it: only your click, or telling it plainly to use that one, sets the default. If you pick one yourself from the list, the Assistant sees your pick and works from that.
Trying things by hand
Inside an experiment, everything works exactly as it does in any other gallery: choose an operation, pick your inputs, run it. The difference is only where the results land. Run the same operation four times with different settings and you get four tiles here and nothing anywhere else.
Inputs are links to media in the gallery you started from, so the original is never moved or copied. Whatever you do in an experiment, the character, location, prop, scene or shot it is about is untouched.
Comparing -- reading two tries side by side
Every result a trial produces is captioned with the model that made it. The Experiment button at the top of the gallery opens the actions that need the gallery beside them: Keep, Decide, and what has been tried, grouped by operation and then by model: "Tried: Remove Background: BiRefNet (2), RMBG 2.0 (1); Upscale: SUPIR (1)". The experiment's title, question, status, what it is about and its conclusion are on its Properties page, the row above Media in the sidebar's Experiment group. That group has three rows -- All experiments, Properties and Media -- and All experiments takes you back to the project's list.
So the comparison is just the gallery, with the model names under the pictures. There is no separate comparison screen to learn, and everything a gallery can do -- fullscreen, zoom, side-by-side selection, the Assistant -- works on trial results too.
Keep -- saving the one that worked
When a result is good enough to use, Keep in
Decide -- writing down the answer
Decide (in the Experiment dialog) asks for one line: "BiRefNet keeps the ink lines." That line is the whole point of the experiment -- it is what you will want six weeks later, when the question comes up again on a different character.
Decide also asks which model won. It lists every operation and model you tried in the experiment, with how many results each produced, plus Don't set a default. If you tried only one, it is already chosen.
Starring a tile inside an experiment is the shortcut: the star says "this one won", so it opens Decide with that result's model already chosen.
And it asks which look the answer belongs to. Once your project has more than one visual style, Decide offers them. An experiment run to settle one scene's look should not quietly change every scene's, which is why it asks rather than assuming.
Deciding with a model in it also makes that model the look's answer for the operation. It is the same decision, so it is one gesture: the Perform operation panel then names the look and the build under that operation, and the model sits at the top of the list next time you open the picker. You never have to go back and pick it again.
Each choice names its operation, so you can see which answer you are about to set. Try an upscaler in the same experiment, choose it, and it settles your upscaler; the background remover you chose earlier is left exactly as it was. One workspace can answer several questions without them treading on each other.
Decide with Don't set a default -- a verdict about settings rather than about models -- records the line alone and leaves your looks exactly as they were.
The experiment then reads Decided on the Experiments list.
You can also let the Assistant do the looking. Select the results inside the experiment and ask it which one wins: it compares the actual pictures against the experiment's own question, gives you a verdict on each, and offers to record the answer. Say yes and the experiment is decided exactly as it would be by hand -- same line, same model, same Decided. See the Assistant for what it can and cannot judge.
Delete -- and why that is safe
A decided experiment is meant to be deleted. Deleting it removes the question, its trial media and the files it produced.
It never removes anything you kept. Files are stored by their content, so a result kept in George's gallery is the same file as the tile it came from -- and that file is left exactly where it is. Delete the experiment and George's picture still opens.
When to reach for one
- You have heard of two models and want to see which suits your artwork.
- An operation is not giving you what you want and you are about to try six variations of the prompt.
- You want to compare the same operation across three characters before committing to it.
For work that belongs to the project but is not a question -- a scratch gallery you keep coming back to -- use the project's Media page instead.