Comparing agents
The experiment page, the runs list, and what the numbers are actually saying.
The experiment page
It answers two questions in the order people ask them.
Is it working? — four tiles across the top: runs settled out of runs created, pass rate, average duration of a finished run, and total spend with a token count.
For whom? — when more than one agent is in the experiment, a bar per agent showing its pass rate and the raw fraction behind it. Underneath, the table: one row per run, with the task, the agent, attempts used, status, score, duration and cost. Click any row to open the run.
The subtitle under the title says how the experiment was configured — when it was launched, whether a schedule launched it, the machine shape, the trials and the concurrency — so the page always explains its own numbers.
What you can do from here
| Action | What it does |
|---|---|
| Rename | A label, changed inline. It is not a decision, so it does not get a dialog. |
| Edit & re-run | Opens the whole configuration in the builder, ready to change. Launching from there makes a new experiment; this one keeps its results. |
| Schedule | Copies this configuration into a new schedule. |
| Cancel | Queued runs stop immediately; ones already in progress stop within a few seconds. Available while it is queued or running. |
| Delete | Removes the experiment and its runs. Not available while it is running. |
While it is queued or running, Attempts each and At once stay editable and apply to runs that have not finished.
All runs
The All runs page is every run in the project, whichever experiment produced it, newest first — with a filter chip per status carrying its own count, and a search over task and agent names. It is the fastest way to answer “what is happening right now” and “what has failed today”.
Reading the numbers honestly
Pass rate needs a rubric behind it
Runs judged with no rubric have no score, and a pass there means only that the agent’s process exited cleanly. A 100% pass rate on an experiment with no criteria is not a result.
Attempts are part of the finding
A run that shows 3/3 attempts passed on its third try. That is a different result from passing first time, and the column is there so the difference is visible rather than averaged away.
Errored is not failed
failed means the agent was judged and did not meet the rubric. errored
means the run did not complete — a missing credential, a clone that failed,
a timeout. Counting the second as evidence about the agent is the most
common way to misread a comparison.
One task is an anecdote too
Two agents on one task tells you which one got lucky. The cross product is cheap for a reason: give them five tasks and read the bars.
The home page
The dashboard is a summary of the same things: what is running now, the totals across the project, anything missing before the next launch — no connected agent, no tasks, no assets — and the most recent experiments and runs.