---
title: Rubrics
description: The standard a run is measured against, written once and pointed at.
---

A rubric is a name and a list of criteria. It lives in your library rather than
on a task, because "the diff is minimal and the tests prove it" is a standard you
want to apply to twenty different tasks — and two copies of a standard are two
standards.

## Writing one

On **Rubrics**, press *New rubric*. Give it a name short enough to pick from a
list, then one check per box. Enter on the last box adds another.

```text title="Flake is actually gone"
The checkout spec passes twenty runs in a row
No bare timeout or sleep was added to the spec
The diff touches only the spec and the code it covers
```

Each criterion is scored on its own, so each one should be a single fact. See
[Rubrics and judging](/concepts/judging) for what the evaluator can and cannot
establish, and how to phrase a criterion so it can fail.

## How it reaches a run

A rubric is chosen **on the experiment**, at launch — not on a task. At that
moment its criteria are read once and copied onto every run in the experiment.

> **Note**
>
> This is why editing a rubric never rewrites history. Past runs keep the exact
> criteria they were judged against, and the Rubrics page keeps the current
> wording for the next launch.

Leaving the rubric as **None** is a valid choice: the run is then judged only on
whether the agent's own process succeeded, has no score, and needs no evaluator
key.

## Reading the page

Each rubric card lists its criteria in order, which experiments have used it, and
how many runs it has judged. Any rubric no experiment has used yet is called out
underneath the list, with a link to the builder — an unused rubric is usually one
somebody wrote and then forgot to pick.