How we test
The harness
A model never runs on its own. A program sends it the prompt, holds the session and hands back the answer: that program is the harness. We do not have a harness of our own (yet). Each model runs in its maker's: Claude models in Claude Code, OpenAI models in the Codex CLI, the way people actually use them.
So every result here is a model inside a harness. The same model can do better or worse in another harness, and a new harness version can move the numbers. Every run records the harness, its version and its settings, and shows them on its page.
| Harness | Made by | Versions in the data | How we run it |
|---|---|---|---|
| Claude Code | Anthropic | 2.1.296 | print mode, safe mode, every tool off (--tools ""), no saved session, empty folder outside Git |
| Codex CLI | OpenAI | 0.162.0 | isolated container with no global instructions, read-only sandbox; shell, browser, apps, multi-agent and web off; empty /work folder |
Our limits are never tighter than the model's own: the output limit is the model's maximum, and a run cut short by one of our limits (time or output) is thrown out and run again.
One shot
Every model gets the same prompt in a fresh session, with no tools, no web and no retries. If a model calls any tool, the run is marked invalid and stays on the site.
Hidden checks
A robot (a real browser, offline) opens the file the model wrote and uses it like a person: clicks, drags, waits, reads the clock, drops food in the tank. The model never sees the checks. Every check is written from the prompt, and every version of the checks has to pass a reference solution and fail on purpose-made broken ones before it counts. A score is always out of the same total, even when a page breaks halfway.
Thinking level
Our videos show every model on high reasoning effort. It is not the factory default: Claude Haiku 5.5 and GPT-6 Luna both default to medium. The site has every level we ran, so you can compare any of them.
Three runs
The setups in a video run three times; reference setups run once. We show "x of 3", the range of cost and time, and the median run as the representative one. The rule is fixed before looking at results.
Cost and time
Cost is API-equivalent: tokens from each request multiplied by the official price on the day of the run, even though the runs use subscriptions. Time is the generation time of the answer.
The clips
The videos you compare are not screen recordings of someone using the page. Our robot opens each model's page and plays the same short script for every model: the same clicks at the same instant, the same clock and the same random seed, recorded frame by frame at 30 fps. That is why two clips play in sync, and why a clip says nothing about smoothness: the "fps live" next to it is what the robot measured in a real browser. "Try it live" runs the model's own file in your browser.
What the checks do not judge
Taste. Each test lists what is left to the human eye, and any human score is marked as such.
Credits
- Galton board: Format inspired by BridgeBench's Design Bench (a fixed visual scene, results side by side with cost). Prompt and checks by Nerdict.
- Browser OS: Inspired by Bijan Bowen's "Browser OS Test". Prompt and checks by Nerdict.
- Astronomical clock: Inspired by Brian Moore's AI World Clocks (clocks.brianmoore.com) and Bijan Bowen's clock site. Prompt, reference astronomy (cross-checked with the US Naval Observatory) and checks by Nerdict.
- Aquarium: Inspired by Theo's "Fish Slop" (t3.gg), the aquarium game he runs on every new model, reduced here to one page. Prompt and checks by Nerdict.
- Motion intro: Format inspired by HeyGen and Kaggle's Code2Video Bench (a timed storyboard turned into code and rendered to video), which names where models fail: timing between beats, colliding text and timid layouts. The automatic checks for those failures are Nerdict's. Numbers in the storyboard: official prices and Artificial Analysis cost per task (Oct 7, 2026).