Tests

5 visual tasks. The same prompt for every model, one shot, checked by a robot. Open a test to see every result running.

Galton boardPhysics simulation12 setups · 15 checks

Browser OSInteractive app12 setups · 19 checks

Astronomical clockReal-world data12 setups · 14 checks

AquariumSimulation and interaction12 setups · 15 checks

Motion introMotion design in code12 setups · 17 checks

Every setup

all 5 tests · middle run of each

sort

SetupChecksCostTime

GPT-6 Lunahigh3 runs79/80$0.0287m50s

Haiku 5.5medium3 runs79/80$0.1418m48s

Haiku 5.5high3 runs79/80$0.2128m23s

GPT-6.1 Solmedium1 run79/80$0.6118m31s

Sonnet 5.5high1 run79/80$3.4139m01s

Opus 5.5medium1 run79/80$5.2738m39s

Haiku 5.5low1 run78/80$0.1115m07s

GPT-6.1 Solhigh1 run78/80$0.8230m27s

Opus 5.5high1 run78/80$9.4061m03s

GPT-6 Lunamedium3 runs77/80$0.0246m49s

GPT-6 Lunamax3 runs77/80$0.06015m01s

Haiku 5.5max1 run9/80$1.25153m16s

Cost and time tracks: a log scale from the cheapest (or fastest) setup on the left to the priciest (or slowest) on the right. Cost is API-equivalent. Thinking level = how long the model may reason before answering.

Every run, one by one110 runs, including the broken ones.→