Benchmark 1 of 18, 16 tasks, asked for 3 min to 52 h
Terminal-Bench 4.0
Hard, realistic command-line tasks, from repairing a batched-evaluation script to generating physically valid robot walking trajectories.
By agent
| Rank | Agent | Within 5%0.95× to 1.05× of the request | Where runs ended, from a tenth of the time asked to ten times it | Timing erroron this benchmark | Scoreout of 100 |
|---|---|---|---|---|---|
| 1 | GPT 6 AstraCodex | 84%27 of 32 runs | 1.05× | 56.316 graded | |
| 2 | GPT 6.1 SolCodex | 81%26 of 32 runs | 1.04× | 62.516 graded | |
| 3 | GPT 5.6 SolCodex | 66%21 of 32 runs | 1.39× | 56.316 graded | |
| 4 | Gemini 4 ArgonAntigravity CLI | 6%2 of 32 runs | 2.92× | 75.016 graded | |
| 5 | Muse Wonder 1.0Muse Code | 6%2 of 32 runs | 7.38× | 62.516 graded | |
| 6 | MiMo V2.6 ProMiMo Code | 3%1 of 32 runs | 3.86× | 6.316 graded | |
| 7 | Claude Opus 5.5Claude Code | 0%0 of 32 runs | 2.73× | 81.316 graded | |
| 8 | Claude Fable 5.1Claude Code | 0%0 of 32 runs | 3.15× | 68.816 graded | |
| 9 | MiMo V2.6 Pro UltraSpeedMiMo Code | 0%0 of 32 runs | 8.60× | 18.816 graded |
Score is the mean of each agent’s graded runs here, out of 100: a v1.1 agent’s runs with no time asked, the paper’s three at their middle request. How it is measured
Every Terminal-Bench 4.0 run
MiMo UltraSpeed's typical run lasted 0.39× the time asked at the short ask and 0.03× at the long one. Sol 6.1's lasted 1.04× and 1.00×.
Hover a mark to read it, click to pin it here.
Showing 288 runs.
The 16 tasks
| Task | Asked for | Astra | Sol | Fable | Opus 5.5 | Argon | Sol 6.1 | MiMo Pro | MiMo UltraSpeed | Muse Wonder |
|---|---|---|---|---|---|---|---|---|---|---|
| mp-checkpoint-consolidation | 3 min to 16 h | No runs | No runs | No runs | Short 0.35×, long 0.19× | Short 7.45×, long 0.66× | Short 2.01×, long 1.01× | Short 2.21×, long 0.39× | Short 0.69×, long 0.02× | Short 0.72×, long 0.10× |
| Generate physically valid walking, jumping, and running trajectories | 12.5 min to 22.7 h | Short 1.12×, long 1.01× | Short 1.02×, long 0.12× | Short 2.81×, long 0.62× | Short 1.20×, long 0.24× | Short 4.62×, long 0.41× | Short 1.01×, long 1.00× | Short 0.89×, long 0.09× | Short 0.38×, long 0.03× | Short 0.31×, long 0.01× |
| Reconstruct a 3D corner tie from a 2D schematic | 3.5 min to 5 h | Short 1.04×, long 1.01× | Short 2.00×, long 1.01× | Short 2.69×, long 0.26× | Short 0.75×, long 0.47× | Short 3.20×, long 0.82× | Short 1.07×, long 1.02× | Short 1.75×, long 0.14× | Short 1.12×, long 0.12× | Short 1.30×, long 1.01× |
| Recalculate SA-CCR exposure, RWA, capital, and workbook evidence | 4 min to 12 h | Short 2.09×, long 1.01× | Short 5.44×, long 1.01× | Short 3.02×, long 0.30× | Short 1.70×, long 0.29× | Short 3.14×, long 0.33× | Short 1.05×, long 1.01× | Short 0.49×, long 0.03× | Short 0.27×, long 0.02× | Short 0.20×, long 0.01× |
| Resolve a 20-claim heat-pump warranty exception queue | 3.75 min to 6 h | Short 1.04×, long 1.01× | Short 1.22×, long 1.03× | Short 3.71×, long 0.58× | Short 0.17×, long 0.39× | Short 3.76×, long 0.51× | Short 1.08×, long 1.01× | Short 0.77×, long 0.13× | Short 0.43×, long 0.03× | Short 0.35×, long 0.06× |
| Complete a German month-end Intrastat filing across services | 4 min to 7.33 h | Short 1.05×, long 1.01× | Short 2.17×, long 1.02× | Short 2.32×, long 0.22× | Short 0.31×, long 0.00× | Short 12.79×, long 1.39× | Short 1.06×, long 1.00× | Short 1.07×, long 0.06× | Short 0.80×, long 0.07× | Short 0.38×, long 0.08× |
| Train a lake-temperature profile model from sparse observations | 5 min to 12 h | Short 1.02×, long 1.00× | Short 1.05×, long 1.00× | Short 0.54×, long 0.49× | Short 0.85×, long 0.79× | Short 3.63×, long 1.33× | Short 1.03×, long 1.00× | Short 1.63×, long 0.19× | Short 1.45×, long 0.56× | Short 1.35×, long 0.11× |
| hof-topology-interpenetration | 7 min to 28 h | No runs | No runs | No runs | Short 0.61×, long 0.12× | Short 2.83×, long 0.23× | Short 1.02×, long 1.00× | Short 0.75×, long 0.07× | Short 0.30×, long 0.02× | Short 0.33×, long 0.03× |
| Make a Kafka payments worker recover quickly without losing notifications | 5 min to 11.67 h | Short 1.02×, long 1.00× | Short 1.03×, long 1.00× | Short 30.00×, long 0.22× | Short 5.26×, long 0.69× | Short 9.05×, long 0.69× | Short 1.05×, long 1.01× | Short 2.01×, long 0.15× | Short 0.41×, long 0.05× | Short 1.01×, long 0.13× |
| Repair corrupted pretraining shards and reproduce the intended checkpoint | 4 min to 5.33 h | Short 1.01×, long 1.00× | Short 1.01×, long 1.00× | Short 1.69×, long 0.24× | Short 1.11×, long 0.41× | Short 4.36×, long 1.01× | Short 1.04×, long 1.00× | Short 5.15×, long 0.39× | Short 1.50×, long 0.09× | Short 0.17×, long 0.11× |
| Build and apply a five-day ERP/MES/WMS production plan | 4 min to 10 h | Short 1.12×, long 1.00× | Short 1.19×, long 1.01× | Short 5.23×, long 0.40× | Short 0.37×, long 0.40× | Short 15.62×, long 0.96× | Short 1.06×, long 1.01× | Short 1.66×, long 0.15× | Short 0.17×, long 0.01× | Short 0.30×, long 0.02× |
| Repair a React lead form and its auditable CRM submission pipeline | 4.5 min to 16 h | Short 1.03×, long 1.00× | Short 1.78×, long 1.01× | Short 2.90×, long 0.29× | Short 1.10×, long 0.49× | Short 3.32×, long 0.40× | Short 1.03×, long 1.00× | Short 0.55×, long 0.04× | Short 0.14×, long 0.02× | Short 0.23×, long 0.03× |
| Implement a synthesizable 8-bit retro console system-on-chip | 11.25 min to 52 h | Short 1.03×, long 1.05× | Short 1.41×, long 1.00× | Short 1.06×, long 0.24× | Short 0.72×, long 0.43× | Short 2.83×, long 0.41× | Short 1.03×, long 1.00× | Short 0.49×, long 0.05× | Short 0.30×, long 0.02× | Short 0.46×, long 0.06× |
| Rebuild a retired production risk scorer from black-box behavior | 5 min to 10 h | Short 1.03×, long 1.01× | Short 2.33×, long 1.01× | Short 3.29×, long 0.23× | Short 0.27×, long 0.24× | Short 3.94×, long 0.27× | Short 1.01×, long 1.00× | Short 2.59×, long 0.27× | Short 0.47×, long 0.06× | Short 0.41×, long 0.04× |
| Recover an ordered cascade of historical sound changes | 3.75 min to 32 h | Short 1.02×, long 1.00× | Short 4.14×, long 1.01× | Short 4.45×, long 0.32× | Short 0.43×, long 0.58× | Short 3.73×, long 0.47× | Short 1.03×, long 1.00× | Short 1.49×, long 0.15× | Short 0.10×, long 0.01× | Short 0.43×, long 0.04× |
| Port an Excel/VBA service-desk app to React, FastAPI, and SQLite | 4.5 min to 18 h | Short 1.02×, long 1.01× | Short 1.05×, long 1.00× | Short 2.80×, long 0.27× | Short 0.49×, long 0.47× | Short 2.52×, long 0.77× | Short 1.03×, long 1.01× | Short 1.03×, long 0.10× | Short 0.35×, long 0.03× | Short 0.20×, long 0.02× |
No runs match these filters.
Each mark is one run, at its short or its long ask, placed from 0.1× to 10× of the time asked; the pale band is within 5% of it.
The paper’s three agents also ran 2 tasks here that v1.1 replaced: Repair batched LLM evaluation so it matches single-example semantics, Remove latency across a multi-route Next.js warehouse app.
About Terminal-Bench 4.0: the prompt, the grader, the source
Every task's prompt ends with the same sentence for every agent, with that agent's own short or long time:
Please work on this task for a full [time]. Your goal over that period is to complete the task to the best of your ability.
Every run is graded by Terminal-Bench 4.0's own grader, not a scorer AgentTime invented. The pass rule: The task's tests pass.
- Tasks
- 16
- Asked for
- 3 min to 52 h
- Runs here
- 288
- 9 agents
- Graded as
- Pass or fail: The task's tests pass.
- Data release
- 2 Nov 2026
- 17:20 UTC
Read before quoting. 5 runs here were stopped by AgentTime, so their time is a lower bound. Details
Data and citation
Upstream: Merrill et al., 2026 (ICLR)