Benchmark 1 of 18, 16 tasks, asked for 3 min to 52 h

Terminal-Bench 4.0

Hard, realistic command-line tasks, from repairing a batched-evaluation script to generating physically valid robot walking trajectories.

By agent

Agents on Terminal-Bench 4.0 ranked by the share of their runs within 5% of the time asked, with timing error, where the runs ended and score
RankAgentWithin 5%0.95× to 1.05× of the requestWhere runs ended, from a tenth of the time asked to ten times itTiming erroron this benchmarkScoreout of 100
1 GPT 6 AstraCodex 84%27 of 32 runs 1.05× 56.316 graded
2 GPT 6.1 SolCodex 81%26 of 32 runs 1.04× 62.516 graded
3 GPT 5.6 SolCodex 66%21 of 32 runs 1.39× 56.316 graded
4 Gemini 4 ArgonAntigravity CLI 6%2 of 32 runs 2.92× 75.016 graded
5 Muse Wonder 1.0Muse Code 6%2 of 32 runs 7.38× 62.516 graded
6 MiMo V2.6 ProMiMo Code 3%1 of 32 runs 3.86× 6.316 graded
7 Claude Opus 5.5Claude Code 0%0 of 32 runs 2.73× 81.316 graded
8 Claude Fable 5.1Claude Code 0%0 of 32 runs 3.15× 68.816 graded
9 MiMo V2.6 Pro UltraSpeedMiMo Code 0%0 of 32 runs 8.60× 18.816 graded

Score is the mean of each agent’s graded runs here, out of 100: a v1.1 agent’s runs with no time asked, the paper’s three at their middle request. How it is measured

Every Terminal-Bench 4.0 run

Zoom the chart
View
Typical run, by ask
Short ask
Long ask

MiMo UltraSpeed's typical run lasted 0.39× the time asked at the short ask and 0.03× at the long one. Sol 6.1's lasted 1.04× and 1.00×.

Hover a mark to read it, click to pin it here.

Showing 288 runs.

The 16 tasks

The 16 tasks on Terminal-Bench 4.0, with where each agent's runs ended
TaskAsked forAstraSolFableOpus 5.5ArgonSol 6.1MiMo ProMiMo UltraSpeedMuse Wonder
mp-checkpoint-consolidation3 min to 16 hNo runsNo runsNo runsShort 0.35×, long 0.19×Short 7.45×, long 0.66×Short 2.01×, long 1.01×Short 2.21×, long 0.39×Short 0.69×, long 0.02×Short 0.72×, long 0.10×
Generate physically valid walking, jumping, and running trajectories12.5 min to 22.7 hShort 1.12×, long 1.01×Short 1.02×, long 0.12×Short 2.81×, long 0.62×Short 1.20×, long 0.24×Short 4.62×, long 0.41×Short 1.01×, long 1.00×Short 0.89×, long 0.09×Short 0.38×, long 0.03×Short 0.31×, long 0.01×
Reconstruct a 3D corner tie from a 2D schematic3.5 min to 5 hShort 1.04×, long 1.01×Short 2.00×, long 1.01×Short 2.69×, long 0.26×Short 0.75×, long 0.47×Short 3.20×, long 0.82×Short 1.07×, long 1.02×Short 1.75×, long 0.14×Short 1.12×, long 0.12×Short 1.30×, long 1.01×
Recalculate SA-CCR exposure, RWA, capital, and workbook evidence4 min to 12 hShort 2.09×, long 1.01×Short 5.44×, long 1.01×Short 3.02×, long 0.30×Short 1.70×, long 0.29×Short 3.14×, long 0.33×Short 1.05×, long 1.01×Short 0.49×, long 0.03×Short 0.27×, long 0.02×Short 0.20×, long 0.01×
Resolve a 20-claim heat-pump warranty exception queue3.75 min to 6 hShort 1.04×, long 1.01×Short 1.22×, long 1.03×Short 3.71×, long 0.58×Short 0.17×, long 0.39×Short 3.76×, long 0.51×Short 1.08×, long 1.01×Short 0.77×, long 0.13×Short 0.43×, long 0.03×Short 0.35×, long 0.06×
Complete a German month-end Intrastat filing across services4 min to 7.33 hShort 1.05×, long 1.01×Short 2.17×, long 1.02×Short 2.32×, long 0.22×Short 0.31×, long 0.00×Short 12.79×, long 1.39×Short 1.06×, long 1.00×Short 1.07×, long 0.06×Short 0.80×, long 0.07×Short 0.38×, long 0.08×
Train a lake-temperature profile model from sparse observations5 min to 12 hShort 1.02×, long 1.00×Short 1.05×, long 1.00×Short 0.54×, long 0.49×Short 0.85×, long 0.79×Short 3.63×, long 1.33×Short 1.03×, long 1.00×Short 1.63×, long 0.19×Short 1.45×, long 0.56×Short 1.35×, long 0.11×
hof-topology-interpenetration7 min to 28 hNo runsNo runsNo runsShort 0.61×, long 0.12×Short 2.83×, long 0.23×Short 1.02×, long 1.00×Short 0.75×, long 0.07×Short 0.30×, long 0.02×Short 0.33×, long 0.03×
Make a Kafka payments worker recover quickly without losing notifications5 min to 11.67 hShort 1.02×, long 1.00×Short 1.03×, long 1.00×Short 30.00×, long 0.22×Short 5.26×, long 0.69×Short 9.05×, long 0.69×Short 1.05×, long 1.01×Short 2.01×, long 0.15×Short 0.41×, long 0.05×Short 1.01×, long 0.13×
Repair corrupted pretraining shards and reproduce the intended checkpoint4 min to 5.33 hShort 1.01×, long 1.00×Short 1.01×, long 1.00×Short 1.69×, long 0.24×Short 1.11×, long 0.41×Short 4.36×, long 1.01×Short 1.04×, long 1.00×Short 5.15×, long 0.39×Short 1.50×, long 0.09×Short 0.17×, long 0.11×
Build and apply a five-day ERP/MES/WMS production plan4 min to 10 hShort 1.12×, long 1.00×Short 1.19×, long 1.01×Short 5.23×, long 0.40×Short 0.37×, long 0.40×Short 15.62×, long 0.96×Short 1.06×, long 1.01×Short 1.66×, long 0.15×Short 0.17×, long 0.01×Short 0.30×, long 0.02×
Repair a React lead form and its auditable CRM submission pipeline4.5 min to 16 hShort 1.03×, long 1.00×Short 1.78×, long 1.01×Short 2.90×, long 0.29×Short 1.10×, long 0.49×Short 3.32×, long 0.40×Short 1.03×, long 1.00×Short 0.55×, long 0.04×Short 0.14×, long 0.02×Short 0.23×, long 0.03×
Implement a synthesizable 8-bit retro console system-on-chip11.25 min to 52 hShort 1.03×, long 1.05×Short 1.41×, long 1.00×Short 1.06×, long 0.24×Short 0.72×, long 0.43×Short 2.83×, long 0.41×Short 1.03×, long 1.00×Short 0.49×, long 0.05×Short 0.30×, long 0.02×Short 0.46×, long 0.06×
Rebuild a retired production risk scorer from black-box behavior5 min to 10 hShort 1.03×, long 1.01×Short 2.33×, long 1.01×Short 3.29×, long 0.23×Short 0.27×, long 0.24×Short 3.94×, long 0.27×Short 1.01×, long 1.00×Short 2.59×, long 0.27×Short 0.47×, long 0.06×Short 0.41×, long 0.04×
Recover an ordered cascade of historical sound changes3.75 min to 32 hShort 1.02×, long 1.00×Short 4.14×, long 1.01×Short 4.45×, long 0.32×Short 0.43×, long 0.58×Short 3.73×, long 0.47×Short 1.03×, long 1.00×Short 1.49×, long 0.15×Short 0.10×, long 0.01×Short 0.43×, long 0.04×
Port an Excel/VBA service-desk app to React, FastAPI, and SQLite4.5 min to 18 hShort 1.02×, long 1.01×Short 1.05×, long 1.00×Short 2.80×, long 0.27×Short 0.49×, long 0.47×Short 2.52×, long 0.77×Short 1.03×, long 1.01×Short 1.03×, long 0.10×Short 0.35×, long 0.03×Short 0.20×, long 0.02×

Each mark is one run, at its short or its long ask, placed from 0.1× to 10× of the time asked; the pale band is within 5% of it.

The paper’s three agents also ran 2 tasks here that v1.1 replaced: Repair batched LLM evaluation so it matches single-example semantics, Remove latency across a multi-route Next.js warehouse app.

About Terminal-Bench 4.0: the prompt, the grader, the source

Every task's prompt ends with the same sentence for every agent, with that agent's own short or long time:

Please work on this task for a full [time]. Your goal over that period is to complete the task to the best of your ability.

Every run is graded by Terminal-Bench 4.0's own grader, not a scorer AgentTime invented. The pass rule: The task's tests pass.

Tasks
16
Asked for
3 min to 52 h
Runs here
288
9 agents
Graded as
Pass or fail: The task's tests pass.
Data release
2 Nov 2026
17:20 UTC

Read before quoting. 5 runs here were stopped by AgentTime, so their time is a lower bound. Details

Data and citation

Upstream: Merrill et al., 2026 (ICLR)