Benchmark 16 of 18, 20 tasks, asked for 5 s to 8 h
Humanity's Last Exam
A very hard, wide-ranging exam of expert-level questions across dozens of academic subjects, built to still be difficult once older benchmarks were saturated.
By agent
| Rank | Agent | Within 5%0.95× to 1.05× of the request | Where runs ended, from a tenth of the time asked to ten times it | Timing erroron this benchmark | Scoreout of 100 |
|---|---|---|---|---|---|
| 1 | GPT 6 AstraCodex | 50%24 of 48 runs | 1.09× | 79.224 graded | |
| 2 | GPT 5.6 SolCodex | 48%23 of 48 runs | 1.28× | 70.824 graded | |
| 3 | GPT 6.1 SolCodex | 43%17 of 40 runs | 1.61× | 55.020 graded | |
| 4 | MiMo V2.6 Pro UltraSpeedMiMo Code | 13%5 of 40 runs | 5.56× | 5.020 graded | |
| 5 | Claude Fable 5.1Claude Code | 6%3 of 48 runs | 3.04× | 87.524 graded | |
| 6 | Muse Wonder 1.0Muse Code | 5%2 of 40 runs | 2.44× | 35.020 graded | |
| 7 | Claude Opus 5.5Claude Code | 5%2 of 40 runs | 2.61× | 63.219 graded | |
| 8 | Gemini 4 ArgonAntigravity CLI | 5%2 of 40 runs | 3.19× | 65.020 graded | |
| 9 | MiMo V2.6 ProMiMo Code | 5%2 of 40 runs | 3.91× | 5.020 graded |
Score is the mean of each agent’s graded runs here, out of 100: a v1.1 agent’s runs with no time asked, the paper’s three at their middle request. How it is measured
Every Humanity's Last Exam run
MiMo UltraSpeed's typical run lasted 0.82× the time asked at the short ask and 0.07× at the long one. Astra's lasted 1.13× and 1.01×.
Hover a mark to read it, click to pin it here.
Showing 384 runs.
The 20 tasks
| Task | Asked for | Opus 5.5 | Argon | Sol 6.1 | MiMo Pro | MiMo UltraSpeed | Muse Wonder |
|---|---|---|---|---|---|---|---|
| Item 25 | 7.5 s to 1 h | Short 3.58×, long 1.47× | Short 8.61×, long 1.29× | Short 2.06×, long 1.04× | Short 1.04×, long 0.17× | Short 0.04×, long 0.01× | Short 1.66×, long 0.18× |
| Item 26 | 30 s to 1.67 h | Short 3.14×, long 0.21× | Short 9.90×, long 1.07× | Short 2.22×, long 1.01× | Short 8.67×, long 0.83× | Short 1.04×, long 0.14× | Short 3.47×, long 0.40× |
| Item 27 | 18 s to 48 min | Short 21.67×, long 1.08× | Short 128.54×, long 3.07× | Short 8.33×, long 1.02× | Short 22.98×, long 1.74× | Short 2.44×, long 0.17× | Short 6.06×, long 0.90× |
| Item 28 | 37.5 s to 1.33 h | Short 4.45×, long 1.40× | Short 5.68×, long 1.08× | Short 1.18×, long 1.02× | Short 8.53×, long 0.47× | Short 1.02×, long 0.08× | Short 2.59×, long 0.37× |
| Item 29 | 7.5 s to 20 min | Short 4.64×, long 1.34× | Short 6.51×, long 1.22× | Short 3.05×, long 1.05× | Short 4.00×, long 0.12× | Short 0.20×, long 0.04× | Short 3.19×, long 0.29× |
| Item 30 | 52.5 s to 2.33 h | Short 1.11×, long 0.97× | Short 1.56×, long 1.04× | Short 3.95×, long 1.01× | Short 7.77×, long 0.57× | Short 0.63×, long 0.09× | Short 1.17×, long 0.17× |
| Item 31 | 52.5 s to 3 h | Short 5.61×, long 0.98× | Short 1.33×, long 3.34× | Short 4.98×, long 1.01× | Short 6.23×, long 0.43× | Short 0.77×, long 0.05× | Short 0.80×, long 0.40× |
| Item 32 | 37.5 s to 8 h | Short 3.76×, long 1.25× | Short 3.82×, long 1.09× | Short 1.57×, long 1.02× | Short 1.22×, long 0.09× | Short 0.21×, long 0.03× | Short 3.17×, long 0.13× |
| Item 33 | 52.5 s to 8 h | Short 5.19×, long 1.21× | Short 4.18×, long 0.97× | Short 2.26×, long 1.01× | Short 7.71×, long 0.83× | Short 0.51×, long 0.02× | Short 3.57×, long 2.36× |
| Item 34 | 37.5 s to 3 h | Short 3.30×, long 1.09× | Short 7.33×, long 1.08× | Short 1.67×, long 1.03× | Short 3.24×, long 0.45× | Short 0.88×, long 0.03× | Short 6.53×, long 0.28× |
| Item 35 | 7.5 s to 20 min | Short 3.49×, long 1.18× | Short 4.84×, long 1.19× | Short 3.05×, long 1.07× | Short 20.01×, long 0.22× | Short 0.17×, long 0.02× | Short 2.25×, long 0.51× |
| Item 36 | 22.5 s to 48 min | Short 1.91×, long 1.06× | Short 4.79×, long 1.10× | Short 1.55×, long 1.02× | Short 9.54×, long 1.04× | Short 0.75×, long 0.23× | Short 2.27×, long 0.58× |
| Item 37 | 6 s to 20 min | Short 4.52×, long 1.16× | Short 75.15×, long 1.16× | Short 3.55×, long 1.05× | Short 12.55×, long 2.82× | Short 0.90×, long 0.06× | Short 2.85×, long 0.40× |
| Item 38 | 6 s to 4 min | Short 20.28×, long 1.99× | Short 17.02×, long 1.55× | Short 2.45×, long 1.12× | Short 7.16×, long 0.51× | Short 0.34×, long 0.02× | Short 3.05×, long 0.91× |
| Item 39 | 6 s to 12 min | Short 2.39×, long 1.49× | Short 4.93×, long 1.62× | Short 1.65×, long 1.03× | Short 6.67×, long 3.94× | Short 0.98×, long 1.18× | Short 2.36×, long 0.57× |
| Item 40 | 7.5 s to 24 min | Short 9.89×, long 1.91× | Short 20.19×, long 1.18× | Short 2.78×, long 1.04× | Short 3.45×, long 0.88× | Short 0.55×, long 0.03× | Short 1.00×, long 0.77× |
| Item 41 | 4.5 s to 8 min | Short 5.82×, long 1.53× | Short 7.07×, long 1.11× | Short 2.61×, long 1.09× | Short 7.45×, long 1.93× | Short 1.79×, long 0.18× | Short 6.07×, long 1.12× |
| Item 42 | 10.5 s to 16 min | Short 2.35×, long 1.26× | Short 3.44×, long 1.11× | Short 1.43×, long 1.02× | Short 5.10×, long 1.96× | Short 2.66×, long 0.16× | Short 1.65×, long 0.53× |
| Item 43 | 12 s to 12 min | Short 10.17×, long 2.15× | Short 7.50×, long 1.07× | Short 3.02×, long 1.04× | Short 35.15×, long 0.57× | Short 0.98×, long 0.09× | Short 1.89×, long 0.41× |
| Item 44 | 7.5 s to 28 min | Short 7.64×, long 1.87× | Short 16.11×, long 1.25× | Short 2.81×, long 1.05× | Short 2.65×, long 1.65× | Short 1.02×, long 0.12× | Short 0.98×, long 0.28× |
No runs match these filters.
Each mark is one run, at its short or its long ask, placed from 0.1× to 10× of the time asked; the pale band is within 5% of it.
The paper’s three agents also ran 24 tasks here that v1.1 replaced: Item 1, Item 2, Item 3, Item 4, Item 5, Item 6, Item 7, Item 8, Item 9, Item 10, Item 11, Item 12, Item 13, Item 14, Item 15, Item 16, Item 17, Item 18, Item 19, Item 20, Item 21, Item 22, Item 23, Item 24.
About Humanity's Last Exam: the prompt, the grader, the source
Every task's prompt ends with the same sentence for every agent, with that agent's own short or long time:
Please work on this task for a full [time]. Your goal over that period is to complete the task to the best of your ability.
Every run is graded by Humanity's Last Exam's own grader, not a scorer AgentTime invented. The pass rule: The benchmark's own judge marks the answer correct.
- Tasks
- 20
- Asked for
- 5 s to 8 h
- Runs here
- 384
- 9 agents
- Graded as
- Pass or fail: The benchmark's own judge marks the answer correct.
- Source
- Phan et al., 2025
- Data release
- 2 Nov 2026
- 17:20 UTC
Read before quoting. 2 runs here were stopped by AgentTime, so their time is a lower bound. Details
Data and citation
Upstream: Phan et al., 2025