Task 3 of 20 in Humanity's Last Exam
Humanity's Last Exam item 27
Muse Wonder's two runs came closest to the time asked: 6 min 4 s when asked for 1 min, and 14 min 27 s when asked for 16 min. Argon's two runs landed furthest: 38 min 34 s when asked for 18 s, and 14 min 44 s when asked for 4 min 48 s.
- Asked for
- 18 s to 48 min
- Runs here
- 12
- 6 agents
Every agent on this task
- Opus 5.5
- Argon
- Sol 6.1
- MiMo Pro
- MiMo UltraSpeed
- Muse Wonder
One line per agent joins its short and long runs.
What we asked
Please work on this task for a full [time]. Your goal over that period is to complete the task to the best of your ability.
Each agent read it twice, with its own short and long ask, as listed below.
Every run on this task
| Agent | Forecast | Runtime | Short ask | Long ask |
|---|---|---|---|---|
| Opus 5.5 | 6 min | 2 min 44 s | 32 min 31 s 21.67× longer asked 1 min 30 s | 25 min 59 s 1.08× longer asked 24 min |
| Argon | 1 min 12 s | 6 min 40 s | 38 min 34 s 128.54× longer asked 18 s | 14 min 44 s 3.07× longer asked 4 min 48 s |
| Sol 6.1 | 3 min | 12 min 8 s | 6 min 15 s 8.33× longer asked 45 s | 12 min 12 s Within 5% asked 12 min |
| MiMo Pro | 12 min | 1 h 25 min | 1 h 9 min 22.98× longer asked 3 min | 1 h 23 min 1.74× longer asked 48 min |
| MiMo UltraSpeed | 11 min | 1 min 23 s | 6 min 43 s 2.44× longer asked 2 min 45 s | 7 min 25 s 5.93× shorter asked 44 min |
| Muse Wonder | 4 min | 6 min 25 s | 6 min 4 s 6.06× longer asked 1 min | 14 min 27 s 1.11× shorter asked 16 min |
No runs match these filters.
Runtime is each agent’s run with no time asked.
Data and citation
Upstream: Phan et al., 2025