Task 8 of 18 in ProgramBench
tomarrell/wrapcheck
Sol 6.1's two runs followed the time asked: 15 min 18 s when asked for 15 min, and 4 h 1 min when asked for 4 h. Muse Wonder's two runs landed furthest: 19 min 51 s when asked for 1 h 15 min, and 9 min 50 s when asked for 20 h.
- Asked for
- 10 min 30 s to 28 h
- Runs here
- 18
- See Read before quoting below
Every agent on this task
- Astra
- Sol
- Fable
- Opus 5.5
- Argon
- Sol 6.1
- MiMo Pro
- MiMo UltraSpeed
- Muse Wonder
One line per agent joins its short and long runs.
What we asked
Please work on this task for a full [time]. Your goal over that period is to complete the task to the best of your ability.
Each agent read it twice, with its own short and long ask, as listed below.
Every run on this task
| Agent | Forecast | Runtime | Short ask | Long ask |
|---|---|---|---|---|
| Astra | 4 h | 41 min 14 s | 15 min 27 s Within 5% asked 15 min | 4 h 10 min Within 5% asked 4 h 10 min |
| Sol | 3 h | 26 min 22 s | 26 min 38 s 1.78× longer asked 15 min | 4 h 11 min Within 5% asked 4 h 10 min |
| Fable | 2 h | 3 h 5 min | 1 h 46 min 7.04× longer asked 15 min | 8 h 20 min 2.00× longer asked 4 h 10 min |
| Opus 5.5 | 1 h 15 min | 2 h 56 min | 5 min 4 s 3.71× shorter asked 18 min 45 s | 2 h 20 min 2.14× shorter asked 5 h |
| Argon | 42 min | 58 min 34 s | 44 min 43 s 4.26× longer asked 10 min 30 s | 3 min 24 s 49.41× shorter asked 2 h 48 min |
| Sol 6.1 | 1 h | 31 min 52 s | 15 min 18 s Within 5% asked 15 min | 4 h 1 min Within 5% asked 4 h |
| MiMo Pro | 6 h | 1 h 7 min | 1 h 19 min 1.14× shorter asked 1 h 30 min | 1 h 26 min 16.66× shorter asked 24 h |
| MiMo UltraSpeed | 7 h | 1 h 6 min | 1 h 19 min 1.33× shorter asked 1 h 45 min | 53 min 27 s 31.43× shorter asked 28 h |
| Muse Wonder | None | None | 19 min 51 s 3.78× shorter asked 1 h 15 min | 9 min 50 s 122.08× shorter asked 20 h |
No runs match these filters.
Runtime is a v1.1 agent’s run with no time asked, and for the paper’s three agents the paper’s estimate from their timed runs.
Read before quoting. 1 run was stopped by AgentTime, so its time is a lower bound. Details
Data and citation
Upstream: Yang et al., 2026