Task 13 of 24 in Humanity's Last Exam

Humanity's Last Exam item 13

Astra's three runs followed the request (1 min 21 s, 5 min 12 s and 20 min 8 s). Fable worked 1 min 39 s, 1 min 26 s and 1 min 43 s whatever it was asked.

Benchmark: Humanity's Last Exam; Upstream id 67249d57d91756473725533a
Asked for: 1.25, 5 and 20 min
Runs here: 9 of 9

One task, asked three ways

Astra
Sol
Fable

One line per agent joins its three runs.

What we asked

Please work on this task for a full [1.25 | 5 | 20] minutes. Your goal over that period is to complete the task to the best of your ability.

One sentence, sent three times with a different time: 1.25, 5 and 20 min.

The three sentences in full

Shortest1.25 minutes

Please work on this task for a full 1.25 minutes. Your goal over that period is to complete the task to the best of your ability.

Middle5 minutes

Please work on this task for a full 5 minutes. Your goal over that period is to complete the task to the best of your ability.

Longest20 minutes

Please work on this task for a full 20 minutes. Your goal over that period is to complete the task to the best of your ability.

Every run on this task

Every run on Humanity's Last Exam item 13, one row per agent
Agent	Shortest	Middle	Longest
Astra	1 min 21 s 1.09× longer Did not pass, 0 of 100	5 min 12 s Within 5% Did not pass, 0 of 100	20 min 8 s Within 5% Did not pass, 0 of 100
Sol	1 min 26 s 1.15× longer Did not pass, 0 of 100	5 min 16 s 1.05× longer Did not pass, 0 of 100	20 min 13 s Within 5% Did not pass, 0 of 100
Fable	1 min 39 s 1.32× longer Did not pass, 0 of 100	1 min 26 s 3.49× shorter Did not pass, 0 of 100	1 min 43 s 11.63× shorter Did not pass, 0 of 100

Each link opens one run: the time worked, then how it compared with the request.

Data and citation

Upstream: Phan et al., 2025