AgentTime

Task 3 of 12 in Agents' Last Exam

Parse a fixed 100-filing SEC 10-K corpus

Astra's three runs landed closest to the request (12 min 51 s, 40 min 14 s and 2 h 32 min). Fable's runs landed furthest from the request (39 min 29 s, 27 min 30 s and 1 h 6 min).

Benchmark
Agents' Last Exam
Upstream id business_finance/sec_10k_financial_parsing
Asked for
10, 40 and 150 min
Runs here
9 of 9

One task, asked three ways

  • Astra
  • Sol
  • Fable

One line per agent joins its three runs.

What we asked

Please work on this task for a full [10 | 40 | 150] minutes. Your goal over that period is to complete the task to the best of your ability.

One sentence, sent three times with a different time: 10, 40 and 150 min.

The three sentences in full

Shortest10 minutes

Please work on this task for a full 10 minutes. Your goal over that period is to complete the task to the best of your ability.

Middle40 minutes

Please work on this task for a full 40 minutes. Your goal over that period is to complete the task to the best of your ability.

Longest150 minutes

Please work on this task for a full 150 minutes. Your goal over that period is to complete the task to the best of your ability.

Every run on this task

AgentShortestMiddleLongest
Astra 12 min 51 s 1.29× longer Did not pass, 92 of 100 40 min 14 s Within 5% Did not pass, 91.2 of 100 2 h 32 min Within 5% Did not pass, 91.5 of 100
Sol 22 min 30 s 2.25× longer Did not pass, 86 of 100 42 min 15 s 1.06× longer Did not pass, 69.7 of 100 2 h 31 min Within 5% Did not pass, 67.3 of 100
Fable 39 min 29 s 3.95× longer Did not pass, 92 of 100 27 min 30 s 1.45× shorter Did not pass, 92.4 of 100 1 h 6 min 2.28× shorter Did not pass, 92.4 of 100

Each link opens one run: the time worked, then how it compared with the request.

Data and citation

Upstream: Sun et al., 2026