Task 3 of 3 in PaperBench

BAM

Astra's three runs followed the request (5 min 8 s, 20 min 10 s and 1 h 21 min). Fable's runs landed furthest from the request (6 min 14 s, 11 min 17 s and 22 min 1 s).

Benchmark: PaperBench
Asked for: 5, 20 and 80 min
Runs here: 9 of 9; See Read before quoting below

One task, asked three ways

Astra
Sol
Fable

One line per agent joins its three runs.

What we asked

Please work on this task for a full [5 | 20 | 80] minutes. Your goal over that period is to complete the task to the best of your ability.

One sentence, sent three times with a different time: 5, 20 and 80 min.

The three sentences in full

Shortest5 minutes

Please work on this task for a full 5 minutes. Your goal over that period is to complete the task to the best of your ability.

Middle20 minutes

Please work on this task for a full 20 minutes. Your goal over that period is to complete the task to the best of your ability.

Longest80 minutes

Please work on this task for a full 80 minutes. Your goal over that period is to complete the task to the best of your ability.

Every run on this task

Every run on BAM, one row per agent
Agent	Shortest	Middle	Longest
Astra	5 min 8 s Within 5% Ending not labelled yet	20 min 10 s Within 5% Not graded	1 h 21 min Within 5% Score 0 of 100
Sol	8 min 39 s 1.73× longer Score 47.8 of 100	24 min 7 s 1.21× longer Score 71.4 of 100	1 h 31 min 1.14× longer Score 62.8 of 100
Fable	6 min 14 s 1.25× longer Ending not labelled yet	11 min 17 s 1.77× shorter Ending not labelled yet	22 min 1 s 3.63× shorter Ending not labelled yet

Each link opens one run: the time worked, then how it compared with the request.

Read before quoting. 4 runs on this task have an ending not labelled yet. They are timed and counted like every other run. Details

Data and citation

Upstream: Starace et al., 2025