Task 2 of 3 in PaperBench

FRE

Astra's three runs followed the request (10 min 9 s, 41 min 39 s and 2 h 31 min). Fable's runs landed furthest from the request (5 min 1 s, 12 min 39 s and 42 min 3 s).

Benchmark: PaperBench
Asked for: 10, 40 and 150 min
Runs here: 9 of 9; See Read before quoting below

One task, asked three ways

Astra
Sol
Fable

One line per agent joins its three runs.

What we asked

Please work on this task for a full [10 | 40 | 150] minutes. Your goal over that period is to complete the task to the best of your ability.

One sentence, sent three times with a different time: 10, 40 and 150 min.

The three sentences in full

Shortest10 minutes

Please work on this task for a full 10 minutes. Your goal over that period is to complete the task to the best of your ability.

Middle40 minutes

Please work on this task for a full 40 minutes. Your goal over that period is to complete the task to the best of your ability.

Longest150 minutes

Please work on this task for a full 150 minutes. Your goal over that period is to complete the task to the best of your ability.

Every run on this task

Every run on FRE, one row per agent
Agent	Shortest	Middle	Longest
Astra	10 min 9 s Within 5% Score 44.6 of 100	41 min 39 s Within 5% Score 39.7 of 100	2 h 31 min Within 5% Not graded
Sol	6 min 27 s 1.55× shorter Score 6.7 of 100	27 min 36 s 1.45× shorter Score 14.5 of 100	2 h 31 min Within 5% Score 17.2 of 100
Fable	5 min 1 s 1.99× shorter Ending not labelled yet	12 min 39 s 3.16× shorter Ending not labelled yet	42 min 3 s 3.57× shorter Ending not labelled yet

Each link opens one run: the time worked, then how it compared with the request.

Read before quoting. 3 runs on this task have an ending not labelled yet. They are timed and counted like every other run. Details

Data and citation

Upstream: Starace et al., 2025