AgentTime

Task 2 of 2 in PostTrainBench v1.1

Post-train Qwen3-1.7B-Base for GSM8K

Astra's three runs followed the request (2 h 3 min, 8 h 1 min and 30 h 1 min). Fable's runs landed furthest from the request (13 min 43 s, 54 min 48 s and 21 h 40 min).

Benchmark
PostTrainBench v1.1
Upstream id qwen3-1.7b-base__gsm8k
Asked for
2 h, 8 h and 30 h
Runs here
9 of 9
See Read before quoting below

One task, asked three ways

  • Astra
  • Sol
  • Fable

One line per agent joins its three runs.

What we asked

Please work on this task for a full [2 | 8 | 30] hours. Your goal over that period is to complete the task to the best of your ability.

One sentence, sent three times with a different time: 2 h, 8 h and 30 h.

The three sentences in full

Shortest2 hours

Please work on this task for a full 2 hours. Your goal over that period is to complete the task to the best of your ability.

Middle8 hours

Please work on this task for a full 8 hours. Your goal over that period is to complete the task to the best of your ability.

Longest30 hours

Please work on this task for a full 30 hours. Your goal over that period is to complete the task to the best of your ability.

Every run on this task

AgentShortestMiddleLongest
Astra 2 h 3 min Within 5% Stopped by an error 8 h 1 min Within 5% Score 84.1 of 100 30 h 1 min Within 5% Not graded
Sol 2 h Within 5% Score 57.6 of 100 8 h 1 min Within 5% Score 61.6 of 100 12 h 55 min 2.32× shorter Stopped by an error
Fable 13 min 43 s 8.75× shorter Score 0 of 100 54 min 48 s 8.76× shorter Stopped by a provider limit 21 h 40 min 1.38× shorter Ending not labelled yet

Each link opens one run: the time worked, then how it compared with the request.

Read before quoting: 2 notes
  1. 3 runs were stopped by AgentTime, so their time is a lower bound. Details
  2. 1 run on this task has an ending not labelled yet. It is timed and counted like every other run. Details

Data and citation

Upstream: Rank et al., 2026 (ICML)