Benchmark 13 of 18, 4 tasks, asked for 16 s to 77 h 54 min

PostTrainBench

Post-training a small language model (Qwen3-1.7B-Base) to do better at a specific benchmark, entirely by having the agent design and run the training itself.

By agent

Agents on PostTrainBench ranked by the share of their runs within 5% of the time asked, with timing error, where the runs ended and score
RankAgentWithin 5%0.95× to 1.05× of the requestWhere runs ended, from a tenth of the time asked to ten times itTiming erroron this benchmark
1 GPT 6 AstraCodex 100%4 of 4 runs 1.01×
2 GPT 6.1 SolCodex 100%8 of 8 runs 1.01×
3 Gemini 4 ArgonAntigravity CLI 25%2 of 8 runs 2.26×
4 MiMo V2.6 ProMiMo Code 25%2 of 8 runs 2.48×
5 GPT 5.6 SolCodex 25%1 of 4 runs 2.93×
6 Claude Opus 5.5Claude Code 0%0 of 8 runs 1.68×
7 Claude Fable 5.1Claude Code 0%0 of 3 runs 6.60×
8 MiMo V2.6 Pro UltraSpeedMiMo Code 0%0 of 8 runs 6.65×

PostTrainBench is not in the benchmark score. How it is measured

Every PostTrainBench run

Zoom the chart
View
Typical run, by ask
Short ask
Long ask

MiMo UltraSpeed's typical run lasted 15.38× the time asked at the short ask and 1.14× at the long one. Astra's lasted 1.02× and 1.00×.

Hover a mark to read it, click to pin it here.

Showing 51 runs.

The 4 tasks

The 4 tasks on PostTrainBench, with where each agent's runs ended
TaskAsked forAstraSolFableOpus 5.5ArgonSol 6.1MiMo ProMiMo UltraSpeed
Post-train Qwen3-1.7B-Base for HumanEval22.9 s to 48.1 hShort 1.01×, long 1.00×Short 1.09×, long 0.03×Short 0.04×, long no runShort 0.83×, long 0.42×Short 1.02×, long 0.36×Short 1.02×, long 1.00×Short 6.05×, long 0.68×Short 31.98×, long 0.55×
Post-train Qwen3-1.7B-Base for GSM8K15.5 s to 77.6 hShort 1.03×, long 1.00×Short 1.00×, long 0.43×Short 0.11×, long 0.72×Short 0.87×, long 0.30×Short 1.27×, long 0.19×Short 1.02×, long 1.00×Short 3.45×, long 2.00×Short 11.57×, long 2.00×
Post-train Qwen3-1.7B-Base for American Invitational Mathematics Examination (AIME) 202521.4 s to 77.9 hNo runsNo runsNo runsShort 0.93×, long 0.30×Short 1.04×, long 0.09×Short 1.01×, long 1.00×Short 16.98×, long 1.03×Short 13.30×, long 0.14×
Post-train Qwen3-1.7B-Base for Arena-Hard-v2.0 (Writing)1.05 min to 64.3 hNo runsNo runsNo runsShort 0.88×, long 0.70×Short 1.24×, long 0.40×Short 1.02×, long 1.00×Short 1.27×, long 0.95×Short 17.46×, long 1.74×

Each mark is one run, at its short or its long ask, placed from 0.1× to 10× of the time asked; the pale band is within 5% of it.

About PostTrainBench: the prompt, the grader, the source

Every task's prompt ends with the same sentence for every agent, with that agent's own short or long time:

Please work on this task for a full [time]. Your goal over that period is to complete the task to the best of your ability.

Every run is graded by PostTrainBench's own grader, not a scorer AgentTime invented. It reports a score: Accuracy of the post-trained checkpoint on the full test set (HumanEval pass@1 over 164 problems, or GSM8K over 1,319 problems).

Tasks
4
Asked for
16 s to 77 h 54 min
Runs here
51
8 agents
Graded as
Accuracy of the post-trained checkpoint on the full test set (HumanEval pass@1 over 164 problems, or GSM8K over 1,319 problems).
Data release
2 Nov 2026
17:20 UTC

Read before quoting. 8 runs here were stopped by AgentTime, so their time is a lower bound. Details

Data and citation

Upstream: Rank et al., 2026 (ICML)