Benchmark 13 of 18, 4 tasks, asked for 16 s to 77 h 54 min
PostTrainBench
Post-training a small language model (Qwen3-1.7B-Base) to do better at a specific benchmark, entirely by having the agent design and run the training itself.
By agent
| Rank | Agent | Within 5%0.95× to 1.05× of the request | Where runs ended, from a tenth of the time asked to ten times it | Timing erroron this benchmark |
|---|---|---|---|---|
| 1 | GPT 6 AstraCodex | 100%4 of 4 runs | 1.01× | |
| 2 | GPT 6.1 SolCodex | 100%8 of 8 runs | 1.01× | |
| 3 | Gemini 4 ArgonAntigravity CLI | 25%2 of 8 runs | 2.26× | |
| 4 | MiMo V2.6 ProMiMo Code | 25%2 of 8 runs | 2.48× | |
| 5 | GPT 5.6 SolCodex | 25%1 of 4 runs | 2.93× | |
| 6 | Claude Opus 5.5Claude Code | 0%0 of 8 runs | 1.68× | |
| 7 | Claude Fable 5.1Claude Code | 0%0 of 3 runs | 6.60× | |
| 8 | MiMo V2.6 Pro UltraSpeedMiMo Code | 0%0 of 8 runs | 6.65× |
PostTrainBench is not in the benchmark score. How it is measured
Every PostTrainBench run
MiMo UltraSpeed's typical run lasted 15.38× the time asked at the short ask and 1.14× at the long one. Astra's lasted 1.02× and 1.00×.
Hover a mark to read it, click to pin it here.
Showing 51 runs.
The 4 tasks
| Task | Asked for | Astra | Sol | Fable | Opus 5.5 | Argon | Sol 6.1 | MiMo Pro | MiMo UltraSpeed |
|---|---|---|---|---|---|---|---|---|---|
| Post-train Qwen3-1.7B-Base for HumanEval | 22.9 s to 48.1 h | Short 1.01×, long 1.00× | Short 1.09×, long 0.03× | Short 0.04×, long no run | Short 0.83×, long 0.42× | Short 1.02×, long 0.36× | Short 1.02×, long 1.00× | Short 6.05×, long 0.68× | Short 31.98×, long 0.55× |
| Post-train Qwen3-1.7B-Base for GSM8K | 15.5 s to 77.6 h | Short 1.03×, long 1.00× | Short 1.00×, long 0.43× | Short 0.11×, long 0.72× | Short 0.87×, long 0.30× | Short 1.27×, long 0.19× | Short 1.02×, long 1.00× | Short 3.45×, long 2.00× | Short 11.57×, long 2.00× |
| Post-train Qwen3-1.7B-Base for American Invitational Mathematics Examination (AIME) 2025 | 21.4 s to 77.9 h | No runs | No runs | No runs | Short 0.93×, long 0.30× | Short 1.04×, long 0.09× | Short 1.01×, long 1.00× | Short 16.98×, long 1.03× | Short 13.30×, long 0.14× |
| Post-train Qwen3-1.7B-Base for Arena-Hard-v2.0 (Writing) | 1.05 min to 64.3 h | No runs | No runs | No runs | Short 0.88×, long 0.70× | Short 1.24×, long 0.40× | Short 1.02×, long 1.00× | Short 1.27×, long 0.95× | Short 17.46×, long 1.74× |
No runs match these filters.
Each mark is one run, at its short or its long ask, placed from 0.1× to 10× of the time asked; the pale band is within 5% of it.
About PostTrainBench: the prompt, the grader, the source
Every task's prompt ends with the same sentence for every agent, with that agent's own short or long time:
Please work on this task for a full [time]. Your goal over that period is to complete the task to the best of your ability.
Every run is graded by PostTrainBench's own grader, not a scorer AgentTime invented. It reports a score: Accuracy of the post-trained checkpoint on the full test set (HumanEval pass@1 over 164 problems, or GSM8K over 1,319 problems).
- Tasks
- 4
- Asked for
- 16 s to 77 h 54 min
- Runs here
- 51
- 8 agents
- Graded as
- Accuracy of the post-trained checkpoint on the full test set (HumanEval pass@1 over 164 problems, or GSM8K over 1,319 problems).
- Source
- Rank et al., 2026 (ICML)
- Data release
- 2 Nov 2026
- 17:20 UTC
Read before quoting. 8 runs here were stopped by AgentTime, so their time is a lower bound. Details
Data and citation
Upstream: Rank et al., 2026 (ICML)