Task 2 of 12 in Agents' Last Exam
Complete an end-to-end Odoo supply-chain workflow
Astra's three runs followed the request (5 min 15 s, 20 min 42 s and 1 h 21 min). Sol worked 1 h 41 min, 2 h 11 min and 2 h 50 min whatever it was asked.
- Asked for
- 5, 20 and 80 min
- Runs here
- 9 of 9
- See Read before quoting below
One task, asked three ways
- Astra
- Sol
- Fable
One line per agent joins its three runs.
What we asked
Please work on this task for a full [5 | 20 | 80] minutes. Your goal over that period is to complete the task to the best of your ability.
One sentence, sent three times with a different time: 5, 20 and 80 min.
The three sentences in full
Shortest5 minutes
Please work on this task for a full 5 minutes. Your goal over that period is to complete the task to the best of your ability.
Middle20 minutes
Please work on this task for a full 20 minutes. Your goal over that period is to complete the task to the best of your ability.
Longest80 minutes
Please work on this task for a full 80 minutes. Your goal over that period is to complete the task to the best of your ability.
Every run on this task
| Agent | Shortest | Middle | Longest |
|---|---|---|---|
| Astra | 5 min 15 s Within 5% Did not pass, 0 of 100 | 20 min 42 s Within 5% Did not pass, 20 of 100 | 1 h 21 min Within 5% Did not pass, 92 of 100 |
| Sol | 1 h 41 min 20.27× longer Did not pass, 88 of 100 | 2 h 11 min 6.55× longer Did not pass, 80 of 100 | 2 h 50 min 2.13× longer Stopped by an error |
| Fable | 1 h 25 min 17.01× longer Did not pass, 96 of 100 | 26 min 5 s 1.30× longer Did not pass, 12 of 100 | 1 h 12 min 1.11× shorter Did not pass, 88 of 100 |
No runs match these filters.
Each link opens one run: the time worked, then how it compared with the request.
Read before quoting. 1 run was stopped by AgentTime, so its time is a lower bound. Details
Data and citation
Upstream: Sun et al., 2026