Task 10 of 18 in AppWorld
Collect every cable bill, normalize filenames, ZIP them, and reply to the request
Astra's three runs landed closest to the request (2 min 42 s, 10 min 39 s and 41 min). Fable worked 5 min 6 s, 5 min 59 s and 5 min whatever it was asked.
- Asked for
- 2.5, 10 and 40 min
- Runs here
- 9 of 9
One task, asked three ways
- Astra
- Sol
- Fable
One line per agent joins its three runs.
What we asked
Please work on this task for a full [2.5 | 10 | 40] minutes. Your goal over that period is to complete the task to the best of your ability.
One sentence, sent three times with a different time: 2.5, 10 and 40 min.
The three sentences in full
Shortest2.5 minutes
Please work on this task for a full 2.5 minutes. Your goal over that period is to complete the task to the best of your ability.
Middle10 minutes
Please work on this task for a full 10 minutes. Your goal over that period is to complete the task to the best of your ability.
Longest40 minutes
Please work on this task for a full 40 minutes. Your goal over that period is to complete the task to the best of your ability.
Every run on this task
| Agent | Shortest | Middle | Longest |
|---|---|---|---|
| Astra | 2 min 42 s 1.08× longer Did not pass, 0 of 100 | 10 min 39 s 1.07× longer Passed, 100 of 100 | 41 min Within 5% Passed, 100 of 100 |
| Sol | 2 min 58 s 1.19× longer Passed, 100 of 100 | 10 min 28 s Within 5% Passed, 100 of 100 | 40 min 59 s Within 5% Passed, 100 of 100 |
| Fable | 5 min 6 s 2.04× longer Passed, 100 of 100 | 5 min 59 s 1.67× shorter Passed, 100 of 100 | 5 min 8.00× shorter Passed, 100 of 100 |
No runs match these filters.
Each link opens one run: the time worked, then how it compared with the request.
Data and citation
Upstream: Trivedi et al., 2024 (ACL)