Benchmarks › τ-bench
τ-bench
Resolve a simulated customer issue using the allowed tools
50.7top score · Grok 4.6
134models ranked
84open-weights models
Pass@1 success ratehigher is better
Oct 20, 2026last updated
How the test works
1
Task
97 tasks
- What the model doesResolve a simulated customer issue using the allowed tools
- Kinds of tasksDual-control customer support
- In → outText → Tool calls, dialogue
2
Run
5 runs per task
- Agent / harnessUpstream tau2-bench harness
- Step limit200 steps
- Token budgetNon-reasoning: up to 16,384 output tokens, adjusted to model limits; reasoning: model-specific maximum
3
Score
Pass@1 (solved on the first try) success rate
- Graded byBackend world/database-state checks
- Better isHigher
Sources: artificialanalysis.ai
τ³-Banking leaderboardtop 10 of 134
Full leaderboard with filters →| Model | τ³-Banking | Per task | Price | Speed | Supports | Context |
|---|---|---|---|---|---|---|
50.7high | $1.48 | $2 / $6 | 65 | 500K | ||
50.5max | $1.60 | $1.25 / $4.25 | 128 | 1M | ||
50.3max | $2.01 | $1.40 / $4.40 | 186 | 1M | ||
49.1thinking | $2.16 | $2 / $6 | 127 | 262K | ||
48xhigh | $1.01 | $0.42 / $2.55 | 114 | 262K | ||
47.8thinking | $5.41 | $2 / $6 | 40 | 984K | ||
47.2max | $7.63 | $10 / $50 | 69 | 1M | ||
47.2max | $0.28 | $0.15 / $0.50 | 179 | 1M | ||
46max | $2.00 | $3 / $15 | 112 | 1M | ||
45.8medium | $0.93 | $0.75 / $3.75 | — | 1M |
Versions and boards on Model Menu
| Board | Models |
|---|---|
τ³-BankingShown above | 134 |
| 21 |
τ-bench questions
What is τ-bench?›
Resolve a simulated customer issue using the allowed tools. It has 97 tasks. Created by Sierra.
Which model is best on τ-bench?›
Grok 4.6 leads τ³-Banking with 50.7, ahead of Muse Spark 1.3 (50.5) and GLM 5.3 (50.3).
What is the best open-weights model on τ-bench?›
GLM 5.3 with 50.3, then Qwen3.8 2.4T A95B (49.1).
How is τ-bench scored?›
Pass@1 (solved on the first try) success rate; backend world/database-state checks; 5 runs per task per task.
Which versions of τ-bench are there?›
τ³-Banking (134 models), τ³-bench · Banking 1.0.1 (21 models).