Benchmarks › AA Analyst Agent
57.5top score · Claude Fable 5.1
43models ranked
15open-weights models
Pass^5 reliabilityhigher is better
Oct 20, 2026last updated
How the test works
1
Task
80 tasks
- What the model doesAnalyze data with Python and provide a final answer
- Kinds of tasksAgentic data analysis
- In → outText, data files → Answer
2
Run
5 runs per task
- Token budgetNon-reasoning: up to 16,384 output tokens, adjusted to model limits; reasoning: model-specific maximum
- Temperature0 for non-reasoning; 0.6 for reasoning, unless the model lab recommends otherwise
- How runs combinePass^5: all five independent attempts must be correct
3
Score
Pass^5 reliability
- Better isHigher
Sources: artificialanalysis.ai
AA Analyst Agent leaderboardtop 10 of 43
Full leaderboard with filters →| Model | AA Analyst Agent | Per task | Price | Speed | Supports | Context |
|---|---|---|---|---|---|---|
57.5max | $7.63 | $10 / $50 | 69 | 1M | ||
57.5max | $5.46 | $2 / $10 | 137 | 1M | ||
56.3max | $5.98 | $4 / $20 | 96 | 1M | ||
55high | $2.73 | $2 / $6 | 87 | 500K | ||
53.8max | $0.21 | $0.10 / $0.50 | 234 | 1M | ||
53.8max | $5.86 | $5 / $25 | 55 | 1M | ||
51.3max | $3.26 | $10 / $50 | 69 | 1.05M | ||
50xhigh | $2.63 | $5 / $30 | 113 | 1.05M | ||
50max | $0.72 | $2 / $10 | 52 | 1.05M | ||
48.8max | $8.75 | $10 / $50 | 67 | 1M |
AA Analyst Agent questions
What is AA Analyst Agent?›
Analyze data with Python and provide a final answer. It has 80 tasks. Created by Artificial Analysis.
Which model is best on AA Analyst Agent?›
Claude Fable 5.1 leads AA Analyst Agent with 57.5, ahead of Claude Sonnet 5.5 (57.5) and Claude Opus 5.5 (56.3).
What is the best open-weights model on AA Analyst Agent?›
Kimi K3 with 38.8, then GLM 5.3 (38.8).
How is AA Analyst Agent scored?›
Pass^5 reliability; 5 runs per task per task.