Benchmarks › Chartography
71.6top score · Gemini 4 Argon
37models ranked
8open-weights models
Mean pass@1higher is better
Oct 20, 2026last updated
How the test works
1
Task
100 chart questions
- What the model doesInterpret specialized professional charts
- Kinds of tasksProfessional chart interpretation
2
Run
10 runs per task
- Agent / harnessInspect AI
- How runs combineMean pass@1 (solved on the first try) over ten trials
3
Score
Mean pass@1 (solved on the first try)
- Graded byGemini 3.5 Flash judges against expert golden answers
- Better isHigher
Sources: surgehq.ai, github.com
Chartography leaderboardtop 10 of 37
Full leaderboard with filters →| Model | Chartography | Per task | Price | Speed | Supports | Context |
|---|---|---|---|---|---|---|
71.6high | $1.99 | $2 / $10 | 84 | 1M | ||
71max | $3.26 | $10 / $50 | 69 | 1.05M | ||
66.3max | $5.98 | $4 / $20 | 96 | 1M | ||
53.6max | $1.04 | $2 / $10 | 128 | 1.05M | ||
46.2max | $7.63 | $10 / $50 | 69 | 1M | ||
45max | $1.99 | $4 / $20 | 114 | 1.05M | ||
40.9high | $1.24 | $0.75 / $3.75 | 149 | 1M | ||
34.8max | $8.75 | $10 / $50 | 67 | 1M | ||
34medium | — | $0.75 / $3.75 | — | 1M | ||
34max | $1.40 | $2 / $12 | 142 | 1.05M |
Chartography questions
What is Chartography?›
Interpret specialized professional charts. It has 100 chart questions. Created by Surge AI.
Which model is best on Chartography?›
Gemini 4 Argon leads Chartography with 71.6, ahead of GPT-6 Astra (71) and Claude Opus 5.5 (66.3).
What is the best open-weights model on Chartography?›
Kimi K3 with 26.6, then Qwen3.8 Flash (17.4).
How is Chartography scored?›
Mean pass@1 (solved on the first try); gemini 3.5 flash judges against expert golden answers; 10 runs per task per task.