Benchmarks › SimpleQA
SimpleQA
By OpenAI; SimpleQA Verified by Google DeepMind and Google Research ↗Results by Epoch AI ↗Since 2025Paper ↗
Answer fact questions with an explicit instruction to make a best guess
77.5top score · Gemini 3.1 Pro Preview
83models ranked
22open-weights models
Accuracyhigher is better
Oct 20, 2026last updated
How the test works
1
Task
1,000 questions
- What the model doesAnswer fact questions with an explicit instruction to make a best guess
- Kinds of tasksFact questions
- In → outText → Short answer
2
Run
Epoch free-answer evaluation with anti-abstention instruction introduced 2026-08-27
- How runs combineProportion of all questions answered correctly; not Google’s F1 of correctness and correctness-given-attempted.
3
Score
Accuracy
- Graded byGoogle ternary autorater: correct / incorrect / not attempted
- Better isHigher
SimpleQA Verified v1 leaderboardtop 10 of 83
Full leaderboard with filters →| Model | SimpleQA Verified v1 | Per task | Price | Speed | Supports | Context |
|---|---|---|---|---|---|---|
77.5unstated | — | $2 / $12 | — | 1M | ||
76high | $1.99 | $2 / $10 | 84 | 1M | ||
75.8unstated | — | $10 / $50 | — | 1.05M | ||
74.6unstated | — | $0.75 / $3.75 | — | 1M | ||
73.9max | $0.72 | $2 / $10 | 52 | 1.05M | ||
72.2max | $5.98 | $4 / $20 | 96 | 1M | ||
70.8max | $7.63 | $10 / $50 | 69 | 1M | ||
70.7xhigh | — | $10 / $50 | — | 1M | ||
69.7max | $1.99 | $4 / $20 | 114 | 1.05M | ||
66.8high | — | $0.50 / $3 | — | 1M |
SimpleQA questions
What is SimpleQA?›
Answer fact questions with an explicit instruction to make a best guess. It has 1,000 questions. Created by OpenAI; SimpleQA Verified by Google DeepMind and Google Research.
Which model is best on SimpleQA?›
Gemini 3.1 Pro Preview leads SimpleQA Verified v1 with 77.5, ahead of Gemini 4 Argon (76) and GPT-6 Astra (75.8).
What is the best open-weights model on SimpleQA?›
DeepSeek V4 Pro 0813 with 52.9, then Kimi K3 (50.6).
How is SimpleQA scored?›
Accuracy; google ternary autorater: correct / incorrect / not attempted.