Benchmarks › GPQA

GPQA

Older benchmarkBy New York University, Cohere and Anthropic ↗Results by Artificial Analysis ↗Since 2023Paper ↗
Answer difficult multiple-choice science questions
96.3top score · GPT-6 Astra
312models ranked
215open-weights models
Accuracy / pass@1higher is better
Oct 20, 2026last updated

How the test works

1

Task

198 questions

  • What the model doesAnswer difficult multiple-choice science questions
  • Kinds of tasksFour-option multiple-choice questions
  • DomainsBiology, physics, chemistry
2

Run

Non-reasoning: up to 16,384 output tokens, adjusted to model limits

  • Temperature0 for non-reasoning; 0.6 for reasoning, unless the model lab recommends otherwise
3

Score

Accuracy / pass@1 (solved on the first try)

  • Graded byRegex answer extraction against the answer key
  • Better isHigher
Sources: artificialanalysis.ai

GPQA Diamond leaderboardtop 10 of 312

Full leaderboard with filters →
ModelGPQA DiamondPer taskPriceSpeedSupportsContext
GPT-6 AstraOpenAI
96.3xhigh
$2.31$10 / $50611.05M
Gemini 3.8 FlashGoogle
95.3high
$1.24$0.75 / $3.751491M
Grok 4.6xAI
94.9high
$1.48$2 / $665500K
Gemini 3.1 Pro PreviewGoogle
94.1thinking
$1.30$2 / $121151M
Muse Spark 1.3Meta
94.1xhigh
$1.37$1.25 / $4.252731M
GPT-5.6 SolOpenAI
94.1max
$1.99$4 / $201141.05M
Claude Fable 5.1Anthropic
93.7max
$7.63$10 / $50691M
Claude Opus 5Anthropic
93.7high
$3.61$5 / $25511M
Kimi K3Moonshot AIOpen
93.5max
$2.00$3 / $151121M
GPT-5.5OpenAI
93.5xhigh
$2.63$5 / $301131.05M

Versions and boards on Model Menu

BoardModels
GPQA DiamondShown above
315
GPQA Diamond · Epoch
133
GPQA Diamond (past run, January 2025 run)
1

GPQA questions

What is GPQA?›

Answer difficult multiple-choice science questions. It has 198 questions. Created by New York University, Cohere and Anthropic.

Which model is best on GPQA?›

GPT-6 Astra leads GPQA Diamond with 96.3, ahead of Gemini 3.8 Flash (95.3) and Grok 4.6 (94.9).

What is the best open-weights model on GPQA?›

Kimi K3 with 93.5, then Qwen3.8 2.4T A95B (93.5).

How is GPQA scored?›

Accuracy / pass@1 (solved on the first try); regex answer extraction against the answer key.

Which versions of GPQA are there?›

GPQA Diamond (315 models), GPQA Diamond · Epoch (133 models), GPQA Diamond (past run, January 2025 run) (1 models).