Benchmarks › MMMU

MMMU

By MMMU team (Ohio State University, University of Waterloo and others) ↗Results by Artificial Analysis ↗Since 2024Paper ↗
Answer visual academic questions with ten answer choices
87.7top score · Claude Opus 5.5
124models ranked
61open-weights models
Pass@1 accuracyhigher is better
Oct 20, 2026last updated

How the test works

1

Task

1,730 questions

  • What the model doesAnswer visual academic questions with ten answer choices
  • Kinds of tasksTen-option multimodal questions
  • DomainsArt & Design, Business, Science, Health & Medicine, Humanities & Social Science, Tech & Engineering
2

Run

Non-reasoning: up to 16,384 output tokens, adjusted to model limits

  • Temperature0 for non-reasoning; 0.6 for reasoning, unless the model lab recommends otherwise
3

Score

Pass@1 (solved on the first try) accuracy

  • Graded byRegex answer extraction
  • Better isHigher
Sources: artificialanalysis.ai, mmmu-benchmark.github.io

MMMU-Pro leaderboardtop 10 of 124

Full leaderboard with filters →
ModelMMMU-ProPer taskPriceSpeedSupportsContext
Claude Opus 5.5Anthropic
87.7max
$5.98$4 / $20961M
GPT-6 AstraOpenAI
86.9max
$3.26$10 / $50691.05M
GPT-6.1 SolOpenAI
86max
$0.72$2 / $10521.05M
Gemini 3.8 FlashGoogle
85.6high
$1.24$0.75 / $3.751491M
Claude Opus 5Anthropic
84.7max
$5.86$5 / $25551M
GPT-5.6 SolOpenAI
83.4max
$1.99$4 / $201141.05M
Gemini 3.6 FlashGoogle
83.2high
$1.60$0.75 / $3.752091M
GPT-6 SolOpenAI
82.9max
$1.04$2 / $101281.05M
Qwen3.8 Max (0902)Alibaba (Qwen)
82.8thinking
$5.41$2 / $640984K
Gemini 3.1 Pro PreviewGoogle
82.4thinking
$1.30$2 / $121151M

Versions and boards on Model Menu

BoardModels
MMMU-ProShown above
127
MMMU validation
41

MMMU questions

What is MMMU?›

Answer visual academic questions with ten answer choices. It has 1,730 questions. Created by MMMU team (Ohio State University, University of Waterloo and others).

Which model is best on MMMU?›

Claude Opus 5.5 leads MMMU-Pro with 87.7, ahead of GPT-6 Astra (86.9) and GPT-6.1 Sol (86).

What is the best open-weights model on MMMU?›

Muse Spark Open with 80.9, then DeepSeek V4.1 Flash 1017 (80.6).

How is MMMU scored?›

Pass@1 (solved on the first try) accuracy; regex answer extraction.

Which versions of MMMU are there?›

MMMU-Pro (127 models), MMMU validation (41 models).