AI benchmarks
54 benchmarks · what each tests, how it is run, who leads
| Benchmark | What it tests | Models | Leader | Top score |
|---|---|---|---|---|
Terminal-BenchCoding | Agents complete tasks in a terminal | 132 |  Claude Sonnet 5.5 | 63.6 |
SWE-benchCoding | Solve long software engineering tasks in a repository | 16 | Claude Opus 5 | 99.4 |
| Answer expert questions using text only | 334 | Claude Opus 5.5 | 61.4 | |
ARC-AGIChat | Infer rules for new colored-grid puzzles | 66 |  GPT-6 Astra | 95 |
GPQAChat | Answer difficult multiple-choice science questions | 312 | GPT-6 Astra | 96.3 |
MMLUChat | Answer harder academic questions with ten answer choices | 32 |  Gemini 3.1 Pro | 91.2 |
LiveCodeBenchCoding | Solve new programming contest problems | 152 | Gemini 3 Flash Preview | 90.8 |
τ-benchAgents | Resolve a simulated customer issue using the allowed tools | 134 |  Grok 4.6 | 50.7 |
SimpleQAChat | Answer fact questions with an explicit instruction to make a best guess | 83 | Gemini 3.1 Pro Preview | 77.5 |
MMMUVision | Answer visual academic questions with ten answer choices | 124 | Claude Opus 5.5 | 87.7 |
GDPval-AAAgents | Create files that satisfy real professional work requests | 175 | Claude Opus 5.5 | 68.3 |
WebDev ArenaCoding | Build an interactive web app from a user prompt | 97 | Claude Opus 5.5 | 1,807 |
LMArena / Arena TextWriting · Chat · Vision · Image Generation · Video Generation | Answer real user prompts in anonymous head-to-head comparisons | 195 | Claude Opus 5.5 | 1,519 |
MTEBEmbeddings | Embed queries and documents, then rank relevant documents for each query | 212 | IYuan Embedding 2.0 En | 70.7 |
AIME / MathArenaChat | Solve twelve hard final-answer problems selected from 2025 mathematics competitions | 34 | Claude Opus 4.8 | 81.3 |
| Combine results from ten knowledge, reasoning, coding and agent evaluations | 334 | Claude Opus 5.5 | 57.6 | |
CritPtChat | Solve unpublished research-level physics problems | 321 |  Muse Wonder Full | 32.9 |
AA OmniscienceChat | Answer knowledge questions or abstain instead of guessing | 318 | Claude Fable 5.1 | 67.2 |
AutomationBench-AAAgents | Complete business workflows by calling app APIs | 133 | Gemini 4 Argon | 77.5 |
DeepSWECoding | Complete long software engineering tasks in open-source repositories | 44 | GPT-6 Astra | 74.1 |
FrontierCodeCoding | Make repository changes that pass tests and a maintainer rubric | 28 | Claude Opus 5.5 | 57.1 |
IFBenchWriting | Follow precise formatting, counting and text instructions | 245 | Grok 4.3 | 83.3 |
EQ-BenchWriting · Chat | Plan, revise and write a story over eight writing turns | 95 | Claude Opus 5 | 86.3 |
Open ASR LeaderboardSpeech | Transcribe speech and compare it with reference transcripts | 61 | ZZoom Scribe V2 Pro | 6.1 |
AA Image ArenaImage Generation | Generate or edit images for blind human preference comparisons | 93 | GPT Image 2.5 Sunburst | 1,222 |
AA Video ArenaVideo Generation | Generate videos for blind human preference comparisons | 24 |  Wan 3.0 | 1,173 |
AA TTS ArenaSpeech | Read text aloud for blind listening comparisons | 98 |  Eleven v4 Turbo | 1,330 |
AA-BriefcaseAgents | Produce professional files for multi-step workplace scenarios | 140 | Claude Sonnet 5.5 | 1,817 |
AA-LCRChat | Answer questions by reasoning across long documents | 273 |  Kimi K3 | 88.7 |
SciCodeCoding | Write Python code to solve scientific computing problems | 140 | Claude Opus 5.5 | 66.9 |
GDP.pdfChat | Answer professional questions grounded in long PDF documents | 133 | Muse Wonder Full | 32.6 |
OTIS Mock AIMEChat | Solve mock AIME mathematics problems | 130 | Claude Fable 5 | 100 |
Community Decision IndexDecision models | Answer typed decision questions across five capability categories | 99 | bmammoth-1 (27B) | 68.9 |
Vals LegalBenchChat | Answer legal reasoning questions | 89 | Claude Fable 5 | 88.6 |
Vals MedScribeChat | Write clinical SOAP notes from encounter information | 80 | Claude Opus 5.5 | 91.4 |
Vals MedCodeChat | Assign medical diagnosis codes from patient records | 78 | Claude Opus 5 | 63.6 |
| Research a legal question using search and document tools | 69 | Claude Fable 5.1 | 55.3 | |
FrontierMathChat | Solve expert-written mathematics problems in FrontierMath Tiers 1–3 v2 | 69 | GPT-6 Astra | 93.7 |
Chess PuzzlesChat | Find strong moves in difficult chess positions | 68 | GPT-6 Astra | 74 |
MLCR-AAChat | Reason across long fragmented medical records | 64 | Claude Sonnet 5.5 | 75 |
| Solve puzzles from an undisclosed game variant | 64 | GPT-6 Astra | 84 | |
AA Speech to TextSpeech | Transcribe audio and count word insertions, deletions and substitutions | 51 | Fun Realtime ASR Preview | 1.7 |
Lech Mazur Short-StoryWriting | Write short stories under the official writing constraints | 44 | Claude Fable 5.1 | 3.8 |
AA Analyst AgentAgents | Analyze data with Python and provide a final answer | 43 | Claude Fable 5.1 | 57.5 |
EnterpriseOps-Gym-AAAgents | Complete enterprise tasks through MCP tools | 43 | Claude Fable 5 | 51.1 |
| Choose and call tools across single-turn, multi-turn, search and memory tasks | 39 | Claude Opus 4.5 | 77.5 | |
ITBenchAgents | Diagnose root causes from offline Kubernetes incident data | 37 | GPT-5.6 Sol | 56.2 |
ChartographyVision | Interpret specialized professional charts | 37 | Gemini 4 Argon | 71.6 |
MATH Level 5Chat | Solve the hardest level of competition mathematics questions | 35 | GPT-5 (2025-08-07) | 98.1 |
Furniture AssemblyAgents | Inspect furniture photos and manuals to identify assembly mistakes and their steps | 31 | Claude Opus 5.5 | 83.3 |
APEX-AgentsAgents | Complete professional services tasks with computer tools | 29 | Qwen3.8 Max (0902) | 42.4 |
AA Speech ReasoningSpeech | Listen to spoken reasoning questions; measure time until response audio starts | 25 | KRaon SpeechChat 9B | 0 |
AA Decision IndexDecision models | Choose answers in classification, tool calling, knowledge and reasoning tests | 15 |  D1 Ultra | 65.2 |
AA Music ArenaMusic | Generate music for blind listening comparisons | 13 |  Suno v6 | 1,146 |