DeepSeek V4 Pro (Preview) vs Qwen3.8 Max
Humanity's Last Exam Humanity's Last Exam: Very hard expert questions across many fields, written to stump today’s models
DeepSeek V4 Pro (Preview)—
Qwen Max43
Arena Text · overall Arena Text · overall: Human pairwise preferences in the stated text vote pool
DeepSeek V4 Pro (Preview)—
Qwen Max · max1,482
Arena Text · Chinese Arena Text · chinese: Human pairwise preferences in the stated text vote pool
DeepSeek V4 Pro (Preview)—
Qwen Max · max1,536
Arena Text · English Arena Text · english: Human pairwise preferences in the stated text vote pool
DeepSeek V4 Pro (Preview)—
Qwen Max · max1,492
Arena Text · French Arena Text · french: Human pairwise preferences in the stated text vote pool
DeepSeek V4 Pro (Preview)—
Qwen Max · max1,502.1
Arena Text · German Arena Text · german: Human pairwise preferences in the stated text vote pool
DeepSeek V4 Pro (Preview)—
Qwen Max · max1,483.2
Arena Text · instruction following Arena Text · instruction following: Human pairwise preferences in the stated text vote pool
DeepSeek V4 Pro (Preview)—
Qwen Max · max1,479.4
Arena Text · Japanese Arena Text · japanese: Human pairwise preferences in the stated text vote pool
DeepSeek V4 Pro (Preview)—
Qwen Max · max1,469.5
Arena Text · Korean Arena Text · korean: Human pairwise preferences in the stated text vote pool
DeepSeek V4 Pro (Preview)—
Qwen Max · max1,463.7
Arena Text · non-English Arena Text · non english: Human pairwise preferences in the stated text vote pool
DeepSeek V4 Pro (Preview)—
Qwen Max · max1,471.7
Arena Text · Polish Arena Text · polish: Human pairwise preferences in the stated text vote pool
DeepSeek V4 Pro (Preview)—
Qwen Max · max1,472.4
Arena Text · Russian Arena Text · russian: Human pairwise preferences in the stated text vote pool
DeepSeek V4 Pro (Preview)—
Qwen Max · max1,481.7
Arena Text · Spanish Arena Text · spanish: Human pairwise preferences in the stated text vote pool
DeepSeek V4 Pro (Preview)—
Qwen Max · max1,491.7
AA Omniscience Index AA Omniscience Index: Scores factual knowledge while penalizing incorrect answers to encourage admitting uncertainty
DeepSeek V4 Pro (Preview)—
Qwen Max3.4
CritPt (physics) CritPt (physics): Research-level physics problems that need many careful reasoning steps
DeepSeek V4 Pro (Preview)—
Qwen Max20
AA Intelligence Index 4.3.2 AA Intelligence Index 4.3.2: Artificial Analysis combines tests of knowledge, reasoning, coding and tool use into one score
DeepSeek V4 Pro (Preview)—
Qwen Max40.2
AA Omniscience · Accuracy AA Omniscience · Accuracy: Measures correct answers on the AA Omniscience factual knowledge test
DeepSeek V4 Pro (Preview)—
Qwen Max31.9
AA Omniscience · Hallucination Rate AA Omniscience · Hallucination Rate: Measures how often attempted answers are incorrect on the AA Omniscience factual knowledge test · lower is better
DeepSeek V4 Pro (Preview)—
Qwen Max41.7
AA-LCR (long context) 1.1 AA-LCR (long context) 1.1: Questions that need reading and connecting facts across very long documents (around 100K tokens)
DeepSeek V4 Pro (Preview)—
Qwen Max78.3
GDP.pdf · AA all-pass GDP.pdf · AA all-pass: Models reason over professional PDF documents; an attempt passes only when every atomic criterion passes
DeepSeek V4 Pro (Preview)—
Qwen Max20.2
MLCR-AA MLCR-AA: Models synthesize long medical records; the headline credits concise answers judged both accurate and complete
DeepSeek V4 Pro (Preview)—
Qwen Max19.4
MathArena · ArXivMath · April 2026 MathArena · arxiv · 04/2026: Answer accuracy on the named mathematics competition under the recorded effort
DeepSeek V4 Pro (Preview)—
Qwen Max · max59.2
MathArena · ArXivMath · August 2026 MathArena · arxiv · 08/2026: Answer accuracy on the named mathematics competition under the recorded effort
DeepSeek V4 Pro (Preview)—
Qwen Max · max57.9
MathArena · ArXivMath · June 2026 MathArena · arxiv · 06/2026: Answer accuracy on the named mathematics competition under the recorded effort
DeepSeek V4 Pro (Preview)—
Qwen Max · max75.7
MathArena · ArXivMath · May 2026 MathArena · arxiv · 05/2026: Answer accuracy on the named mathematics competition under the recorded effort
DeepSeek V4 Pro (Preview)—
Qwen Max · max69.2
MathArena · BrokenArXiv · April 2026 MathArena · arxiv_false · 04/2026: Answer accuracy on the named mathematics competition under the recorded effort
DeepSeek V4 Pro (Preview)—
Qwen Max · max45.5
MathArena · BrokenArXiv · August 2026 MathArena · arxiv_false · 08/2026: Answer accuracy on the named mathematics competition under the recorded effort
DeepSeek V4 Pro (Preview)—
Qwen Max · max69.6
MathArena · BrokenArXiv · June 2026 MathArena · arxiv_false · 06/2026: Answer accuracy on the named mathematics competition under the recorded effort
DeepSeek V4 Pro (Preview)—
Qwen Max · max44
MathArena · BrokenArXiv · May 2026 MathArena · arxiv_false · 05/2026: Answer accuracy on the named mathematics competition under the recorded effort
DeepSeek V4 Pro (Preview)—
Qwen Max · max32.5
MathArena · ArXivLean · June 2026 MathArena · arxivlean · 06/2026: Answer accuracy on the named mathematics competition under the recorded effort
DeepSeek V4 Pro (Preview)—
Qwen Max · max45.7
Vals LegalBench Vals LegalBench: Legal reasoning and legal-language understanding on LegalBench
DeepSeek V4 Pro (Preview)—
Qwen Max · max83.6
Vals MedCode Vals MedCode: Medical coding accuracy on patient encounters
DeepSeek V4 Pro (Preview)—
Qwen Max · max40.7
Vals MedScribe Vals MedScribe: Accuracy of clinical notes produced from consultation transcripts
DeepSeek V4 Pro (Preview)—
Qwen Max · max85
Vals Legal Research Bench Vals Legal Research Bench: Grounded legal research; a task passes only if all criteria pass
DeepSeek V4 Pro (Preview)—
Qwen Max · max47.6
| Benchmark | DeepSeek V4 Pro (Preview) | Qwen Max |
|---|---|---|
| Humanity's Last Exam | — | 43 |
| Arena Text · overall | — | 1,482 · max |
| Arena Text · Chinese | — | 1,536 · max |
| Arena Text · English | — | 1,492 · max |
| Arena Text · French | — | 1,502.1 · max |
| Arena Text · German | — | 1,483.2 · max |
| Arena Text · instruction following | — | 1,479.4 · max |
| Arena Text · Japanese | — | 1,469.5 · max |
| Arena Text · Korean | — | 1,463.7 · max |
| Arena Text · non-English | — | 1,471.7 · max |
| Arena Text · Polish | — | 1,472.4 · max |
| Arena Text · Russian | — | 1,481.7 · max |
| Arena Text · Spanish | — | 1,491.7 · max |
| AA Omniscience Index | — | 3.4 |
| CritPt (physics) | — | 20 |
| AA Intelligence Index 4.3.2 | — | 40.2 |
| AA Omniscience · Accuracy | — | 31.9 |
| AA Omniscience · Hallucination Rate | — | 41.7 |
| AA-LCR (long context) 1.1 | — | 78.3 |
| GDP.pdf · AA all-pass | — | 20.2 |
| MLCR-AA | — | 19.4 |
| MathArena · ArXivMath · April 2026 | — | 59.2 · max |
| MathArena · ArXivMath · August 2026 | — | 57.9 · max |
| MathArena · ArXivMath · June 2026 | — | 75.7 · max |
| MathArena · ArXivMath · May 2026 | — | 69.2 · max |
| MathArena · BrokenArXiv · April 2026 | — | 45.5 · max |
| MathArena · BrokenArXiv · August 2026 | — | 69.6 · max |
| MathArena · BrokenArXiv · June 2026 | — | 44 · max |
| MathArena · BrokenArXiv · May 2026 | — | 32.5 · max |
| MathArena · ArXivLean · June 2026 | — | 45.7 · max |
| Vals LegalBench | — | 83.6 · max |
| Vals MedCode | — | 40.7 · max |
| Vals MedScribe | — | 85 · max |
| Vals Legal Research Bench | — | 47.6 · max |
Specs
| Spec | DeepSeek V4 Pro (Preview) | Qwen Max |
|---|---|---|
| Input, per 1M | $1.30 | $2 |
| Output, per 1M | $2.60 | $6 |
| Cost per task? | — | $2.67 |
| Speed, tok/s | — | 38 |
| Context | — | 1M |
| Weights | Not verified | Closed |
| Released | — | Aug 2026 |
| See images | — | ✓ |
| Read PDFs | — | Partial |
| Hear audio | — | — |
| Watch video | — | ✓ |
| Call tools | Partial | ✓ |
| Force a tool call | ✓ | Partial |
| Return strict JSON | ✓ | ✓ |
| Search the web | ? | ? |
| Run code | ? | ? |
| Use a computer | ? | ? |
What breaks when you switch
DeepSeek V4 Pro (Preview) → Qwen Max
max_tokens range · QwenCloud›
With thinking enabled, max_tokens outside 1–32768 returns HTTP 400; use max_completion_tokens to budget reasoning plus answer without that cap.
File is not delivered · QwenCloud / Responses›
PDF input through Responses returns HTTP 200 without delivering the file to the model; use Chat Completions or DashScope.
Not simultaneously · QwenCloud›
Setting reasoning_effort and thinking_budget together returns an error; set only one.
tool_choice only supports auto or none · QwenCloud›
With thinking enabled, tool_choice supports only auto or none.
Include the word JSON · QwenCloud›
JSON Object mode requires the word JSON in a system or user message; otherwise the API returns an error.
Agent search_strategy not supported · QwenCloud / Chat Completions›
Chat Completions does not support Qwen3.8 agent search_strategy; use the Responses web_search tool for multi-turn retrieval.
Qwen Max → DeepSeek V4 Pro (Preview)