Step 3.7 Flash vs Step 5 Preview
Different efforts: high · thinking · Compare all at high
Humanity's Last Exam Humanity's Last Exam: Very hard expert questions across many fields, written to stump today’s models
Step 3.7 Flash21.4
Step 5 Preview46.5
Terminal-Bench 4.0 Terminal-Bench 4.0 · source setting 2: Agents use a terminal to complete executable coding, system administration and research tasks in the 4.0 suite
Step 3.7 Flash—
Step 5 Preview33.3
AutomationBench-AA AutomationBench-AA: Agents orchestrate SaaS workflows through APIs; objectives receive credit only when the task has no guardrail violation
Step 3.7 Flash—
Step 5 Preview51
Arena Text · creative writing Arena Text · creative writing: Human pairwise preferences in the stated text vote pool
Step 3.7 Flash—
Step 5 Preview1,403
MMMU-Pro (vision) MMMU-Pro (vision): College-level questions that require understanding images: charts, diagrams, photos
Step 3.7 Flash75.3
Step 5 Preview76.4
| Benchmark | Step 3.7 Flash | Step 5 Preview |
|---|---|---|
| Humanity's Last Exam | 21.4 | 46.5 |
| Terminal-Bench 4.0 | — | 33.3 |
| AutomationBench-AA | — | 51 |
| Arena Text · creative writing | — | 1,403 |
| MMMU-Pro (vision) | 75.3 | 76.4 |
Specs
| Spec | Step 3.7 Flash | Step 5 Preview |
|---|---|---|
| Input, per 1M | $0.20 | $1 |
| Output, per 1M | $1.15 | $2.70 |
| Cost per task? | — | $1.03 |
| Speed, tok/s | 123 | 93 |
| Context | 256K | 1M |
| Weights | Open | Open |
| Released | May 2026 | Sep 2026 |
| Memory (4-bit) | 121 GB | 363 GB |
| Licence | Apache-2.0 | custom licence |
| Commercial use | Allowed | Restricted |
| See images | ✓ | ✓ |
| Read PDFs | — | — |
| Hear audio | — | — |
| Watch video | ✓ | ✓ |
| Call tools | ✓ | ✓ |
| Force a tool call | Partial | Partial |
| Return strict JSON | ✓ | ✓ |
| Search the web | ? | — |
| Run code | ? | — |
| Use a computer | ? | ? |
What breaks when you switch
Step 3.7 Flash → Step 5 Preview
Memory (4-bit): 121 GB → 363 GB
web_search not available · Chat Completions›
Built-in Chat Completions web_search is unavailable; integrate external search through a function tool and return the results.
Output includes reasoning›
Output billing includes the reasoning process as well as the final answer.
LRU eviction during peak traffic›
LRU eviction removes unused cache entries more aggressively during peak traffic, so cached prefixes may not survive.
Step 5 Preview → Step 3.7 Flash