Benchmarks › MLCR-AA
MLCR-AA
Reason across long fragmented medical records
75top score · Claude Sonnet 5.5
64models ranked
24open-weights models
Pass@1 pass rate (complete and accurate)higher is better
Oct 20, 2026last updated
How the test works
1
Task
60 tasks
- What the model doesReason across long fragmented medical records
- Kinds of tasksMedical record reasoning
- In → outText → Answer
2
Run
3 runs per task
- Agent / harnessAnswer each part in one paragraph and cite record evidence unless the question specifies another format.
- Token budgetNon-reasoning: up to 16,384 output tokens, adjusted to model limits; reasoning: model-specific maximum
- Temperature0 for non-reasoning; 0.6 for reasoning, unless the model lab recommends otherwise
3
Score
Pass@1 (solved on the first try) pass rate (complete and accurate)
- Graded byThree-judge majority: Claude Opus 5.5, GPT-6.1 Sol and Grok 4.7 (medium)
- Better isHigher
- Test setPrivate held-out expert/compound set, separate from public Wisedocs MLCR dataset
Sources: artificialanalysis.ai
MLCR-AA leaderboardtop 10 of 64
Full leaderboard with filters →| Model | MLCR-AA | Per task | Price | Speed | Supports | Context |
|---|---|---|---|---|---|---|
75max | $5.46 | $2 / $10 | 137 | 1M | ||
71.1max | $7.63 | $10 / $50 | 69 | 1M | ||
66.7max | $5.98 | $4 / $20 | 96 | 1M | ||
64.4max | $8.75 | $10 / $50 | 67 | 1M | ||
63.3max | $0.21 | $0.10 / $0.50 | 234 | 1M | ||
59.4high | $3.61 | $5 / $25 | 51 | 1M | ||
55max | $5.09 | $2 / $10 | 80 | 1M | ||
51.1max | $0.28 | $0.15 / $0.50 | 179 | 1M | ||
48.3max | $2.01 | $1.40 / $4.40 | 186 | 1M | ||
45.6max | $4.08 | $5 / $25 | 66 | 1M |
MLCR-AA questions
What is MLCR-AA?›
Reason across long fragmented medical records. It has 60 tasks. Created by Wisedocs (MLCR); Artificial Analysis (AA evaluation).
Which model is best on MLCR-AA?›
Claude Sonnet 5.5 leads MLCR-AA with 75, ahead of Claude Fable 5.1 (71.1) and Claude Opus 5.5 (66.7).
What is the best open-weights model on MLCR-AA?›
GLM 5.3 Flash with 51.1, then GLM 5.3 (48.3).
How is MLCR-AA scored?›
Pass@1 (solved on the first try) pass rate (complete and accurate); three-judge majority: claude opus 5.5, gpt-6.1 sol and grok 4.7 (medium); compare expert ground truth and medical record, including contradictions and documentation gaps.; 3 runs per task per task.