ModelMMLU-Pro · Official boardPer taskPriceSpeedSupportsContext
91.2unstated
—$2 / $12—1M
GPT-4oOpenAI
74.7unstated
—$2.50 / $10—128K
GPT OSS 20BOpenAIOpen
73.1medium
—$0.018 / $0.09—131K
QwQ 32BAlibaba (Qwen)Open
69.1
—$0.15 / $0.40—131K
65.9unstated
—$0.10 / $0.32—128K
63.1unstated
—$0.15 / $0.60—128K
62.8unstated
—$0.40 / $0.40—131K
60.4
—$0.35 / $0.40—8K
LEXAONE 3.5 32B InstructLGAI-EXAONEOpen
58.9
——33K
Gemma 2 27BGoogleOpen
56.5unstated
—$0.27 / $0.27—8K
56.3unstated
—$2 / $6—66K
56.2
——8K
52.8
—$2.65 / $3.50——
52.5
—$0.72 / $0.72——
Gemma 2 9B ItGoogleOpen
52.1
—$0.03 / $0.06—8K
Jamba 1.5 LargeAI21 LabsOpen
49.5
——256K
47.9
——131K
Qwen 2 7B InstructAlibaba (Qwen)Open
47.2
—$0.055 / $0.055—33K
LEXAONE 3.5 7.8B InstructLGAI-EXAONEOpen
46.2
——33K
Mistral NemoMistralOpen
44.8unstated
—$0.15 / $0.15—131K
44.3unstated
—$0.02 / $0.04—131K
43.3
—$0.54 / $0.54—33K
41
——8K
39.2unstated
—$0.48 / $0.48—66K
LEXAONE 3.5 2.4B InstructLGAI-EXAONEOpen
39.1
——33K
36.6
—$0.22 / $0.22——
35.4
—$0.30 / $0.60——
CAya Expanse 8BCohereLabsOpen
33.7
———
Mistral 7B v0.1MistralOpen
30.9
—$0.40 / $0.80—33K
30.8
—$0.055 / $0.055—33K
25.8
—$0.055 / $0.055—33K
13.3
——131K