Gemma 2 27B
Jun 2024
Gemma 2 27BCompare inon
Benchmarks of Gemma 2 27B20
BenchmarkByScoreRank
Math1
Writing15
Health & law1
Legacy · saturated or no longer reported3
No benchmark matches
Rankingconfiguration unstated?
Alternatives?
ModelScoreCostPer taskWhat changes
CheaperGLM 5.3 FlashunstatedOpenwins 69%-12%$0.15 / $0.50Memory (4-bit): 16 GB → 193 GB+5
BetterGPT-5 Minihighwins 54%+155%$0.053OpenGemma 4 31Bunstatedwins 67%-41%$0.10 / $0.34Memory (4-bit): 16 GB → 19 GB
CheaperHy3unstatedOpenwins 68%-14%$0.13 / $0.53Memory (4-bit): 16 GB → 179 GBCheaperGPT-6 Lunamaxwins 66%-26%$0.068Function calling only with none · Chat Completions+3ModelScoreCostPer taskWhat changes
GLM 5.3 FlashunstatedOpenwins 69%-12%$0.15 / $0.50Memory (4-bit): 16 GB → 193 GB+5Hy3unstatedOpenwins 68%-14%$0.13 / $0.53Memory (4-bit): 16 GB → 179 GBGemma 4 31BunstatedOpenwins 67%-41%$0.10 / $0.34Memory (4-bit): 16 GB → 19 GBGPT-6 Lunamaxwins 66%-26%$0.068Function calling only with none · Chat Completions+3
MiMo v2 Omniunstatedwins 64%-35%$0.14 / $0.28ModelScoreCostPer taskWhat changes
GPT-5 Minihighwins 54%+155%$0.053GPT-6 Lunamaxwins 66%-26%$0.068Function calling only with none · Chat Completions+3GPT-5.6 Lunaxhighwins 66%+67%$0.085GPT-5.6 Terraxhighwins 69%+1,567%$0.63Prompts with >272K input tokensGPT-6.1 Solmaxwins 72%+1,381%$0.72Remove unsupported parameters+5ModelScoreCostPer taskWhat changes
Gemma 4 31BunstatedOpenwins 67%-41%$0.10 / $0.34Memory (4-bit): 16 GB → 19 GBGemma 4 26B A4BunstatedOpenwins 65%same$0.15 / $0.60
Qwen3.8 27BunstatedOpenwins 60%+254%$0.42 / $2.55Memory (4-bit): 16 GB → 17 GB+5
Muse Glimmer 30BunstatedOpenwins 60%+85%$0.30 / $1.10Memory (4-bit): 16 GB → 18 GBQwen 3.5 27BunstatedOpenwins 59%+206%$0.30 / $2.40Memory (4-bit): 16 GB → 17 GBModelScoreCostPer taskWhat changes
Gemini 4 Argonhighwins 78%—*$1.99Synchronous output stops at 65,536+2Gemini 3.8 Flashhighwins 75%—*$1.24Sampling parameters are ignored+1Gemini 3.1 Pro Previewunstatedwins 75%—$2 / $12Cannot use both thinking controlsGemini 3.6 Flashhighwins 74%—*$1.60Gemini 3 Flash Previewunstatedwins 72%—$0.50 / $3Gemini 2.5 Prounstatedwins 70%—$1.25 / $10Gemini 3.5 Flash Liteunstatedwins 69%—$0.30 / $2.50Gemma 4 31BunstatedOpenwins 67%—$0.10 / $0.34Memory (4-bit): 16 GB → 19 GBAll Google models →Run it yourself
estimateFits on✗ 8 GB (phone, laptop)✗ 16 GB✓ 24 GB GPU · compressed✓ 48 GB GPU · 8-bit✓ 80 GB GPU · full quality
LicenceGemma termscommercial use with conditions
History
2 versionsBenchmarks
all categoriesChat & Knowledge
Arena Text · overall1,290#159
Arena Text · Chinese1,220.9#153
Arena Text · English1,241.8#161
Arena Text · French1,247.4#129
Arena Text · German1,208.9#131
Arena Text · instruction following1,205.9#160
Arena Text · Japanese1,175.4#112
Arena Text · Korean1,173.8#124
Arena Text · non-English1,217#156
Arena Text · Polish1,277.3#117
Arena Text · Russian1,233.9#153
Arena Text · Spanish1,228.2#139
MATH Level 527.9#26
Vals LegalBench68#84
Arena Text · overall · full cohort1,231.3#159
GPQA Diamond · Epoch36.5#118
MMLU-Pro · official board56.5#10
OTIS Mock AIME 2024 20251.4#123
Writing & Language
Arena Text · creative writing1,293#134
Arena Text · creative writing · full cohort1,240.5#149
FAQ
How much does Gemma 2 27B cost?›
$0.27 per 1M input tokens and $0.27 per 1M output tokens (cheapest host: DeepInfra).
Is Gemma 2 27B free to use?›
There is no free API; the cheapest paid price is $0.27 per 1M input tokens. The weights are open, so you can run it yourself.
How good is Gemma 2 27B, and what is it best at?›
#159 of 196 in Chat & Knowledge, #134 of 195 in Writing & Language.
Is Gemma 2 27B better than GPT-4o Mini?›
On Arena Text · creative writing, Gemma 2 27B scores 1,293 vs 1,294 — about the same.
How can I use Gemma 2 27B?›
Through the API of DeepInfra, NextBit.
What is the context window of Gemma 2 27B?›
8K tokens, with up to 2K output tokens.
When was Gemma 2 27B released? Is there a newer version?›
Released 2024-06-24. It is the current version in its line.
Is Gemma 2 27B open source?›
Yes, the weights are open (gemma licence).
Can I run Gemma 2 27B locally?›
The weights are open and can be self-hosted; hardware needs are not confirmed yet.
Can Gemma 2 27B read images?›
No, it accepts text only.
What is Gemma 2 27B?›
Gemma 2 27B is an open-weights language model from Google, released Jun 2024.
What are the best alternatives to Gemma 2 27B?›
GLM 5.3 Flash (cheaper), GPT-5 Mini (better), Gemma 4 31B (open weights).