Benchmarks › EnterpriseOps-Gym-AA

EnterpriseOps-Gym-AA

By ServiceNow AI Research, Mila and Université de Montréal ↗Results by Artificial Analysis ↗
Complete enterprise tasks through MCP tools
51.1top score · Claude Fable 5
43models ranked
22open-weights models
Strict pass@1 success ratehigher is better
Oct 20, 2026last updated

How the test works

1

Task

1,117 tasks

  • What the model doesComplete enterprise tasks through MCP tools
  • Kinds of tasksEnterprise operations
  • In → outText → MCP tool calls
2

Run

3 runs per task

  • Token budgetNon-reasoning: up to 16,384 output tokens, adjusted to model limits; reasoning: model-specific maximum
  • Temperature0 for non-reasoning; 0.6 for reasoning, unless the model lab recommends otherwise
3

Score

Strict pass@1 (solved on the first try) success rate

  • Graded byOutcome-based SQL state verifiers
  • Better isHigher
Sources: artificialanalysis.ai

EnterpriseOps-Gym-AA leaderboardtop 10 of 43

Full leaderboard with filters →
ModelEnterpriseOps-Gym-AAPer taskPriceSpeedSupportsContext
Claude Fable 5Anthropic
51.1max
$8.75$10 / $50671M
DeepSeek V4 Pro 0813DeepSeekOpen
49.6max
$0.67$1.32 / $3.96901M
Grok 4.6xAI
48.3high
$1.48$2 / $665500K
Qwen3.8 Max (0902)Alibaba (Qwen)
47.6thinking
$5.41$2 / $640984K
Claude Opus 5Anthropic
47.5max
$5.86$5 / $25551M
Qwen3.8 2.4T A95BAlibaba (Qwen)Open
47.4thinking
$2.16$2 / $6127262K
Muse Spark 1.2Meta
47.3xhigh
$0.97$1.25 / $4.253551M
Step 5 PreviewStepFunOpen
47.2thinking
$1.03$1 / $2.70931M
GPT-5.5OpenAI
46.6xhigh
$2.63$5 / $301131.05M
Kimi K3Moonshot AIOpen
45.3max
$2.00$3 / $151121M

EnterpriseOps-Gym-AA questions

What is EnterpriseOps-Gym-AA?›

Complete enterprise tasks through MCP tools. It has 1,117 tasks. Created by ServiceNow AI Research, Mila and Université de Montréal.

Which model is best on EnterpriseOps-Gym-AA?›

Claude Fable 5 leads EnterpriseOps-Gym-AA with 51.1, ahead of DeepSeek V4 Pro 0813 (49.6) and Grok 4.6 (48.3).

What is the best open-weights model on EnterpriseOps-Gym-AA?›

DeepSeek V4 Pro 0813 with 49.6, then Qwen3.8 2.4T A95B (47.4).

How is EnterpriseOps-Gym-AA scored?›

Strict pass@1 (solved on the first try) success rate; outcome-based sql state verifiers; 3 runs per task per task.