Recommendations by role
We select benchmark categories to emphasize for each role, then show the performance-first top result and candidates on the cost-efficiency frontier for each comparison group among models available in Claude Code / Codex. The summary counts these by model.
Summary
N is the number of comparison groups with at least two distinct models and a value for that model (the denominator). A plateauing effort costs more than the next lower effort but does not improve the score. We do not give a single “recommended” model (see About for the method).
Models available in Claude Code
Advisor / design decisionsReasoning · Knowledge · Math
| Model | Top score in a of N comparison groups | Groups with a plateauing effort (of N) | Plateauing efforts |
|---|---|---|---|
| Claude Fable 5.2 | 8 of 11 | 0 of 111 assessed | — |
| Claude Opus 5.5 | 7 of 16 | 0 of 163 assessed | — |
| Claude Sonnet 5.5 | 6 of 14 | 0 of 143 assessed | — |
| Claude Haiku 5.5 | 0 of 11 | 0 of 112 assessed | — |
Manager / oversightCoding · Reasoning
| Model | Top score in a of N comparison groups | Groups with a plateauing effort (of N) | Plateauing efforts |
|---|---|---|---|
| Claude Fable 5.2 | 2 of 3 | 0 of 31 assessed | — |
| Claude Opus 5.5 | 3 of 7 | 0 of 71 assessed | — |
| Claude Sonnet 5.5 | 4 of 7 | 0 of 71 assessed | — |
| Claude Haiku 5.5 | 0 of 6 | 0 of 61 assessed | — |
Implementer / task executorAgentic coding · Coding · Tool use
| Model | Top score in a of N comparison groups | Groups with a plateauing effort (of N) | Plateauing efforts |
|---|---|---|---|
| Claude Fable 5.2 | 5 of 5 | 1 of 52 assessed | Max 1 |
| Claude Opus 5.5 | 7 of 14 | 4 of 148 assessed | High 1 · Xhigh 3 · Max 1 |
| Claude Sonnet 5.5 | 4 of 7 | 1 of 74 assessed | Max 1 |
| Claude Haiku 5.5 | 0 of 4 | 0 of 41 assessed | — |
Simple tasksCoding · Tool use
| Model | Top score in a of N comparison groups | Groups with a plateauing effort (of N) | Plateauing efforts |
|---|---|---|---|
| Claude Fable 5.2 | 2 of 2 | 0 of 21 assessed | — |
| Claude Opus 5.5 | 3 of 6 | 0 of 63 assessed | — |
| Claude Sonnet 5.5 | 1 of 2 | 0 of 20 assessed | — |
| Claude Haiku 5.5 | 0 of 2 | 0 of 20 assessed | — |
Design / UIDesign and UI
No data
Models available in Codex
Advisor / design decisionsReasoning · Knowledge · Math
No effort pairs have source costs, so plateauing effort cannot be determined for any model (zero does not mean no plateau).
| Model | Top score in a of N comparison groups | Groups with a plateauing effort (of N) | Plateauing efforts |
|---|---|---|---|
| GPT-6 Astra | 5 of 6 | 0 of 60 assessed | — |
| GPT-6.1 Sol | 5 of 8 | 0 of 80 assessed | — |
| GPT-6.1 Terra | 1 of 7 | 0 of 70 assessed | — |
| GPT-6 Luna | 0 of 8 | 0 of 80 assessed | — |
Manager / oversightCoding · Reasoning
No effort pairs have source costs, so plateauing effort cannot be determined for any model (zero does not mean no plateau).
| Model | Top score in a of N comparison groups | Groups with a plateauing effort (of N) | Plateauing efforts |
|---|---|---|---|
| GPT-6 Astra | 3 of 4 | 0 of 40 assessed | — |
| GPT-6.1 Sol | 0 of 4 | 0 of 40 assessed | — |
| GPT-6.1 Terra | 1 of 2 | 0 of 20 assessed | — |
| GPT-6 Luna | 0 of 4 | 0 of 40 assessed | — |
Implementer / task executorAgentic coding · Coding · Tool use
Comparable groups per model: at most 2. This is too few to read differences in counts as a trend.
No effort pairs have source costs, so plateauing effort cannot be determined for any model (zero does not mean no plateau).
| Model | Top score in a of N comparison groups | Groups with a plateauing effort (of N) | Plateauing efforts |
|---|---|---|---|
| GPT-6 Astra | 1 of 2 | 0 of 20 assessed | — |
| GPT-6.1 Sol | 0 of 2 | 0 of 20 assessed | — |
| GPT-6.1 Terra | 1 of 1 | 0 of 10 assessed | — |
| GPT-6 Luna | 0 of 2 | 0 of 20 assessed | — |
Simple tasksCoding · Tool use
Comparable groups per model: at most 2. This is too few to read differences in counts as a trend.
No effort pairs have source costs, so plateauing effort cannot be determined for any model (zero does not mean no plateau).
| Model | Top score in a of N comparison groups | Groups with a plateauing effort (of N) | Plateauing efforts |
|---|---|---|---|
| GPT-6 Astra | 1 of 2 | 0 of 20 assessed | — |
| GPT-6.1 Sol | 0 of 2 | 0 of 20 assessed | — |
| GPT-6.1 Terra | 1 of 1 | 0 of 10 assessed | — |
| GPT-6 Luna | 0 of 2 | 0 of 20 assessed | — |
Design / UIDesign and UI
No data
The roles and benchmark categories emphasized for each role are editorial choices made by this site. They are not determined automatically from benchmark results.
- Advisor / design decisions
- Reasoning · Knowledge · MathA role that advises on choices of direction and design decisions. Look at results in reasoning, knowledge, and mathematics.
- Manager / oversight
- Coding · ReasoningA role that directs work and reviews deliverables for acceptance. Look at the ability to read code and make judgments, along with reasoning results.
- Implementer / task executor
- Agentic coding · Coding · Tool useA role that writes code and uses tools to complete tasks.
- Simple tasks
- Coding · Tool useSmall, routine tasks. Focus on low-cost candidates on the frontier.
- Design / UI
- Design and UIA role that creates screens and designs.
Candidates are shown by comparison group (the same source, or sources verified to use the same measurement conditions), not combined across groups. The “frontier” consists of candidates for which no better score exists at the same or lower source cost. Only models confirmed available in official documentation for each tool are included; roles without matching results show “No data”. A group with only one value has no comparison, so its value is shown as “No comparison” rather than first place. A group with only different efforts of the same model is not a comparison between models, so no first place or frontier is shown; values by effort and their differences are shown for reference as “Only effort comparisons for the same model”.
Models available in Claude Code (details by group)
Advisor / design decisions
Emphasized categories: Reasoning · Knowledge · Math
Details by group (groups: 24)
| Supporting comparison group | Top result (performance first) | Candidates on the cost-efficiency frontier (lowest cost first) |
|---|---|---|
| Chess Puzzles(Epoch AI)Reasoning · third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, chess_puzzles.csv) CC BY 4.0 · Entries: 19 | Claude Fable 5.2 Max · 79.0% · Source cost unavailablemeasured/reported by: Epoch AI | Cannot be shown because no values have source costs |
| OTIS Mock AIME 2024-2025Math · third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0 · Entries: 18 | Claude Sonnet 5.5 Max · 100.0% · Source cost unavailablemeasured/reported by: Epoch AIClaude Opus 5.5 Max · 100.0% · Source cost unavailablemeasured/reported by: Epoch AIClaude Fable 5 High · 100.0% · Source cost unavailablemeasured/reported by: Epoch AI | Cannot be shown because no values have source costs |
| GPQA DiamondReasoning · third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, gpqa_diamond.csv) CC BY 4.0 · Entries: 17 | Claude Sonnet 5.5 Max · 95.6% · Source cost unavailablemeasured/reported by: Epoch AI | Cannot be shown because no values have source costs |
| SimpleQA VerifiedKnowledge · third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, simpleqa_verified.csv) CC BY 4.0 · Entries: 11 | Claude Fable 5.2 Max · 94.0% · Source cost unavailablemeasured/reported by: Epoch AI | Cannot be shown because no values have source costs |
| FrontierMath Tiers 1-3(v2)Math · third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, frontiermath_tiers_1_3_v2.csv) CC BY 4.0 · Entries: 8 | Claude Fable 5.2 Max · 95.1% · Source cost unavailablemeasured/reported by: Epoch AI | Cannot be shown because no values have source costs |
| FrontierMath Tier 4(v2)Math · third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, frontiermath_tier_4_v2.csv) CC BY 4.0 · Entries: 8 | Claude Fable 5.2 Max · 100.0% · Source cost unavailablemeasured/reported by: Epoch AI | Cannot be shown because no values have source costs |
| HealthBench Professional (length-adjusted score)Knowledge · official · Claude Fable 5.2 and Claude Mythos 5.2 System Card · Entries: 4 | Claude Fable 5.2 Max · 67.7 · Source cost unavailablemeasured/reported by: Artificial Analysis | Cannot be shown because no values have source costs |
| GDPval-AA v2.1Knowledge · official · Introducing Claude Opus 5.5 · Entries: 15 | Claude Opus 5.5 Max · 1,846 · $8.92 (Estimated cost per task)measured/reported by: Anthropic | Claude Opus 5.5 Low · 1,224 · $0.21 (Estimated cost per task)measured/reported by: AnthropicClaude Opus 5 Low · 1,294 · $0.58 (Estimated cost per task)measured/reported by: AnthropicClaude Opus 5.5 Med · 1,576 · $0.86 (Estimated cost per task)measured/reported by: AnthropicClaude Opus 5.5 High · 1,692 · $1.54 (Estimated cost per task)measured/reported by: AnthropicClaude Opus 5.5 Xhigh · 1,820 · $4.21 (Estimated cost per task)measured/reported by: Anthropic |
| AA-Briefcase v1.1(Elo)Knowledge · official · Introducing Claude Sonnet 5.5 · Entries: 15 | Claude Opus 5.5 Max · 1,822 · $21.05 (Cost per task)measured/reported by: Anthropic | Claude Sonnet 5 Low · 923 · $0.82 (Cost per task)measured/reported by: AnthropicClaude Sonnet 5.5 Low · 1,264 · $0.87 (Cost per task)measured/reported by: AnthropicClaude Opus 5.5 Low · 1,285 · $1.15 (Cost per task)measured/reported by: AnthropicClaude Sonnet 5.5 Med · 1,461 · $1.64 (Cost per task)measured/reported by: AnthropicClaude Sonnet 5.5 High · 1,634 · $3.95 (Cost per task)measured/reported by: AnthropicClaude Opus 5.5 Med · 1,642 · $4.40 (Cost per task)measured/reported by: AnthropicClaude Opus 5.5 High · 1,705 · $6.27 (Cost per task)measured/reported by: AnthropicClaude Sonnet 5.5 Xhigh · 1,746 · $9.63 (Cost per task)measured/reported by: AnthropicClaude Opus 5.5 Xhigh · 1,780 · $12.27 (Cost per task)measured/reported by: Anthropic |
| Humanity's Last Exam (with tools)Reasoning · official · Introducing Claude Fable 5.2 and Claude Mythos 5.2 · Entries: 15 | Claude Fable 5.2 Max · 74.2% · $4.51 (Cost per attempt)measured/reported by: Anthropic | Claude Opus 5.5 Low · 57.4% · $0.08 (Cost per attempt)measured/reported by: AnthropicClaude Opus 5.5 Med · 63.0% · $0.17 (Cost per attempt)measured/reported by: AnthropicClaude Fable 5.2 Low · 68.4% · $0.19 (Cost per attempt)measured/reported by: AnthropicClaude Fable 5.2 Med · 68.6% · $0.21 (Cost per attempt)measured/reported by: AnthropicClaude Fable 5.2 High · 71.5% · $0.63 (Cost per attempt)measured/reported by: AnthropicClaude Fable 5.2 Xhigh · 73.4% · $1.48 (Cost per attempt)measured/reported by: Anthropic |
| GDPval-AA v2.1Knowledge · official · Introducing Claude Haiku 5.5 · Entries: 11 | Claude Sonnet 5.5 Max · 1,840 · $6.78 (Cost per task)measured/reported by: Anthropic | Claude Haiku 5.5 Low · 1,125 · $0.01 (Cost per task)measured/reported by: AnthropicClaude Haiku 5.5 Med · 1,277 · $0.03 (Cost per task)measured/reported by: AnthropicClaude Haiku 5.5 High · 1,420 · $0.09 (Cost per task)measured/reported by: AnthropicClaude Haiku 5.5 Xhigh · 1,513 · $0.27 (Cost per task)measured/reported by: AnthropicClaude Sonnet 5.5 High · 1,551 · $0.62 (Cost per task)measured/reported by: AnthropicClaude Haiku 5.5 Max · 1,620 · $0.87 (Cost per task)measured/reported by: AnthropicClaude Sonnet 5.5 Xhigh · 1,731 · $1.88 (Cost per task)measured/reported by: Anthropic |
| Humanity's Last Exam (without tools)Reasoning · official · Introducing Claude Haiku 5.5 · Entries: 11 | Claude Sonnet 5.5 Max · 56.9% · $0.89 (Cost per attempt)measured/reported by: Anthropic | Claude Haiku 5.5 Low · 30.9% · $0.003 (Cost per attempt)measured/reported by: AnthropicClaude Haiku 5.5 Med · 35.5% · $0.007 (Cost per attempt)measured/reported by: AnthropicClaude Haiku 5.5 High · 39.7% · $0.02 (Cost per attempt)measured/reported by: AnthropicClaude Sonnet 5.5 Med · 43.2% · $0.03 (Cost per attempt)measured/reported by: AnthropicClaude Haiku 5.5 Xhigh · 44.1% · $0.06 (Cost per attempt)measured/reported by: AnthropicClaude Sonnet 5.5 High · 47.9% · $0.08 (Cost per attempt)measured/reported by: AnthropicClaude Sonnet 5.5 Xhigh · 53.0% · $0.20 (Cost per attempt)measured/reported by: Anthropic |
| Humanity's Last Exam (with tools)Reasoning · official · Introducing Claude Opus 5.5 · Entries: 3 | Claude Opus 5.5 Max · 67.7% · Source cost unavailablemeasured/reported by: Anthropic | Cannot be shown because no values have source costs |
| GDPval-AA v2.1Knowledge · official · Introducing Claude Sonnet 5.5 · Entries: 3 | Claude Opus 5.5 not stated · 1,846 · Source cost unavailablemeasured/reported by: Artificial Analysis | Cannot be shown because no values have source costs |
| Humanity's Last Exam (with tools)Reasoning · official · Introducing Claude Sonnet 5.5 · Entries: 3 | Claude Opus 5.5 not stated · 67.7% · Source cost unavailablemeasured/reported by: Anthropic | Cannot be shown because no values have source costs |
| AA-Briefcase v1.1(Elo)Knowledge · official · Introducing Claude Haiku 5.5 · Entries: 3 | Claude Sonnet 5.5 not stated · 1,824 · Source cost unavailablemeasured/reported by: Anthropic | Cannot be shown because no values have source costs |
| Humanity's Last Exam (with tools)Reasoning · official · Introducing Claude Haiku 5.5 · Entries: 3 | Claude Sonnet 5.5 not stated · 64.5% · Source cost unavailablemeasured/reported by: Anthropic | Cannot be shown because no values have source costs |
| AA-Briefcase v1.1(rubric)Knowledge · official · Introducing Claude Fable 5.2 and Claude Mythos 5.2 · Entries: 3 | Claude Fable 5.2 not stated · 66.1% · Source cost unavailablemeasured/reported by: Anthropic | Cannot be shown because no values have source costs |
| AA-Briefcase v1.1(Elo)Knowledge · official · Introducing Claude Fable 5.2 and Claude Mythos 5.2 · Entries: 3 | Claude Opus 5.5 not stated · 1,800 · Source cost unavailablemeasured/reported by: Anthropic | Cannot be shown because no values have source costs |
| Remote Labor Index(RLI)Knowledge · official · Introducing Claude Fable 5.2 and Claude Mythos 5.2 · Entries: 3 | Claude Fable 5.2 High · 41.3% · Source cost unavailablemeasured/reported by: Anthropic | Cannot be shown because no values have source costs |
| GDPval-AA v2.1Knowledge · official · Claude Fable 5.2 and Claude Mythos 5.2 System Card · Entries: 2 | Only effort comparisons for the same model (entries: 2)Claude Fable 5.2 Max · 1,831 · Source cost unavailablemeasured/reported by: Artificial AnalysisClaude Fable 5.2 Xhigh · 1,751 · Source cost unavailablemeasured/reported by: Artificial Analysis
| |
| Humanity's Last Exam (with tools)Reasoning · official · Introducing Claude Opus 4.6 · Entries: 1 | No comparison (entries: 1)Claude Opus 4.6 Max · 53.0% · Source cost unavailablemeasured/reported by: Anthropic | |
| Humanity's Last Exam (without tools)Reasoning · official · Claude Fable 5.2 and Claude Mythos 5.2 System Card · Entries: 1 | No comparison (entries: 1)Claude Fable 5.2 Max · 73.2% · Source cost unavailablemeasured/reported by: Anthropic | |
| Humanity's Last Exam (without tools)Reasoning · official · Claude Opus 5.5 System Card · Entries: 1 | No comparison (entries: 1)Claude Opus 5.5 Max · 64.4% · Source cost unavailablemeasured/reported by: Anthropic | |
Manager / oversight
Emphasized categories: Coding · Reasoning
Details by group (groups: 12)
| Supporting comparison group | Top result (performance first) | Candidates on the cost-efficiency frontier (lowest cost first) |
|---|---|---|
| Chess Puzzles(Epoch AI)Reasoning · third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, chess_puzzles.csv) CC BY 4.0 · Entries: 19 | Claude Fable 5.2 Max · 79.0% · Source cost unavailablemeasured/reported by: Epoch AI | Cannot be shown because no values have source costs |
| GPQA DiamondReasoning · third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, gpqa_diamond.csv) CC BY 4.0 · Entries: 17 | Claude Sonnet 5.5 Max · 95.6% · Source cost unavailablemeasured/reported by: Epoch AI | Cannot be shown because no values have source costs |
| MirrorCodeCoding · third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, mirrorcode.csv) CC BY 4.0 · Entries: 9 | Claude Sonnet 5.5 Max · 97.8% · Source cost unavailablemeasured/reported by: Epoch AI | Cannot be shown because no values have source costs |
| Humanity's Last Exam (with tools)Reasoning · official · Introducing Claude Fable 5.2 and Claude Mythos 5.2 · Entries: 15 | Claude Fable 5.2 Max · 74.2% · $4.51 (Cost per attempt)measured/reported by: Anthropic | Claude Opus 5.5 Low · 57.4% · $0.08 (Cost per attempt)measured/reported by: AnthropicClaude Opus 5.5 Med · 63.0% · $0.17 (Cost per attempt)measured/reported by: AnthropicClaude Fable 5.2 Low · 68.4% · $0.19 (Cost per attempt)measured/reported by: AnthropicClaude Fable 5.2 Med · 68.6% · $0.21 (Cost per attempt)measured/reported by: AnthropicClaude Fable 5.2 High · 71.5% · $0.63 (Cost per attempt)measured/reported by: AnthropicClaude Fable 5.2 Xhigh · 73.4% · $1.48 (Cost per attempt)measured/reported by: Anthropic |
| Humanity's Last Exam (without tools)Reasoning · official · Introducing Claude Haiku 5.5 · Entries: 11 | Claude Sonnet 5.5 Max · 56.9% · $0.89 (Cost per attempt)measured/reported by: Anthropic | Claude Haiku 5.5 Low · 30.9% · $0.003 (Cost per attempt)measured/reported by: AnthropicClaude Haiku 5.5 Med · 35.5% · $0.007 (Cost per attempt)measured/reported by: AnthropicClaude Haiku 5.5 High · 39.7% · $0.02 (Cost per attempt)measured/reported by: AnthropicClaude Sonnet 5.5 Med · 43.2% · $0.03 (Cost per attempt)measured/reported by: AnthropicClaude Haiku 5.5 Xhigh · 44.1% · $0.06 (Cost per attempt)measured/reported by: AnthropicClaude Sonnet 5.5 High · 47.9% · $0.08 (Cost per attempt)measured/reported by: AnthropicClaude Sonnet 5.5 Xhigh · 53.0% · $0.20 (Cost per attempt)measured/reported by: Anthropic |
| Humanity's Last Exam (with tools)Reasoning · official · Introducing Claude Opus 5.5 · Entries: 3 | Claude Opus 5.5 Max · 67.7% · Source cost unavailablemeasured/reported by: Anthropic | Cannot be shown because no values have source costs |
| Humanity's Last Exam (with tools)Reasoning · official · Introducing Claude Sonnet 5.5 · Entries: 3 | Claude Opus 5.5 not stated · 67.7% · Source cost unavailablemeasured/reported by: Anthropic | Cannot be shown because no values have source costs |
| Humanity's Last Exam (with tools)Reasoning · official · Introducing Claude Haiku 5.5 · Entries: 3 | Claude Sonnet 5.5 not stated · 64.5% · Source cost unavailablemeasured/reported by: Anthropic | Cannot be shown because no values have source costs |
| LMArena Text Coding (without style control)Coding · third-party · lmarena-ai/leaderboard-dataset text/latest-00000-of-00001.parquet (CC BY 4.0) · Entries: 3 | Claude Opus 5.5 High · 1,542 · Source cost unavailablemeasured/reported by: LMArena | Cannot be shown because no values have source costs |
| Humanity's Last Exam (with tools)Reasoning · official · Introducing Claude Opus 4.6 · Entries: 1 | No comparison (entries: 1)Claude Opus 4.6 Max · 53.0% · Source cost unavailablemeasured/reported by: Anthropic | |
| Humanity's Last Exam (without tools)Reasoning · official · Claude Fable 5.2 and Claude Mythos 5.2 System Card · Entries: 1 | No comparison (entries: 1)Claude Fable 5.2 Max · 73.2% · Source cost unavailablemeasured/reported by: Anthropic | |
| Humanity's Last Exam (without tools)Reasoning · official · Claude Opus 5.5 System Card · Entries: 1 | No comparison (entries: 1)Claude Opus 5.5 Max · 64.4% · Source cost unavailablemeasured/reported by: Anthropic | |
Implementer / task executor
Emphasized categories: Agentic coding · Coding · Tool use
Details by group (groups: 27)
| Supporting comparison group | Top result (performance first) | Candidates on the cost-efficiency frontier (lowest cost first) |
|---|---|---|
| MirrorCodeCoding · third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, mirrorcode.csv) CC BY 4.0 · Entries: 9 | Claude Sonnet 5.5 Max · 97.8% · Source cost unavailablemeasured/reported by: Epoch AI | Cannot be shown because no values have source costs |
| Terminal-Bench 4.0Agentic coding · official · Accuracy vs Cost figure announced by Anthropic (Opus 5.5 and Sonnet 5.5 pages) · Entries: 25 | Claude Sonnet 5.5 Max · 70.6% · $12.54 (Cost per attempt)measured/reported by: Anthropic | Claude Sonnet 5.5 Low · 20.0% · $0.76 (Cost per attempt)measured/reported by: AnthropicClaude Sonnet 5.5 Med · 28.8% · $0.83 (Cost per attempt)measured/reported by: AnthropicClaude Opus 5.5 Low · 38.5% · $1.29 (Cost per attempt)measured/reported by: AnthropicClaude Sonnet 5.5 High · 43.0% · $1.94 (Cost per attempt)measured/reported by: AnthropicClaude Opus 5.5 Med · 57.6% · $2.94 (Cost per attempt)measured/reported by: AnthropicClaude Opus 5.5 High · 64.2% · $3.88 (Cost per attempt)measured/reported by: AnthropicClaude Opus 5.5 Xhigh · 66.4% · $7.35 (Cost per attempt)measured/reported by: Anthropic |
| FrontierCode v1.1(main set)Agentic coding · official · Accuracy vs Cost figure announced by Anthropic (Opus 5.5 and Sonnet 5.5 pages) · Entries: 25 | Claude Opus 5.5 Med · 54.6% · $0.80 (Cost per task)measured/reported by: Anthropic | Claude Sonnet 5.5 Low · 29.3% · $0.19 (Cost per task)measured/reported by: AnthropicClaude Sonnet 5.5 Med · 36.5% · $0.24 (Cost per task)measured/reported by: AnthropicClaude Opus 5.5 Low · 47.3% · $0.40 (Cost per task)measured/reported by: AnthropicClaude Sonnet 5.5 High · 49.4% · $0.42 (Cost per task)measured/reported by: Anthropic |
| τ²-bench banking_knowledge (v1.0.1)Tool use · third-party · τ²-bench v1.0.1 banking_knowledge (run by Sierra, search alltools) · Entries: 5 | Claude Opus 5 Max · 48.7% · Source cost unavailablemeasured/reported by: Sierra | Cannot be shown because no values have source costs |
| CursorBench 4.0Agentic coding · official · Introducing Claude Opus 5.5 · Entries: 15 | Claude Opus 5.5 Max · 57.8% · $13.43 (Cost per task)measured/reported by: Anthropic | Claude Opus 5.5 Low · 43.7% · $1.18 (Cost per task)measured/reported by: AnthropicClaude Opus 5.5 Med · 52.5% · $2.90 (Cost per task)measured/reported by: AnthropicClaude Opus 5.5 High · 56.0% · $3.97 (Cost per task)measured/reported by: AnthropicClaude Opus 5.5 Xhigh · 56.0% · $6.99 (Cost per task)measured/reported by: Anthropic |
| WANDRTool use · official · Introducing Claude Opus 5.5 · Entries: 15 | Claude Opus 5.5 Max · 72.3% · $37.92 (Cost per attempt)measured/reported by: Anthropic | Claude Opus 5.5 Low · 31.2% · $1.20 (Cost per attempt)measured/reported by: AnthropicClaude Opus 5 Low · 50.5% · $10.65 (Cost per attempt)measured/reported by: AnthropicClaude Opus 5.5 Med · 62.8% · $11.20 (Cost per attempt)measured/reported by: AnthropicClaude Opus 5.5 High · 67.3% · $16.88 (Cost per attempt)measured/reported by: AnthropicClaude Opus 5.5 Xhigh · 71.3% · $29.06 (Cost per attempt)measured/reported by: Anthropic |
| CursorBench 4.0Agentic coding · official · Introducing Claude Sonnet 5.5 · Entries: 15 | Claude Opus 5.5 Max · 57.8% · $13.43 (Cost per task)measured/reported by: Anthropic | Claude Sonnet 5.5 Low · 35.8% · $0.50 (Cost per task)measured/reported by: AnthropicClaude Sonnet 5.5 Med · 39.2% · $0.70 (Cost per task)measured/reported by: AnthropicClaude Opus 5.5 Low · 43.7% · $1.17 (Cost per task)measured/reported by: AnthropicClaude Sonnet 5.5 High · 47.8% · $1.67 (Cost per task)measured/reported by: AnthropicClaude Opus 5.5 Med · 52.5% · $2.91 (Cost per task)measured/reported by: AnthropicClaude Sonnet 5.5 Xhigh · 53.1% · $3.88 (Cost per task)measured/reported by: AnthropicClaude Opus 5.5 High · 56.0% · $3.97 (Cost per task)measured/reported by: AnthropicClaude Opus 5.5 Xhigh · 56.0% · $6.98 (Cost per task)measured/reported by: Anthropic |
| FrontierCode 2.0Agentic coding · official · Introducing Claude Fable 5.2 and Claude Mythos 5.2 · Entries: 15 | Claude Fable 5.2 Xhigh · 49.1% · $14.19 (Cost per task)measured/reported by: Anthropic | Claude Opus 5.5 Low · 14.7% · $0.75 (Cost per task)measured/reported by: AnthropicClaude Opus 5.5 Med · 30.1% · $1.70 (Cost per task)measured/reported by: AnthropicClaude Fable 5.2 Low · 34.2% · $2.10 (Cost per task)measured/reported by: AnthropicClaude Opus 5.5 High · 34.5% · $2.42 (Cost per task)measured/reported by: AnthropicClaude Fable 5.2 Med · 37.6% · $2.48 (Cost per task)measured/reported by: AnthropicClaude Opus 5.5 Xhigh · 43.4% · $5.18 (Cost per task)measured/reported by: AnthropicClaude Fable 5.2 High · 45.3% · $6.72 (Cost per task)measured/reported by: AnthropicClaude Opus 5.5 Max · 45.5% · $12.82 (Cost per task)measured/reported by: Anthropic |
| WANDR(relaxed score)Tool use · official · Introducing Claude Fable 5.2 and Claude Mythos 5.2 · Entries: 15 | Claude Fable 5.2 Max · 82.1% · $53.11 (Cost per attempt)measured/reported by: Anthropic | Claude Opus 5.5 Low · 31.2% · $1.25 (Cost per attempt)measured/reported by: AnthropicClaude Opus 5.5 Med · 62.8% · $11.40 (Cost per attempt)measured/reported by: AnthropicClaude Fable 5.2 Low · 71.4% · $12.66 (Cost per attempt)measured/reported by: AnthropicClaude Fable 5.2 Med · 73.6% · $15.71 (Cost per attempt)measured/reported by: AnthropicClaude Fable 5.2 High · 79.3% · $29.20 (Cost per attempt)measured/reported by: AnthropicClaude Fable 5.2 Xhigh · 80.6% · $39.84 (Cost per attempt)measured/reported by: Anthropic |
| AutomationBenchTool use · official · Introducing Claude Opus 5.5 · Entries: 11 | Claude Opus 5.5 Max · 40.0% · $1.37 (Cost per task)measured/reported by: Zapier | Claude Opus 5.5 Low · 23.3% · $0.50 (Cost per task)measured/reported by: ZapierClaude Opus 5.5 Med · 28.6% · $0.64 (Cost per task)measured/reported by: ZapierClaude Opus 5.5 High · 32.0% · $0.70 (Cost per task)measured/reported by: ZapierClaude Opus 5.5 Xhigh · 34.4% · $0.86 (Cost per task)measured/reported by: ZapierEntries without source costs excluded: 1 |
| Terminal-Bench 4.0Agentic coding · official · Introducing Claude Haiku 5.5 · Entries: 11 | Claude Sonnet 5.5 Max · 70.6% · $10.44 (Cost per attempt)measured/reported by: Anthropic | Claude Haiku 5.5 Low · 12.7% · $0.42 (Cost per attempt)measured/reported by: AnthropicClaude Sonnet 5.5 Low · 20.0% · $0.62 (Cost per attempt)measured/reported by: AnthropicClaude Sonnet 5.5 Med · 28.8% · $0.68 (Cost per attempt)measured/reported by: AnthropicClaude Sonnet 5.5 High · 43.0% · $1.46 (Cost per attempt)measured/reported by: AnthropicClaude Sonnet 5.5 Xhigh · 61.5% · $4.34 (Cost per attempt)measured/reported by: Anthropic |
| Terminal-Bench-Science 0.1Agentic coding · official · Introducing Claude Opus 5.5 · Entries: 3 | Claude Opus 5.5 Max · 58.7% · Source cost unavailablemeasured/reported by: Anthropic | Cannot be shown because no values have source costs |
| FrontierSWE 2Agentic coding · official · Introducing Claude Fable 5.2 and Claude Mythos 5.2 · Entries: 3 | Claude Fable 5.2 not stated · 69.9% · Source cost unavailablemeasured/reported by: Anthropic | Cannot be shown because no values have source costs |
| AutomationBenchTool use · official · Introducing Claude Fable 5.2 and Claude Mythos 5.2 · Entries: 3 | Claude Fable 5.2 not stated · 51.4% · Source cost unavailablemeasured/reported by: Anthropic | Cannot be shown because no values have source costs |
| Terminal-Bench-Science 0.1Agentic coding · official · Introducing Claude Fable 5.2 and Claude Mythos 5.2 · Entries: 3 | Claude Fable 5.2 not stated · 74.6% · Source cost unavailablemeasured/reported by: Anthropic | Cannot be shown because no values have source costs |
| LMArena Text Coding (without style control)Coding · third-party · lmarena-ai/leaderboard-dataset text/latest-00000-of-00001.parquet (CC BY 4.0) · Entries: 3 | Claude Opus 5.5 High · 1,542 · Source cost unavailablemeasured/reported by: LMArena | Cannot be shown because no values have source costs |
| FrontierCode v1.1(main set)Agentic coding · official · Introducing Claude Haiku 5.5 · Entries: 2 | Claude Sonnet 5.5 Xhigh · 52.1% · Source cost unavailablemeasured/reported by: Anthropic | Cannot be shown because no values have source costs |
| CursorBench 4.0Agentic coding · official · Claude Fable 5.2 and Claude Mythos 5.2 System Card · Entries: 5 | Only effort comparisons for the same model (entries: 5)Claude Fable 5.2 Low · 53.2% · Source cost unavailablemeasured/reported by: CursorClaude Fable 5.2 Med · 54.8% · Source cost unavailablemeasured/reported by: CursorClaude Fable 5.2 High · 56.9% · Source cost unavailablemeasured/reported by: CursorClaude Fable 5.2 Xhigh · 56.3% · Source cost unavailablemeasured/reported by: CursorClaude Fable 5.2 Max · 60.0% · Source cost unavailablemeasured/reported by: Cursor
| |
| Terminal-Bench 4.0Agentic coding · official · Claude Fable 5.2 and Claude Mythos 5.2 System Card · Entries: 1 | No comparison (entries: 1)Claude Fable 5.2 Max · 67.6% · Source cost unavailablemeasured/reported by: Anthropic | |
| Terminal-Bench-Science 0.1Agentic coding · official · Claude Sonnet 5.5 System Card · Entries: 1 | No comparison (entries: 1)Claude Sonnet 5.5 Max · 59.9% · Source cost unavailablemeasured/reported by: Anthropic | |
| Terminal-Bench-Science 0.1Agentic coding · official · Claude Haiku 5.5 System Card · Entries: 1 | No comparison (entries: 1)Claude Haiku 5.5 Max · 20.6% · Source cost unavailablemeasured/reported by: Anthropic | |
| FrontierSWE 2Agentic coding · official · Claude Sonnet 5.5 System Card · Entries: 1 | No comparison (entries: 1)Claude Sonnet 5.5 Max · 61.9% · Source cost unavailablemeasured/reported by: Proximal | |
| FrontierSWE 2Agentic coding · official · Claude Haiku 5.5 System Card · Entries: 1 | No comparison (entries: 1)Claude Haiku 5.5 Max · 43.8% · Source cost unavailablemeasured/reported by: Proximal | |
| τ²-bench airline (v1.0.1)Tool use · third-party · τ²-bench v1.0.1 (run by Sierra) · Entries: 1 | No comparison (entries: 1)Claude Sonnet 4.5 not stated · 72.0% · $0.30 (Average cost in USD to run one trajectory in this domain)measured/reported by: Sierra | |
| τ²-bench retail (v1.0.1)Tool use · third-party · τ²-bench v1.0.1 (run by Sierra) · Entries: 1 | No comparison (entries: 1)Claude Sonnet 4.5 not stated · 72.4% · $0.25 (Average cost in USD to run one trajectory in this domain)measured/reported by: Sierra | |
| τ²-bench telecom (v1.0.1)Tool use · third-party · τ²-bench v1.0.1 (run by Sierra) · Entries: 1 | No comparison (entries: 1)Claude Sonnet 4.5 not stated · 84.9% · $0.49 (Average cost in USD to run one trajectory in this domain)measured/reported by: Sierra | |
| τ²-bench banking_knowledge (v1.0.1)Tool use · third-party · τ²-bench v1.0.1 banking_knowledge (run by Sierra, search terminal) · Entries: 1 | No comparison (entries: 1)Claude Sonnet 4.5 not stated · 25.3% · $4.05 (Average cost in USD to run one trajectory in this domain)measured/reported by: Sierra | |
Simple tasks
Emphasized categories: Coding · Tool use
Details by group (groups: 11)
| Supporting comparison group | Top result (performance first) | Candidates on the cost-efficiency frontier (lowest cost first) |
|---|---|---|
| MirrorCodeCoding · third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, mirrorcode.csv) CC BY 4.0 · Entries: 9 | Claude Sonnet 5.5 Max · 97.8% · Source cost unavailablemeasured/reported by: Epoch AI | Cannot be shown because no values have source costs |
| τ²-bench banking_knowledge (v1.0.1)Tool use · third-party · τ²-bench v1.0.1 banking_knowledge (run by Sierra, search alltools) · Entries: 5 | Claude Opus 5 Max · 48.7% · Source cost unavailablemeasured/reported by: Sierra | Cannot be shown because no values have source costs |
| WANDRTool use · official · Introducing Claude Opus 5.5 · Entries: 15 | Claude Opus 5.5 Max · 72.3% · $37.92 (Cost per attempt)measured/reported by: Anthropic | Claude Opus 5.5 Low · 31.2% · $1.20 (Cost per attempt)measured/reported by: AnthropicClaude Opus 5 Low · 50.5% · $10.65 (Cost per attempt)measured/reported by: AnthropicClaude Opus 5.5 Med · 62.8% · $11.20 (Cost per attempt)measured/reported by: AnthropicClaude Opus 5.5 High · 67.3% · $16.88 (Cost per attempt)measured/reported by: AnthropicClaude Opus 5.5 Xhigh · 71.3% · $29.06 (Cost per attempt)measured/reported by: Anthropic |
| WANDR(relaxed score)Tool use · official · Introducing Claude Fable 5.2 and Claude Mythos 5.2 · Entries: 15 | Claude Fable 5.2 Max · 82.1% · $53.11 (Cost per attempt)measured/reported by: Anthropic | Claude Opus 5.5 Low · 31.2% · $1.25 (Cost per attempt)measured/reported by: AnthropicClaude Opus 5.5 Med · 62.8% · $11.40 (Cost per attempt)measured/reported by: AnthropicClaude Fable 5.2 Low · 71.4% · $12.66 (Cost per attempt)measured/reported by: AnthropicClaude Fable 5.2 Med · 73.6% · $15.71 (Cost per attempt)measured/reported by: AnthropicClaude Fable 5.2 High · 79.3% · $29.20 (Cost per attempt)measured/reported by: AnthropicClaude Fable 5.2 Xhigh · 80.6% · $39.84 (Cost per attempt)measured/reported by: Anthropic |
| AutomationBenchTool use · official · Introducing Claude Opus 5.5 · Entries: 11 | Claude Opus 5.5 Max · 40.0% · $1.37 (Cost per task)measured/reported by: Zapier | Claude Opus 5.5 Low · 23.3% · $0.50 (Cost per task)measured/reported by: ZapierClaude Opus 5.5 Med · 28.6% · $0.64 (Cost per task)measured/reported by: ZapierClaude Opus 5.5 High · 32.0% · $0.70 (Cost per task)measured/reported by: ZapierClaude Opus 5.5 Xhigh · 34.4% · $0.86 (Cost per task)measured/reported by: ZapierEntries without source costs excluded: 1 |
| AutomationBenchTool use · official · Introducing Claude Fable 5.2 and Claude Mythos 5.2 · Entries: 3 | Claude Fable 5.2 not stated · 51.4% · Source cost unavailablemeasured/reported by: Anthropic | Cannot be shown because no values have source costs |
| LMArena Text Coding (without style control)Coding · third-party · lmarena-ai/leaderboard-dataset text/latest-00000-of-00001.parquet (CC BY 4.0) · Entries: 3 | Claude Opus 5.5 High · 1,542 · Source cost unavailablemeasured/reported by: LMArena | Cannot be shown because no values have source costs |
| τ²-bench airline (v1.0.1)Tool use · third-party · τ²-bench v1.0.1 (run by Sierra) · Entries: 1 | No comparison (entries: 1)Claude Sonnet 4.5 not stated · 72.0% · $0.30 (Average cost in USD to run one trajectory in this domain)measured/reported by: Sierra | |
| τ²-bench retail (v1.0.1)Tool use · third-party · τ²-bench v1.0.1 (run by Sierra) · Entries: 1 | No comparison (entries: 1)Claude Sonnet 4.5 not stated · 72.4% · $0.25 (Average cost in USD to run one trajectory in this domain)measured/reported by: Sierra | |
| τ²-bench telecom (v1.0.1)Tool use · third-party · τ²-bench v1.0.1 (run by Sierra) · Entries: 1 | No comparison (entries: 1)Claude Sonnet 4.5 not stated · 84.9% · $0.49 (Average cost in USD to run one trajectory in this domain)measured/reported by: Sierra | |
| τ²-bench banking_knowledge (v1.0.1)Tool use · third-party · τ²-bench v1.0.1 banking_knowledge (run by Sierra, search terminal) · Entries: 1 | No comparison (entries: 1)Claude Sonnet 4.5 not stated · 25.3% · $4.05 (Average cost in USD to run one trajectory in this domain)measured/reported by: Sierra | |
Design / UI
Emphasized categories: Design and UI
No data
There are no benchmark results in these categories yet for models available in Claude Code.
Models available in Codex (details by group)
Advisor / design decisions
Emphasized categories: Reasoning · Knowledge · Math
Details by group (groups: 20)
| Supporting comparison group | Top result (performance first) | Candidates on the cost-efficiency frontier (lowest cost first) |
|---|---|---|
| FrontierMath Tier 4(v2)Math · third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, frontiermath_tier_4_v2.csv) CC BY 4.0 · Entries: 10 | GPT-6.1 Sol Max · 100.0% · Source cost unavailablemeasured/reported by: Epoch AI | Cannot be shown because no values have source costs |
| GPQA DiamondReasoning · third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, gpqa_diamond.csv) CC BY 4.0 · Entries: 6 | GPT-6 Astra Max · 95.8% · Source cost unavailablemeasured/reported by: Epoch AI | Cannot be shown because no values have source costs |
| OTIS Mock AIME 2024-2025Math · third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0 · Entries: 6 | GPT-6.1 Terra Max · 100.0% · Source cost unavailablemeasured/reported by: Epoch AIGPT-6.1 Sol Max · 100.0% · Source cost unavailablemeasured/reported by: Epoch AIGPT-6 Astra Max · 100.0% · Source cost unavailablemeasured/reported by: Epoch AI | Cannot be shown because no values have source costs |
| FrontierMath Tiers 1-3(v2)Math · third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, frontiermath_tiers_1_3_v2.csv) CC BY 4.0 · Entries: 5 | GPT-6.1 Sol Max · 93.7% · Source cost unavailablemeasured/reported by: Epoch AIGPT-6 Astra Max · 93.7% · Source cost unavailablemeasured/reported by: Epoch AI | Cannot be shown because no values have source costs |
| SimpleQA VerifiedKnowledge · third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, simpleqa_verified.csv) CC BY 4.0 · Entries: 5 | GPT-6 Astra Max · 75.6% · Source cost unavailablemeasured/reported by: Epoch AI | Cannot be shown because no values have source costs |
| Chess Puzzles(Epoch AI)Reasoning · third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, chess_puzzles.csv) CC BY 4.0 · Entries: 6 | GPT-6 Astra Max · 74.0% · Source cost unavailablemeasured/reported by: Epoch AI | Cannot be shown because no values have source costs |
| HealthBench Professional (length-adjusted score)Knowledge · official · Addendum to GPT-6 Astra System Card: GPT-6.1 Terra · Entries: 3 | GPT-6.1 Sol not stated · 64.2 · Source cost unavailablemeasured/reported by: OpenAI | Cannot be shown because no values have source costs |
| HealthBench Hard (length-adjusted score)Knowledge · official · Addendum to GPT-6 Astra System Card: GPT-6.1 Terra · Entries: 3 | GPT-6.1 Sol not stated · 36.2 · Source cost unavailablemeasured/reported by: OpenAI | Cannot be shown because no values have source costs |
| GDPval-AA v2.1Knowledge · official · Introducing Claude Opus 5.5 · Entries: 5 | Only effort comparisons for the same model (entries: 5)GPT-6 Astra Low · 1,366 · $0.85 (Estimated cost per task)measured/reported by: not statedGPT-6 Astra Med · 1,468 · $1.82 (Estimated cost per task)measured/reported by: not statedGPT-6 Astra High · 1,485 · $2.43 (Estimated cost per task)measured/reported by: not statedGPT-6 Astra Xhigh · 1,516 · $3.04 (Estimated cost per task)measured/reported by: not statedGPT-6 Astra Max · 1,542 · $4.53 (Estimated cost per task)measured/reported by: not stated
| |
| GDPval-AA v2.1Knowledge · official · Introducing Claude Haiku 5.5 · Entries: 5 | Only effort comparisons for the same model (entries: 5)GPT-6 Luna Low · 1,036 · $0.004 (Cost per task)measured/reported by: not statedGPT-6 Luna Med · 1,262 · $0.02 (Cost per task)measured/reported by: not statedGPT-6 Luna High · 1,344 · $0.03 (Cost per task)measured/reported by: not statedGPT-6 Luna Xhigh · 1,364 · $0.05 (Cost per task)measured/reported by: not statedGPT-6 Luna Max · 1,437 · $0.09 (Cost per task)measured/reported by: not stated
| |
| Humanity's Last Exam (with tools)Reasoning · official · Introducing Claude Opus 5.5 · Entries: 1 | No comparison (entries: 1)GPT-6 Astra not stated · 57.2% · Source cost unavailablemeasured/reported by: not stated | |
| AA-Briefcase v1.1(Elo)Knowledge · official · Introducing Claude Haiku 5.5 · Entries: 1 | No comparison (entries: 1)GPT-6 Luna not stated · 1,336 · Source cost unavailablemeasured/reported by: not stated | |
| AA-Briefcase v1.1(rubric)Knowledge · official · Introducing Claude Fable 5.2 and Claude Mythos 5.2 · Entries: 1 | No comparison (entries: 1)GPT-6 Astra not stated · 52.0% · Source cost unavailablemeasured/reported by: not stated | |
| AA-Briefcase v1.1(Elo)Knowledge · official · Introducing Claude Fable 5.2 and Claude Mythos 5.2 · Entries: 1 | No comparison (entries: 1)GPT-6 Astra not stated · 1,569 · Source cost unavailablemeasured/reported by: not stated | |
| Remote Labor Index(RLI)Knowledge · official · Introducing Claude Fable 5.2 and Claude Mythos 5.2 · Entries: 1 | No comparison (entries: 1)GPT-6 Astra not stated · 20.8% · Source cost unavailablemeasured/reported by: not stated | |
| Humanity's Last Exam (with tools)Reasoning · official · Introducing Claude Fable 5.2 and Claude Mythos 5.2 · Entries: 1 | No comparison (entries: 1)GPT-6 Astra not stated · 57.2% · Source cost unavailablemeasured/reported by: not stated | |
| HealthBench Professional (length-adjusted score)Knowledge · official · GPT-6 Astra System Card (Table 6. HealthBench Evaluations) · Entries: 1 | No comparison (entries: 1)GPT-6 Astra not stated · 64.7 · Source cost unavailablemeasured/reported by: OpenAI | |
| HealthBench Hard (length-adjusted score)Knowledge · official · GPT-6 Astra System Card (Table 6. HealthBench Evaluations) · Entries: 1 | No comparison (entries: 1)GPT-6 Astra not stated · 36.6 · Source cost unavailablemeasured/reported by: OpenAI | |
| GDPval-AA v2.1Knowledge · official · Claude Fable 5.2 and Claude Mythos 5.2 System Card · Entries: 1 | No comparison (entries: 1)GPT-6.1 Sol not stated · 1,609 · Source cost unavailablemeasured/reported by: Artificial Analysis | |
| AA-Briefcase v1.1(Elo)Knowledge · official · Claude Fable 5.2 and Claude Mythos 5.2 System Card · Entries: 1 | No comparison (entries: 1)GPT-6.1 Sol not stated · 1,555 · Source cost unavailablemeasured/reported by: Artificial Analysis | |
Manager / oversight
Emphasized categories: Coding · Reasoning
Details by group (groups: 6)
| Supporting comparison group | Top result (performance first) | Candidates on the cost-efficiency frontier (lowest cost first) |
|---|---|---|
| MirrorCodeCoding · third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, mirrorcode.csv) CC BY 4.0 · Entries: 7 | GPT-6.1 Terra Max · 76.7% · Source cost unavailablemeasured/reported by: Epoch AI | Cannot be shown because no values have source costs |
| GPQA DiamondReasoning · third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, gpqa_diamond.csv) CC BY 4.0 · Entries: 6 | GPT-6 Astra Max · 95.8% · Source cost unavailablemeasured/reported by: Epoch AI | Cannot be shown because no values have source costs |
| Chess Puzzles(Epoch AI)Reasoning · third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, chess_puzzles.csv) CC BY 4.0 · Entries: 6 | GPT-6 Astra Max · 74.0% · Source cost unavailablemeasured/reported by: Epoch AI | Cannot be shown because no values have source costs |
| LMArena Text Coding (without style control)Coding · third-party · lmarena-ai/leaderboard-dataset text/latest-00000-of-00001.parquet (CC BY 4.0) · Entries: 3 | GPT-6 Astra Max · 1,494 · Source cost unavailablemeasured/reported by: LMArena | Cannot be shown because no values have source costs |
| Humanity's Last Exam (with tools)Reasoning · official · Introducing Claude Opus 5.5 · Entries: 1 | No comparison (entries: 1)GPT-6 Astra not stated · 57.2% · Source cost unavailablemeasured/reported by: not stated | |
| Humanity's Last Exam (with tools)Reasoning · official · Introducing Claude Fable 5.2 and Claude Mythos 5.2 · Entries: 1 | No comparison (entries: 1)GPT-6 Astra not stated · 57.2% · Source cost unavailablemeasured/reported by: not stated | |
Implementer / task executor
Emphasized categories: Agentic coding · Coding · Tool use
Details by group (groups: 14)
| Supporting comparison group | Top result (performance first) | Candidates on the cost-efficiency frontier (lowest cost first) |
|---|---|---|
| MirrorCodeCoding · third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, mirrorcode.csv) CC BY 4.0 · Entries: 7 | GPT-6.1 Terra Max · 76.7% · Source cost unavailablemeasured/reported by: Epoch AI | Cannot be shown because no values have source costs |
| LMArena Text Coding (without style control)Coding · third-party · lmarena-ai/leaderboard-dataset text/latest-00000-of-00001.parquet (CC BY 4.0) · Entries: 3 | GPT-6 Astra Max · 1,494 · Source cost unavailablemeasured/reported by: LMArena | Cannot be shown because no values have source costs |
| Terminal-Bench 4.0Agentic coding · official · Accuracy vs Cost figure announced by Anthropic (Opus 5.5 and Sonnet 5.5 pages) · Entries: 5 | Only effort comparisons for the same model (entries: 5)GPT-6 Astra Low · 49.7% · $4.95 (Cost per attempt)measured/reported by: OpenAIGPT-6 Astra Med · 53.9% · $6.15 (Cost per attempt)measured/reported by: OpenAIGPT-6 Astra High · 57.9% · $7.21 (Cost per attempt)measured/reported by: OpenAIGPT-6 Astra Xhigh · 57.6% · $7.48 (Cost per attempt)measured/reported by: OpenAIGPT-6 Astra Max · 56.7% · $10.35 (Cost per attempt)measured/reported by: OpenAI
| |
| FrontierCode v1.1(main set)Agentic coding · official · Accuracy vs Cost figure announced by Anthropic (Opus 5.5 and Sonnet 5.5 pages) · Entries: 5 | Only effort comparisons for the same model (entries: 5)GPT-6 Astra Low · 45.3% · $1.59 (Cost per task)measured/reported by: not statedGPT-6 Astra Med · 48.8% · $2.28 (Cost per task)measured/reported by: not statedGPT-6 Astra High · 50.9% · $2.85 (Cost per task)measured/reported by: not statedGPT-6 Astra Xhigh · 50.6% · $3.10 (Cost per task)measured/reported by: not statedGPT-6 Astra Max · 53.3% · $4.36 (Cost per task)measured/reported by: not stated
| |
| AutomationBenchTool use · official · Introducing Claude Opus 5.5 · Entries: 5 | Only effort comparisons for the same model (entries: 5)GPT-6 Astra Low · 30.3% · $1.08 (Cost per task)measured/reported by: ZapierGPT-6 Astra Med · 34.1% · $1.28 (Cost per task)measured/reported by: ZapierGPT-6 Astra High · 37.1% · $1.45 (Cost per task)measured/reported by: ZapierGPT-6 Astra Xhigh · 39.0% · $1.53 (Cost per task)measured/reported by: ZapierGPT-6 Astra Max · 41.4% · $1.77 (Cost per task)measured/reported by: Zapier
| |
| FrontierCode 2.0Agentic coding · official · Introducing Claude Fable 5.2 and Claude Mythos 5.2 · Entries: 5 | Only effort comparisons for the same model (entries: 5)GPT-6 Astra Low · 31.1% · $2.44 (Cost per task)measured/reported by: not statedGPT-6 Astra Med · 36.8% · $3.49 (Cost per task)measured/reported by: not statedGPT-6 Astra High · 38.3% · $4.28 (Cost per task)measured/reported by: not statedGPT-6 Astra Xhigh · 41.4% · $4.54 (Cost per task)measured/reported by: not statedGPT-6 Astra Max · 42.8% · $6.68 (Cost per task)measured/reported by: not stated
| |
| Terminal-Bench-Science 0.1Agentic coding · official · Introducing Claude Opus 5.5 · Entries: 1 | No comparison (entries: 1)GPT-6 Astra not stated · 64.6% · Source cost unavailablemeasured/reported by: OpenAI | |
| Terminal-Bench 4.0Agentic coding · official · Introducing Claude Haiku 5.5 · Entries: 1 | No comparison (entries: 1)GPT-6 Luna not stated · 16.4% · Source cost unavailablemeasured/reported by: not stated | |
| FrontierCode v1.1(main set)Agentic coding · official · Introducing Claude Haiku 5.5 · Entries: 1 | No comparison (entries: 1)GPT-6 Luna not stated · 42.4% · Source cost unavailablemeasured/reported by: not stated | |
| FrontierSWE 2Agentic coding · official · Introducing Claude Fable 5.2 and Claude Mythos 5.2 · Entries: 1 | No comparison (entries: 1)GPT-6 Astra not stated · 65.5% · Source cost unavailablemeasured/reported by: not stated | |
| AutomationBenchTool use · official · Introducing Claude Fable 5.2 and Claude Mythos 5.2 · Entries: 1 | No comparison (entries: 1)GPT-6 Astra not stated · 41.4% · Source cost unavailablemeasured/reported by: not stated | |
| Terminal-Bench-Science 0.1Agentic coding · official · Introducing Claude Fable 5.2 and Claude Mythos 5.2 · Entries: 1 | No comparison (entries: 1)GPT-6 Astra not stated · 68.1% · Source cost unavailablemeasured/reported by: not stated | |
| Terminal-Bench-Science 0.1Agentic coding · official · Claude Haiku 5.5 System Card · Entries: 1 | No comparison (entries: 1)GPT-6.1 Sol Max · 57.0% · Source cost unavailablemeasured/reported by: OpenAI | |
| τ²-bench banking_knowledge (v1.0.1)Tool use · third-party · τ²-bench v1.0.1 banking_knowledge (run by Sierra, search alltools) · Entries: 1 | No comparison (entries: 1)GPT-5.5 Xhigh · 44.6% · Source cost unavailablemeasured/reported by: Sierra | |
Simple tasks
Emphasized categories: Coding · Tool use
Details by group (groups: 5)
| Supporting comparison group | Top result (performance first) | Candidates on the cost-efficiency frontier (lowest cost first) |
|---|---|---|
| MirrorCodeCoding · third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, mirrorcode.csv) CC BY 4.0 · Entries: 7 | GPT-6.1 Terra Max · 76.7% · Source cost unavailablemeasured/reported by: Epoch AI | Cannot be shown because no values have source costs |
| LMArena Text Coding (without style control)Coding · third-party · lmarena-ai/leaderboard-dataset text/latest-00000-of-00001.parquet (CC BY 4.0) · Entries: 3 | GPT-6 Astra Max · 1,494 · Source cost unavailablemeasured/reported by: LMArena | Cannot be shown because no values have source costs |
| AutomationBenchTool use · official · Introducing Claude Opus 5.5 · Entries: 5 | Only effort comparisons for the same model (entries: 5)GPT-6 Astra Low · 30.3% · $1.08 (Cost per task)measured/reported by: ZapierGPT-6 Astra Med · 34.1% · $1.28 (Cost per task)measured/reported by: ZapierGPT-6 Astra High · 37.1% · $1.45 (Cost per task)measured/reported by: ZapierGPT-6 Astra Xhigh · 39.0% · $1.53 (Cost per task)measured/reported by: ZapierGPT-6 Astra Max · 41.4% · $1.77 (Cost per task)measured/reported by: Zapier
| |
| AutomationBenchTool use · official · Introducing Claude Fable 5.2 and Claude Mythos 5.2 · Entries: 1 | No comparison (entries: 1)GPT-6 Astra not stated · 41.4% · Source cost unavailablemeasured/reported by: not stated | |
| τ²-bench banking_knowledge (v1.0.1)Tool use · third-party · τ²-bench v1.0.1 banking_knowledge (run by Sierra, search alltools) · Entries: 1 | No comparison (entries: 1)GPT-5.5 Xhigh · 44.6% · Source cost unavailablemeasured/reported by: Sierra | |
Design / UI
Emphasized categories: Design and UI
No data
There are no benchmark results in these categories yet for models available in Codex.