All model rankings (records)
Values with different source types or measurement conditions (comparison groups) are ranked separately. Overall rankings aggregate only models with values for the same set of comparison groups, and costs are not combined across benchmarks. See About for the methods.
These records show ranks, percentiles, and frontiers for all listed models (models: 57), including legacy models and those unavailable in Claude Code / Codex. For rankings of currently available models only, see Rankings. Ranks differ because the set of models differs.
Overall (performance first)
Only model × effort entries with values for the same set of comparison groups are aggregated. Within each group, included entries are ranked and converted to percentiles (first = 100, last = 0), averaged across groups of the same benchmark, then averaged across benchmarks for ranking. Multiple groups for one benchmark do not give it extra weight. Aggregations cover different groups, so their scores are not comparable.
Aggregations are ordered by descending model count, benchmark count, then entry count. The aggregation containing the most models currently available in Claude Code / Codex opens first. The rest are collapsed under “Other aggregations” by source type.
Aggregation criteria: minimum models: 3 (different efforts count as one model); minimum benchmarks: 3 (multiple comparison groups for the same benchmark count as one). Coverage n/N counts benchmarks; N is the number of benchmarks with values of the same source type. “measured/reported by” for each group shows the parties and counts for values included in that aggregation. Even in official announcements, the publisher may have measured values for other companies’ models.
third-party · Models: 36 · Entries: 36 × Groups: 3 (benchmark coverage 3/14)GPQA Diamond, FrontierMath Tiers 1-3(v2), FrontierMath Tier 4(v2)
- GPQA Diamondthird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, gpqa_diamond.csv) CC BY 4.0measured/reported by: Epoch AI (36)
- FrontierMath Tiers 1-3(v2)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, frontiermath_tiers_1_3_v2.csv) CC BY 4.0measured/reported by: Epoch AI (36)
- FrontierMath Tier 4(v2)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, frontiermath_tier_4_v2.csv) CC BY 4.0measured/reported by: Epoch AI (36)
| Rank | Model | effort | Mean percentile | Benchmarks with values | 1. GPQA Diamond | 2. FrontierMath Tiers 1-3(v2) | 3. FrontierMath Tier 4(v2) |
|---|---|---|---|---|---|---|---|
| 1 | GPT-6 Astra | Max | 95 | 9/14 | 95.8% | 93.7% | 97.6% |
| 2 | GPT-6.1 Sol | Max | 94 | 9/14 | 95.4% | 93.7% | 100.0% |
| 3 | Gemini 4 Carbon | High | 92 | 8/14 | 96.5% | 90.5% | 95.1% |
| 4 | Claude Fable 5.2 | Max | 88 | 6/14 | 93.9% | 95.1% | 100.0% |
| 5 | GPT-6.1 Terra | Max | 84 | 6/14 | 94.3% | 90.9% | 92.7% |
| 6 | Claude Sonnet 5.5 | Max | 81 | 7/14 | 95.6% | 88.8% | 80.5% |
| 7 | GPT-6 Sol | Max | 80 | 7/14 | 94.3% | 89.8% | 90.0% |
| 8 | Gemini 4 Argon | High | 80 | 9/14 | 96.4% | 86.0% | 78.1% |
| 9 | GPT-5.6 Sol | Max | 73 | 6/14 | 93.5% | 89.1% | 82.9% |
| 10 | Claude Opus 5.5 | Max | 70 | 7/14 | 90.6% | 91.2% | 95.0% |
| 11 | GPT-5.6 Terra | Max | 66 | 7/14 | 93.3% | 86.0% | 70.7% |
| 12 | Kimi K3.1 Pro | Max | 66 | 6/14 | 94.2% | 78.3% | 65.9% |
| 13 | Gemini 3.7 Flash | High | 57 | 7/14 | 94.8% | 71.6% | 36.6% |
| 14 | GPT-5.6 Luna | Max | 56 | 7/14 | 91.6% | 82.1% | 61.0% |
| 15 | Claude Fable 5 | Max | 55 | 7/14 | 85.9% | 87.0% | 90.2% |
| 16 | Claude Opus 4.8 | Max | 53 | 8/14 | 91.0% | 80.0% | 56.1% |
| 17 | DeepSeek-V4.1-Flash-1017 | Max | 50 | 6/14 | 93.2% | 71.2% | 46.3% |
| 18 | Qwen3.8-Max | Xhigh | 50 | 8/14 | 92.7% | 74.7% | 46.3% |
| 19 | GPT-6 Luna | Max | 49 | 9/14 | 90.5% | 79.0% | 56.1% |
| 20 | Kimi K3 | Max | 49 | 8/14 | 93.1% | 72.2% | 39.0% |
| 21 | Gemini 3.8 Flash | High | 46 | 7/14 | 95.4% | 68.4% | 22.0% |
| 22 | Claude Haiku 5.5 | Max | 42 | 6/14 | 89.7% | 75.1% | 46.3% |
| 23 | Grok 4.6 | Xhigh | 40 | 7/14 | 93.2% | 66.0% | 31.7% |
| 24 | Gemini 3.1 Pro Preview | not stated | 38 | 5/14 | 94.1% | 59.7% | 26.8% |
| 25 | GLM-5.3 | Max | 35 | 6/14 | 90.9% | 68.8% | 29.3% |
| 26 | Gemini 3.6 Flash | High | 35 | 7/14 | 94.1% | 59.0% | 22.0% |
| 27 | DeepSeek-V4-Pro-0813 | Max | 30 | 7/14 | 91.7% | 64.6% | 26.8% |
| 28 | Claude Opus 4.7 | Max | 30 | 6/14 | 86.4% | 70.2% | 31.7% |
| 29 | DeepSeek-V4.1-Flash | not stated | 27 | 8/14 | 89.8% | 67.4% | 26.8% |
| 30 | Claude Opus 4.6 | Max | 23 | 8/14 | 88.4% | 66.0% | 26.8% |
| 31 | Grok 4.7 | Xhigh | 22 | 7/14 | 92.7% | 53.0% | 17.1% |
| 32 | Mistral Large 4 | High | 18 | 7/14 | 88.1% | 58.6% | 26.8% |
| 33 | GLM-5.3-Flash | Max | 15 | 5/14 | 90.2% | 55.8% | 17.1% |
| 34 | Gemini 3 Flash Preview | not stated | 5 | 5/14 | 83.2% | 51.2% | 17.1% |
| 35 | Gemini 3.5 Flash-Lite | High | 2 | 5/14 | 83.3% | 26.0% | 0.0% |
| 35 | Gemini 2.5 Pro | not stated | 2 | 5/14 | 85.3% | 24.6% | 0.0% |
Other aggregations (entries: 74)
official
official · Models: 5 · Entries: 5 × Groups: 6 (benchmark coverage 6/90)FrontierSWE 2, AutomationBench, Terminal-Bench-Science 0.1, OSWorld 2.1 (offline set, partial), AA-Briefcase v1.1(rubric), AA-Briefcase v1.1(Elo)
- FrontierSWE 2official · Introducing Claude Fable 5.2 and Claude Mythos 5.2measured/reported by: Anthropic (3) · not stated (2)
- AutomationBenchofficial · Introducing Claude Fable 5.2 and Claude Mythos 5.2measured/reported by: Anthropic (3) · not stated (2)
- Terminal-Bench-Science 0.1official · Introducing Claude Fable 5.2 and Claude Mythos 5.2measured/reported by: Anthropic (3) · not stated (2)
- OSWorld 2.1 (offline set, partial)official · Introducing Claude Fable 5.2 and Claude Mythos 5.2measured/reported by: Anthropic (3) · not stated (2)
- AA-Briefcase v1.1(rubric)official · Introducing Claude Fable 5.2 and Claude Mythos 5.2measured/reported by: Anthropic (3) · not stated (2)
- AA-Briefcase v1.1(Elo)official · Introducing Claude Fable 5.2 and Claude Mythos 5.2measured/reported by: Anthropic (3) · not stated (2)
| Rank | Model | effort | Mean percentile | Benchmarks with values | 1. FrontierSWE 2 | 2. AutomationBench | 3. Terminal-Bench-Science 0.1 | 4. OSWorld 2.1 (offline set, partial) | 5. AA-Briefcase v1.1(rubric) | 6. AA-Briefcase v1.1(Elo) |
|---|---|---|---|---|---|---|---|---|---|---|
| 1 | Claude Fable 5.2 | not stated | 92 | 7/90 | 69.9% | 51.4% | 74.6% | 89.3% | 66.1% | 1,785 |
| 2 | Claude Opus 5.5 | not stated | 54 | 14/90 | 62.3% | 42.5% | 58.7% | 87.2% | 61.2% | 1,800 |
| 3 | Gemini 4 Carbon | not stated | 50 | 7/90 | 61.2% | 53.9% | 64.8% | 81.5% | 61.8% | 1,611 |
| 4 | GPT-6 Astra | not stated | 29 | 12/90 | 65.5% | 41.4% | 68.1% | 79.5% | 52.0% | 1,569 |
| 5 | Claude Fable 5.1 | not stated | 25 | 11/90 | 56.3% | 31.4% | 52.6% | 83.5% | 61.5% | 1,669 |
official · Models: 5 · Entries: 25 × Groups: 3 (benchmark coverage 3/90)Terminal-Bench 4.0, FrontierCode v1.1(main set), GDPval-AA v2.1
- Terminal-Bench 4.0official · Accuracy vs Cost figure announced by Anthropic (Opus 5.5 and Sonnet 5.5 pages)measured/reported by: Anthropic (15) · OpenAI (10)
- FrontierCode v1.1(main set)official · Accuracy vs Cost figure announced by Anthropic (Opus 5.5 and Sonnet 5.5 pages)measured/reported by: Anthropic (15) · not stated (10)
- GDPval-AA v2.1official · Introducing Claude Opus 5.5measured/reported by: Anthropic (15) · not stated (10)
| Rank | Model | effort | Mean percentile | Benchmarks with values | 1. Terminal-Bench 4.0 | 2. FrontierCode v1.1(main set) | 3. GDPval-AA v2.1 |
|---|---|---|---|---|---|---|---|
| 1 | Claude Opus 5.5 | Max | 97 | 18/90 | 64.8% | 54.4% | 1,846 |
| 2 | Claude Opus 5.5 | Xhigh | 90 | 13/90 | 66.4% | 51.4% | 1,820 |
| 3 | Claude Opus 5.5 | High | 88 | 13/90 | 64.2% | 54.0% | 1,692 |
| 4 | Claude Opus 5.5 | Med | 80 | 13/90 | 57.6% | 54.6% | 1,576 |
| 5 | Claude Fable 5.1 | Max | 73 | 10/90 | 55.8% | 50.3% | 1,735 |
| 6 | GPT-6 Astra | Max | 69 | 5/90 | 56.7% | 53.3% | 1,542 |
| 7 | GPT-6 Astra | High | 65 | 5/90 | 57.9% | 50.9% | 1,485 |
| 8 | Claude Fable 5.1 | Xhigh | 64 | 10/90 | 51.3% | 48.7% | 1,721 |
| 9 | Claude Opus 5 | Max | 62 | 6/90 | 52.3% | 48.0% | 1,708 |
| 9 | GPT-6 Astra | Xhigh | 62 | 5/90 | 57.6% | 50.6% | 1,516 |
| 11 | Claude Fable 5.1 | High | 58 | 10/90 | 49.4% | 50.3% | 1,617 |
| 12 | Claude Fable 5.1 | Med | 51 | 10/90 | 43.4% | 50.9% | 1,536 |
| 13 | Claude Opus 5 | Med | 50 | 6/90 | 41.2% | 53.4% | 1,476 |
| 14 | Claude Opus 5 | High | 48 | 6/90 | 47.0% | 48.0% | 1,581 |
| 15 | Claude Opus 5 | Xhigh | 47 | 6/90 | 50.6% | 43.6% | 1,676 |
| 15 | GPT-6 Astra | Med | 47 | 5/90 | 53.9% | 48.8% | 1,468 |
| 17 | Claude Fable 5.1 | Low | 43 | 10/90 | 40.2% | 52.8% | 1,450 |
| 18 | GPT-5.6 Sol | Max | 40 | 5/90 | 37.3% | 47.5% | 1,588 |
| 19 | GPT-5.6 Sol | Xhigh | 31 | 5/90 | 28.5% | 46.8% | 1,548 |
| 20 | GPT-6 Astra | Low | 28 | 5/90 | 49.7% | 45.3% | 1,366 |
| 21 | GPT-5.6 Sol | High | 19 | 5/90 | 26.1% | 45.1% | 1,480 |
| 22 | Claude Opus 5.5 | Low | 18 | 13/90 | 38.5% | 47.3% | 1,224 |
| 23 | Claude Opus 5 | Low | 10 | 6/90 | 28.5% | 42.0% | 1,294 |
| 24 | GPT-5.6 Sol | Med | 8 | 5/90 | 20.9% | 39.9% | 1,403 |
| 25 | GPT-5.6 Sol | Low | 1 | 5/90 | 7.9% | 35.4% | 1,289 |
official · Models: 4 · Entries: 4 × Groups: 7 (benchmark coverage 7/90)FrontierSWE 2, AutomationBench, Terminal-Bench-Science 0.1, OSWorld 2.1 (offline set, partial), RoboDojo Sim, AA-Briefcase v1.1(rubric), AA-Briefcase v1.1(Elo)
- FrontierSWE 2official · Introducing Claude Fable 5.2 and Claude Mythos 5.2measured/reported by: Anthropic (3) · not stated (1)
- AutomationBenchofficial · Introducing Claude Fable 5.2 and Claude Mythos 5.2measured/reported by: Anthropic (3) · not stated (1)
- Terminal-Bench-Science 0.1official · Introducing Claude Fable 5.2 and Claude Mythos 5.2measured/reported by: Anthropic (3) · not stated (1)
- OSWorld 2.1 (offline set, partial)official · Introducing Claude Fable 5.2 and Claude Mythos 5.2measured/reported by: Anthropic (3) · not stated (1)
- RoboDojo Simofficial · Introducing Claude Fable 5.2 and Claude Mythos 5.2measured/reported by: Anthropic (4)
- AA-Briefcase v1.1(rubric)official · Introducing Claude Fable 5.2 and Claude Mythos 5.2measured/reported by: Anthropic (3) · not stated (1)
- AA-Briefcase v1.1(Elo)official · Introducing Claude Fable 5.2 and Claude Mythos 5.2measured/reported by: Anthropic (3) · not stated (1)
| Rank | Model | effort | Mean percentile | Benchmarks with values | 1. FrontierSWE 2 | 2. AutomationBench | 3. Terminal-Bench-Science 0.1 | 4. OSWorld 2.1 (offline set, partial) | 5. RoboDojo Sim | 6. AA-Briefcase v1.1(rubric) | 7. AA-Briefcase v1.1(Elo) |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | Claude Fable 5.2 | not stated | 95 | 7/90 | 69.9% | 51.4% | 74.6% | 89.3% | 64.9% | 66.1% | 1,785 |
| 2 | Claude Opus 5.5 | not stated | 57 | 14/90 | 62.3% | 42.5% | 58.7% | 87.2% | 39.4% | 61.2% | 1,800 |
| 3 | GPT-6 Astra | not stated | 29 | 12/90 | 65.5% | 41.4% | 68.1% | 79.5% | 29.8% | 52.0% | 1,569 |
| 4 | Claude Fable 5.1 | not stated | 19 | 11/90 | 56.3% | 31.4% | 52.6% | 83.5% | 27.6% | 61.5% | 1,669 |
official · Models: 4 · Entries: 20 × Groups: 4 (benchmark coverage 4/90)Terminal-Bench 4.0, FrontierCode v1.1(main set), CursorBench 4.0, GDPval-AA v2.1
- Terminal-Bench 4.0official · Accuracy vs Cost figure announced by Anthropic (Opus 5.5 and Sonnet 5.5 pages)measured/reported by: Anthropic (15) · OpenAI (5)
- FrontierCode v1.1(main set)official · Accuracy vs Cost figure announced by Anthropic (Opus 5.5 and Sonnet 5.5 pages)measured/reported by: Anthropic (15) · not stated (5)
- CursorBench 4.0official · Introducing Claude Opus 5.5measured/reported by: Anthropic (15) · not stated (5)
- GDPval-AA v2.1official · Introducing Claude Opus 5.5measured/reported by: Anthropic (15) · not stated (5)
| Rank | Model | effort | Mean percentile | Benchmarks with values | 1. Terminal-Bench 4.0 | 2. FrontierCode v1.1(main set) | 3. CursorBench 4.0 | 4. GDPval-AA v2.1 |
|---|---|---|---|---|---|---|---|---|
| 1 | Claude Opus 5.5 | Max | 97 | 18/90 | 64.8% | 54.4% | 57.8% | 1,846 |
| 2 | Claude Opus 5.5 | Xhigh | 90 | 13/90 | 66.4% | 51.4% | 56.0% | 1,820 |
| 3 | Claude Opus 5.5 | High | 86 | 13/90 | 64.2% | 54.0% | 56.0% | 1,692 |
| 4 | Claude Opus 5.5 | Med | 79 | 13/90 | 57.6% | 54.6% | 52.5% | 1,576 |
| 5 | Claude Fable 5.1 | Max | 77 | 10/90 | 55.8% | 50.3% | 51.8% | 1,735 |
| 6 | Claude Fable 5.1 | Xhigh | 70 | 10/90 | 51.3% | 48.7% | 51.6% | 1,721 |
| 7 | Claude Opus 5 | Max | 64 | 6/90 | 52.3% | 48.0% | 46.6% | 1,708 |
| 8 | Claude Fable 5.1 | High | 63 | 10/90 | 49.4% | 50.3% | 49.2% | 1,617 |
| 9 | Claude Fable 5.1 | Med | 54 | 10/90 | 43.4% | 50.9% | 46.8% | 1,536 |
| 10 | Claude Opus 5 | Xhigh | 50 | 6/90 | 50.6% | 43.6% | 46.1% | 1,676 |
| 11 | Claude Opus 5 | High | 48 | 6/90 | 47.0% | 48.0% | 44.7% | 1,581 |
| 12 | Claude Fable 5.1 | Low | 46 | 10/90 | 40.2% | 52.8% | 45.1% | 1,450 |
| 12 | Claude Opus 5 | Med | 46 | 6/90 | 41.2% | 53.4% | 43.3% | 1,476 |
| 14 | GPT-5.6 Sol | Max | 37 | 5/90 | 37.3% | 47.5% | 41.7% | 1,588 |
| 15 | GPT-5.6 Sol | Xhigh | 26 | 5/90 | 28.5% | 46.8% | 37.7% | 1,548 |
| 16 | Claude Opus 5.5 | Low | 25 | 13/90 | 38.5% | 47.3% | 43.7% | 1,224 |
| 17 | GPT-5.6 Sol | High | 18 | 5/90 | 26.1% | 45.1% | 35.7% | 1,480 |
| 18 | Claude Opus 5 | Low | 15 | 6/90 | 28.5% | 42.0% | 40.7% | 1,294 |
| 19 | GPT-5.6 Sol | Med | 8 | 5/90 | 20.9% | 39.9% | 31.1% | 1,403 |
| 20 | GPT-5.6 Sol | Low | 1 | 5/90 | 7.9% | 35.4% | 24.6% | 1,289 |
official · Models: 4 · Entries: 20 × Groups: 4 (benchmark coverage 4/90)Terminal-Bench 4.0, FrontierCode v1.1(main set), GDPval-AA v2.1, AutomationBench
- Terminal-Bench 4.0official · Accuracy vs Cost figure announced by Anthropic (Opus 5.5 and Sonnet 5.5 pages)measured/reported by: Anthropic (10) · OpenAI (10)
- FrontierCode v1.1(main set)official · Accuracy vs Cost figure announced by Anthropic (Opus 5.5 and Sonnet 5.5 pages)measured/reported by: Anthropic (10) · not stated (10)
- GDPval-AA v2.1official · Introducing Claude Opus 5.5measured/reported by: Anthropic (10) · not stated (10)
- AutomationBenchofficial · Introducing Claude Opus 5.5measured/reported by: Zapier (20)
| Rank | Model | effort | Mean percentile | Benchmarks with values | 1. Terminal-Bench 4.0 | 2. FrontierCode v1.1(main set) | 3. GDPval-AA v2.1 | 4. AutomationBench |
|---|---|---|---|---|---|---|---|---|
| 1 | Claude Opus 5.5 | Max | 96 | 18/90 | 64.8% | 54.4% | 1,846 | 40.0% |
| 2 | Claude Opus 5.5 | Xhigh | 87 | 13/90 | 66.4% | 51.4% | 1,820 | 34.4% |
| 3 | Claude Opus 5.5 | High | 83 | 13/90 | 64.2% | 54.0% | 1,692 | 32.0% |
| 4 | GPT-6 Astra | Max | 75 | 5/90 | 56.7% | 53.3% | 1,542 | 41.4% |
| 5 | Claude Opus 5.5 | Med | 73 | 13/90 | 57.6% | 54.6% | 1,576 | 28.6% |
| 6 | GPT-6 Astra | High | 70 | 5/90 | 57.9% | 50.9% | 1,485 | 37.1% |
| 7 | GPT-6 Astra | Xhigh | 69 | 5/90 | 57.6% | 50.6% | 1,516 | 39.0% |
| 8 | Claude Opus 5 | Max | 61 | 6/90 | 52.3% | 48.0% | 1,708 | 26.9% |
| 9 | GPT-6 Astra | Med | 55 | 5/90 | 53.9% | 48.8% | 1,468 | 34.1% |
| 10 | GPT-5.6 Sol | Max | 50 | 5/90 | 37.3% | 47.5% | 1,588 | 28.8% |
| 11 | Claude Opus 5 | Xhigh | 46 | 6/90 | 50.6% | 43.6% | 1,676 | 25.3% |
| 12 | Claude Opus 5 | Med | 45 | 6/90 | 41.2% | 53.4% | 1,476 | 23.9% |
| 13 | Claude Opus 5 | High | 44 | 6/90 | 47.0% | 48.0% | 1,581 | 20.6% |
| 14 | GPT-6 Astra | Low | 38 | 5/90 | 49.7% | 45.3% | 1,366 | 30.3% |
| 15 | GPT-5.6 Sol | Xhigh | 38 | 5/90 | 28.5% | 46.8% | 1,548 | 26.3% |
| 16 | GPT-5.6 Sol | High | 25 | 5/90 | 26.1% | 45.1% | 1,480 | 24.8% |
| 17 | Claude Opus 5.5 | Low | 22 | 13/90 | 38.5% | 47.3% | 1,224 | 23.3% |
| 18 | Claude Opus 5 | Low | 13 | 6/90 | 28.5% | 42.0% | 1,294 | 20.4% |
| 19 | GPT-5.6 Sol | Med | 9 | 5/90 | 20.9% | 39.9% | 1,403 | 19.6% |
| 20 | GPT-5.6 Sol | Low | 1 | 5/90 | 7.9% | 35.4% | 1,289 | 11.7% |
official · Models: 4 · Entries: 20 × Groups: 3 (benchmark coverage 3/90)Terminal-Bench 4.0, FrontierCode v1.1(main set), CursorBench 4.0
- Terminal-Bench 4.0official · Accuracy vs Cost figure announced by Anthropic (Opus 5.5 and Sonnet 5.5 pages)measured/reported by: Anthropic (15) · OpenAI (5)
- FrontierCode v1.1(main set)official · Accuracy vs Cost figure announced by Anthropic (Opus 5.5 and Sonnet 5.5 pages)measured/reported by: Anthropic (15) · not stated (5)
- CursorBench 4.0official · Introducing Claude Sonnet 5.5measured/reported by: Anthropic (15) · not stated (5)
| Rank | Model | effort | Mean percentile | Benchmarks with values | 1. Terminal-Bench 4.0 | 2. FrontierCode v1.1(main set) | 3. CursorBench 4.0 |
|---|---|---|---|---|---|---|---|
| 1 | Claude Opus 5.5 | Max | 95 | 18/90 | 64.8% | 54.4% | 57.8% |
| 2 | Claude Opus 5.5 | High | 89 | 13/90 | 64.2% | 54.0% | 56.0% |
| 2 | Claude Opus 5.5 | Xhigh | 89 | 13/90 | 66.4% | 51.4% | 56.0% |
| 4 | Claude Opus 5.5 | Med | 82 | 13/90 | 57.6% | 54.6% | 52.5% |
| 5 | Claude Sonnet 5.5 | Xhigh | 81 | 7/90 | 61.5% | 52.1% | 53.1% |
| 6 | Claude Sonnet 5.5 | Max | 79 | 11/90 | 70.6% | 46.2% | 55.5% |
| 7 | Claude Sonnet 5.5 | High | 70 | 7/90 | 43.0% | 49.4% | 47.8% |
| 8 | Claude Opus 5.5 | Low | 63 | 13/90 | 38.5% | 47.3% | 43.7% |
| 9 | GPT-5.6 Sol | Max | 61 | 5/90 | 37.3% | 47.5% | 41.7% |
| 10 | GPT-5.6 Sol | Xhigh | 51 | 5/90 | 28.5% | 46.8% | 37.7% |
| 11 | Claude Sonnet 5.5 | Med | 42 | 7/90 | 28.8% | 36.5% | 39.2% |
| 11 | GPT-5.6 Sol | High | 42 | 5/90 | 26.1% | 45.1% | 35.7% |
| 13 | Claude Sonnet 5 | Max | 32 | 4/90 | 10.3% | 42.4% | 34.1% |
| 14 | GPT-5.6 Sol | Med | 30 | 5/90 | 20.9% | 39.9% | 31.1% |
| 15 | Claude Sonnet 5 | Xhigh | 28 | 4/90 | 7.0% | 42.7% | 32.0% |
| 16 | Claude Sonnet 5.5 | Low | 26 | 7/90 | 20.0% | 29.3% | 35.8% |
| 17 | Claude Sonnet 5 | High | 18 | 4/90 | 4.5% | 39.4% | 30.8% |
| 18 | GPT-5.6 Sol | Low | 14 | 5/90 | 7.9% | 35.4% | 24.6% |
| 19 | Claude Sonnet 5 | Med | 9 | 4/90 | 4.4% | 35.2% | 28.0% |
| 20 | Claude Sonnet 5 | Low | 0 | 4/90 | 3.2% | 28.7% | 24.1% |
official · Models: 3 · Entries: 3 × Groups: 8 (benchmark coverage 8/90)FrontierSWE 2, AutomationBench, Terminal-Bench-Science 0.1, OSWorld 2.1 (offline set, partial), RoboDojo Sim, AA-Briefcase v1.1(rubric), AA-Briefcase v1.1(Elo), Remote Labor Index(RLI)
- FrontierSWE 2official · Introducing Claude Fable 5.2 and Claude Mythos 5.2measured/reported by: Anthropic (2) · not stated (1)
- AutomationBenchofficial · Introducing Claude Fable 5.2 and Claude Mythos 5.2measured/reported by: Anthropic (2) · not stated (1)
- Terminal-Bench-Science 0.1official · Introducing Claude Fable 5.2 and Claude Mythos 5.2measured/reported by: Anthropic (2) · not stated (1)
- OSWorld 2.1 (offline set, partial)official · Introducing Claude Fable 5.2 and Claude Mythos 5.2measured/reported by: Anthropic (2) · not stated (1)
- RoboDojo Simofficial · Introducing Claude Fable 5.2 and Claude Mythos 5.2measured/reported by: Anthropic (3)
- AA-Briefcase v1.1(rubric)official · Introducing Claude Fable 5.2 and Claude Mythos 5.2measured/reported by: Anthropic (2) · not stated (1)
- AA-Briefcase v1.1(Elo)official · Introducing Claude Fable 5.2 and Claude Mythos 5.2measured/reported by: Anthropic (2) · not stated (1)
- Remote Labor Index(RLI)official · Introducing Claude Fable 5.2 and Claude Mythos 5.2measured/reported by: Anthropic (2) · not stated (1)
| Rank | Model | effort | Mean percentile | Benchmarks with values | 1. FrontierSWE 2 | 2. AutomationBench | 3. Terminal-Bench-Science 0.1 | 4. OSWorld 2.1 (offline set, partial) | 5. RoboDojo Sim | 6. AA-Briefcase v1.1(rubric) | 7. AA-Briefcase v1.1(Elo) | 8. Remote Labor Index(RLI) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | Claude Opus 5.5 | not stated | 81 | 14/90 | 62.3% | 42.5% | 58.7% | 87.2% | 39.4% | 61.2% | 1,800 | 21.3% |
| 2 | GPT-6 Astra | not stated | 44 | 12/90 | 65.5% | 41.4% | 68.1% | 79.5% | 29.8% | 52.0% | 1,569 | 20.8% |
| 3 | Claude Fable 5.1 | not stated | 25 | 11/90 | 56.3% | 31.4% | 52.6% | 83.5% | 27.6% | 61.5% | 1,669 | 17.9% |
official · Models: 3 · Entries: 3 × Groups: 6 (benchmark coverage 6/90)SWE-Bench Pro, OSWorld-Verified, CharXiv Reasoning, MMMU-Pro, MRCR v2 (8-needle, 128k), MRCR v2 (8-needle, 1M)
- SWE-Bench Proofficial · Gemini 3.5 Flash model cardmeasured/reported by: Google (3)
- OSWorld-Verifiedofficial · Gemini 3.5 Flash model cardmeasured/reported by: Google (3)
- CharXiv Reasoningofficial · Gemini 3.5 Flash model cardmeasured/reported by: Google (3)
- MMMU-Proofficial · Gemini 3.5 Flash model cardmeasured/reported by: Google (3)
- MRCR v2 (8-needle, 128k)official · Gemini 3.5 Flash model cardmeasured/reported by: Google (3)
- MRCR v2 (8-needle, 1M)official · Gemini 3.5 Flash model cardmeasured/reported by: Google (3)
| Rank | Model | effort | Mean percentile | Benchmarks with values | 1. SWE-Bench Pro | 2. OSWorld-Verified | 3. CharXiv Reasoning | 4. MMMU-Pro | 5. MRCR v2 (8-needle, 128k) | 6. MRCR v2 (8-needle, 1M) |
|---|---|---|---|---|---|---|---|---|---|---|
| 1 | Gemini 3.5 Flash | not stated | 92 | 7/90 | 55.1% | 78.4% | 84.2% | 83.6% | 77.3% | 26.6% |
| 2 | Gemini 3.1 Pro Preview | not stated | 50 | 6/90 | 54.2% | 76.2% | 83.3% | 80.5% | 84.9% | 26.3% |
| 3 | Gemini 3 Flash Preview | not stated | 8 | 7/90 | 49.6% | 65.1% | 80.3% | 81.2% | 67.2% | 22.1% |
official · Models: 3 · Entries: 15 × Groups: 5 (benchmark coverage 5/90)Terminal-Bench 4.0, FrontierCode v1.1(main set), CursorBench 4.0, GDPval-AA v2.1, AutomationBench
- Terminal-Bench 4.0official · Accuracy vs Cost figure announced by Anthropic (Opus 5.5 and Sonnet 5.5 pages)measured/reported by: Anthropic (10) · OpenAI (5)
- FrontierCode v1.1(main set)official · Accuracy vs Cost figure announced by Anthropic (Opus 5.5 and Sonnet 5.5 pages)measured/reported by: Anthropic (10) · not stated (5)
- CursorBench 4.0official · Introducing Claude Opus 5.5measured/reported by: Anthropic (10) · not stated (5)
- GDPval-AA v2.1official · Introducing Claude Opus 5.5measured/reported by: Anthropic (10) · not stated (5)
- AutomationBenchofficial · Introducing Claude Opus 5.5measured/reported by: Zapier (15)
| Rank | Model | effort | Mean percentile | Benchmarks with values | 1. Terminal-Bench 4.0 | 2. FrontierCode v1.1(main set) | 3. CursorBench 4.0 | 4. GDPval-AA v2.1 | 5. AutomationBench |
|---|---|---|---|---|---|---|---|---|---|
| 1 | Claude Opus 5.5 | Max | 97 | 18/90 | 64.8% | 54.4% | 57.8% | 1,846 | 40.0% |
| 2 | Claude Opus 5.5 | Xhigh | 89 | 13/90 | 66.4% | 51.4% | 56.0% | 1,820 | 34.4% |
| 3 | Claude Opus 5.5 | High | 85 | 13/90 | 64.2% | 54.0% | 56.0% | 1,692 | 32.0% |
| 4 | Claude Opus 5.5 | Med | 76 | 13/90 | 57.6% | 54.6% | 52.5% | 1,576 | 28.6% |
| 5 | Claude Opus 5 | Max | 71 | 6/90 | 52.3% | 48.0% | 46.6% | 1,708 | 26.9% |
| 6 | Claude Opus 5 | Xhigh | 54 | 6/90 | 50.6% | 43.6% | 46.1% | 1,676 | 25.3% |
| 7 | GPT-5.6 Sol | Max | 53 | 5/90 | 37.3% | 47.5% | 41.7% | 1,588 | 28.8% |
| 8 | Claude Opus 5 | High | 51 | 6/90 | 47.0% | 48.0% | 44.7% | 1,581 | 20.6% |
| 9 | Claude Opus 5 | Med | 47 | 6/90 | 41.2% | 53.4% | 43.3% | 1,476 | 23.9% |
| 10 | GPT-5.6 Sol | Xhigh | 36 | 5/90 | 28.5% | 46.8% | 37.7% | 1,548 | 26.3% |
| 11 | Claude Opus 5.5 | Low | 33 | 13/90 | 38.5% | 47.3% | 43.7% | 1,224 | 23.3% |
| 12 | GPT-5.6 Sol | High | 27 | 5/90 | 26.1% | 45.1% | 35.7% | 1,480 | 24.8% |
| 13 | Claude Opus 5 | Low | 19 | 6/90 | 28.5% | 42.0% | 40.7% | 1,294 | 20.4% |
| 14 | GPT-5.6 Sol | Med | 10 | 5/90 | 20.9% | 39.9% | 31.1% | 1,403 | 19.6% |
| 15 | GPT-5.6 Sol | Low | 1 | 5/90 | 7.9% | 35.4% | 24.6% | 1,289 | 11.7% |
official · Models: 3 · Entries: 15 × Groups: 5 (benchmark coverage 5/90)Terminal-Bench 4.0, FrontierCode v1.1(main set), CursorBench 4.0, GDPval-AA v2.1, WANDR
- Terminal-Bench 4.0official · Accuracy vs Cost figure announced by Anthropic (Opus 5.5 and Sonnet 5.5 pages)measured/reported by: Anthropic (15)
- FrontierCode v1.1(main set)official · Accuracy vs Cost figure announced by Anthropic (Opus 5.5 and Sonnet 5.5 pages)measured/reported by: Anthropic (15)
- CursorBench 4.0official · Introducing Claude Opus 5.5measured/reported by: Anthropic (15)
- GDPval-AA v2.1official · Introducing Claude Opus 5.5measured/reported by: Anthropic (15)
- WANDRofficial · Introducing Claude Opus 5.5measured/reported by: Anthropic (15)
| Rank | Model | effort | Mean percentile | Benchmarks with values | 1. Terminal-Bench 4.0 | 2. FrontierCode v1.1(main set) | 3. CursorBench 4.0 | 4. GDPval-AA v2.1 | 5. WANDR |
|---|---|---|---|---|---|---|---|---|---|
| 1 | Claude Opus 5.5 | Max | 97 | 18/90 | 64.8% | 54.4% | 57.8% | 1,846 | 72.3% |
| 2 | Claude Opus 5.5 | Xhigh | 88 | 13/90 | 66.4% | 51.4% | 56.0% | 1,820 | 71.3% |
| 3 | Claude Opus 5.5 | High | 79 | 13/90 | 64.2% | 54.0% | 56.0% | 1,692 | 67.3% |
| 4 | Claude Fable 5.1 | Max | 72 | 10/90 | 55.8% | 50.3% | 51.8% | 1,735 | 68.7% |
| 5 | Claude Opus 5.5 | Med | 63 | 13/90 | 57.6% | 54.6% | 52.5% | 1,576 | 62.8% |
| 5 | Claude Fable 5.1 | Xhigh | 63 | 10/90 | 51.3% | 48.7% | 51.6% | 1,721 | 67.7% |
| 7 | Claude Opus 5 | Max | 54 | 6/90 | 52.3% | 48.0% | 46.6% | 1,708 | 67.2% |
| 8 | Claude Fable 5.1 | High | 49 | 10/90 | 49.4% | 50.3% | 49.2% | 1,617 | 66.7% |
| 9 | Claude Opus 5 | Xhigh | 41 | 6/90 | 50.6% | 43.6% | 46.1% | 1,676 | 67.0% |
| 10 | Claude Fable 5.1 | Med | 40 | 10/90 | 43.4% | 50.9% | 46.8% | 1,536 | 64.5% |
| 11 | Claude Opus 5 | High | 34 | 6/90 | 47.0% | 48.0% | 44.7% | 1,581 | 64.6% |
| 12 | Claude Fable 5.1 | Low | 31 | 10/90 | 40.2% | 52.8% | 45.1% | 1,450 | 63.3% |
| 13 | Claude Opus 5 | Med | 29 | 6/90 | 41.2% | 53.4% | 43.3% | 1,476 | 58.1% |
| 14 | Claude Opus 5.5 | Low | 7 | 13/90 | 38.5% | 47.3% | 43.7% | 1,224 | 31.2% |
| 15 | Claude Opus 5 | Low | 3 | 6/90 | 28.5% | 42.0% | 40.7% | 1,294 | 50.5% |
official · Models: 3 · Entries: 15 × Groups: 5 (benchmark coverage 5/90)FrontierCode 2.0, WANDR(relaxed score), BenchCAD 2.1 (with tools), Humanity's Last Exam (with tools), Chartography (without tools)
- FrontierCode 2.0official · Introducing Claude Fable 5.2 and Claude Mythos 5.2measured/reported by: Anthropic (15)
- WANDR(relaxed score)official · Introducing Claude Fable 5.2 and Claude Mythos 5.2measured/reported by: Anthropic (15)
- BenchCAD 2.1 (with tools)official · Introducing Claude Fable 5.2 and Claude Mythos 5.2measured/reported by: Anthropic (15)
- Humanity's Last Exam (with tools)official · Introducing Claude Fable 5.2 and Claude Mythos 5.2measured/reported by: Anthropic (15)
- Chartography (without tools)official · Introducing Claude Fable 5.2 and Claude Mythos 5.2measured/reported by: Anthropic (15)
| Rank | Model | effort | Mean percentile | Benchmarks with values | 1. FrontierCode 2.0 | 2. WANDR(relaxed score) | 3. BenchCAD 2.1 (with tools) | 4. Humanity's Last Exam (with tools) | 5. Chartography (without tools) |
|---|---|---|---|---|---|---|---|---|---|
| 1 | Claude Fable 5.2 | Max | 99 | 11/90 | 45.6% | 82.1% | 83.5 | 74.2% | 73.4% |
| 2 | Claude Fable 5.2 | Xhigh | 94 | 7/90 | 49.1% | 80.6% | 74.1 | 73.4% | 71.4% |
| 3 | Claude Fable 5.2 | High | 83 | 7/90 | 45.3% | 79.3% | 61 | 71.5% | 69.8% |
| 4 | Claude Opus 5.5 | Max | 76 | 18/90 | 45.5% | 72.3% | 65.3 | 67.7% | 64.4% |
| 5 | Claude Fable 5.2 | Med | 70 | 6/90 | 37.6% | 73.6% | 32.4 | 68.6% | 66.4% |
| 6 | Claude Opus 5.5 | Xhigh | 63 | 13/90 | 43.4% | 71.3% | 54.5 | 66.4% | 62.4% |
| 7 | Claude Fable 5.2 | Low | 57 | 6/90 | 34.2% | 71.4% | 26.9 | 68.4% | 64.2% |
| 8 | Claude Opus 5.5 | High | 44 | 13/90 | 34.5% | 67.3% | 29.7 | 63.9% | 59.2% |
| 9 | Claude Fable 5.1 | Xhigh | 43 | 10/90 | 32.4% | 67.7% | 40.5 | 65.2% | 47.0% |
| 9 | Claude Fable 5.1 | Max | 43 | 10/90 | 31.7% | 68.7% | 44.4 | 65.6% | 44.8% |
| 11 | Claude Fable 5.1 | High | 26 | 10/90 | 25.7% | 66.7% | 25.8 | 63.1% | 46.0% |
| 12 | Claude Opus 5.5 | Med | 24 | 13/90 | 30.1% | 62.8% | 19.1 | 63.0% | 59.0% |
| 13 | Claude Fable 5.1 | Med | 14 | 10/90 | 21.5% | 64.5% | 15.7 | 62.4% | 42.2% |
| 14 | Claude Opus 5.5 | Low | 7 | 13/90 | 14.7% | 31.2% | 6.3 | 57.4% | 57.0% |
| 14 | Claude Fable 5.1 | Low | 7 | 10/90 | 20.6% | 63.3% | 12.1 | 60.2% | 36.6% |
official · Models: 3 · Entries: 15 × Groups: 4 (benchmark coverage 4/90)Terminal-Bench 4.0, FrontierCode v1.1(main set), GDPval-AA v2.1, FrontierCode 2.0
- Terminal-Bench 4.0official · Accuracy vs Cost figure announced by Anthropic (Opus 5.5 and Sonnet 5.5 pages)measured/reported by: Anthropic (10) · OpenAI (5)
- FrontierCode v1.1(main set)official · Accuracy vs Cost figure announced by Anthropic (Opus 5.5 and Sonnet 5.5 pages)measured/reported by: Anthropic (10) · not stated (5)
- GDPval-AA v2.1official · Introducing Claude Opus 5.5measured/reported by: Anthropic (10) · not stated (5)
- FrontierCode 2.0official · Introducing Claude Fable 5.2 and Claude Mythos 5.2measured/reported by: Anthropic (10) · not stated (5)
| Rank | Model | effort | Mean percentile | Benchmarks with values | 1. Terminal-Bench 4.0 | 2. FrontierCode v1.1(main set) | 3. GDPval-AA v2.1 | 4. FrontierCode 2.0 |
|---|---|---|---|---|---|---|---|---|
| 1 | Claude Opus 5.5 | Max | 96 | 18/90 | 64.8% | 54.4% | 1,846 | 45.5% |
| 2 | Claude Opus 5.5 | Xhigh | 88 | 13/90 | 66.4% | 51.4% | 1,820 | 43.4% |
| 3 | Claude Opus 5.5 | High | 75 | 13/90 | 64.2% | 54.0% | 1,692 | 34.5% |
| 4 | GPT-6 Astra | Max | 68 | 5/90 | 56.7% | 53.3% | 1,542 | 42.8% |
| 5 | Claude Opus 5.5 | Med | 63 | 13/90 | 57.6% | 54.6% | 1,576 | 30.1% |
| 6 | GPT-6 Astra | High | 58 | 5/90 | 57.9% | 50.9% | 1,485 | 38.3% |
| 7 | GPT-6 Astra | Xhigh | 56 | 5/90 | 57.6% | 50.6% | 1,516 | 41.4% |
| 8 | Claude Fable 5.1 | Max | 53 | 10/90 | 55.8% | 50.3% | 1,735 | 31.7% |
| 9 | Claude Fable 5.1 | Xhigh | 45 | 10/90 | 51.3% | 48.7% | 1,721 | 32.4% |
| 10 | GPT-6 Astra | Med | 38 | 5/90 | 53.9% | 48.8% | 1,468 | 36.8% |
| 11 | Claude Fable 5.1 | High | 35 | 10/90 | 49.4% | 50.3% | 1,617 | 25.7% |
| 12 | Claude Fable 5.1 | Med | 31 | 10/90 | 43.4% | 50.9% | 1,536 | 21.5% |
| 13 | Claude Fable 5.1 | Low | 25 | 10/90 | 40.2% | 52.8% | 1,450 | 20.6% |
| 14 | GPT-6 Astra | Low | 18 | 5/90 | 49.7% | 45.3% | 1,366 | 31.1% |
| 15 | Claude Opus 5.5 | Low | 2 | 13/90 | 38.5% | 47.3% | 1,224 | 14.7% |
official · Models: 3 · Entries: 15 × Groups: 4 (benchmark coverage 4/90)Terminal-Bench 4.0, FrontierCode v1.1(main set), CursorBench 4.0, AA-Briefcase v1.1(Elo)
- Terminal-Bench 4.0official · Accuracy vs Cost figure announced by Anthropic (Opus 5.5 and Sonnet 5.5 pages)measured/reported by: Anthropic (15)
- FrontierCode v1.1(main set)official · Accuracy vs Cost figure announced by Anthropic (Opus 5.5 and Sonnet 5.5 pages)measured/reported by: Anthropic (15)
- CursorBench 4.0official · Introducing Claude Sonnet 5.5measured/reported by: Anthropic (15)
- AA-Briefcase v1.1(Elo)official · Introducing Claude Sonnet 5.5measured/reported by: Anthropic (15)
| Rank | Model | effort | Mean percentile | Benchmarks with values | 1. Terminal-Bench 4.0 | 2. FrontierCode v1.1(main set) | 3. CursorBench 4.0 | 4. AA-Briefcase v1.1(Elo) |
|---|---|---|---|---|---|---|---|---|
| 1 | Claude Opus 5.5 | Max | 95 | 18/90 | 64.8% | 54.4% | 57.8% | 1,822 |
| 2 | Claude Opus 5.5 | Xhigh | 85 | 13/90 | 66.4% | 51.4% | 56.0% | 1,780 |
| 3 | Claude Opus 5.5 | High | 81 | 13/90 | 64.2% | 54.0% | 56.0% | 1,705 |
| 4 | Claude Sonnet 5.5 | Max | 80 | 11/90 | 70.6% | 46.2% | 55.5% | 1,811 |
| 5 | Claude Sonnet 5.5 | Xhigh | 75 | 7/90 | 61.5% | 52.1% | 53.1% | 1,746 |
| 6 | Claude Opus 5.5 | Med | 73 | 13/90 | 57.6% | 54.6% | 52.5% | 1,642 |
| 7 | Claude Sonnet 5.5 | High | 59 | 7/90 | 43.0% | 49.4% | 47.8% | 1,634 |
| 8 | Claude Opus 5.5 | Low | 48 | 13/90 | 38.5% | 47.3% | 43.7% | 1,285 |
| 9 | Claude Sonnet 5.5 | Med | 39 | 7/90 | 28.8% | 36.5% | 39.2% | 1,461 |
| 10 | Claude Sonnet 5 | Max | 34 | 4/90 | 10.3% | 42.4% | 34.1% | 1,359 |
| 11 | Claude Sonnet 5 | Xhigh | 29 | 4/90 | 7.0% | 42.7% | 32.0% | 1,274 |
| 12 | Claude Sonnet 5.5 | Low | 25 | 7/90 | 20.0% | 29.3% | 35.8% | 1,264 |
| 13 | Claude Sonnet 5 | High | 18 | 4/90 | 4.5% | 39.4% | 30.8% | 1,177 |
| 14 | Claude Sonnet 5 | Med | 9 | 4/90 | 4.4% | 35.2% | 28.0% | 1,056 |
| 15 | Claude Sonnet 5 | Low | 0 | 4/90 | 3.2% | 28.7% | 24.1% | 923 |
official · Models: 3 · Entries: 11 × Groups: 4 (benchmark coverage 4/90)OSWorld 2.1(offline subset), GDPval-AA v2.1, Humanity's Last Exam (without tools), Terminal-Bench 4.0
- OSWorld 2.1(offline subset)official · Introducing Claude Haiku 5.5measured/reported by: Anthropic (11)
- GDPval-AA v2.1official · Introducing Claude Haiku 5.5measured/reported by: Anthropic (11)
- Humanity's Last Exam (without tools)official · Introducing Claude Haiku 5.5measured/reported by: Anthropic (11)
- Terminal-Bench 4.0official · Introducing Claude Haiku 5.5measured/reported by: Anthropic (11)
| Rank | Model | effort | Mean percentile | Benchmarks with values | 1. OSWorld 2.1(offline subset) | 2. GDPval-AA v2.1 | 3. Humanity's Last Exam (without tools) | 4. Terminal-Bench 4.0 |
|---|---|---|---|---|---|---|---|---|
| 1 | Claude Sonnet 5.5 | Max | 100 | 11/90 | 83.9% | 1,840 | 56.9% | 70.6% |
| 2 | Claude Sonnet 5.5 | Xhigh | 90 | 7/90 | 81.1% | 1,731 | 53.0% | 61.5% |
| 3 | Claude Sonnet 5.5 | High | 78 | 7/90 | 73.2% | 1,551 | 47.9% | 43.0% |
| 4 | Claude Haiku 5.5 | Max | 73 | 8/90 | 72.4% | 1,620 | 45.9% | 39.2% |
| 5 | Claude Haiku 5.5 | Xhigh | 60 | 4/90 | 67.6% | 1,513 | 44.1% | 31.5% |
| 6 | Claude Sonnet 5.5 | Med | 48 | 7/90 | 66.0% | 1,324 | 43.2% | 28.8% |
| 7 | Claude Haiku 5.5 | High | 40 | 4/90 | 61.3% | 1,420 | 39.7% | 24.8% |
| 8 | Claude Sonnet 5.5 | Low | 28 | 7/90 | 57.9% | 1,179 | 41.4% | 20.0% |
| 9 | Claude Haiku 5.5 | Med | 25 | 4/90 | 53.3% | 1,277 | 35.5% | 20.3% |
| 10 | Claude Haiku 5.5 | Low | 10 | 4/90 | 42.0% | 1,125 | 30.9% | 12.7% |
| 11 | Claude Haiku 4.5 | not stated | 0 | 7/90 | 15.7% | 735 | 10.2% | 0.0% |
official · Models: 3 · Entries: 3 × Groups: 4 (benchmark coverage 4/90)Humanity's Last Exam (with tools), Terminal-Bench-Science 0.1, OSWorld 2.1(partial), Chartography (with tools)
- Humanity's Last Exam (with tools)official · Introducing Claude Opus 5.5measured/reported by: Anthropic (3)
- Terminal-Bench-Science 0.1official · Introducing Claude Opus 5.5measured/reported by: Anthropic (3)
- OSWorld 2.1(partial)official · Introducing Claude Opus 5.5measured/reported by: Anthropic (3)
- Chartography (with tools)official · Introducing Claude Opus 5.5measured/reported by: Anthropic (3)
| Rank | Model | effort | Mean percentile | Benchmarks with values | 1. Humanity's Last Exam (with tools) | 2. Terminal-Bench-Science 0.1 | 3. OSWorld 2.1(partial) | 4. Chartography (with tools) |
|---|---|---|---|---|---|---|---|---|
| 1 | Claude Opus 5.5 | Max | 100 | 18/90 | 67.7% | 58.7% | 81.8% | 89.0% |
| 2 | Claude Fable 5.1 | not stated | 50 | 11/90 | 65.6% | 52.6% | 80.7% | 88.4% |
| 3 | Claude Opus 5 | not stated | 0 | 4/90 | 63.6% | 29.0% | 74.0% | 83.4% |
official · Models: 3 · Entries: 3 × Groups: 4 (benchmark coverage 4/90)GDPval-AA v2.1, Humanity's Last Exam (with tools), OSWorld 2.1(partial), Chartography (without tools)
- GDPval-AA v2.1official · Introducing Claude Sonnet 5.5measured/reported by: Artificial Analysis (3)
- Humanity's Last Exam (with tools)official · Introducing Claude Sonnet 5.5measured/reported by: Anthropic (3)
- OSWorld 2.1(partial)official · Introducing Claude Sonnet 5.5measured/reported by: Anthropic (3)
- Chartography (without tools)official · Introducing Claude Sonnet 5.5measured/reported by: Anthropic (3)
| Rank | Model | effort | Mean percentile | Benchmarks with values | 1. GDPval-AA v2.1 | 2. Humanity's Last Exam (with tools) | 3. OSWorld 2.1(partial) | 4. Chartography (without tools) |
|---|---|---|---|---|---|---|---|---|
| 1 | Claude Opus 5.5 | not stated | 100 | 14/90 | 1,846 | 67.7% | 81.8% | 64.4% |
| 2 | Claude Sonnet 5.5 | not stated | 50 | 5/90 | 1,844 | 64.5% | 80.1% | 61.6% |
| 3 | Claude Sonnet 5 | not stated | 0 | 4/90 | 1,449 | 54.9% | 57.0% | 15.6% |
official · Models: 3 · Entries: 3 × Groups: 3 (benchmark coverage 3/90)AA-Briefcase v1.1(Elo), Humanity's Last Exam (with tools), Chartography (without tools)
- AA-Briefcase v1.1(Elo)official · Introducing Claude Haiku 5.5measured/reported by: Anthropic (3)
- Humanity's Last Exam (with tools)official · Introducing Claude Haiku 5.5measured/reported by: Anthropic (3)
- Chartography (without tools)official · Introducing Claude Haiku 5.5measured/reported by: Anthropic (3)
| Rank | Model | effort | Mean percentile | Benchmarks with values | 1. AA-Briefcase v1.1(Elo) | 2. Humanity's Last Exam (with tools) | 3. Chartography (without tools) |
|---|---|---|---|---|---|---|---|
| 1 | Claude Sonnet 5.5 | not stated | 100 | 5/90 | 1,824 | 64.5% | 61.6% |
| 2 | Claude Haiku 5.5 | not stated | 50 | 4/90 | 1,578 | 57.4% | 46.4% |
| 3 | Claude Haiku 4.5 | not stated | 0 | 7/90 | 614 | 18.7% | 6.4% |
third-party
third-party · Models: 37 · Entries: 58 × Groups: 3 (benchmark coverage 3/14)GPQA Diamond, OTIS Mock AIME 2024-2025, Chess Puzzles(Epoch AI)
- GPQA Diamondthird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, gpqa_diamond.csv) CC BY 4.0measured/reported by: Epoch AI (58)
- OTIS Mock AIME 2024-2025third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0measured/reported by: Epoch AI (58)
- Chess Puzzles(Epoch AI)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, chess_puzzles.csv) CC BY 4.0measured/reported by: Epoch AI (58)
| Rank | Model | effort | Mean percentile | Benchmarks with values | 1. GPQA Diamond | 2. OTIS Mock AIME 2024-2025 | 3. Chess Puzzles(Epoch AI) |
|---|---|---|---|---|---|---|---|
| 1 | Gemini 4 Carbon | High | 97 | 8/14 | 96.5% | 100.0% | 78.0% |
| 2 | GPT-6 Astra | Max | 95 | 9/14 | 95.8% | 100.0% | 74.0% |
| 3 | GPT-6.1 Sol | Max | 93 | 9/14 | 95.4% | 100.0% | 68.0% |
| 4 | Gemini 4 Argon | High | 93 | 9/14 | 96.4% | 100.0% | 59.0% |
| 5 | Claude Sonnet 5.5 | Max | 92 | 7/14 | 95.6% | 100.0% | 59.0% |
| 6 | GPT-6 Sol | Max | 87 | 7/14 | 94.3% | 100.0% | 56.0% |
| 7 | Claude Fable 5.2 | Max | 87 | 6/14 | 93.9% | 99.7% | 79.0% |
| 8 | GPT-5.6 Sol | Max | 86 | 6/14 | 93.5% | 100.0% | 59.0% |
| 9 | Gemini 3.8 Flash | High | 82 | 7/14 | 95.4% | 98.9% | 58.0% |
| 10 | Kimi K3.1 Pro | Max | 81 | 6/14 | 94.2% | 100.0% | 45.0% |
| 11 | Claude Opus 5.5 | Max | 80 | 7/14 | 90.6% | 100.0% | 67.0% |
| 12 | GPT-5.6 Terra | Max | 77 | 7/14 | 93.3% | 99.7% | 47.0% |
| 12 | Gemini 3.7 Flash | High | 77 | 7/14 | 94.8% | 97.2% | 66.0% |
| 14 | DeepSeek-V4.1-Flash-1017 | Max | 73 | 6/14 | 93.2% | 99.4% | 46.0% |
| 15 | Qwen3.8-Max | Xhigh | 71 | 8/14 | 92.7% | 99.4% | 46.0% |
| 16 | Gemini 3.1 Pro Preview | High | 71 | 6/14 | 94.4% | 95.6% | 51.0% |
| 17 | Claude Opus 4.8 | Max | 68 | 8/14 | 91.0% | 98.3% | 52.0% |
| 18 | DeepSeek-V4-Pro-0813 | Max | 67 | 7/14 | 91.7% | 98.6% | 46.0% |
| 19 | Grok 4.6 | Xhigh | 63 | 7/14 | 93.2% | 99.2% | 33.0% |
| 19 | GPT-5.6 Luna | Max | 63 | 7/14 | 91.6% | 98.3% | 43.0% |
| 21 | Grok 4.7 | Xhigh | 62 | 7/14 | 92.7% | 98.1% | 41.0% |
| 21 | DeepSeek-V4.1-Flash | not stated | 62 | 8/14 | 89.8% | 98.3% | 48.0% |
| 23 | Claude Fable 5 | Max | 62 | 7/14 | 85.9% | 99.7% | 47.0% |
| 24 | Claude Fable 5 | High | 60 | 4/14 | 83.3% | 100.0% | 45.0% |
| 24 | Gemini 3.6 Flash | High | 60 | 7/14 | 94.1% | 94.2% | 40.0% |
| 26 | Grok 4.6 | High | 59 | 4/14 | 94.0% | 97.8% | 33.0% |
| 27 | GPT-6 Luna | Max | 57 | 9/14 | 90.5% | 98.9% | 36.0% |
| 28 | Kimi K3 | Max | 56 | 8/14 | 93.1% | 97.2% | 36.0% |
| 29 | Claude Opus 4.7 | Xhigh | 52 | 4/14 | 90.2% | 97.8% | 36.0% |
| 29 | Kimi K3 | High | 52 | 3/14 | 91.9% | 93.3% | 37.0% |
| 31 | GLM-5.3 | Max | 49 | 6/14 | 90.9% | 91.1% | 37.0% |
| 32 | Mistral Large 4 | High | 48 | 7/14 | 88.1% | 98.3% | 33.0% |
| 32 | Gemini 3 Flash Preview | High | 48 | 8/14 | 89.4% | 95.6% | 40.0% |
| 34 | GPT-5.6 Sol | Low | 44 | 3/14 | 89.9% | 95.6% | 31.0% |
| 35 | Claude Opus 4.8 | Low | 43 | 3/14 | 88.4% | 97.8% | 29.0% |
| 36 | Claude Opus 4.7 | Max | 42 | 6/14 | 86.4% | 86.7% | 41.0% |
| 37 | Gemini 3.6 Flash | Low | 40 | 3/14 | 86.4% | 82.2% | 42.0% |
| 38 | GPT-5.5 | Low | 40 | 3/14 | 90.7% | 84.4% | 30.0% |
| 39 | GLM-5.3-Flash | Max | 38 | 5/14 | 90.2% | 93.9% | 21.0% |
| 40 | Claude Fable 5 | Low | 35 | 3/14 | 78.8% | 97.8% | 31.0% |
| 41 | Claude Opus 4.6 | Max | 34 | 8/14 | 88.4% | 91.1% | 21.0% |
| 42 | GPT-5.6 Terra | Low | 33 | 3/14 | 87.4% | 88.9% | 27.0% |
| 43 | Gemini 3.6 Flash | Minimal | 29 | 3/14 | 85.9% | 80.0% | 29.0% |
| 44 | Claude Opus 4.8 | not stated | 26 | 4/14 | 85.4% | 84.4% | 19.0% |
| 45 | Claude Sonnet 4.6 | High | 21 | 4/14 | 83.3% | 75.6% | 19.0% |
| 46 | Claude Sonnet 4.6 | Med | 20 | 3/14 | 83.3% | 82.2% | 16.0% |
| 47 | Kimi K3 | Low | 20 | 3/14 | 84.9% | 68.9% | 18.0% |
| 48 | Claude Sonnet 4.6 | Max | 19 | 4/14 | 78.8% | 71.1% | 23.0% |
| 49 | Gemini 3.5 Flash-Lite | High | 18 | 5/14 | 83.3% | 71.1% | 16.0% |
| 50 | GPT-5.6 Luna | Low | 15 | 4/14 | 82.3% | 66.7% | 17.0% |
| 51 | GPT-5.6 Sol | None | 13 | 3/14 | 82.8% | 68.9% | 7.0% |
| 51 | Gemini 3.5 Flash-Lite | Low | 13 | 3/14 | 75.8% | 60.0% | 21.0% |
| 53 | GPT-5.5 | None | 9 | 3/14 | 77.3% | 57.8% | 9.0% |
| 54 | Claude Opus 4.5 | not stated | 7 | 4/14 | 80.7% | 48.1% | 5.0% |
| 55 | GPT-5.6 Terra | None | 6 | 3/14 | 77.3% | 53.3% | 5.0% |
| 56 | Gemini 3.5 Flash-Lite | Minimal | 6 | 3/14 | 74.2% | 51.1% | 7.0% |
| 57 | Claude Sonnet 4.5 | not stated | 2 | 9/14 | 73.7% | 35.6% | 6.0% |
| 58 | GPT-5.6 Luna | None | 1 | 4/14 | 63.6% | 40.0% | 3.0% |
third-party · Models: 36 · Entries: 40 × Groups: 3 (benchmark coverage 3/14)GPQA Diamond, FrontierMath Tiers 1-3(v2), OTIS Mock AIME 2024-2025
- GPQA Diamondthird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, gpqa_diamond.csv) CC BY 4.0measured/reported by: Epoch AI (40)
- FrontierMath Tiers 1-3(v2)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, frontiermath_tiers_1_3_v2.csv) CC BY 4.0measured/reported by: Epoch AI (40)
- OTIS Mock AIME 2024-2025third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0measured/reported by: Epoch AI (40)
| Rank | Model | effort | Mean percentile | Benchmarks with values | 1. GPQA Diamond | 2. FrontierMath Tiers 1-3(v2) | 3. OTIS Mock AIME 2024-2025 |
|---|---|---|---|---|---|---|---|
| 1 | GPT-6 Astra | Max | 93 | 9/14 | 95.8% | 93.7% | 100.0% |
| 2 | Gemini 4 Carbon | High | 92 | 8/14 | 96.5% | 90.5% | 100.0% |
| 3 | GPT-6.1 Sol | Max | 91 | 9/14 | 95.4% | 93.7% | 100.0% |
| 4 | Claude Sonnet 5.5 | Max | 87 | 7/14 | 95.6% | 88.8% | 100.0% |
| 5 | GPT-6.1 Terra | Max | 86 | 6/14 | 94.3% | 90.9% | 100.0% |
| 5 | Gemini 4 Argon | High | 86 | 9/14 | 96.4% | 86.0% | 100.0% |
| 7 | GPT-6 Sol | Max | 85 | 7/14 | 94.3% | 89.8% | 100.0% |
| 8 | Claude Fable 5.2 | Max | 80 | 6/14 | 93.9% | 95.1% | 99.7% |
| 9 | GPT-5.6 Sol | Max | 79 | 6/14 | 93.5% | 89.1% | 100.0% |
| 10 | Kimi K3.1 Pro | Max | 76 | 6/14 | 94.2% | 78.3% | 100.0% |
| 11 | Claude Opus 5.5 | Max | 73 | 7/14 | 90.6% | 91.2% | 100.0% |
| 12 | GPT-5.6 Terra | Max | 70 | 7/14 | 93.3% | 86.0% | 99.7% |
| 13 | Gemini 3.8 Flash | High | 63 | 7/14 | 95.4% | 68.4% | 98.9% |
| 14 | DeepSeek-V4.1-Flash-1017 | Max | 59 | 6/14 | 93.2% | 71.2% | 99.4% |
| 15 | Gemini 3.7 Flash | High | 59 | 7/14 | 94.8% | 71.6% | 97.2% |
| 15 | Qwen3.8-Max | Xhigh | 59 | 8/14 | 92.7% | 74.7% | 99.4% |
| 17 | Claude Fable 5 | Max | 56 | 7/14 | 85.9% | 87.0% | 99.7% |
| 18 | GPT-5.6 Luna | Max | 54 | 7/14 | 91.6% | 82.1% | 98.3% |
| 19 | Grok 4.6 | Xhigh | 53 | 7/14 | 93.2% | 66.0% | 99.2% |
| 20 | GPT-6 Luna | Max | 53 | 9/14 | 90.5% | 79.0% | 98.9% |
| 20 | Claude Opus 4.8 | Max | 53 | 8/14 | 91.0% | 80.0% | 98.3% |
| 22 | Kimi K3 | Max | 50 | 8/14 | 93.1% | 72.2% | 97.2% |
| 23 | Gemini 3.1 Pro Preview | not stated | 45 | 5/14 | 94.1% | 59.7% | 95.6% |
| 23 | DeepSeek-V4-Pro-0813 | Max | 45 | 7/14 | 91.7% | 64.6% | 98.6% |
| 25 | Gemini 3.6 Flash | High | 44 | 7/14 | 94.1% | 59.0% | 94.2% |
| 26 | DeepSeek-V4.1-Flash | not stated | 40 | 8/14 | 89.8% | 67.4% | 98.3% |
| 27 | Grok 4.7 | Xhigh | 38 | 7/14 | 92.7% | 53.0% | 98.1% |
| 28 | GLM-5.3 | Max | 36 | 6/14 | 90.9% | 68.8% | 91.1% |
| 29 | Mistral Large 4 | High | 33 | 7/14 | 88.1% | 58.6% | 98.3% |
| 30 | Claude Opus 4.7 | Max | 30 | 6/14 | 86.4% | 70.2% | 86.7% |
| 31 | Claude Opus 4.6 | Max | 29 | 8/14 | 88.4% | 66.0% | 91.1% |
| 32 | GLM-5.3-Flash | Max | 28 | 5/14 | 90.2% | 55.8% | 93.9% |
| 33 | Gemini 3 Flash Preview | not stated | 19 | 5/14 | 83.2% | 51.2% | 92.8% |
| 34 | Gemini 2.5 Pro | not stated | 13 | 5/14 | 85.3% | 24.6% | 84.7% |
| 35 | GPT-5.6 Luna | Low | 11 | 4/14 | 82.3% | 41.4% | 66.7% |
| 35 | Gemini 3.5 Flash-Lite | High | 11 | 5/14 | 83.3% | 26.0% | 71.1% |
| 37 | Gemini 3.1 Flash-Lite | High | 10 | 3/14 | 81.8% | 27.7% | 80.0% |
| 38 | GPT-5.6 Luna | None | 5 | 4/14 | 63.6% | 39.7% | 40.0% |
| 39 | Gemini 3.1 Flash-Lite | Low | 4 | 3/14 | 74.2% | 22.5% | 44.4% |
| 40 | Gemini 3.1 Flash-Lite | Minimal | 1 | 3/14 | 73.7% | 21.4% | 37.8% |
third-party · Models: 35 · Entries: 35 × Groups: 4 (benchmark coverage 4/14)GPQA Diamond, FrontierMath Tiers 1-3(v2), FrontierMath Tier 4(v2), OTIS Mock AIME 2024-2025
- GPQA Diamondthird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, gpqa_diamond.csv) CC BY 4.0measured/reported by: Epoch AI (35)
- FrontierMath Tiers 1-3(v2)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, frontiermath_tiers_1_3_v2.csv) CC BY 4.0measured/reported by: Epoch AI (35)
- FrontierMath Tier 4(v2)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, frontiermath_tier_4_v2.csv) CC BY 4.0measured/reported by: Epoch AI (35)
- OTIS Mock AIME 2024-2025third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0measured/reported by: Epoch AI (35)
| Rank | Model | effort | Mean percentile | Benchmarks with values | 1. GPQA Diamond | 2. FrontierMath Tiers 1-3(v2) | 3. FrontierMath Tier 4(v2) | 4. OTIS Mock AIME 2024-2025 |
|---|---|---|---|---|---|---|---|---|
| 1 | GPT-6 Astra | Max | 93 | 9/14 | 95.8% | 93.7% | 97.6% | 100.0% |
| 2 | GPT-6.1 Sol | Max | 92 | 9/14 | 95.4% | 93.7% | 100.0% | 100.0% |
| 3 | Gemini 4 Carbon | High | 91 | 8/14 | 96.5% | 90.5% | 95.1% | 100.0% |
| 4 | GPT-6.1 Terra | Max | 85 | 6/14 | 94.3% | 90.9% | 92.7% | 100.0% |
| 5 | Claude Fable 5.2 | Max | 83 | 6/14 | 93.9% | 95.1% | 100.0% | 99.7% |
| 6 | Claude Sonnet 5.5 | Max | 82 | 7/14 | 95.6% | 88.8% | 80.5% | 100.0% |
| 7 | GPT-6 Sol | Max | 82 | 7/14 | 94.3% | 89.8% | 90.0% | 100.0% |
| 8 | Gemini 4 Argon | High | 81 | 9/14 | 96.4% | 86.0% | 78.1% | 100.0% |
| 9 | GPT-5.6 Sol | Max | 76 | 6/14 | 93.5% | 89.1% | 82.9% | 100.0% |
| 10 | Claude Opus 5.5 | Max | 74 | 7/14 | 90.6% | 91.2% | 95.0% | 100.0% |
| 11 | Kimi K3.1 Pro | Max | 70 | 6/14 | 94.2% | 78.3% | 65.9% | 100.0% |
| 12 | GPT-5.6 Terra | Max | 66 | 7/14 | 93.3% | 86.0% | 70.7% | 99.7% |
| 13 | Claude Fable 5 | Max | 58 | 7/14 | 85.9% | 87.0% | 90.2% | 99.7% |
| 14 | DeepSeek-V4.1-Flash-1017 | Max | 53 | 6/14 | 93.2% | 71.2% | 46.3% | 99.4% |
| 15 | Qwen3.8-Max | Xhigh | 52 | 8/14 | 92.7% | 74.7% | 46.3% | 99.4% |
| 16 | GPT-5.6 Luna | Max | 51 | 7/14 | 91.6% | 82.1% | 61.0% | 98.3% |
| 17 | Gemini 3.7 Flash | High | 50 | 7/14 | 94.8% | 71.6% | 36.6% | 97.2% |
| 18 | GPT-6 Luna | Max | 49 | 9/14 | 90.5% | 79.0% | 56.1% | 98.9% |
| 18 | Claude Opus 4.8 | Max | 49 | 8/14 | 91.0% | 80.0% | 56.1% | 98.3% |
| 20 | Gemini 3.8 Flash | High | 47 | 7/14 | 95.4% | 68.4% | 22.0% | 98.9% |
| 21 | Grok 4.6 | Xhigh | 44 | 7/14 | 93.2% | 66.0% | 31.7% | 99.2% |
| 22 | Kimi K3 | Max | 44 | 8/14 | 93.1% | 72.2% | 39.0% | 97.2% |
| 23 | Gemini 3.1 Pro Preview | not stated | 35 | 5/14 | 94.1% | 59.7% | 26.8% | 95.6% |
| 23 | DeepSeek-V4-Pro-0813 | Max | 35 | 7/14 | 91.7% | 64.6% | 26.8% | 98.6% |
| 25 | Gemini 3.6 Flash | High | 31 | 7/14 | 94.1% | 59.0% | 22.0% | 94.2% |
| 26 | DeepSeek-V4.1-Flash | not stated | 30 | 8/14 | 89.8% | 67.4% | 26.8% | 98.3% |
| 27 | GLM-5.3 | Max | 29 | 6/14 | 90.9% | 68.8% | 29.3% | 91.1% |
| 28 | Claude Opus 4.7 | Max | 25 | 6/14 | 86.4% | 70.2% | 31.7% | 86.7% |
| 29 | Grok 4.7 | Xhigh | 24 | 7/14 | 92.7% | 53.0% | 17.1% | 98.1% |
| 30 | Mistral Large 4 | High | 24 | 7/14 | 88.1% | 58.6% | 26.8% | 98.3% |
| 31 | Claude Opus 4.6 | Max | 21 | 8/14 | 88.4% | 66.0% | 26.8% | 91.1% |
| 32 | GLM-5.3-Flash | Max | 15 | 5/14 | 90.2% | 55.8% | 17.1% | 93.9% |
| 33 | Gemini 3 Flash Preview | not stated | 7 | 5/14 | 83.2% | 51.2% | 17.1% | 92.8% |
| 34 | Gemini 2.5 Pro | not stated | 3 | 5/14 | 85.3% | 24.6% | 0.0% | 84.7% |
| 35 | Gemini 3.5 Flash-Lite | High | 2 | 5/14 | 83.3% | 26.0% | 0.0% | 71.1% |
third-party · Models: 33 · Entries: 35 × Groups: 3 (benchmark coverage 3/14)GPQA Diamond, OTIS Mock AIME 2024-2025, SimpleQA Verified
- GPQA Diamondthird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, gpqa_diamond.csv) CC BY 4.0measured/reported by: Epoch AI (35)
- OTIS Mock AIME 2024-2025third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0measured/reported by: Epoch AI (35)
- SimpleQA Verifiedthird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, simpleqa_verified.csv) CC BY 4.0measured/reported by: Epoch AI (35)
| Rank | Model | effort | Mean percentile | Benchmarks with values | 1. GPQA Diamond | 2. OTIS Mock AIME 2024-2025 | 3. SimpleQA Verified |
|---|---|---|---|---|---|---|---|
| 1 | Gemini 4 Carbon | High | 95 | 8/14 | 96.5% | 100.0% | 77.7% |
| 2 | Gemini 4 Argon | High | 93 | 9/14 | 96.4% | 100.0% | 76.0% |
| 3 | GPT-6 Astra | Max | 91 | 9/14 | 95.8% | 100.0% | 75.6% |
| 4 | GPT-6.1 Sol | Max | 87 | 9/14 | 95.4% | 100.0% | 73.9% |
| 5 | Claude Fable 5.2 | Max | 77 | 6/14 | 93.9% | 99.7% | 94.0% |
| 6 | GPT-6 Sol | Max | 75 | 7/14 | 94.3% | 100.0% | 60.7% |
| 6 | GPT-5.6 Sol | Max | 75 | 6/14 | 93.5% | 100.0% | 69.7% |
| 8 | Kimi K3.1 Pro | Max | 74 | 6/14 | 94.2% | 100.0% | 62.2% |
| 9 | GPT-6.1 Terra | Max | 74 | 6/14 | 94.3% | 100.0% | 57.3% |
| 10 | Gemini 3.8 Flash | High | 73 | 7/14 | 95.4% | 98.9% | 69.7% |
| 11 | Claude Sonnet 5.5 | Max | 70 | 7/14 | 95.6% | 100.0% | 46.5% |
| 12 | Claude Opus 5.5 | Max | 65 | 7/14 | 90.6% | 100.0% | 72.2% |
| 13 | Gemini 3.1 Pro Preview | High | 61 | 6/14 | 94.4% | 95.6% | 73.5% |
| 14 | Gemini 3.7 Flash | High | 60 | 7/14 | 94.8% | 97.2% | 69.2% |
| 15 | GPT-5.6 Terra | Max | 50 | 7/14 | 93.3% | 99.7% | 43.2% |
| 16 | Gemini 3.6 Flash | High | 50 | 7/14 | 94.1% | 94.2% | 66.2% |
| 17 | Grok 4.6 | Xhigh | 50 | 7/14 | 93.2% | 99.2% | 48.9% |
| 18 | DeepSeek-V4-Pro-0813 | Max | 46 | 7/14 | 91.7% | 98.6% | 52.9% |
| 19 | Grok 4.7 | Xhigh | 45 | 7/14 | 92.7% | 98.1% | 56.0% |
| 19 | Grok 4.6 | High | 45 | 4/14 | 94.0% | 97.8% | 49.3% |
| 19 | DeepSeek-V4.1-Flash-1017 | Max | 45 | 6/14 | 93.2% | 99.4% | 42.5% |
| 22 | Qwen3.8-Max | Xhigh | 45 | 8/14 | 92.7% | 99.4% | 45.8% |
| 23 | Claude Opus 4.8 | Max | 43 | 8/14 | 91.0% | 98.3% | 53.0% |
| 24 | Kimi K3 | Max | 39 | 8/14 | 93.1% | 97.2% | 50.6% |
| 25 | Gemini 3 Flash Preview | High | 35 | 8/14 | 89.4% | 95.6% | 66.8% |
| 26 | Claude Opus 4.7 | Xhigh | 33 | 4/14 | 90.2% | 97.8% | 51.7% |
| 27 | GPT-6 Luna | Max | 32 | 9/14 | 90.5% | 98.9% | 41.4% |
| 28 | GPT-5.6 Luna | Max | 30 | 7/14 | 91.6% | 98.3% | 41.0% |
| 29 | DeepSeek-V4.1-Flash | not stated | 28 | 8/14 | 89.8% | 98.3% | 42.9% |
| 30 | Claude Opus 4.6 | Max | 19 | 8/14 | 88.4% | 91.1% | 47.0% |
| 31 | GLM-5.3 | Max | 18 | 6/14 | 90.9% | 91.1% | 41.0% |
| 32 | Mistral Large 4 | High | 17 | 7/14 | 88.1% | 98.3% | 20.0% |
| 33 | Claude Sonnet 4.6 | High | 7 | 4/14 | 83.3% | 75.6% | 35.5% |
| 34 | Claude Sonnet 4.6 | Max | 4 | 4/14 | 78.8% | 71.1% | 32.8% |
| 35 | Claude Sonnet 4.5 | not stated | 1 | 9/14 | 73.7% | 35.6% | 23.7% |
third-party · Models: 33 · Entries: 35 × Groups: 3 (benchmark coverage 3/14)GPQA Diamond, SimpleQA Verified, Chess Puzzles(Epoch AI)
- GPQA Diamondthird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, gpqa_diamond.csv) CC BY 4.0measured/reported by: Epoch AI (35)
- SimpleQA Verifiedthird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, simpleqa_verified.csv) CC BY 4.0measured/reported by: Epoch AI (35)
- Chess Puzzles(Epoch AI)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, chess_puzzles.csv) CC BY 4.0measured/reported by: Epoch AI (35)
| Rank | Model | effort | Mean percentile | Benchmarks with values | 1. GPQA Diamond | 2. SimpleQA Verified | 3. Chess Puzzles(Epoch AI) |
|---|---|---|---|---|---|---|---|
| 1 | Gemini 4 Carbon | High | 98 | 8/14 | 96.5% | 77.7% | 78.0% |
| 2 | GPT-6 Astra | Max | 93 | 9/14 | 95.8% | 75.6% | 74.0% |
| 3 | Gemini 4 Argon | High | 90 | 9/14 | 96.4% | 76.0% | 59.0% |
| 4 | GPT-6.1 Sol | Max | 89 | 9/14 | 95.4% | 73.9% | 68.0% |
| 5 | Claude Fable 5.2 | Max | 88 | 6/14 | 93.9% | 94.0% | 79.0% |
| 6 | Gemini 3.7 Flash | High | 80 | 7/14 | 94.8% | 69.2% | 66.0% |
| 7 | Gemini 3.8 Flash | High | 79 | 7/14 | 95.4% | 69.7% | 58.0% |
| 8 | Gemini 3.1 Pro Preview | High | 75 | 6/14 | 94.4% | 73.5% | 51.0% |
| 9 | GPT-5.6 Sol | Max | 73 | 6/14 | 93.5% | 69.7% | 59.0% |
| 10 | GPT-6 Sol | Max | 70 | 7/14 | 94.3% | 60.7% | 56.0% |
| 11 | Claude Sonnet 5.5 | Max | 69 | 7/14 | 95.6% | 46.5% | 59.0% |
| 12 | Claude Opus 5.5 | Max | 67 | 7/14 | 90.6% | 72.2% | 67.0% |
| 13 | Kimi K3.1 Pro | Max | 61 | 6/14 | 94.2% | 62.2% | 45.0% |
| 14 | Gemini 3.6 Flash | High | 57 | 7/14 | 94.1% | 66.2% | 40.0% |
| 15 | Claude Opus 4.8 | Max | 52 | 8/14 | 91.0% | 53.0% | 52.0% |
| 16 | Grok 4.7 | Xhigh | 48 | 7/14 | 92.7% | 56.0% | 41.0% |
| 16 | GPT-5.6 Terra | Max | 48 | 7/14 | 93.3% | 43.2% | 47.0% |
| 16 | DeepSeek-V4-Pro-0813 | Max | 48 | 7/14 | 91.7% | 52.9% | 46.0% |
| 19 | DeepSeek-V4.1-Flash-1017 | Max | 43 | 6/14 | 93.2% | 42.5% | 46.0% |
| 20 | Grok 4.6 | High | 42 | 4/14 | 94.0% | 49.3% | 33.0% |
| 20 | Qwen3.8-Max | Xhigh | 42 | 8/14 | 92.7% | 45.8% | 46.0% |
| 22 | Kimi K3 | Max | 40 | 8/14 | 93.1% | 50.6% | 36.0% |
| 23 | Gemini 3 Flash Preview | High | 40 | 8/14 | 89.4% | 66.8% | 40.0% |
| 24 | Grok 4.6 | Xhigh | 37 | 7/14 | 93.2% | 48.9% | 33.0% |
| 25 | DeepSeek-V4.1-Flash | not stated | 35 | 8/14 | 89.8% | 42.9% | 48.0% |
| 26 | Claude Opus 4.7 | Xhigh | 32 | 4/14 | 90.2% | 51.7% | 36.0% |
| 27 | GPT-5.6 Luna | Max | 32 | 7/14 | 91.6% | 41.0% | 43.0% |
| 28 | Claude Haiku 5.5 | Max | 30 | 6/14 | 89.7% | 23.8% | 55.0% |
| 29 | GLM-5.3 | Max | 26 | 6/14 | 90.9% | 41.0% | 37.0% |
| 30 | GPT-6 Luna | Max | 24 | 9/14 | 90.5% | 41.4% | 36.0% |
| 31 | Claude Opus 4.6 | Max | 19 | 8/14 | 88.4% | 47.0% | 21.0% |
| 32 | Mistral Large 4 | High | 8 | 7/14 | 88.1% | 20.0% | 33.0% |
| 33 | Claude Sonnet 4.6 | Max | 7 | 4/14 | 78.8% | 32.8% | 23.0% |
| 33 | Claude Sonnet 4.6 | High | 7 | 4/14 | 83.3% | 35.5% | 19.0% |
| 35 | Claude Sonnet 4.5 | not stated | 1 | 9/14 | 73.7% | 23.7% | 6.0% |
third-party · Models: 33 · Entries: 33 × Groups: 3 (benchmark coverage 3/14)FrontierMath Tiers 1-3(v2), FrontierMath Tier 4(v2), Chess Puzzles(Epoch AI)
- FrontierMath Tiers 1-3(v2)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, frontiermath_tiers_1_3_v2.csv) CC BY 4.0measured/reported by: Epoch AI (33)
- FrontierMath Tier 4(v2)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, frontiermath_tier_4_v2.csv) CC BY 4.0measured/reported by: Epoch AI (33)
- Chess Puzzles(Epoch AI)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, chess_puzzles.csv) CC BY 4.0measured/reported by: Epoch AI (33)
| Rank | Model | effort | Mean percentile | Benchmarks with values | 1. FrontierMath Tiers 1-3(v2) | 2. FrontierMath Tier 4(v2) | 3. Chess Puzzles(Epoch AI) |
|---|---|---|---|---|---|---|---|
| 1 | Claude Fable 5.2 | Max | 99 | 6/14 | 95.1% | 100.0% | 79.0% |
| 2 | GPT-6.1 Sol | Max | 95 | 9/14 | 93.7% | 100.0% | 68.0% |
| 3 | GPT-6 Astra | Max | 94 | 9/14 | 93.7% | 97.6% | 74.0% |
| 4 | Gemini 4 Carbon | High | 92 | 8/14 | 90.5% | 95.1% | 78.0% |
| 5 | Claude Opus 5.5 | Max | 89 | 7/14 | 91.2% | 95.0% | 67.0% |
| 6 | GPT-5.6 Sol | Max | 78 | 6/14 | 89.1% | 82.9% | 59.0% |
| 7 | GPT-6 Sol | Max | 77 | 7/14 | 89.8% | 90.0% | 56.0% |
| 8 | Claude Sonnet 5.5 | Max | 76 | 7/14 | 88.8% | 80.5% | 59.0% |
| 9 | Gemini 4 Argon | High | 72 | 9/14 | 86.0% | 78.1% | 59.0% |
| 10 | GPT-5.5 | Xhigh | 72 | 6/14 | 85.3% | 72.5% | 63.0% |
| 11 | Claude Fable 5 | Max | 70 | 7/14 | 87.0% | 90.2% | 47.0% |
| 12 | GPT-5.6 Terra | Max | 63 | 7/14 | 86.0% | 70.7% | 47.0% |
| 13 | Claude Opus 4.8 | Max | 58 | 8/14 | 80.0% | 56.1% | 52.0% |
| 14 | Gemini 3.7 Flash | High | 54 | 7/14 | 71.6% | 36.6% | 66.0% |
| 15 | Claude Haiku 5.5 | Max | 53 | 6/14 | 75.1% | 46.3% | 55.0% |
| 16 | GPT-5.6 Luna | Max | 52 | 7/14 | 82.1% | 61.0% | 43.0% |
| 17 | Kimi K3.1 Pro | Max | 51 | 6/14 | 78.3% | 65.9% | 45.0% |
| 18 | Qwen3.8-Max | Xhigh | 46 | 8/14 | 74.7% | 46.3% | 46.0% |
| 19 | GPT-6 Luna | Max | 43 | 9/14 | 79.0% | 56.1% | 36.0% |
| 19 | DeepSeek-V4.1-Flash-1017 | Max | 43 | 6/14 | 71.2% | 46.3% | 46.0% |
| 21 | Gemini 3.8 Flash | High | 36 | 7/14 | 68.4% | 22.0% | 58.0% |
| 22 | Kimi K3 | Max | 34 | 8/14 | 72.2% | 39.0% | 36.0% |
| 22 | DeepSeek-V4.1-Flash | not stated | 34 | 8/14 | 67.4% | 26.8% | 48.0% |
| 24 | Claude Opus 4.7 | Max | 32 | 6/14 | 70.2% | 31.7% | 41.0% |
| 25 | GLM-5.3 | Max | 27 | 6/14 | 68.8% | 29.3% | 37.0% |
| 26 | DeepSeek-V4-Pro-0813 | Max | 27 | 7/14 | 64.6% | 26.8% | 46.0% |
| 27 | Grok 4.6 | Xhigh | 21 | 7/14 | 66.0% | 31.7% | 33.0% |
| 28 | Gemini 3.6 Flash | High | 16 | 7/14 | 59.0% | 22.0% | 40.0% |
| 29 | Claude Opus 4.6 | Max | 15 | 8/14 | 66.0% | 26.8% | 21.0% |
| 30 | Mistral Large 4 | High | 14 | 7/14 | 58.6% | 26.8% | 33.0% |
| 31 | Grok 4.7 | Xhigh | 13 | 7/14 | 53.0% | 17.1% | 41.0% |
| 32 | GLM-5.3-Flash | Max | 5 | 5/14 | 55.8% | 17.1% | 21.0% |
| 33 | Gemini 3.5 Flash-Lite | High | 0 | 5/14 | 26.0% | 0.0% | 16.0% |
third-party · Models: 32 · Entries: 34 × Groups: 4 (benchmark coverage 4/14)GPQA Diamond, OTIS Mock AIME 2024-2025, SimpleQA Verified, Chess Puzzles(Epoch AI)
- GPQA Diamondthird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, gpqa_diamond.csv) CC BY 4.0measured/reported by: Epoch AI (34)
- OTIS Mock AIME 2024-2025third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0measured/reported by: Epoch AI (34)
- SimpleQA Verifiedthird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, simpleqa_verified.csv) CC BY 4.0measured/reported by: Epoch AI (34)
- Chess Puzzles(Epoch AI)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, chess_puzzles.csv) CC BY 4.0measured/reported by: Epoch AI (34)
| Rank | Model | effort | Mean percentile | Benchmarks with values | 1. GPQA Diamond | 2. OTIS Mock AIME 2024-2025 | 3. SimpleQA Verified | 4. Chess Puzzles(Epoch AI) |
|---|---|---|---|---|---|---|---|---|
| 1 | Gemini 4 Carbon | High | 95 | 8/14 | 96.5% | 100.0% | 77.7% | 78.0% |
| 2 | GPT-6 Astra | Max | 92 | 9/14 | 95.8% | 100.0% | 75.6% | 74.0% |
| 3 | Gemini 4 Argon | High | 89 | 9/14 | 96.4% | 100.0% | 76.0% | 59.0% |
| 4 | GPT-6.1 Sol | Max | 88 | 9/14 | 95.4% | 100.0% | 73.9% | 68.0% |
| 5 | Claude Fable 5.2 | Max | 84 | 6/14 | 93.9% | 99.7% | 94.0% | 79.0% |
| 6 | GPT-5.6 Sol | Max | 76 | 6/14 | 93.5% | 100.0% | 69.7% | 59.0% |
| 7 | GPT-6 Sol | Max | 73 | 7/14 | 94.3% | 100.0% | 60.7% | 56.0% |
| 8 | Gemini 3.8 Flash | High | 73 | 7/14 | 95.4% | 98.9% | 69.7% | 58.0% |
| 9 | Claude Sonnet 5.5 | Max | 73 | 7/14 | 95.6% | 100.0% | 46.5% | 59.0% |
| 10 | Claude Opus 5.5 | Max | 71 | 7/14 | 90.6% | 100.0% | 72.2% | 67.0% |
| 11 | Kimi K3.1 Pro | Max | 67 | 6/14 | 94.2% | 100.0% | 62.2% | 45.0% |
| 12 | Gemini 3.7 Flash | High | 66 | 7/14 | 94.8% | 97.2% | 69.2% | 66.0% |
| 13 | Gemini 3.1 Pro Preview | High | 62 | 6/14 | 94.4% | 95.6% | 73.5% | 51.0% |
| 14 | GPT-5.6 Terra | Max | 53 | 7/14 | 93.3% | 99.7% | 43.2% | 47.0% |
| 15 | Claude Opus 4.8 | Max | 50 | 8/14 | 91.0% | 98.3% | 53.0% | 52.0% |
| 16 | DeepSeek-V4-Pro-0813 | Max | 48 | 7/14 | 91.7% | 98.6% | 52.9% | 46.0% |
| 17 | DeepSeek-V4.1-Flash-1017 | Max | 48 | 6/14 | 93.2% | 99.4% | 42.5% | 46.0% |
| 18 | Qwen3.8-Max | Xhigh | 47 | 8/14 | 92.7% | 99.4% | 45.8% | 46.0% |
| 19 | Gemini 3.6 Flash | High | 47 | 7/14 | 94.1% | 94.2% | 66.2% | 40.0% |
| 20 | Grok 4.7 | Xhigh | 45 | 7/14 | 92.7% | 98.1% | 56.0% | 41.0% |
| 21 | Grok 4.6 | Xhigh | 42 | 7/14 | 93.2% | 99.2% | 48.9% | 33.0% |
| 22 | Grok 4.6 | High | 39 | 4/14 | 94.0% | 97.8% | 49.3% | 33.0% |
| 23 | DeepSeek-V4.1-Flash | not stated | 37 | 8/14 | 89.8% | 98.3% | 42.9% | 48.0% |
| 24 | Kimi K3 | Max | 36 | 8/14 | 93.1% | 97.2% | 50.6% | 36.0% |
| 25 | Gemini 3 Flash Preview | High | 35 | 8/14 | 89.4% | 95.6% | 66.8% | 40.0% |
| 26 | GPT-5.6 Luna | Max | 34 | 7/14 | 91.6% | 98.3% | 41.0% | 43.0% |
| 27 | Claude Opus 4.7 | Xhigh | 32 | 4/14 | 90.2% | 97.8% | 51.7% | 36.0% |
| 28 | GPT-6 Luna | Max | 31 | 9/14 | 90.5% | 98.9% | 41.4% | 36.0% |
| 29 | GLM-5.3 | Max | 21 | 6/14 | 90.9% | 91.1% | 41.0% | 37.0% |
| 30 | Mistral Large 4 | High | 17 | 7/14 | 88.1% | 98.3% | 20.0% | 33.0% |
| 31 | Claude Opus 4.6 | Max | 16 | 8/14 | 88.4% | 91.1% | 47.0% | 21.0% |
| 32 | Claude Sonnet 4.6 | High | 6 | 4/14 | 83.3% | 75.6% | 35.5% | 19.0% |
| 33 | Claude Sonnet 4.6 | Max | 5 | 4/14 | 78.8% | 71.1% | 32.8% | 23.0% |
| 34 | Claude Sonnet 4.5 | not stated | 1 | 9/14 | 73.7% | 35.6% | 23.7% | 6.0% |
third-party · Models: 32 · Entries: 32 × Groups: 4 (benchmark coverage 4/14)GPQA Diamond, FrontierMath Tiers 1-3(v2), FrontierMath Tier 4(v2), Chess Puzzles(Epoch AI)
- GPQA Diamondthird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, gpqa_diamond.csv) CC BY 4.0measured/reported by: Epoch AI (32)
- FrontierMath Tiers 1-3(v2)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, frontiermath_tiers_1_3_v2.csv) CC BY 4.0measured/reported by: Epoch AI (32)
- FrontierMath Tier 4(v2)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, frontiermath_tier_4_v2.csv) CC BY 4.0measured/reported by: Epoch AI (32)
- Chess Puzzles(Epoch AI)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, chess_puzzles.csv) CC BY 4.0measured/reported by: Epoch AI (32)
| Rank | Model | effort | Mean percentile | Benchmarks with values | 1. GPQA Diamond | 2. FrontierMath Tiers 1-3(v2) | 3. FrontierMath Tier 4(v2) | 4. Chess Puzzles(Epoch AI) |
|---|---|---|---|---|---|---|---|---|
| 1 | GPT-6 Astra | Max | 94 | 9/14 | 95.8% | 93.7% | 97.6% | 74.0% |
| 2 | Gemini 4 Carbon | High | 94 | 8/14 | 96.5% | 90.5% | 95.1% | 78.0% |
| 3 | GPT-6.1 Sol | Max | 92 | 9/14 | 95.4% | 93.7% | 100.0% | 68.0% |
| 4 | Claude Fable 5.2 | Max | 92 | 6/14 | 93.9% | 95.1% | 100.0% | 79.0% |
| 5 | Claude Sonnet 5.5 | Max | 80 | 7/14 | 95.6% | 88.8% | 80.5% | 59.0% |
| 6 | Gemini 4 Argon | High | 79 | 9/14 | 96.4% | 86.0% | 78.1% | 59.0% |
| 7 | GPT-6 Sol | Max | 77 | 7/14 | 94.3% | 89.8% | 90.0% | 56.0% |
| 8 | GPT-5.6 Sol | Max | 75 | 6/14 | 93.5% | 89.1% | 82.9% | 59.0% |
| 9 | Claude Opus 5.5 | Max | 73 | 7/14 | 90.6% | 91.2% | 95.0% | 67.0% |
| 10 | GPT-5.6 Terra | Max | 63 | 7/14 | 93.3% | 86.0% | 70.7% | 47.0% |
| 11 | Gemini 3.7 Flash | High | 61 | 7/14 | 94.8% | 71.6% | 36.6% | 66.0% |
| 12 | Kimi K3.1 Pro | Max | 58 | 6/14 | 94.2% | 78.3% | 65.9% | 45.0% |
| 13 | Claude Fable 5 | Max | 54 | 7/14 | 85.9% | 87.0% | 90.2% | 47.0% |
| 13 | Claude Opus 4.8 | Max | 54 | 8/14 | 91.0% | 80.0% | 56.1% | 52.0% |
| 15 | GPT-5.6 Luna | Max | 50 | 7/14 | 91.6% | 82.1% | 61.0% | 43.0% |
| 16 | Gemini 3.8 Flash | High | 49 | 7/14 | 95.4% | 68.4% | 22.0% | 58.0% |
| 17 | DeepSeek-V4.1-Flash-1017 | Max | 47 | 6/14 | 93.2% | 71.2% | 46.3% | 46.0% |
| 18 | Qwen3.8-Max | Xhigh | 47 | 8/14 | 92.7% | 74.7% | 46.3% | 46.0% |
| 19 | Claude Haiku 5.5 | Max | 45 | 6/14 | 89.7% | 75.1% | 46.3% | 55.0% |
| 20 | GPT-6 Luna | Max | 40 | 9/14 | 90.5% | 79.0% | 56.1% | 36.0% |
| 21 | Kimi K3 | Max | 39 | 8/14 | 93.1% | 72.2% | 39.0% | 36.0% |
| 22 | DeepSeek-V4-Pro-0813 | Max | 31 | 7/14 | 91.7% | 64.6% | 26.8% | 46.0% |
| 22 | DeepSeek-V4.1-Flash | not stated | 31 | 8/14 | 89.8% | 67.4% | 26.8% | 48.0% |
| 24 | Grok 4.6 | Xhigh | 31 | 7/14 | 93.2% | 66.0% | 31.7% | 33.0% |
| 25 | Gemini 3.6 Flash | High | 30 | 7/14 | 94.1% | 59.0% | 22.0% | 40.0% |
| 26 | GLM-5.3 | Max | 29 | 6/14 | 90.9% | 68.8% | 29.3% | 37.0% |
| 27 | Claude Opus 4.7 | Max | 27 | 6/14 | 86.4% | 70.2% | 31.7% | 41.0% |
| 28 | Grok 4.7 | Xhigh | 22 | 7/14 | 92.7% | 53.0% | 17.1% | 41.0% |
| 29 | Claude Opus 4.6 | Max | 15 | 8/14 | 88.4% | 66.0% | 26.8% | 21.0% |
| 30 | Mistral Large 4 | High | 13 | 7/14 | 88.1% | 58.6% | 26.8% | 33.0% |
| 31 | GLM-5.3-Flash | Max | 10 | 5/14 | 90.2% | 55.8% | 17.1% | 21.0% |
| 32 | Gemini 3.5 Flash-Lite | High | 0 | 5/14 | 83.3% | 26.0% | 0.0% | 16.0% |
third-party · Models: 32 · Entries: 34 × Groups: 3 (benchmark coverage 3/14)GPQA Diamond, FrontierMath Tiers 1-3(v2), Chess Puzzles(Epoch AI)
- GPQA Diamondthird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, gpqa_diamond.csv) CC BY 4.0measured/reported by: Epoch AI (34)
- FrontierMath Tiers 1-3(v2)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, frontiermath_tiers_1_3_v2.csv) CC BY 4.0measured/reported by: Epoch AI (34)
- Chess Puzzles(Epoch AI)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, chess_puzzles.csv) CC BY 4.0measured/reported by: Epoch AI (34)
| Rank | Model | effort | Mean percentile | Benchmarks with values | 1. GPQA Diamond | 2. FrontierMath Tiers 1-3(v2) | 3. Chess Puzzles(Epoch AI) |
|---|---|---|---|---|---|---|---|
| 1 | Gemini 4 Carbon | High | 95 | 8/14 | 96.5% | 90.5% | 78.0% |
| 2 | GPT-6 Astra | Max | 94 | 9/14 | 95.8% | 93.7% | 74.0% |
| 3 | GPT-6.1 Sol | Max | 91 | 9/14 | 95.4% | 93.7% | 68.0% |
| 4 | Claude Fable 5.2 | Max | 90 | 6/14 | 93.9% | 95.1% | 79.0% |
| 5 | Claude Sonnet 5.5 | Max | 83 | 7/14 | 95.6% | 88.8% | 59.0% |
| 6 | Gemini 4 Argon | High | 82 | 9/14 | 96.4% | 86.0% | 59.0% |
| 7 | GPT-6 Sol | Max | 78 | 7/14 | 94.3% | 89.8% | 56.0% |
| 8 | GPT-5.6 Sol | Max | 76 | 6/14 | 93.5% | 89.1% | 59.0% |
| 9 | Claude Opus 5.5 | Max | 71 | 7/14 | 90.6% | 91.2% | 67.0% |
| 9 | Gemini 3.7 Flash | High | 71 | 7/14 | 94.8% | 71.6% | 66.0% |
| 11 | Gemini 3.8 Flash | High | 64 | 7/14 | 95.4% | 68.4% | 58.0% |
| 12 | GPT-5.6 Terra | Max | 64 | 7/14 | 93.3% | 86.0% | 47.0% |
| 13 | Kimi K3.1 Pro | Max | 59 | 6/14 | 94.2% | 78.3% | 45.0% |
| 14 | Claude Opus 4.8 | Max | 56 | 8/14 | 91.0% | 80.0% | 52.0% |
| 15 | DeepSeek-V4.1-Flash-1017 | Max | 50 | 6/14 | 93.2% | 71.2% | 46.0% |
| 16 | GPT-5.6 Luna | Max | 49 | 7/14 | 91.6% | 82.1% | 43.0% |
| 16 | Qwen3.8-Max | Xhigh | 49 | 8/14 | 92.7% | 74.7% | 46.0% |
| 18 | Claude Haiku 5.5 | Max | 47 | 6/14 | 89.7% | 75.1% | 55.0% |
| 19 | Claude Fable 5 | Max | 47 | 7/14 | 85.9% | 87.0% | 47.0% |
| 20 | Kimi K3 | Max | 42 | 8/14 | 93.1% | 72.2% | 36.0% |
| 21 | Gemini 3.6 Flash | High | 40 | 7/14 | 94.1% | 59.0% | 40.0% |
| 22 | DeepSeek-V4-Pro-0813 | Max | 38 | 7/14 | 91.7% | 64.6% | 46.0% |
| 22 | DeepSeek-V4.1-Flash | not stated | 38 | 8/14 | 89.8% | 67.4% | 48.0% |
| 24 | GPT-6 Luna | Max | 38 | 9/14 | 90.5% | 79.0% | 36.0% |
| 25 | Grok 4.6 | Xhigh | 34 | 7/14 | 93.2% | 66.0% | 33.0% |
| 26 | GLM-5.3 | Max | 33 | 6/14 | 90.9% | 68.8% | 37.0% |
| 27 | Grok 4.7 | Xhigh | 32 | 7/14 | 92.7% | 53.0% | 41.0% |
| 28 | Claude Opus 4.7 | Max | 29 | 6/14 | 86.4% | 70.2% | 41.0% |
| 29 | Claude Opus 4.6 | Max | 18 | 8/14 | 88.4% | 66.0% | 21.0% |
| 30 | GLM-5.3-Flash | Max | 17 | 5/14 | 90.2% | 55.8% | 21.0% |
| 31 | Mistral Large 4 | High | 16 | 7/14 | 88.1% | 58.6% | 33.0% |
| 32 | GPT-5.6 Luna | Low | 5 | 4/14 | 82.3% | 41.4% | 17.0% |
| 33 | Gemini 3.5 Flash-Lite | High | 3 | 5/14 | 83.3% | 26.0% | 16.0% |
| 34 | GPT-5.6 Luna | None | 1 | 4/14 | 63.6% | 39.7% | 3.0% |
third-party · Models: 31 · Entries: 31 × Groups: 5 (benchmark coverage 5/14)GPQA Diamond, FrontierMath Tiers 1-3(v2), FrontierMath Tier 4(v2), OTIS Mock AIME 2024-2025, Chess Puzzles(Epoch AI)
- GPQA Diamondthird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, gpqa_diamond.csv) CC BY 4.0measured/reported by: Epoch AI (31)
- FrontierMath Tiers 1-3(v2)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, frontiermath_tiers_1_3_v2.csv) CC BY 4.0measured/reported by: Epoch AI (31)
- FrontierMath Tier 4(v2)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, frontiermath_tier_4_v2.csv) CC BY 4.0measured/reported by: Epoch AI (31)
- OTIS Mock AIME 2024-2025third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0measured/reported by: Epoch AI (31)
- Chess Puzzles(Epoch AI)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, chess_puzzles.csv) CC BY 4.0measured/reported by: Epoch AI (31)
| Rank | Model | effort | Mean percentile | Benchmarks with values | 1. GPQA Diamond | 2. FrontierMath Tiers 1-3(v2) | 3. FrontierMath Tier 4(v2) | 4. OTIS Mock AIME 2024-2025 | 5. Chess Puzzles(Epoch AI) |
|---|---|---|---|---|---|---|---|---|---|
| 1 | GPT-6 Astra | Max | 92 | 9/14 | 95.8% | 93.7% | 97.6% | 100.0% | 74.0% |
| 2 | Gemini 4 Carbon | High | 92 | 8/14 | 96.5% | 90.5% | 95.1% | 100.0% | 78.0% |
| 3 | GPT-6.1 Sol | Max | 91 | 9/14 | 95.4% | 93.7% | 100.0% | 100.0% | 68.0% |
| 4 | Claude Fable 5.2 | Max | 86 | 6/14 | 93.9% | 95.1% | 100.0% | 99.7% | 79.0% |
| 5 | Claude Sonnet 5.5 | Max | 81 | 7/14 | 95.6% | 88.8% | 80.5% | 100.0% | 59.0% |
| 6 | Gemini 4 Argon | High | 80 | 9/14 | 96.4% | 86.0% | 78.1% | 100.0% | 59.0% |
| 7 | GPT-6 Sol | Max | 79 | 7/14 | 94.3% | 89.8% | 90.0% | 100.0% | 56.0% |
| 8 | GPT-5.6 Sol | Max | 77 | 6/14 | 93.5% | 89.1% | 82.9% | 100.0% | 59.0% |
| 9 | Claude Opus 5.5 | Max | 75 | 7/14 | 90.6% | 91.2% | 95.0% | 100.0% | 67.0% |
| 10 | Kimi K3.1 Pro | Max | 63 | 6/14 | 94.2% | 78.3% | 65.9% | 100.0% | 45.0% |
| 10 | GPT-5.6 Terra | Max | 63 | 7/14 | 93.3% | 86.0% | 70.7% | 99.7% | 47.0% |
| 12 | Claude Fable 5 | Max | 56 | 7/14 | 85.9% | 87.0% | 90.2% | 99.7% | 47.0% |
| 13 | Gemini 3.7 Flash | High | 54 | 7/14 | 94.8% | 71.6% | 36.6% | 97.2% | 66.0% |
| 14 | DeepSeek-V4.1-Flash-1017 | Max | 50 | 6/14 | 93.2% | 71.2% | 46.3% | 99.4% | 46.0% |
| 15 | Claude Opus 4.8 | Max | 49 | 8/14 | 91.0% | 80.0% | 56.1% | 98.3% | 52.0% |
| 15 | Qwen3.8-Max | Xhigh | 49 | 8/14 | 92.7% | 74.7% | 46.3% | 99.4% | 46.0% |
| 17 | Gemini 3.8 Flash | High | 49 | 7/14 | 95.4% | 68.4% | 22.0% | 98.9% | 58.0% |
| 18 | GPT-5.6 Luna | Max | 46 | 7/14 | 91.6% | 82.1% | 61.0% | 98.3% | 43.0% |
| 19 | GPT-6 Luna | Max | 40 | 9/14 | 90.5% | 79.0% | 56.1% | 98.9% | 36.0% |
| 20 | Kimi K3 | Max | 36 | 8/14 | 93.1% | 72.2% | 39.0% | 97.2% | 36.0% |
| 21 | Grok 4.6 | Xhigh | 35 | 7/14 | 93.2% | 66.0% | 31.7% | 99.2% | 33.0% |
| 22 | DeepSeek-V4-Pro-0813 | Max | 34 | 7/14 | 91.7% | 64.6% | 26.8% | 98.6% | 46.0% |
| 23 | DeepSeek-V4.1-Flash | not stated | 32 | 8/14 | 89.8% | 67.4% | 26.8% | 98.3% | 48.0% |
| 24 | Gemini 3.6 Flash | High | 28 | 7/14 | 94.1% | 59.0% | 22.0% | 94.2% | 40.0% |
| 25 | GLM-5.3 | Max | 25 | 6/14 | 90.9% | 68.8% | 29.3% | 91.1% | 37.0% |
| 26 | Grok 4.7 | Xhigh | 23 | 7/14 | 92.7% | 53.0% | 17.1% | 98.1% | 41.0% |
| 26 | Claude Opus 4.7 | Max | 23 | 6/14 | 86.4% | 70.2% | 31.7% | 86.7% | 41.0% |
| 28 | Mistral Large 4 | High | 18 | 7/14 | 88.1% | 58.6% | 26.8% | 98.3% | 33.0% |
| 29 | Claude Opus 4.6 | Max | 14 | 8/14 | 88.4% | 66.0% | 26.8% | 91.1% | 21.0% |
| 30 | GLM-5.3-Flash | Max | 10 | 5/14 | 90.2% | 55.8% | 17.1% | 93.9% | 21.0% |
| 31 | Gemini 3.5 Flash-Lite | High | 0 | 5/14 | 83.3% | 26.0% | 0.0% | 71.1% | 16.0% |
third-party · Models: 31 · Entries: 33 × Groups: 4 (benchmark coverage 4/14)GPQA Diamond, FrontierMath Tiers 1-3(v2), OTIS Mock AIME 2024-2025, Chess Puzzles(Epoch AI)
- GPQA Diamondthird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, gpqa_diamond.csv) CC BY 4.0measured/reported by: Epoch AI (33)
- FrontierMath Tiers 1-3(v2)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, frontiermath_tiers_1_3_v2.csv) CC BY 4.0measured/reported by: Epoch AI (33)
- OTIS Mock AIME 2024-2025third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0measured/reported by: Epoch AI (33)
- Chess Puzzles(Epoch AI)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, chess_puzzles.csv) CC BY 4.0measured/reported by: Epoch AI (33)
| Rank | Model | effort | Mean percentile | Benchmarks with values | 1. GPQA Diamond | 2. FrontierMath Tiers 1-3(v2) | 3. OTIS Mock AIME 2024-2025 | 4. Chess Puzzles(Epoch AI) |
|---|---|---|---|---|---|---|---|---|
| 1 | Gemini 4 Carbon | High | 93 | 8/14 | 96.5% | 90.5% | 100.0% | 78.0% |
| 2 | GPT-6 Astra | Max | 93 | 9/14 | 95.8% | 93.7% | 100.0% | 74.0% |
| 3 | GPT-6.1 Sol | Max | 90 | 9/14 | 95.4% | 93.7% | 100.0% | 68.0% |
| 4 | Claude Fable 5.2 | Max | 84 | 6/14 | 93.9% | 95.1% | 99.7% | 79.0% |
| 5 | Claude Sonnet 5.5 | Max | 84 | 7/14 | 95.6% | 88.8% | 100.0% | 59.0% |
| 6 | Gemini 4 Argon | High | 83 | 9/14 | 96.4% | 86.0% | 100.0% | 59.0% |
| 7 | GPT-6 Sol | Max | 80 | 7/14 | 94.3% | 89.8% | 100.0% | 56.0% |
| 8 | GPT-5.6 Sol | Max | 78 | 6/14 | 93.5% | 89.1% | 100.0% | 59.0% |
| 9 | Claude Opus 5.5 | Max | 74 | 7/14 | 90.6% | 91.2% | 100.0% | 67.0% |
| 10 | Kimi K3.1 Pro | Max | 66 | 6/14 | 94.2% | 78.3% | 100.0% | 45.0% |
| 11 | GPT-5.6 Terra | Max | 65 | 7/14 | 93.3% | 86.0% | 99.7% | 47.0% |
| 12 | Gemini 3.8 Flash | High | 61 | 7/14 | 95.4% | 68.4% | 98.9% | 58.0% |
| 13 | Gemini 3.7 Flash | High | 60 | 7/14 | 94.8% | 71.6% | 97.2% | 66.0% |
| 14 | DeepSeek-V4.1-Flash-1017 | Max | 53 | 6/14 | 93.2% | 71.2% | 99.4% | 46.0% |
| 15 | Claude Fable 5 | Max | 53 | 7/14 | 85.9% | 87.0% | 99.7% | 47.0% |
| 15 | Qwen3.8-Max | Xhigh | 53 | 8/14 | 92.7% | 74.7% | 99.4% | 46.0% |
| 17 | Claude Opus 4.8 | Max | 51 | 8/14 | 91.0% | 80.0% | 98.3% | 52.0% |
| 18 | GPT-5.6 Luna | Max | 46 | 7/14 | 91.6% | 82.1% | 98.3% | 43.0% |
| 19 | GPT-6 Luna | Max | 41 | 9/14 | 90.5% | 79.0% | 98.9% | 36.0% |
| 19 | DeepSeek-V4-Pro-0813 | Max | 41 | 7/14 | 91.7% | 64.6% | 98.6% | 46.0% |
| 21 | Grok 4.6 | Xhigh | 39 | 7/14 | 93.2% | 66.0% | 99.2% | 33.0% |
| 22 | DeepSeek-V4.1-Flash | not stated | 39 | 8/14 | 89.8% | 67.4% | 98.3% | 48.0% |
| 23 | Kimi K3 | Max | 38 | 8/14 | 93.1% | 72.2% | 97.2% | 36.0% |
| 24 | Gemini 3.6 Flash | High | 36 | 7/14 | 94.1% | 59.0% | 94.2% | 40.0% |
| 25 | Grok 4.7 | Xhigh | 32 | 7/14 | 92.7% | 53.0% | 98.1% | 41.0% |
| 26 | GLM-5.3 | Max | 29 | 6/14 | 90.9% | 68.8% | 91.1% | 37.0% |
| 27 | Claude Opus 4.7 | Max | 25 | 6/14 | 86.4% | 70.2% | 86.7% | 41.0% |
| 28 | Mistral Large 4 | High | 22 | 7/14 | 88.1% | 58.6% | 98.3% | 33.0% |
| 29 | Claude Opus 4.6 | Max | 18 | 8/14 | 88.4% | 66.0% | 91.1% | 21.0% |
| 30 | GLM-5.3-Flash | Max | 17 | 5/14 | 90.2% | 55.8% | 93.9% | 21.0% |
| 31 | GPT-5.6 Luna | Low | 5 | 4/14 | 82.3% | 41.4% | 66.7% | 17.0% |
| 32 | Gemini 3.5 Flash-Lite | High | 4 | 5/14 | 83.3% | 26.0% | 71.1% | 16.0% |
| 33 | GPT-5.6 Luna | None | 1 | 4/14 | 63.6% | 39.7% | 40.0% | 3.0% |
third-party · Models: 30 · Entries: 30 × Groups: 3 (benchmark coverage 3/14)FrontierMath Tiers 1-3(v2), FrontierMath Tier 4(v2), SimpleQA Verified
- FrontierMath Tiers 1-3(v2)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, frontiermath_tiers_1_3_v2.csv) CC BY 4.0measured/reported by: Epoch AI (30)
- FrontierMath Tier 4(v2)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, frontiermath_tier_4_v2.csv) CC BY 4.0measured/reported by: Epoch AI (30)
- SimpleQA Verifiedthird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, simpleqa_verified.csv) CC BY 4.0measured/reported by: Epoch AI (30)
| Rank | Model | effort | Mean percentile | Benchmarks with values | 1. FrontierMath Tiers 1-3(v2) | 2. FrontierMath Tier 4(v2) | 3. SimpleQA Verified |
|---|---|---|---|---|---|---|---|
| 1 | Claude Fable 5.2 | Max | 99 | 6/14 | 95.1% | 100.0% | 94.0% |
| 2 | GPT-6.1 Sol | Max | 93 | 9/14 | 93.7% | 100.0% | 73.9% |
| 3 | GPT-6 Astra | Max | 93 | 9/14 | 93.7% | 97.6% | 75.6% |
| 4 | Gemini 4 Carbon | High | 90 | 8/14 | 90.5% | 95.1% | 77.7% |
| 5 | Claude Opus 5.5 | Max | 86 | 7/14 | 91.2% | 95.0% | 72.2% |
| 6 | Gemini 4 Argon | High | 76 | 9/14 | 86.0% | 78.1% | 76.0% |
| 6 | GPT-5.6 Sol | Max | 76 | 6/14 | 89.1% | 82.9% | 69.7% |
| 8 | GPT-6.1 Terra | Max | 75 | 6/14 | 90.9% | 92.7% | 57.3% |
| 9 | GPT-6 Sol | Max | 72 | 7/14 | 89.8% | 90.0% | 60.7% |
| 10 | GPT-5.5 | Xhigh | 64 | 6/14 | 85.3% | 72.5% | 63.0% |
| 11 | Claude Sonnet 5.5 | Max | 59 | 7/14 | 88.8% | 80.5% | 46.5% |
| 12 | Kimi K3.1 Pro | Max | 56 | 6/14 | 78.3% | 65.9% | 62.2% |
| 13 | GPT-5.6 Terra | Max | 51 | 7/14 | 86.0% | 70.7% | 43.2% |
| 13 | Claude Opus 4.8 | Max | 51 | 8/14 | 80.0% | 56.1% | 53.0% |
| 15 | Gemini 3.7 Flash | High | 46 | 7/14 | 71.6% | 36.6% | 69.2% |
| 16 | GPT-5.6 Luna | Max | 41 | 7/14 | 82.1% | 61.0% | 41.0% |
| 17 | GPT-6 Luna | Max | 39 | 9/14 | 79.0% | 56.1% | 41.4% |
| 18 | Kimi K3 | Max | 38 | 8/14 | 72.2% | 39.0% | 50.6% |
| 19 | Qwen3.8-Max | Xhigh | 37 | 8/14 | 74.7% | 46.3% | 45.8% |
| 20 | Gemini 3.8 Flash | High | 36 | 7/14 | 68.4% | 22.0% | 69.7% |
| 21 | Claude Haiku 5.5 | Max | 30 | 6/14 | 75.1% | 46.3% | 23.8% |
| 21 | DeepSeek-V4.1-Flash-1017 | Max | 30 | 6/14 | 71.2% | 46.3% | 42.5% |
| 23 | Grok 4.6 | Xhigh | 27 | 7/14 | 66.0% | 31.7% | 48.9% |
| 23 | Gemini 3.6 Flash | High | 27 | 7/14 | 59.0% | 22.0% | 66.2% |
| 25 | DeepSeek-V4-Pro-0813 | Max | 24 | 7/14 | 64.6% | 26.8% | 52.9% |
| 26 | Claude Opus 4.6 | Max | 22 | 8/14 | 66.0% | 26.8% | 47.0% |
| 27 | GLM-5.3 | Max | 20 | 6/14 | 68.8% | 29.3% | 41.0% |
| 28 | DeepSeek-V4.1-Flash | not stated | 19 | 8/14 | 67.4% | 26.8% | 42.9% |
| 29 | Grok 4.7 | Xhigh | 17 | 7/14 | 53.0% | 17.1% | 56.0% |
| 30 | Mistral Large 4 | High | 6 | 7/14 | 58.6% | 26.8% | 20.0% |
third-party · Models: 29 · Entries: 29 × Groups: 4 (benchmark coverage 4/14)GPQA Diamond, FrontierMath Tiers 1-3(v2), FrontierMath Tier 4(v2), SimpleQA Verified
- GPQA Diamondthird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, gpqa_diamond.csv) CC BY 4.0measured/reported by: Epoch AI (29)
- FrontierMath Tiers 1-3(v2)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, frontiermath_tiers_1_3_v2.csv) CC BY 4.0measured/reported by: Epoch AI (29)
- FrontierMath Tier 4(v2)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, frontiermath_tier_4_v2.csv) CC BY 4.0measured/reported by: Epoch AI (29)
- SimpleQA Verifiedthird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, simpleqa_verified.csv) CC BY 4.0measured/reported by: Epoch AI (29)
| Rank | Model | effort | Mean percentile | Benchmarks with values | 1. GPQA Diamond | 2. FrontierMath Tiers 1-3(v2) | 3. FrontierMath Tier 4(v2) | 4. SimpleQA Verified |
|---|---|---|---|---|---|---|---|---|
| 1 | GPT-6 Astra | Max | 92 | 9/14 | 95.8% | 93.7% | 97.6% | 75.6% |
| 2 | Gemini 4 Carbon | High | 92 | 8/14 | 96.5% | 90.5% | 95.1% | 77.7% |
| 3 | GPT-6.1 Sol | Max | 91 | 9/14 | 95.4% | 93.7% | 100.0% | 73.9% |
| 4 | Claude Fable 5.2 | Max | 90 | 6/14 | 93.9% | 95.1% | 100.0% | 94.0% |
| 5 | Gemini 4 Argon | High | 81 | 9/14 | 96.4% | 86.0% | 78.1% | 76.0% |
| 6 | GPT-6.1 Terra | Max | 75 | 6/14 | 94.3% | 90.9% | 92.7% | 57.3% |
| 7 | GPT-6 Sol | Max | 73 | 7/14 | 94.3% | 89.8% | 90.0% | 60.7% |
| 8 | GPT-5.6 Sol | Max | 71 | 6/14 | 93.5% | 89.1% | 82.9% | 69.7% |
| 9 | Claude Opus 5.5 | Max | 69 | 7/14 | 90.6% | 91.2% | 95.0% | 72.2% |
| 10 | Claude Sonnet 5.5 | Max | 66 | 7/14 | 95.6% | 88.8% | 80.5% | 46.5% |
| 11 | Kimi K3.1 Pro | Max | 61 | 6/14 | 94.2% | 78.3% | 65.9% | 62.2% |
| 12 | Gemini 3.7 Flash | High | 54 | 7/14 | 94.8% | 71.6% | 36.6% | 69.2% |
| 13 | GPT-5.6 Terra | Max | 52 | 7/14 | 93.3% | 86.0% | 70.7% | 43.2% |
| 14 | Gemini 3.8 Flash | High | 48 | 7/14 | 95.4% | 68.4% | 22.0% | 69.7% |
| 15 | Claude Opus 4.8 | Max | 46 | 8/14 | 91.0% | 80.0% | 56.1% | 53.0% |
| 16 | Kimi K3 | Max | 40 | 8/14 | 93.1% | 72.2% | 39.0% | 50.6% |
| 17 | GPT-5.6 Luna | Max | 39 | 7/14 | 91.6% | 82.1% | 61.0% | 41.0% |
| 18 | Qwen3.8-Max | Xhigh | 38 | 8/14 | 92.7% | 74.7% | 46.3% | 45.8% |
| 19 | Gemini 3.6 Flash | High | 36 | 7/14 | 94.1% | 59.0% | 22.0% | 66.2% |
| 20 | DeepSeek-V4.1-Flash-1017 | Max | 35 | 6/14 | 93.2% | 71.2% | 46.3% | 42.5% |
| 21 | GPT-6 Luna | Max | 33 | 9/14 | 90.5% | 79.0% | 56.1% | 41.4% |
| 22 | Grok 4.6 | Xhigh | 33 | 7/14 | 93.2% | 66.0% | 31.7% | 48.9% |
| 23 | DeepSeek-V4-Pro-0813 | Max | 26 | 7/14 | 91.7% | 64.6% | 26.8% | 52.9% |
| 24 | Claude Haiku 5.5 | Max | 25 | 6/14 | 89.7% | 75.1% | 46.3% | 23.8% |
| 25 | Grok 4.7 | Xhigh | 23 | 7/14 | 92.7% | 53.0% | 17.1% | 56.0% |
| 26 | GLM-5.3 | Max | 21 | 6/14 | 90.9% | 68.8% | 29.3% | 41.0% |
| 27 | Claude Opus 4.6 | Max | 18 | 8/14 | 88.4% | 66.0% | 26.8% | 47.0% |
| 28 | DeepSeek-V4.1-Flash | not stated | 17 | 8/14 | 89.8% | 67.4% | 26.8% | 42.9% |
| 29 | Mistral Large 4 | High | 5 | 7/14 | 88.1% | 58.6% | 26.8% | 20.0% |
third-party · Models: 29 · Entries: 29 × Groups: 4 (benchmark coverage 4/14)FrontierMath Tiers 1-3(v2), FrontierMath Tier 4(v2), SimpleQA Verified, Chess Puzzles(Epoch AI)
- FrontierMath Tiers 1-3(v2)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, frontiermath_tiers_1_3_v2.csv) CC BY 4.0measured/reported by: Epoch AI (29)
- FrontierMath Tier 4(v2)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, frontiermath_tier_4_v2.csv) CC BY 4.0measured/reported by: Epoch AI (29)
- SimpleQA Verifiedthird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, simpleqa_verified.csv) CC BY 4.0measured/reported by: Epoch AI (29)
- Chess Puzzles(Epoch AI)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, chess_puzzles.csv) CC BY 4.0measured/reported by: Epoch AI (29)
| Rank | Model | effort | Mean percentile | Benchmarks with values | 1. FrontierMath Tiers 1-3(v2) | 2. FrontierMath Tier 4(v2) | 3. SimpleQA Verified | 4. Chess Puzzles(Epoch AI) |
|---|---|---|---|---|---|---|---|---|
| 1 | Claude Fable 5.2 | Max | 100 | 6/14 | 95.1% | 100.0% | 94.0% | 79.0% |
| 2 | GPT-6 Astra | Max | 92 | 9/14 | 93.7% | 97.6% | 75.6% | 74.0% |
| 3 | Gemini 4 Carbon | High | 92 | 8/14 | 90.5% | 95.1% | 77.7% | 78.0% |
| 3 | GPT-6.1 Sol | Max | 92 | 9/14 | 93.7% | 100.0% | 73.9% | 68.0% |
| 5 | Claude Opus 5.5 | Max | 86 | 7/14 | 91.2% | 95.0% | 72.2% | 67.0% |
| 6 | Gemini 4 Argon | High | 76 | 9/14 | 86.0% | 78.1% | 76.0% | 59.0% |
| 6 | GPT-5.6 Sol | Max | 76 | 6/14 | 89.1% | 82.9% | 69.7% | 59.0% |
| 8 | GPT-6 Sol | Max | 71 | 7/14 | 89.8% | 90.0% | 60.7% | 56.0% |
| 9 | GPT-5.5 | Xhigh | 69 | 6/14 | 85.3% | 72.5% | 63.0% | 63.0% |
| 10 | Claude Sonnet 5.5 | Max | 63 | 7/14 | 88.8% | 80.5% | 46.5% | 59.0% |
| 11 | Gemini 3.7 Flash | High | 55 | 7/14 | 71.6% | 36.6% | 69.2% | 66.0% |
| 12 | Claude Opus 4.8 | Max | 53 | 8/14 | 80.0% | 56.1% | 53.0% | 52.0% |
| 13 | GPT-5.6 Terra | Max | 51 | 7/14 | 86.0% | 70.7% | 43.2% | 47.0% |
| 14 | Kimi K3.1 Pro | Max | 51 | 6/14 | 78.3% | 65.9% | 62.2% | 45.0% |
| 15 | Gemini 3.8 Flash | High | 43 | 7/14 | 68.4% | 22.0% | 69.7% | 58.0% |
| 16 | GPT-5.6 Luna | Max | 39 | 7/14 | 82.1% | 61.0% | 41.0% | 43.0% |
| 17 | Qwen3.8-Max | Xhigh | 38 | 8/14 | 74.7% | 46.3% | 45.8% | 46.0% |
| 18 | Claude Haiku 5.5 | Max | 38 | 6/14 | 75.1% | 46.3% | 23.8% | 55.0% |
| 19 | GPT-6 Luna | Max | 33 | 9/14 | 79.0% | 56.1% | 41.4% | 36.0% |
| 19 | DeepSeek-V4.1-Flash-1017 | Max | 33 | 6/14 | 71.2% | 46.3% | 42.5% | 46.0% |
| 21 | Kimi K3 | Max | 33 | 8/14 | 72.2% | 39.0% | 50.6% | 36.0% |
| 22 | DeepSeek-V4-Pro-0813 | Max | 28 | 7/14 | 64.6% | 26.8% | 52.9% | 46.0% |
| 23 | DeepSeek-V4.1-Flash | not stated | 27 | 8/14 | 67.4% | 26.8% | 42.9% | 48.0% |
| 24 | Gemini 3.6 Flash | High | 25 | 7/14 | 59.0% | 22.0% | 66.2% | 40.0% |
| 25 | Grok 4.6 | Xhigh | 22 | 7/14 | 66.0% | 31.7% | 48.9% | 33.0% |
| 26 | GLM-5.3 | Max | 20 | 6/14 | 68.8% | 29.3% | 41.0% | 37.0% |
| 27 | Grok 4.7 | Xhigh | 20 | 7/14 | 53.0% | 17.1% | 56.0% | 41.0% |
| 28 | Claude Opus 4.6 | Max | 17 | 8/14 | 66.0% | 26.8% | 47.0% | 21.0% |
| 29 | Mistral Large 4 | High | 6 | 7/14 | 58.6% | 26.8% | 20.0% | 33.0% |
third-party · Models: 28 · Entries: 28 × Groups: 5 (benchmark coverage 5/14)GPQA Diamond, FrontierMath Tiers 1-3(v2), FrontierMath Tier 4(v2), OTIS Mock AIME 2024-2025, SimpleQA Verified
- GPQA Diamondthird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, gpqa_diamond.csv) CC BY 4.0measured/reported by: Epoch AI (28)
- FrontierMath Tiers 1-3(v2)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, frontiermath_tiers_1_3_v2.csv) CC BY 4.0measured/reported by: Epoch AI (28)
- FrontierMath Tier 4(v2)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, frontiermath_tier_4_v2.csv) CC BY 4.0measured/reported by: Epoch AI (28)
- OTIS Mock AIME 2024-2025third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0measured/reported by: Epoch AI (28)
- SimpleQA Verifiedthird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, simpleqa_verified.csv) CC BY 4.0measured/reported by: Epoch AI (28)
| Rank | Model | effort | Mean percentile | Benchmarks with values | 1. GPQA Diamond | 2. FrontierMath Tiers 1-3(v2) | 3. FrontierMath Tier 4(v2) | 4. OTIS Mock AIME 2024-2025 | 5. SimpleQA Verified |
|---|---|---|---|---|---|---|---|---|---|
| 1 | GPT-6 Astra | Max | 90 | 9/14 | 95.8% | 93.7% | 97.6% | 100.0% | 75.6% |
| 2 | Gemini 4 Carbon | High | 90 | 8/14 | 96.5% | 90.5% | 95.1% | 100.0% | 77.7% |
| 3 | GPT-6.1 Sol | Max | 89 | 9/14 | 95.4% | 93.7% | 100.0% | 100.0% | 73.9% |
| 4 | Claude Fable 5.2 | Max | 84 | 6/14 | 93.9% | 95.1% | 100.0% | 99.7% | 94.0% |
| 5 | Gemini 4 Argon | High | 81 | 9/14 | 96.4% | 86.0% | 78.1% | 100.0% | 76.0% |
| 6 | GPT-6.1 Terra | Max | 76 | 6/14 | 94.3% | 90.9% | 92.7% | 100.0% | 57.3% |
| 7 | GPT-6 Sol | Max | 74 | 7/14 | 94.3% | 89.8% | 90.0% | 100.0% | 60.7% |
| 8 | GPT-5.6 Sol | Max | 73 | 6/14 | 93.5% | 89.1% | 82.9% | 100.0% | 69.7% |
| 9 | Claude Opus 5.5 | Max | 71 | 7/14 | 90.6% | 91.2% | 95.0% | 100.0% | 72.2% |
| 10 | Claude Sonnet 5.5 | Max | 69 | 7/14 | 95.6% | 88.8% | 80.5% | 100.0% | 46.5% |
| 11 | Kimi K3.1 Pro | Max | 64 | 6/14 | 94.2% | 78.3% | 65.9% | 100.0% | 62.2% |
| 12 | GPT-5.6 Terra | Max | 53 | 7/14 | 93.3% | 86.0% | 70.7% | 99.7% | 43.2% |
| 13 | Gemini 3.8 Flash | High | 47 | 7/14 | 95.4% | 68.4% | 22.0% | 98.9% | 69.7% |
| 14 | Gemini 3.7 Flash | High | 46 | 7/14 | 94.8% | 71.6% | 36.6% | 97.2% | 69.2% |
| 15 | Claude Opus 4.8 | Max | 41 | 8/14 | 91.0% | 80.0% | 56.1% | 98.3% | 53.0% |
| 16 | Qwen3.8-Max | Xhigh | 40 | 8/14 | 92.7% | 74.7% | 46.3% | 99.4% | 45.8% |
| 17 | DeepSeek-V4.1-Flash-1017 | Max | 38 | 6/14 | 93.2% | 71.2% | 46.3% | 99.4% | 42.5% |
| 18 | Grok 4.6 | Xhigh | 36 | 7/14 | 93.2% | 66.0% | 31.7% | 99.2% | 48.9% |
| 19 | GPT-5.6 Luna | Max | 35 | 7/14 | 91.6% | 82.1% | 61.0% | 98.3% | 41.0% |
| 20 | Kimi K3 | Max | 34 | 8/14 | 93.1% | 72.2% | 39.0% | 97.2% | 50.6% |
| 21 | GPT-6 Luna | Max | 33 | 9/14 | 90.5% | 79.0% | 56.1% | 98.9% | 41.4% |
| 22 | Gemini 3.6 Flash | High | 30 | 7/14 | 94.1% | 59.0% | 22.0% | 94.2% | 66.2% |
| 23 | DeepSeek-V4-Pro-0813 | Max | 28 | 7/14 | 91.7% | 64.6% | 26.8% | 98.6% | 52.9% |
| 24 | Grok 4.7 | Xhigh | 21 | 7/14 | 92.7% | 53.0% | 17.1% | 98.1% | 56.0% |
| 25 | DeepSeek-V4.1-Flash | not stated | 19 | 8/14 | 89.8% | 67.4% | 26.8% | 98.3% | 42.9% |
| 26 | GLM-5.3 | Max | 16 | 6/14 | 90.9% | 68.8% | 29.3% | 91.1% | 41.0% |
| 27 | Claude Opus 4.6 | Max | 14 | 8/14 | 88.4% | 66.0% | 26.8% | 91.1% | 47.0% |
| 28 | Mistral Large 4 | High | 10 | 7/14 | 88.1% | 58.6% | 26.8% | 98.3% | 20.0% |
third-party · Models: 28 · Entries: 28 × Groups: 5 (benchmark coverage 5/14)GPQA Diamond, FrontierMath Tiers 1-3(v2), FrontierMath Tier 4(v2), SimpleQA Verified, Chess Puzzles(Epoch AI)
- GPQA Diamondthird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, gpqa_diamond.csv) CC BY 4.0measured/reported by: Epoch AI (28)
- FrontierMath Tiers 1-3(v2)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, frontiermath_tiers_1_3_v2.csv) CC BY 4.0measured/reported by: Epoch AI (28)
- FrontierMath Tier 4(v2)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, frontiermath_tier_4_v2.csv) CC BY 4.0measured/reported by: Epoch AI (28)
- SimpleQA Verifiedthird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, simpleqa_verified.csv) CC BY 4.0measured/reported by: Epoch AI (28)
- Chess Puzzles(Epoch AI)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, chess_puzzles.csv) CC BY 4.0measured/reported by: Epoch AI (28)
| Rank | Model | effort | Mean percentile | Benchmarks with values | 1. GPQA Diamond | 2. FrontierMath Tiers 1-3(v2) | 3. FrontierMath Tier 4(v2) | 4. SimpleQA Verified | 5. Chess Puzzles(Epoch AI) |
|---|---|---|---|---|---|---|---|---|---|
| 1 | Gemini 4 Carbon | High | 93 | 8/14 | 96.5% | 90.5% | 95.1% | 77.7% | 78.0% |
| 2 | Claude Fable 5.2 | Max | 92 | 6/14 | 93.9% | 95.1% | 100.0% | 94.0% | 79.0% |
| 2 | GPT-6 Astra | Max | 92 | 9/14 | 95.8% | 93.7% | 97.6% | 75.6% | 74.0% |
| 4 | GPT-6.1 Sol | Max | 90 | 9/14 | 95.4% | 93.7% | 100.0% | 73.9% | 68.0% |
| 5 | Gemini 4 Argon | High | 80 | 9/14 | 96.4% | 86.0% | 78.1% | 76.0% | 59.0% |
| 6 | GPT-5.6 Sol | Max | 73 | 6/14 | 93.5% | 89.1% | 82.9% | 69.7% | 59.0% |
| 7 | Claude Opus 5.5 | Max | 72 | 7/14 | 90.6% | 91.2% | 95.0% | 72.2% | 67.0% |
| 7 | GPT-6 Sol | Max | 72 | 7/14 | 94.3% | 89.8% | 90.0% | 60.7% | 56.0% |
| 9 | Claude Sonnet 5.5 | Max | 69 | 7/14 | 95.6% | 88.8% | 80.5% | 46.5% | 59.0% |
| 10 | Gemini 3.7 Flash | High | 60 | 7/14 | 94.8% | 71.6% | 36.6% | 69.2% | 66.0% |
| 11 | Kimi K3.1 Pro | Max | 56 | 6/14 | 94.2% | 78.3% | 65.9% | 62.2% | 45.0% |
| 12 | GPT-5.6 Terra | Max | 53 | 7/14 | 93.3% | 86.0% | 70.7% | 43.2% | 47.0% |
| 13 | Gemini 3.8 Flash | High | 51 | 7/14 | 95.4% | 68.4% | 22.0% | 69.7% | 58.0% |
| 14 | Claude Opus 4.8 | Max | 49 | 8/14 | 91.0% | 80.0% | 56.1% | 53.0% | 52.0% |
| 15 | Qwen3.8-Max | Xhigh | 39 | 8/14 | 92.7% | 74.7% | 46.3% | 45.8% | 46.0% |
| 16 | GPT-5.6 Luna | Max | 38 | 7/14 | 91.6% | 82.1% | 61.0% | 41.0% | 43.0% |
| 17 | DeepSeek-V4.1-Flash-1017 | Max | 37 | 6/14 | 93.2% | 71.2% | 46.3% | 42.5% | 46.0% |
| 18 | Kimi K3 | Max | 36 | 8/14 | 93.1% | 72.2% | 39.0% | 50.6% | 36.0% |
| 19 | Gemini 3.6 Flash | High | 34 | 7/14 | 94.1% | 59.0% | 22.0% | 66.2% | 40.0% |
| 20 | Claude Haiku 5.5 | Max | 33 | 6/14 | 89.7% | 75.1% | 46.3% | 23.8% | 55.0% |
| 21 | GPT-6 Luna | Max | 30 | 9/14 | 90.5% | 79.0% | 56.1% | 41.4% | 36.0% |
| 22 | DeepSeek-V4-Pro-0813 | Max | 30 | 7/14 | 91.7% | 64.6% | 26.8% | 52.9% | 46.0% |
| 23 | Grok 4.6 | Xhigh | 29 | 7/14 | 93.2% | 66.0% | 31.7% | 48.9% | 33.0% |
| 24 | DeepSeek-V4.1-Flash | not stated | 25 | 8/14 | 89.8% | 67.4% | 26.8% | 42.9% | 48.0% |
| 25 | Grok 4.7 | Xhigh | 24 | 7/14 | 92.7% | 53.0% | 17.1% | 56.0% | 41.0% |
| 26 | GLM-5.3 | Max | 21 | 6/14 | 90.9% | 68.8% | 29.3% | 41.0% | 37.0% |
| 27 | Claude Opus 4.6 | Max | 15 | 8/14 | 88.4% | 66.0% | 26.8% | 47.0% | 21.0% |
| 28 | Mistral Large 4 | High | 5 | 7/14 | 88.1% | 58.6% | 26.8% | 20.0% | 33.0% |
third-party · Models: 27 · Entries: 27 × Groups: 6 (benchmark coverage 6/14)GPQA Diamond, FrontierMath Tiers 1-3(v2), FrontierMath Tier 4(v2), OTIS Mock AIME 2024-2025, SimpleQA Verified, Chess Puzzles(Epoch AI)
- GPQA Diamondthird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, gpqa_diamond.csv) CC BY 4.0measured/reported by: Epoch AI (27)
- FrontierMath Tiers 1-3(v2)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, frontiermath_tiers_1_3_v2.csv) CC BY 4.0measured/reported by: Epoch AI (27)
- FrontierMath Tier 4(v2)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, frontiermath_tier_4_v2.csv) CC BY 4.0measured/reported by: Epoch AI (27)
- OTIS Mock AIME 2024-2025third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0measured/reported by: Epoch AI (27)
- SimpleQA Verifiedthird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, simpleqa_verified.csv) CC BY 4.0measured/reported by: Epoch AI (27)
- Chess Puzzles(Epoch AI)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, chess_puzzles.csv) CC BY 4.0measured/reported by: Epoch AI (27)
| Rank | Model | effort | Mean percentile | Benchmarks with values | 1. GPQA Diamond | 2. FrontierMath Tiers 1-3(v2) | 3. FrontierMath Tier 4(v2) | 4. OTIS Mock AIME 2024-2025 | 5. SimpleQA Verified | 6. Chess Puzzles(Epoch AI) |
|---|---|---|---|---|---|---|---|---|---|---|
| 1 | Gemini 4 Carbon | High | 92 | 8/14 | 96.5% | 90.5% | 95.1% | 100.0% | 77.7% | 78.0% |
| 2 | GPT-6 Astra | Max | 91 | 9/14 | 95.8% | 93.7% | 97.6% | 100.0% | 75.6% | 74.0% |
| 3 | GPT-6.1 Sol | Max | 89 | 9/14 | 95.4% | 93.7% | 100.0% | 100.0% | 73.9% | 68.0% |
| 4 | Claude Fable 5.2 | Max | 87 | 6/14 | 93.9% | 95.1% | 100.0% | 99.7% | 94.0% | 79.0% |
| 5 | Gemini 4 Argon | High | 80 | 9/14 | 96.4% | 86.0% | 78.1% | 100.0% | 76.0% | 59.0% |
| 6 | GPT-5.6 Sol | Max | 74 | 6/14 | 93.5% | 89.1% | 82.9% | 100.0% | 69.7% | 59.0% |
| 7 | Claude Opus 5.5 | Max | 73 | 7/14 | 90.6% | 91.2% | 95.0% | 100.0% | 72.2% | 67.0% |
| 7 | GPT-6 Sol | Max | 73 | 7/14 | 94.3% | 89.8% | 90.0% | 100.0% | 60.7% | 56.0% |
| 9 | Claude Sonnet 5.5 | Max | 71 | 7/14 | 95.6% | 88.8% | 80.5% | 100.0% | 46.5% | 59.0% |
| 10 | Kimi K3.1 Pro | Max | 60 | 6/14 | 94.2% | 78.3% | 65.9% | 100.0% | 62.2% | 45.0% |
| 11 | GPT-5.6 Terra | Max | 54 | 7/14 | 93.3% | 86.0% | 70.7% | 99.7% | 43.2% | 47.0% |
| 12 | Gemini 3.7 Flash | High | 52 | 7/14 | 94.8% | 71.6% | 36.6% | 97.2% | 69.2% | 66.0% |
| 13 | Gemini 3.8 Flash | High | 50 | 7/14 | 95.4% | 68.4% | 22.0% | 98.9% | 69.7% | 58.0% |
| 14 | Claude Opus 4.8 | Max | 45 | 8/14 | 91.0% | 80.0% | 56.1% | 98.3% | 53.0% | 52.0% |
| 15 | Qwen3.8-Max | Xhigh | 42 | 8/14 | 92.7% | 74.7% | 46.3% | 99.4% | 45.8% | 46.0% |
| 16 | DeepSeek-V4.1-Flash-1017 | Max | 40 | 6/14 | 93.2% | 71.2% | 46.3% | 99.4% | 42.5% | 46.0% |
| 17 | GPT-5.6 Luna | Max | 35 | 7/14 | 91.6% | 82.1% | 61.0% | 98.3% | 41.0% | 43.0% |
| 18 | Kimi K3 | Max | 32 | 8/14 | 93.1% | 72.2% | 39.0% | 97.2% | 50.6% | 36.0% |
| 19 | Grok 4.6 | Xhigh | 32 | 7/14 | 93.2% | 66.0% | 31.7% | 99.2% | 48.9% | 33.0% |
| 20 | GPT-6 Luna | Max | 31 | 9/14 | 90.5% | 79.0% | 56.1% | 98.9% | 41.4% | 36.0% |
| 20 | DeepSeek-V4-Pro-0813 | Max | 31 | 7/14 | 91.7% | 64.6% | 26.8% | 98.6% | 52.9% | 46.0% |
| 22 | Gemini 3.6 Flash | High | 29 | 7/14 | 94.1% | 59.0% | 22.0% | 94.2% | 66.2% | 40.0% |
| 23 | DeepSeek-V4.1-Flash | not stated | 25 | 8/14 | 89.8% | 67.4% | 26.8% | 98.3% | 42.9% | 48.0% |
| 24 | Grok 4.7 | Xhigh | 23 | 7/14 | 92.7% | 53.0% | 17.1% | 98.1% | 56.0% | 41.0% |
| 25 | GLM-5.3 | Max | 17 | 6/14 | 90.9% | 68.8% | 29.3% | 91.1% | 41.0% | 37.0% |
| 26 | Claude Opus 4.6 | Max | 13 | 8/14 | 88.4% | 66.0% | 26.8% | 91.1% | 47.0% | 21.0% |
| 27 | Mistral Large 4 | High | 9 | 7/14 | 88.1% | 58.6% | 26.8% | 98.3% | 20.0% | 33.0% |
third-party · Models: 25 · Entries: 25 × Groups: 3 (benchmark coverage 3/14)FrontierMath Tiers 1-3(v2), FrontierMath Tier 4(v2), MirrorCode
- FrontierMath Tiers 1-3(v2)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, frontiermath_tiers_1_3_v2.csv) CC BY 4.0measured/reported by: Epoch AI (25)
- FrontierMath Tier 4(v2)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, frontiermath_tier_4_v2.csv) CC BY 4.0measured/reported by: Epoch AI (25)
- MirrorCodethird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, mirrorcode.csv) CC BY 4.0measured/reported by: Epoch AI (25)
| Rank | Model | effort | Mean percentile | Benchmarks with values | 1. FrontierMath Tiers 1-3(v2) | 2. FrontierMath Tier 4(v2) | 3. MirrorCode |
|---|---|---|---|---|---|---|---|
| 1 | Claude Opus 5.5 | Max | 93 | 7/14 | 91.2% | 95.0% | 95.6% |
| 2 | GPT-6.1 Sol | Max | 90 | 9/14 | 93.7% | 100.0% | 61.1% |
| 3 | GPT-6.1 Terra | Max | 88 | 6/14 | 90.9% | 92.7% | 76.7% |
| 4 | GPT-6 Astra | Max | 87 | 9/14 | 93.7% | 97.6% | 60.0% |
| 5 | Claude Sonnet 5.5 | Max | 85 | 7/14 | 88.8% | 80.5% | 97.8% |
| 6 | Claude Fable 5 | Max | 81 | 7/14 | 87.0% | 90.2% | 70.0% |
| 7 | GPT-6 Sol | Max | 69 | 7/14 | 89.8% | 90.0% | 48.9% |
| 8 | Gemini 4 Argon | High | 67 | 9/14 | 86.0% | 78.1% | 58.9% |
| 9 | Claude Opus 4.8 | Max | 62 | 8/14 | 80.0% | 56.1% | 65.6% |
| 10 | Claude Haiku 5.5 | Max | 60 | 6/14 | 75.1% | 46.3% | 78.9% |
| 11 | GPT-5.6 Terra | Max | 56 | 7/14 | 86.0% | 70.7% | 41.1% |
| 12 | Qwen3.8-Max | Xhigh | 45 | 8/14 | 74.7% | 46.3% | 52.2% |
| 12 | GPT-5.5 | Xhigh | 45 | 6/14 | 85.3% | 72.5% | 23.3% |
| 14 | Kimi K3 | Max | 44 | 8/14 | 72.2% | 39.0% | 54.4% |
| 15 | DeepSeek-V4.1-Flash | not stated | 40 | 8/14 | 67.4% | 26.8% | 64.4% |
| 16 | GPT-5.6 Luna | Max | 39 | 7/14 | 82.1% | 61.0% | 21.1% |
| 17 | GPT-6 Luna | Max | 38 | 9/14 | 79.0% | 56.1% | 26.7% |
| 18 | Gemini 3.7 Flash | High | 35 | 7/14 | 71.6% | 36.6% | 41.1% |
| 19 | Grok 4.6 | Xhigh | 28 | 7/14 | 66.0% | 31.7% | 41.1% |
| 20 | Gemini 3.8 Flash | High | 22 | 7/14 | 68.4% | 22.0% | 38.9% |
| 21 | Claude Opus 4.6 | Max | 20 | 8/14 | 66.0% | 26.8% | 33.3% |
| 22 | Grok 4.7 | Xhigh | 19 | 7/14 | 53.0% | 17.1% | 54.4% |
| 23 | DeepSeek-V4-Pro-0813 | Max | 18 | 7/14 | 64.6% | 26.8% | 33.3% |
| 24 | Mistral Large 4 | High | 13 | 7/14 | 58.6% | 26.8% | 27.8% |
| 25 | Gemini 3.6 Flash | High | 7 | 7/14 | 59.0% | 22.0% | 23.3% |
third-party · Models: 24 · Entries: 24 × Groups: 4 (benchmark coverage 4/14)GPQA Diamond, FrontierMath Tiers 1-3(v2), FrontierMath Tier 4(v2), MirrorCode
- GPQA Diamondthird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, gpqa_diamond.csv) CC BY 4.0measured/reported by: Epoch AI (24)
- FrontierMath Tiers 1-3(v2)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, frontiermath_tiers_1_3_v2.csv) CC BY 4.0measured/reported by: Epoch AI (24)
- FrontierMath Tier 4(v2)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, frontiermath_tier_4_v2.csv) CC BY 4.0measured/reported by: Epoch AI (24)
- MirrorCodethird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, mirrorcode.csv) CC BY 4.0measured/reported by: Epoch AI (24)
| Rank | Model | effort | Mean percentile | Benchmarks with values | 1. GPQA Diamond | 2. FrontierMath Tiers 1-3(v2) | 3. FrontierMath Tier 4(v2) | 4. MirrorCode |
|---|---|---|---|---|---|---|---|---|
| 1 | GPT-6 Astra | Max | 89 | 9/14 | 95.8% | 93.7% | 97.6% | 60.0% |
| 2 | GPT-6.1 Sol | Max | 88 | 9/14 | 95.4% | 93.7% | 100.0% | 61.1% |
| 3 | Claude Sonnet 5.5 | Max | 86 | 7/14 | 95.6% | 88.8% | 80.5% | 97.8% |
| 4 | GPT-6.1 Terra | Max | 83 | 6/14 | 94.3% | 90.9% | 92.7% | 76.7% |
| 5 | Claude Opus 5.5 | Max | 76 | 7/14 | 90.6% | 91.2% | 95.0% | 95.6% |
| 6 | Gemini 4 Argon | High | 74 | 9/14 | 96.4% | 86.0% | 78.1% | 58.9% |
| 7 | GPT-6 Sol | Max | 69 | 7/14 | 94.3% | 89.8% | 90.0% | 48.9% |
| 8 | Claude Fable 5 | Max | 60 | 7/14 | 85.9% | 87.0% | 90.2% | 70.0% |
| 9 | GPT-5.6 Terra | Max | 57 | 7/14 | 93.3% | 86.0% | 70.7% | 41.1% |
| 10 | Claude Opus 4.8 | Max | 55 | 8/14 | 91.0% | 80.0% | 56.1% | 65.6% |
| 11 | Claude Haiku 5.5 | Max | 49 | 6/14 | 89.7% | 75.1% | 46.3% | 78.9% |
| 12 | Kimi K3 | Max | 46 | 8/14 | 93.1% | 72.2% | 39.0% | 54.4% |
| 13 | Gemini 3.7 Flash | High | 46 | 7/14 | 94.8% | 71.6% | 36.6% | 41.1% |
| 14 | Qwen3.8-Max | Xhigh | 45 | 8/14 | 92.7% | 74.7% | 46.3% | 52.2% |
| 15 | GPT-5.6 Luna | Max | 39 | 7/14 | 91.6% | 82.1% | 61.0% | 21.1% |
| 16 | Gemini 3.8 Flash | High | 37 | 7/14 | 95.4% | 68.4% | 22.0% | 38.9% |
| 17 | Grok 4.6 | Xhigh | 35 | 7/14 | 93.2% | 66.0% | 31.7% | 41.1% |
| 18 | GPT-6 Luna | Max | 34 | 9/14 | 90.5% | 79.0% | 56.1% | 26.7% |
| 18 | DeepSeek-V4.1-Flash | not stated | 34 | 8/14 | 89.8% | 67.4% | 26.8% | 64.4% |
| 20 | Grok 4.7 | Xhigh | 26 | 7/14 | 92.7% | 53.0% | 17.1% | 54.4% |
| 21 | DeepSeek-V4-Pro-0813 | Max | 23 | 7/14 | 91.7% | 64.6% | 26.8% | 33.3% |
| 22 | Gemini 3.6 Flash | High | 21 | 7/14 | 94.1% | 59.0% | 22.0% | 23.3% |
| 23 | Claude Opus 4.6 | Max | 17 | 8/14 | 88.4% | 66.0% | 26.8% | 33.3% |
| 24 | Mistral Large 4 | High | 10 | 7/14 | 88.1% | 58.6% | 26.8% | 27.8% |
third-party · Models: 24 · Entries: 24 × Groups: 4 (benchmark coverage 4/14)FrontierMath Tiers 1-3(v2), FrontierMath Tier 4(v2), SimpleQA Verified, MirrorCode
- FrontierMath Tiers 1-3(v2)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, frontiermath_tiers_1_3_v2.csv) CC BY 4.0measured/reported by: Epoch AI (24)
- FrontierMath Tier 4(v2)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, frontiermath_tier_4_v2.csv) CC BY 4.0measured/reported by: Epoch AI (24)
- SimpleQA Verifiedthird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, simpleqa_verified.csv) CC BY 4.0measured/reported by: Epoch AI (24)
- MirrorCodethird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, mirrorcode.csv) CC BY 4.0measured/reported by: Epoch AI (24)
| Rank | Model | effort | Mean percentile | Benchmarks with values | 1. FrontierMath Tiers 1-3(v2) | 2. FrontierMath Tier 4(v2) | 3. SimpleQA Verified | 4. MirrorCode |
|---|---|---|---|---|---|---|---|---|
| 1 | Claude Opus 5.5 | Max | 91 | 7/14 | 91.2% | 95.0% | 72.2% | 95.6% |
| 2 | GPT-6.1 Sol | Max | 91 | 9/14 | 93.7% | 100.0% | 73.9% | 61.1% |
| 3 | GPT-6 Astra | Max | 90 | 9/14 | 93.7% | 97.6% | 75.6% | 60.0% |
| 4 | GPT-6.1 Terra | Max | 80 | 6/14 | 90.9% | 92.7% | 57.3% | 76.7% |
| 5 | Gemini 4 Argon | High | 78 | 9/14 | 86.0% | 78.1% | 76.0% | 58.9% |
| 6 | Claude Sonnet 5.5 | Max | 72 | 7/14 | 88.8% | 80.5% | 46.5% | 97.8% |
| 7 | GPT-6 Sol | Max | 70 | 7/14 | 89.8% | 90.0% | 60.7% | 48.9% |
| 8 | Claude Opus 4.8 | Max | 61 | 8/14 | 80.0% | 56.1% | 53.0% | 65.6% |
| 9 | GPT-5.5 | Xhigh | 53 | 6/14 | 85.3% | 72.5% | 63.0% | 23.3% |
| 10 | GPT-5.6 Terra | Max | 49 | 7/14 | 86.0% | 70.7% | 43.2% | 41.1% |
| 11 | Claude Haiku 5.5 | Max | 47 | 6/14 | 75.1% | 46.3% | 23.8% | 78.9% |
| 12 | Gemini 3.7 Flash | High | 47 | 7/14 | 71.6% | 36.6% | 69.2% | 41.1% |
| 13 | Kimi K3 | Max | 45 | 8/14 | 72.2% | 39.0% | 50.6% | 54.4% |
| 14 | Qwen3.8-Max | Xhigh | 42 | 8/14 | 74.7% | 46.3% | 45.8% | 52.2% |
| 15 | Gemini 3.8 Flash | High | 38 | 7/14 | 68.4% | 22.0% | 69.7% | 38.9% |
| 16 | DeepSeek-V4.1-Flash | not stated | 35 | 8/14 | 67.4% | 26.8% | 42.9% | 64.4% |
| 17 | GPT-6 Luna | Max | 33 | 9/14 | 79.0% | 56.1% | 41.4% | 26.7% |
| 18 | GPT-5.6 Luna | Max | 33 | 7/14 | 82.1% | 61.0% | 41.0% | 21.1% |
| 19 | Grok 4.6 | Xhigh | 32 | 7/14 | 66.0% | 31.7% | 48.9% | 41.1% |
| 20 | Grok 4.7 | Xhigh | 29 | 7/14 | 53.0% | 17.1% | 56.0% | 54.4% |
| 21 | DeepSeek-V4-Pro-0813 | Max | 26 | 7/14 | 64.6% | 26.8% | 52.9% | 33.3% |
| 22 | Claude Opus 4.6 | Max | 24 | 8/14 | 66.0% | 26.8% | 47.0% | 33.3% |
| 23 | Gemini 3.6 Flash | High | 24 | 7/14 | 59.0% | 22.0% | 66.2% | 23.3% |
| 24 | Mistral Large 4 | High | 10 | 7/14 | 58.6% | 26.8% | 20.0% | 27.8% |
third-party · Models: 24 · Entries: 24 × Groups: 4 (benchmark coverage 4/14)FrontierMath Tiers 1-3(v2), FrontierMath Tier 4(v2), Chess Puzzles(Epoch AI), MirrorCode
- FrontierMath Tiers 1-3(v2)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, frontiermath_tiers_1_3_v2.csv) CC BY 4.0measured/reported by: Epoch AI (24)
- FrontierMath Tier 4(v2)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, frontiermath_tier_4_v2.csv) CC BY 4.0measured/reported by: Epoch AI (24)
- Chess Puzzles(Epoch AI)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, chess_puzzles.csv) CC BY 4.0measured/reported by: Epoch AI (24)
- MirrorCodethird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, mirrorcode.csv) CC BY 4.0measured/reported by: Epoch AI (24)
| Rank | Model | effort | Mean percentile | Benchmarks with values | 1. FrontierMath Tiers 1-3(v2) | 2. FrontierMath Tier 4(v2) | 3. Chess Puzzles(Epoch AI) | 4. MirrorCode |
|---|---|---|---|---|---|---|---|---|
| 1 | Claude Opus 5.5 | Max | 92 | 7/14 | 91.2% | 95.0% | 67.0% | 95.6% |
| 2 | GPT-6.1 Sol | Max | 92 | 9/14 | 93.7% | 100.0% | 68.0% | 61.1% |
| 3 | GPT-6 Astra | Max | 91 | 9/14 | 93.7% | 97.6% | 74.0% | 60.0% |
| 4 | Claude Sonnet 5.5 | Max | 84 | 7/14 | 88.8% | 80.5% | 59.0% | 97.8% |
| 5 | Claude Fable 5 | Max | 74 | 7/14 | 87.0% | 90.2% | 47.0% | 70.0% |
| 6 | Gemini 4 Argon | High | 72 | 9/14 | 86.0% | 78.1% | 59.0% | 58.9% |
| 7 | GPT-6 Sol | Max | 71 | 7/14 | 89.8% | 90.0% | 56.0% | 48.9% |
| 8 | Claude Opus 4.8 | Max | 63 | 8/14 | 80.0% | 56.1% | 52.0% | 65.6% |
| 9 | Claude Haiku 5.5 | Max | 61 | 6/14 | 75.1% | 46.3% | 55.0% | 78.9% |
| 10 | GPT-5.5 | Xhigh | 56 | 6/14 | 85.3% | 72.5% | 63.0% | 23.3% |
| 11 | GPT-5.6 Terra | Max | 55 | 7/14 | 86.0% | 70.7% | 47.0% | 41.1% |
| 12 | Gemini 3.7 Flash | High | 49 | 7/14 | 71.6% | 36.6% | 66.0% | 41.1% |
| 13 | Qwen3.8-Max | Xhigh | 45 | 8/14 | 74.7% | 46.3% | 46.0% | 52.2% |
| 14 | DeepSeek-V4.1-Flash | not stated | 44 | 8/14 | 67.4% | 26.8% | 48.0% | 64.4% |
| 15 | GPT-5.6 Luna | Max | 38 | 7/14 | 82.1% | 61.0% | 43.0% | 21.1% |
| 15 | Kimi K3 | Max | 38 | 8/14 | 72.2% | 39.0% | 36.0% | 54.4% |
| 17 | Gemini 3.8 Flash | High | 34 | 7/14 | 68.4% | 22.0% | 58.0% | 38.9% |
| 18 | GPT-6 Luna | Max | 34 | 9/14 | 79.0% | 56.1% | 36.0% | 26.7% |
| 19 | Grok 4.6 | Xhigh | 24 | 7/14 | 66.0% | 31.7% | 33.0% | 41.1% |
| 20 | DeepSeek-V4-Pro-0813 | Max | 23 | 7/14 | 64.6% | 26.8% | 46.0% | 33.3% |
| 21 | Grok 4.7 | Xhigh | 21 | 7/14 | 53.0% | 17.1% | 41.0% | 54.4% |
| 22 | Claude Opus 4.6 | Max | 16 | 8/14 | 66.0% | 26.8% | 21.0% | 33.3% |
| 23 | Mistral Large 4 | High | 12 | 7/14 | 58.6% | 26.8% | 33.0% | 27.8% |
| 24 | Gemini 3.6 Flash | High | 11 | 7/14 | 59.0% | 22.0% | 40.0% | 23.3% |
third-party · Models: 24 · Entries: 25 × Groups: 3 (benchmark coverage 3/14)GPQA Diamond, OTIS Mock AIME 2024-2025, MirrorCode
- GPQA Diamondthird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, gpqa_diamond.csv) CC BY 4.0measured/reported by: Epoch AI (25)
- OTIS Mock AIME 2024-2025third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0measured/reported by: Epoch AI (25)
- MirrorCodethird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, mirrorcode.csv) CC BY 4.0measured/reported by: Epoch AI (25)
| Rank | Model | effort | Mean percentile | Benchmarks with values | 1. GPQA Diamond | 2. OTIS Mock AIME 2024-2025 | 3. MirrorCode |
|---|---|---|---|---|---|---|---|
| 1 | Claude Sonnet 5.5 | Max | 92 | 7/14 | 95.6% | 100.0% | 97.8% |
| 2 | Gemini 4 Argon | High | 83 | 9/14 | 96.4% | 100.0% | 58.9% |
| 2 | GPT-6 Astra | Max | 83 | 9/14 | 95.8% | 100.0% | 60.0% |
| 4 | GPT-6.1 Terra | Max | 82 | 6/14 | 94.3% | 100.0% | 76.7% |
| 5 | GPT-6.1 Sol | Max | 81 | 9/14 | 95.4% | 100.0% | 61.1% |
| 6 | Claude Opus 5.5 | Max | 69 | 7/14 | 90.6% | 100.0% | 95.6% |
| 7 | GPT-6 Sol | Max | 67 | 7/14 | 94.3% | 100.0% | 48.9% |
| 8 | Gemini 3.8 Flash | High | 54 | 7/14 | 95.4% | 98.9% | 38.9% |
| 9 | Claude Fable 5 | High | 53 | 4/14 | 83.3% | 100.0% | 63.9% |
| 9 | GPT-5.6 Terra | Max | 53 | 7/14 | 93.3% | 99.7% | 41.1% |
| 11 | Claude Fable 5 | Max | 52 | 7/14 | 85.9% | 99.7% | 70.0% |
| 12 | Qwen3.8-Max | Xhigh | 50 | 8/14 | 92.7% | 99.4% | 52.2% |
| 13 | Grok 4.6 | Xhigh | 49 | 7/14 | 93.2% | 99.2% | 41.1% |
| 14 | Claude Opus 4.8 | Max | 48 | 8/14 | 91.0% | 98.3% | 65.6% |
| 15 | Gemini 3.7 Flash | High | 44 | 7/14 | 94.8% | 97.2% | 41.1% |
| 16 | DeepSeek-V4.1-Flash | not stated | 42 | 8/14 | 89.8% | 98.3% | 64.4% |
| 17 | Grok 4.7 | Xhigh | 41 | 7/14 | 92.7% | 98.1% | 54.4% |
| 18 | Kimi K3 | Max | 40 | 8/14 | 93.1% | 97.2% | 54.4% |
| 19 | DeepSeek-V4-Pro-0813 | Max | 34 | 7/14 | 91.7% | 98.6% | 33.3% |
| 20 | Gemini 3.1 Pro Preview | High | 28 | 6/14 | 94.4% | 95.6% | 8.9% |
| 21 | GPT-6 Luna | Max | 27 | 9/14 | 90.5% | 98.9% | 26.7% |
| 22 | Gemini 3.6 Flash | High | 25 | 7/14 | 94.1% | 94.2% | 23.3% |
| 23 | GPT-5.6 Luna | Max | 23 | 7/14 | 91.6% | 98.3% | 21.1% |
| 24 | Mistral Large 4 | High | 19 | 7/14 | 88.1% | 98.3% | 27.8% |
| 25 | Claude Opus 4.6 | Max | 12 | 8/14 | 88.4% | 91.1% | 33.3% |
third-party · Models: 24 · Entries: 25 × Groups: 3 (benchmark coverage 3/14)GPQA Diamond, Chess Puzzles(Epoch AI), MirrorCode
- GPQA Diamondthird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, gpqa_diamond.csv) CC BY 4.0measured/reported by: Epoch AI (25)
- Chess Puzzles(Epoch AI)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, chess_puzzles.csv) CC BY 4.0measured/reported by: Epoch AI (25)
- MirrorCodethird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, mirrorcode.csv) CC BY 4.0measured/reported by: Epoch AI (25)
| Rank | Model | effort | Mean percentile | Benchmarks with values | 1. GPQA Diamond | 2. Chess Puzzles(Epoch AI) | 3. MirrorCode |
|---|---|---|---|---|---|---|---|
| 1 | Claude Sonnet 5.5 | Max | 91 | 7/14 | 95.6% | 59.0% | 97.8% |
| 2 | GPT-6 Astra | Max | 88 | 9/14 | 95.8% | 74.0% | 60.0% |
| 3 | GPT-6.1 Sol | Max | 84 | 9/14 | 95.4% | 68.0% | 61.1% |
| 4 | Gemini 4 Argon | High | 81 | 9/14 | 96.4% | 59.0% | 58.9% |
| 5 | Claude Opus 5.5 | Max | 72 | 7/14 | 90.6% | 67.0% | 95.6% |
| 6 | Gemini 3.7 Flash | High | 68 | 7/14 | 94.8% | 66.0% | 41.1% |
| 7 | Gemini 3.8 Flash | High | 63 | 7/14 | 95.4% | 58.0% | 38.9% |
| 8 | GPT-6 Sol | Max | 63 | 7/14 | 94.3% | 56.0% | 48.9% |
| 9 | Claude Opus 4.8 | Max | 60 | 8/14 | 91.0% | 52.0% | 65.6% |
| 10 | Claude Haiku 5.5 | Max | 58 | 6/14 | 89.7% | 55.0% | 78.9% |
| 11 | DeepSeek-V4.1-Flash | not stated | 51 | 8/14 | 89.8% | 48.0% | 64.4% |
| 12 | GPT-5.6 Terra | Max | 49 | 7/14 | 93.3% | 47.0% | 41.1% |
| 13 | Claude Fable 5 | Max | 47 | 7/14 | 85.9% | 47.0% | 70.0% |
| 14 | Qwen3.8-Max | Xhigh | 45 | 8/14 | 92.7% | 46.0% | 52.2% |
| 15 | Gemini 3.1 Pro Preview | High | 44 | 6/14 | 94.4% | 51.0% | 8.9% |
| 16 | Grok 4.7 | Xhigh | 44 | 7/14 | 92.7% | 41.0% | 54.4% |
| 17 | Kimi K3 | Max | 42 | 8/14 | 93.1% | 36.0% | 54.4% |
| 18 | Claude Fable 5 | High | 36 | 4/14 | 83.3% | 45.0% | 63.9% |
| 19 | DeepSeek-V4-Pro-0813 | Max | 35 | 7/14 | 91.7% | 46.0% | 33.3% |
| 20 | Grok 4.6 | Xhigh | 34 | 7/14 | 93.2% | 33.0% | 41.1% |
| 21 | Gemini 3.6 Flash | High | 32 | 7/14 | 94.1% | 40.0% | 23.3% |
| 22 | GPT-5.6 Luna | Max | 24 | 7/14 | 91.6% | 43.0% | 21.1% |
| 23 | GPT-6 Luna | Max | 17 | 9/14 | 90.5% | 36.0% | 26.7% |
| 24 | Claude Opus 4.6 | Max | 12 | 8/14 | 88.4% | 21.0% | 33.3% |
| 25 | Mistral Large 4 | High | 10 | 7/14 | 88.1% | 33.0% | 27.8% |
third-party · Models: 24 · Entries: 24 × Groups: 3 (benchmark coverage 3/14)GPQA Diamond, SimpleQA Verified, MirrorCode
- GPQA Diamondthird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, gpqa_diamond.csv) CC BY 4.0measured/reported by: Epoch AI (24)
- SimpleQA Verifiedthird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, simpleqa_verified.csv) CC BY 4.0measured/reported by: Epoch AI (24)
- MirrorCodethird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, mirrorcode.csv) CC BY 4.0measured/reported by: Epoch AI (24)
| Rank | Model | effort | Mean percentile | Benchmarks with values | 1. GPQA Diamond | 2. SimpleQA Verified | 3. MirrorCode |
|---|---|---|---|---|---|---|---|
| 1 | Gemini 4 Argon | High | 88 | 9/14 | 96.4% | 76.0% | 58.9% |
| 2 | GPT-6 Astra | Max | 87 | 9/14 | 95.8% | 75.6% | 60.0% |
| 3 | GPT-6.1 Sol | Max | 83 | 9/14 | 95.4% | 73.9% | 61.1% |
| 4 | Claude Sonnet 5.5 | Max | 74 | 7/14 | 95.6% | 46.5% | 97.8% |
| 5 | GPT-6.1 Terra | Max | 72 | 6/14 | 94.3% | 57.3% | 76.7% |
| 6 | Claude Opus 5.5 | Max | 67 | 7/14 | 90.6% | 72.2% | 95.6% |
| 7 | Gemini 3.8 Flash | High | 64 | 7/14 | 95.4% | 69.7% | 38.9% |
| 8 | Gemini 3.7 Flash | High | 64 | 7/14 | 94.8% | 69.2% | 41.1% |
| 9 | GPT-6 Sol | Max | 60 | 7/14 | 94.3% | 60.7% | 48.9% |
| 10 | Gemini 3.1 Pro Preview | High | 54 | 6/14 | 94.4% | 73.5% | 8.9% |
| 10 | Claude Opus 4.8 | Max | 54 | 8/14 | 91.0% | 53.0% | 65.6% |
| 12 | Grok 4.7 | Xhigh | 53 | 7/14 | 92.7% | 56.0% | 54.4% |
| 13 | Kimi K3 | Max | 50 | 8/14 | 93.1% | 50.6% | 54.4% |
| 14 | Gemini 3.6 Flash | High | 46 | 7/14 | 94.1% | 66.2% | 23.3% |
| 15 | Grok 4.6 | Xhigh | 43 | 7/14 | 93.2% | 48.9% | 41.1% |
| 16 | GPT-5.6 Terra | Max | 39 | 7/14 | 93.3% | 43.2% | 41.1% |
| 16 | Qwen3.8-Max | Xhigh | 39 | 8/14 | 92.7% | 45.8% | 52.2% |
| 18 | DeepSeek-V4.1-Flash | not stated | 36 | 8/14 | 89.8% | 42.9% | 64.4% |
| 19 | DeepSeek-V4-Pro-0813 | Max | 36 | 7/14 | 91.7% | 52.9% | 33.3% |
| 20 | Claude Haiku 5.5 | Max | 35 | 6/14 | 89.7% | 23.8% | 78.9% |
| 21 | Claude Opus 4.6 | Max | 21 | 8/14 | 88.4% | 47.0% | 33.3% |
| 22 | GPT-6 Luna | Max | 14 | 9/14 | 90.5% | 41.4% | 26.7% |
| 22 | GPT-5.6 Luna | Max | 14 | 7/14 | 91.6% | 41.0% | 21.1% |
| 24 | Mistral Large 4 | High | 6 | 7/14 | 88.1% | 20.0% | 27.8% |
third-party · Models: 24 · Entries: 24 × Groups: 3 (benchmark coverage 3/14)SimpleQA Verified, Chess Puzzles(Epoch AI), MirrorCode
- SimpleQA Verifiedthird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, simpleqa_verified.csv) CC BY 4.0measured/reported by: Epoch AI (24)
- Chess Puzzles(Epoch AI)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, chess_puzzles.csv) CC BY 4.0measured/reported by: Epoch AI (24)
- MirrorCodethird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, mirrorcode.csv) CC BY 4.0measured/reported by: Epoch AI (24)
| Rank | Model | effort | Mean percentile | Benchmarks with values | 1. SimpleQA Verified | 2. Chess Puzzles(Epoch AI) | 3. MirrorCode |
|---|---|---|---|---|---|---|---|
| 1 | Claude Opus 5.5 | Max | 90 | 7/14 | 72.2% | 67.0% | 95.6% |
| 1 | GPT-6 Astra | Max | 90 | 9/14 | 75.6% | 74.0% | 60.0% |
| 3 | GPT-6.1 Sol | Max | 88 | 9/14 | 73.9% | 68.0% | 61.1% |
| 4 | Gemini 4 Argon | High | 82 | 9/14 | 76.0% | 59.0% | 58.9% |
| 5 | Claude Sonnet 5.5 | Max | 69 | 7/14 | 46.5% | 59.0% | 97.8% |
| 6 | Gemini 3.7 Flash | High | 68 | 7/14 | 69.2% | 66.0% | 41.1% |
| 7 | Claude Opus 4.8 | Max | 65 | 8/14 | 53.0% | 52.0% | 65.6% |
| 8 | Gemini 3.8 Flash | High | 61 | 7/14 | 69.7% | 58.0% | 38.9% |
| 9 | GPT-6 Sol | Max | 59 | 7/14 | 60.7% | 56.0% | 48.9% |
| 10 | GPT-5.5 | Xhigh | 53 | 6/14 | 63.0% | 63.0% | 23.3% |
| 11 | Claude Haiku 5.5 | Max | 52 | 6/14 | 23.8% | 55.0% | 78.9% |
| 12 | DeepSeek-V4.1-Flash | not stated | 49 | 8/14 | 42.9% | 48.0% | 64.4% |
| 13 | Grok 4.7 | Xhigh | 49 | 7/14 | 56.0% | 41.0% | 54.4% |
| 14 | Gemini 3.1 Pro Preview | High | 46 | 6/14 | 73.5% | 51.0% | 8.9% |
| 15 | Kimi K3 | Max | 41 | 8/14 | 50.6% | 36.0% | 54.4% |
| 16 | Qwen3.8-Max | Xhigh | 40 | 8/14 | 45.8% | 46.0% | 52.2% |
| 17 | DeepSeek-V4-Pro-0813 | Max | 38 | 7/14 | 52.9% | 46.0% | 33.3% |
| 18 | GPT-5.6 Terra | Max | 36 | 7/14 | 43.2% | 47.0% | 41.1% |
| 19 | Gemini 3.6 Flash | High | 34 | 7/14 | 66.2% | 40.0% | 23.3% |
| 20 | Grok 4.6 | Xhigh | 30 | 7/14 | 48.9% | 33.0% | 41.1% |
| 21 | Claude Opus 4.6 | Max | 21 | 8/14 | 47.0% | 21.0% | 33.3% |
| 22 | GPT-6 Luna | Max | 15 | 9/14 | 41.4% | 36.0% | 26.7% |
| 23 | GPT-5.6 Luna | Max | 14 | 7/14 | 41.0% | 43.0% | 21.1% |
| 24 | Mistral Large 4 | High | 9 | 7/14 | 20.0% | 33.0% | 27.8% |
third-party · Models: 23 · Entries: 23 × Groups: 5 (benchmark coverage 5/14)GPQA Diamond, FrontierMath Tiers 1-3(v2), FrontierMath Tier 4(v2), OTIS Mock AIME 2024-2025, MirrorCode
- GPQA Diamondthird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, gpqa_diamond.csv) CC BY 4.0measured/reported by: Epoch AI (23)
- FrontierMath Tiers 1-3(v2)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, frontiermath_tiers_1_3_v2.csv) CC BY 4.0measured/reported by: Epoch AI (23)
- FrontierMath Tier 4(v2)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, frontiermath_tier_4_v2.csv) CC BY 4.0measured/reported by: Epoch AI (23)
- OTIS Mock AIME 2024-2025third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0measured/reported by: Epoch AI (23)
- MirrorCodethird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, mirrorcode.csv) CC BY 4.0measured/reported by: Epoch AI (23)
| Rank | Model | effort | Mean percentile | Benchmarks with values | 1. GPQA Diamond | 2. FrontierMath Tiers 1-3(v2) | 3. FrontierMath Tier 4(v2) | 4. OTIS Mock AIME 2024-2025 | 5. MirrorCode |
|---|---|---|---|---|---|---|---|---|---|
| 1 | GPT-6 Astra | Max | 89 | 9/14 | 95.8% | 93.7% | 97.6% | 100.0% | 60.0% |
| 2 | GPT-6.1 Sol | Max | 88 | 9/14 | 95.4% | 93.7% | 100.0% | 100.0% | 61.1% |
| 3 | Claude Sonnet 5.5 | Max | 85 | 7/14 | 95.6% | 88.8% | 80.5% | 100.0% | 97.8% |
| 4 | GPT-6.1 Terra | Max | 84 | 6/14 | 94.3% | 90.9% | 92.7% | 100.0% | 76.7% |
| 5 | Claude Opus 5.5 | Max | 77 | 7/14 | 90.6% | 91.2% | 95.0% | 100.0% | 95.6% |
| 6 | Gemini 4 Argon | High | 77 | 9/14 | 96.4% | 86.0% | 78.1% | 100.0% | 58.9% |
| 7 | GPT-6 Sol | Max | 72 | 7/14 | 94.3% | 89.8% | 90.0% | 100.0% | 48.9% |
| 8 | Claude Fable 5 | Max | 61 | 7/14 | 85.9% | 87.0% | 90.2% | 99.7% | 70.0% |
| 9 | GPT-5.6 Terra | Max | 58 | 7/14 | 93.3% | 86.0% | 70.7% | 99.7% | 41.1% |
| 10 | Claude Opus 4.8 | Max | 49 | 8/14 | 91.0% | 80.0% | 56.1% | 98.3% | 65.6% |
| 11 | Qwen3.8-Max | Xhigh | 48 | 8/14 | 92.7% | 74.7% | 46.3% | 99.4% | 52.2% |
| 12 | Kimi K3 | Max | 40 | 8/14 | 93.1% | 72.2% | 39.0% | 97.2% | 54.4% |
| 13 | Gemini 3.8 Flash | High | 40 | 7/14 | 95.4% | 68.4% | 22.0% | 98.9% | 38.9% |
| 13 | Grok 4.6 | Xhigh | 40 | 7/14 | 93.2% | 66.0% | 31.7% | 99.2% | 41.1% |
| 13 | Gemini 3.7 Flash | High | 40 | 7/14 | 94.8% | 71.6% | 36.6% | 97.2% | 41.1% |
| 16 | GPT-5.6 Luna | Max | 36 | 7/14 | 91.6% | 82.1% | 61.0% | 98.3% | 21.1% |
| 17 | GPT-6 Luna | Max | 35 | 9/14 | 90.5% | 79.0% | 56.1% | 98.9% | 26.7% |
| 18 | DeepSeek-V4.1-Flash | not stated | 34 | 8/14 | 89.8% | 67.4% | 26.8% | 98.3% | 64.4% |
| 19 | DeepSeek-V4-Pro-0813 | Max | 26 | 7/14 | 91.7% | 64.6% | 26.8% | 98.6% | 33.3% |
| 20 | Grok 4.7 | Xhigh | 24 | 7/14 | 92.7% | 53.0% | 17.1% | 98.1% | 54.4% |
| 21 | Gemini 3.6 Flash | High | 18 | 7/14 | 94.1% | 59.0% | 22.0% | 94.2% | 23.3% |
| 22 | Mistral Large 4 | High | 15 | 7/14 | 88.1% | 58.6% | 26.8% | 98.3% | 27.8% |
| 23 | Claude Opus 4.6 | Max | 14 | 8/14 | 88.4% | 66.0% | 26.8% | 91.1% | 33.3% |
third-party · Models: 23 · Entries: 23 × Groups: 5 (benchmark coverage 5/14)GPQA Diamond, FrontierMath Tiers 1-3(v2), FrontierMath Tier 4(v2), SimpleQA Verified, MirrorCode
- GPQA Diamondthird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, gpqa_diamond.csv) CC BY 4.0measured/reported by: Epoch AI (23)
- FrontierMath Tiers 1-3(v2)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, frontiermath_tiers_1_3_v2.csv) CC BY 4.0measured/reported by: Epoch AI (23)
- FrontierMath Tier 4(v2)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, frontiermath_tier_4_v2.csv) CC BY 4.0measured/reported by: Epoch AI (23)
- SimpleQA Verifiedthird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, simpleqa_verified.csv) CC BY 4.0measured/reported by: Epoch AI (23)
- MirrorCodethird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, mirrorcode.csv) CC BY 4.0measured/reported by: Epoch AI (23)
| Rank | Model | effort | Mean percentile | Benchmarks with values | 1. GPQA Diamond | 2. FrontierMath Tiers 1-3(v2) | 3. FrontierMath Tier 4(v2) | 4. SimpleQA Verified | 5. MirrorCode |
|---|---|---|---|---|---|---|---|---|---|
| 1 | GPT-6 Astra | Max | 90 | 9/14 | 95.8% | 93.7% | 97.6% | 75.6% | 60.0% |
| 2 | GPT-6.1 Sol | Max | 89 | 9/14 | 95.4% | 93.7% | 100.0% | 73.9% | 61.1% |
| 3 | Gemini 4 Argon | High | 81 | 9/14 | 96.4% | 86.0% | 78.1% | 76.0% | 58.9% |
| 4 | GPT-6.1 Terra | Max | 79 | 6/14 | 94.3% | 90.9% | 92.7% | 57.3% | 76.7% |
| 5 | Claude Opus 5.5 | Max | 77 | 7/14 | 90.6% | 91.2% | 95.0% | 72.2% | 95.6% |
| 6 | Claude Sonnet 5.5 | Max | 75 | 7/14 | 95.6% | 88.8% | 80.5% | 46.5% | 97.8% |
| 7 | GPT-6 Sol | Max | 70 | 7/14 | 94.3% | 89.8% | 90.0% | 60.7% | 48.9% |
| 8 | Claude Opus 4.8 | Max | 56 | 8/14 | 91.0% | 80.0% | 56.1% | 53.0% | 65.6% |
| 9 | Gemini 3.7 Flash | High | 53 | 7/14 | 94.8% | 71.6% | 36.6% | 69.2% | 41.1% |
| 10 | GPT-5.6 Terra | Max | 51 | 7/14 | 93.3% | 86.0% | 70.7% | 43.2% | 41.1% |
| 11 | Kimi K3 | Max | 47 | 8/14 | 93.1% | 72.2% | 39.0% | 50.6% | 54.4% |
| 12 | Gemini 3.8 Flash | High | 46 | 7/14 | 95.4% | 68.4% | 22.0% | 69.7% | 38.9% |
| 13 | Qwen3.8-Max | Xhigh | 42 | 8/14 | 92.7% | 74.7% | 46.3% | 45.8% | 52.2% |
| 14 | Claude Haiku 5.5 | Max | 40 | 6/14 | 89.7% | 75.1% | 46.3% | 23.8% | 78.9% |
| 15 | Grok 4.6 | Xhigh | 37 | 7/14 | 93.2% | 66.0% | 31.7% | 48.9% | 41.1% |
| 16 | GPT-5.6 Luna | Max | 34 | 7/14 | 91.6% | 82.1% | 61.0% | 41.0% | 21.1% |
| 17 | Grok 4.7 | Xhigh | 32 | 7/14 | 92.7% | 53.0% | 17.1% | 56.0% | 54.4% |
| 18 | Gemini 3.6 Flash | High | 31 | 7/14 | 94.1% | 59.0% | 22.0% | 66.2% | 23.3% |
| 18 | DeepSeek-V4.1-Flash | not stated | 31 | 8/14 | 89.8% | 67.4% | 26.8% | 42.9% | 64.4% |
| 20 | GPT-6 Luna | Max | 30 | 9/14 | 90.5% | 79.0% | 56.1% | 41.4% | 26.7% |
| 21 | DeepSeek-V4-Pro-0813 | Max | 28 | 7/14 | 91.7% | 64.6% | 26.8% | 52.9% | 33.3% |
| 22 | Claude Opus 4.6 | Max | 20 | 8/14 | 88.4% | 66.0% | 26.8% | 47.0% | 33.3% |
| 23 | Mistral Large 4 | High | 8 | 7/14 | 88.1% | 58.6% | 26.8% | 20.0% | 27.8% |
third-party · Models: 23 · Entries: 23 × Groups: 5 (benchmark coverage 5/14)GPQA Diamond, FrontierMath Tiers 1-3(v2), FrontierMath Tier 4(v2), Chess Puzzles(Epoch AI), MirrorCode
- GPQA Diamondthird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, gpqa_diamond.csv) CC BY 4.0measured/reported by: Epoch AI (23)
- FrontierMath Tiers 1-3(v2)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, frontiermath_tiers_1_3_v2.csv) CC BY 4.0measured/reported by: Epoch AI (23)
- FrontierMath Tier 4(v2)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, frontiermath_tier_4_v2.csv) CC BY 4.0measured/reported by: Epoch AI (23)
- Chess Puzzles(Epoch AI)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, chess_puzzles.csv) CC BY 4.0measured/reported by: Epoch AI (23)
- MirrorCodethird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, mirrorcode.csv) CC BY 4.0measured/reported by: Epoch AI (23)
| Rank | Model | effort | Mean percentile | Benchmarks with values | 1. GPQA Diamond | 2. FrontierMath Tiers 1-3(v2) | 3. FrontierMath Tier 4(v2) | 4. Chess Puzzles(Epoch AI) | 5. MirrorCode |
|---|---|---|---|---|---|---|---|---|---|
| 1 | GPT-6 Astra | Max | 91 | 9/14 | 95.8% | 93.7% | 97.6% | 74.0% | 60.0% |
| 2 | GPT-6.1 Sol | Max | 90 | 9/14 | 95.4% | 93.7% | 100.0% | 68.0% | 61.1% |
| 3 | Claude Sonnet 5.5 | Max | 86 | 7/14 | 95.6% | 88.8% | 80.5% | 59.0% | 97.8% |
| 4 | Claude Opus 5.5 | Max | 79 | 7/14 | 90.6% | 91.2% | 95.0% | 67.0% | 95.6% |
| 5 | Gemini 4 Argon | High | 77 | 9/14 | 96.4% | 86.0% | 78.1% | 59.0% | 58.9% |
| 6 | GPT-6 Sol | Max | 71 | 7/14 | 94.3% | 89.8% | 90.0% | 56.0% | 48.9% |
| 7 | Claude Fable 5 | Max | 60 | 7/14 | 85.9% | 87.0% | 90.2% | 47.0% | 70.0% |
| 8 | Claude Opus 4.8 | Max | 58 | 8/14 | 91.0% | 80.0% | 56.1% | 52.0% | 65.6% |
| 9 | GPT-5.6 Terra | Max | 57 | 7/14 | 93.3% | 86.0% | 70.7% | 47.0% | 41.1% |
| 10 | Gemini 3.7 Flash | High | 55 | 7/14 | 94.8% | 71.6% | 36.6% | 66.0% | 41.1% |
| 11 | Claude Haiku 5.5 | Max | 53 | 6/14 | 89.7% | 75.1% | 46.3% | 55.0% | 78.9% |
| 12 | Qwen3.8-Max | Xhigh | 45 | 8/14 | 92.7% | 74.7% | 46.3% | 46.0% | 52.2% |
| 13 | Gemini 3.8 Flash | High | 45 | 7/14 | 95.4% | 68.4% | 22.0% | 58.0% | 38.9% |
| 14 | Kimi K3 | Max | 42 | 8/14 | 93.1% | 72.2% | 39.0% | 36.0% | 54.4% |
| 15 | DeepSeek-V4.1-Flash | not stated | 40 | 8/14 | 89.8% | 67.4% | 26.8% | 48.0% | 64.4% |
| 16 | GPT-5.6 Luna | Max | 39 | 7/14 | 91.6% | 82.1% | 61.0% | 43.0% | 21.1% |
| 17 | GPT-6 Luna | Max | 32 | 9/14 | 90.5% | 79.0% | 56.1% | 36.0% | 26.7% |
| 18 | Grok 4.6 | Xhigh | 31 | 7/14 | 93.2% | 66.0% | 31.7% | 33.0% | 41.1% |
| 19 | Grok 4.7 | Xhigh | 27 | 7/14 | 92.7% | 53.0% | 17.1% | 41.0% | 54.4% |
| 19 | DeepSeek-V4-Pro-0813 | Max | 27 | 7/14 | 91.7% | 64.6% | 26.8% | 46.0% | 33.3% |
| 21 | Gemini 3.6 Flash | High | 22 | 7/14 | 94.1% | 59.0% | 22.0% | 40.0% | 23.3% |
| 22 | Claude Opus 4.6 | Max | 14 | 8/14 | 88.4% | 66.0% | 26.8% | 21.0% | 33.3% |
| 23 | Mistral Large 4 | High | 10 | 7/14 | 88.1% | 58.6% | 26.8% | 33.0% | 27.8% |
third-party · Models: 23 · Entries: 23 × Groups: 5 (benchmark coverage 5/14)FrontierMath Tiers 1-3(v2), FrontierMath Tier 4(v2), SimpleQA Verified, Chess Puzzles(Epoch AI), MirrorCode
- FrontierMath Tiers 1-3(v2)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, frontiermath_tiers_1_3_v2.csv) CC BY 4.0measured/reported by: Epoch AI (23)
- FrontierMath Tier 4(v2)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, frontiermath_tier_4_v2.csv) CC BY 4.0measured/reported by: Epoch AI (23)
- SimpleQA Verifiedthird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, simpleqa_verified.csv) CC BY 4.0measured/reported by: Epoch AI (23)
- Chess Puzzles(Epoch AI)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, chess_puzzles.csv) CC BY 4.0measured/reported by: Epoch AI (23)
- MirrorCodethird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, mirrorcode.csv) CC BY 4.0measured/reported by: Epoch AI (23)
| Rank | Model | effort | Mean percentile | Benchmarks with values | 1. FrontierMath Tiers 1-3(v2) | 2. FrontierMath Tier 4(v2) | 3. SimpleQA Verified | 4. Chess Puzzles(Epoch AI) | 5. MirrorCode |
|---|---|---|---|---|---|---|---|---|---|
| 1 | GPT-6.1 Sol | Max | 92 | 9/14 | 93.7% | 100.0% | 73.9% | 68.0% | 61.1% |
| 1 | GPT-6 Astra | Max | 92 | 9/14 | 93.7% | 97.6% | 75.6% | 74.0% | 60.0% |
| 3 | Claude Opus 5.5 | Max | 91 | 7/14 | 91.2% | 95.0% | 72.2% | 67.0% | 95.6% |
| 4 | Gemini 4 Argon | High | 79 | 9/14 | 86.0% | 78.1% | 76.0% | 59.0% | 58.9% |
| 5 | Claude Sonnet 5.5 | Max | 74 | 7/14 | 88.8% | 80.5% | 46.5% | 59.0% | 97.8% |
| 6 | GPT-6 Sol | Max | 70 | 7/14 | 89.8% | 90.0% | 60.7% | 56.0% | 48.9% |
| 7 | Claude Opus 4.8 | Max | 62 | 8/14 | 80.0% | 56.1% | 53.0% | 52.0% | 65.6% |
| 8 | GPT-5.5 | Xhigh | 60 | 6/14 | 85.3% | 72.5% | 63.0% | 63.0% | 23.3% |
| 9 | Gemini 3.7 Flash | High | 55 | 7/14 | 71.6% | 36.6% | 69.2% | 66.0% | 41.1% |
| 10 | Claude Haiku 5.5 | Max | 50 | 6/14 | 75.1% | 46.3% | 23.8% | 55.0% | 78.9% |
| 10 | GPT-5.6 Terra | Max | 50 | 7/14 | 86.0% | 70.7% | 43.2% | 47.0% | 41.1% |
| 12 | Gemini 3.8 Flash | High | 44 | 7/14 | 68.4% | 22.0% | 69.7% | 58.0% | 38.9% |
| 13 | Qwen3.8-Max | Xhigh | 43 | 8/14 | 74.7% | 46.3% | 45.8% | 46.0% | 52.2% |
| 14 | Kimi K3 | Max | 41 | 8/14 | 72.2% | 39.0% | 50.6% | 36.0% | 54.4% |
| 15 | DeepSeek-V4.1-Flash | not stated | 40 | 8/14 | 67.4% | 26.8% | 42.9% | 48.0% | 64.4% |
| 16 | GPT-5.6 Luna | Max | 34 | 7/14 | 82.1% | 61.0% | 41.0% | 43.0% | 21.1% |
| 17 | GPT-6 Luna | Max | 31 | 9/14 | 79.0% | 56.1% | 41.4% | 36.0% | 26.7% |
| 18 | Grok 4.7 | Xhigh | 30 | 7/14 | 53.0% | 17.1% | 56.0% | 41.0% | 54.4% |
| 18 | DeepSeek-V4-Pro-0813 | Max | 30 | 7/14 | 64.6% | 26.8% | 52.9% | 46.0% | 33.3% |
| 20 | Grok 4.6 | Xhigh | 28 | 7/14 | 66.0% | 31.7% | 48.9% | 33.0% | 41.1% |
| 21 | Gemini 3.6 Flash | High | 24 | 7/14 | 59.0% | 22.0% | 66.2% | 40.0% | 23.3% |
| 22 | Claude Opus 4.6 | Max | 20 | 8/14 | 66.0% | 26.8% | 47.0% | 21.0% | 33.3% |
| 23 | Mistral Large 4 | High | 10 | 7/14 | 58.6% | 26.8% | 20.0% | 33.0% | 27.8% |
third-party · Models: 23 · Entries: 24 × Groups: 4 (benchmark coverage 4/14)GPQA Diamond, OTIS Mock AIME 2024-2025, Chess Puzzles(Epoch AI), MirrorCode
- GPQA Diamondthird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, gpqa_diamond.csv) CC BY 4.0measured/reported by: Epoch AI (24)
- OTIS Mock AIME 2024-2025third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0measured/reported by: Epoch AI (24)
- Chess Puzzles(Epoch AI)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, chess_puzzles.csv) CC BY 4.0measured/reported by: Epoch AI (24)
- MirrorCodethird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, mirrorcode.csv) CC BY 4.0measured/reported by: Epoch AI (24)
| Rank | Model | effort | Mean percentile | Benchmarks with values | 1. GPQA Diamond | 2. OTIS Mock AIME 2024-2025 | 3. Chess Puzzles(Epoch AI) | 4. MirrorCode |
|---|---|---|---|---|---|---|---|---|
| 1 | Claude Sonnet 5.5 | Max | 90 | 7/14 | 95.6% | 100.0% | 59.0% | 97.8% |
| 2 | GPT-6 Astra | Max | 88 | 9/14 | 95.8% | 100.0% | 74.0% | 60.0% |
| 3 | GPT-6.1 Sol | Max | 85 | 9/14 | 95.4% | 100.0% | 68.0% | 61.1% |
| 4 | Gemini 4 Argon | High | 83 | 9/14 | 96.4% | 100.0% | 59.0% | 58.9% |
| 5 | Claude Opus 5.5 | Max | 75 | 7/14 | 90.6% | 100.0% | 67.0% | 95.6% |
| 6 | GPT-6 Sol | Max | 68 | 7/14 | 94.3% | 100.0% | 56.0% | 48.9% |
| 7 | Gemini 3.8 Flash | High | 60 | 7/14 | 95.4% | 98.9% | 58.0% | 38.9% |
| 8 | Gemini 3.7 Flash | High | 55 | 7/14 | 94.8% | 97.2% | 66.0% | 41.1% |
| 9 | GPT-5.6 Terra | Max | 54 | 7/14 | 93.3% | 99.7% | 47.0% | 41.1% |
| 10 | Claude Opus 4.8 | Max | 54 | 8/14 | 91.0% | 98.3% | 52.0% | 65.6% |
| 11 | Claude Fable 5 | Max | 53 | 7/14 | 85.9% | 99.7% | 47.0% | 70.0% |
| 12 | Claude Fable 5 | High | 50 | 4/14 | 83.3% | 100.0% | 45.0% | 63.9% |
| 13 | Qwen3.8-Max | Xhigh | 49 | 8/14 | 92.7% | 99.4% | 46.0% | 52.2% |
| 14 | DeepSeek-V4.1-Flash | not stated | 47 | 8/14 | 89.8% | 98.3% | 48.0% | 64.4% |
| 15 | Grok 4.6 | Xhigh | 40 | 7/14 | 93.2% | 99.2% | 33.0% | 41.1% |
| 16 | Grok 4.7 | Xhigh | 39 | 7/14 | 92.7% | 98.1% | 41.0% | 54.4% |
| 17 | DeepSeek-V4-Pro-0813 | Max | 37 | 7/14 | 91.7% | 98.6% | 46.0% | 33.3% |
| 18 | Gemini 3.1 Pro Preview | High | 36 | 6/14 | 94.4% | 95.6% | 51.0% | 8.9% |
| 19 | Kimi K3 | Max | 35 | 8/14 | 93.1% | 97.2% | 36.0% | 54.4% |
| 20 | GPT-5.6 Luna | Max | 26 | 7/14 | 91.6% | 98.3% | 43.0% | 21.1% |
| 21 | GPT-6 Luna | Max | 25 | 9/14 | 90.5% | 98.9% | 36.0% | 26.7% |
| 21 | Gemini 3.6 Flash | High | 25 | 7/14 | 94.1% | 94.2% | 40.0% | 23.3% |
| 23 | Mistral Large 4 | High | 16 | 7/14 | 88.1% | 98.3% | 33.0% | 27.8% |
| 24 | Claude Opus 4.6 | Max | 9 | 8/14 | 88.4% | 91.1% | 21.0% | 33.3% |
third-party · Models: 23 · Entries: 23 × Groups: 4 (benchmark coverage 4/14)GPQA Diamond, OTIS Mock AIME 2024-2025, SimpleQA Verified, MirrorCode
- GPQA Diamondthird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, gpqa_diamond.csv) CC BY 4.0measured/reported by: Epoch AI (23)
- OTIS Mock AIME 2024-2025third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0measured/reported by: Epoch AI (23)
- SimpleQA Verifiedthird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, simpleqa_verified.csv) CC BY 4.0measured/reported by: Epoch AI (23)
- MirrorCodethird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, mirrorcode.csv) CC BY 4.0measured/reported by: Epoch AI (23)
| Rank | Model | effort | Mean percentile | Benchmarks with values | 1. GPQA Diamond | 2. OTIS Mock AIME 2024-2025 | 3. SimpleQA Verified | 4. MirrorCode |
|---|---|---|---|---|---|---|---|---|
| 1 | Gemini 4 Argon | High | 89 | 9/14 | 96.4% | 100.0% | 76.0% | 58.9% |
| 2 | GPT-6 Astra | Max | 88 | 9/14 | 95.8% | 100.0% | 75.6% | 60.0% |
| 3 | GPT-6.1 Sol | Max | 85 | 9/14 | 95.4% | 100.0% | 73.9% | 61.1% |
| 4 | Claude Sonnet 5.5 | Max | 76 | 7/14 | 95.6% | 100.0% | 46.5% | 97.8% |
| 5 | GPT-6.1 Terra | Max | 76 | 6/14 | 94.3% | 100.0% | 57.3% | 76.7% |
| 6 | Claude Opus 5.5 | Max | 70 | 7/14 | 90.6% | 100.0% | 72.2% | 95.6% |
| 7 | GPT-6 Sol | Max | 66 | 7/14 | 94.3% | 100.0% | 60.7% | 48.9% |
| 8 | Gemini 3.8 Flash | High | 61 | 7/14 | 95.4% | 98.9% | 69.7% | 38.9% |
| 9 | Gemini 3.7 Flash | High | 52 | 7/14 | 94.8% | 97.2% | 69.2% | 41.1% |
| 10 | Claude Opus 4.8 | Max | 48 | 8/14 | 91.0% | 98.3% | 53.0% | 65.6% |
| 11 | Grok 4.6 | Xhigh | 47 | 7/14 | 93.2% | 99.2% | 48.9% | 41.1% |
| 12 | GPT-5.6 Terra | Max | 45 | 7/14 | 93.3% | 99.7% | 43.2% | 41.1% |
| 13 | Grok 4.7 | Xhigh | 45 | 7/14 | 92.7% | 98.1% | 56.0% | 54.4% |
| 14 | Qwen3.8-Max | Xhigh | 44 | 8/14 | 92.7% | 99.4% | 45.8% | 52.2% |
| 15 | Gemini 3.1 Pro Preview | High | 42 | 6/14 | 94.4% | 95.6% | 73.5% | 8.9% |
| 16 | Kimi K3 | Max | 41 | 8/14 | 93.1% | 97.2% | 50.6% | 54.4% |
| 17 | DeepSeek-V4-Pro-0813 | Max | 37 | 7/14 | 91.7% | 98.6% | 52.9% | 33.3% |
| 18 | Gemini 3.6 Flash | High | 35 | 7/14 | 94.1% | 94.2% | 66.2% | 23.3% |
| 19 | DeepSeek-V4.1-Flash | not stated | 35 | 8/14 | 89.8% | 98.3% | 42.9% | 64.4% |
| 20 | GPT-6 Luna | Max | 22 | 9/14 | 90.5% | 98.9% | 41.4% | 26.7% |
| 21 | GPT-5.6 Luna | Max | 18 | 7/14 | 91.6% | 98.3% | 41.0% | 21.1% |
| 22 | Claude Opus 4.6 | Max | 15 | 8/14 | 88.4% | 91.1% | 47.0% | 33.3% |
| 23 | Mistral Large 4 | High | 13 | 7/14 | 88.1% | 98.3% | 20.0% | 27.8% |
third-party · Models: 23 · Entries: 23 × Groups: 4 (benchmark coverage 4/14)GPQA Diamond, SimpleQA Verified, Chess Puzzles(Epoch AI), MirrorCode
- GPQA Diamondthird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, gpqa_diamond.csv) CC BY 4.0measured/reported by: Epoch AI (23)
- SimpleQA Verifiedthird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, simpleqa_verified.csv) CC BY 4.0measured/reported by: Epoch AI (23)
- Chess Puzzles(Epoch AI)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, chess_puzzles.csv) CC BY 4.0measured/reported by: Epoch AI (23)
- MirrorCodethird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, mirrorcode.csv) CC BY 4.0measured/reported by: Epoch AI (23)
| Rank | Model | effort | Mean percentile | Benchmarks with values | 1. GPQA Diamond | 2. SimpleQA Verified | 3. Chess Puzzles(Epoch AI) | 4. MirrorCode |
|---|---|---|---|---|---|---|---|---|
| 1 | GPT-6 Astra | Max | 91 | 9/14 | 95.8% | 75.6% | 74.0% | 60.0% |
| 2 | Gemini 4 Argon | High | 87 | 9/14 | 96.4% | 76.0% | 59.0% | 58.9% |
| 2 | GPT-6.1 Sol | Max | 87 | 9/14 | 95.4% | 73.9% | 68.0% | 61.1% |
| 4 | Claude Sonnet 5.5 | Max | 76 | 7/14 | 95.6% | 46.5% | 59.0% | 97.8% |
| 5 | Claude Opus 5.5 | Max | 73 | 7/14 | 90.6% | 72.2% | 67.0% | 95.6% |
| 6 | Gemini 3.7 Flash | High | 69 | 7/14 | 94.8% | 69.2% | 66.0% | 41.1% |
| 7 | Gemini 3.8 Flash | High | 66 | 7/14 | 95.4% | 69.7% | 58.0% | 38.9% |
| 8 | GPT-6 Sol | Max | 63 | 7/14 | 94.3% | 60.7% | 56.0% | 48.9% |
| 9 | Claude Opus 4.8 | Max | 57 | 8/14 | 91.0% | 53.0% | 52.0% | 65.6% |
| 10 | Gemini 3.1 Pro Preview | High | 53 | 6/14 | 94.4% | 73.5% | 51.0% | 8.9% |
| 11 | Grok 4.7 | Xhigh | 48 | 7/14 | 92.7% | 56.0% | 41.0% | 54.4% |
| 12 | Kimi K3 | Max | 43 | 8/14 | 93.1% | 50.6% | 36.0% | 54.4% |
| 13 | Claude Haiku 5.5 | Max | 42 | 6/14 | 89.7% | 23.8% | 55.0% | 78.9% |
| 13 | GPT-5.6 Terra | Max | 42 | 7/14 | 93.3% | 43.2% | 47.0% | 41.1% |
| 15 | Gemini 3.6 Flash | High | 41 | 7/14 | 94.1% | 66.2% | 40.0% | 23.3% |
| 15 | DeepSeek-V4.1-Flash | not stated | 41 | 8/14 | 89.8% | 42.9% | 48.0% | 64.4% |
| 17 | Qwen3.8-Max | Xhigh | 40 | 8/14 | 92.7% | 45.8% | 46.0% | 52.2% |
| 18 | DeepSeek-V4-Pro-0813 | Max | 38 | 7/14 | 91.7% | 52.9% | 46.0% | 33.3% |
| 19 | Grok 4.6 | Xhigh | 36 | 7/14 | 93.2% | 48.9% | 33.0% | 41.1% |
| 20 | GPT-5.6 Luna | Max | 19 | 7/14 | 91.6% | 41.0% | 43.0% | 21.1% |
| 21 | Claude Opus 4.6 | Max | 16 | 8/14 | 88.4% | 47.0% | 21.0% | 33.3% |
| 22 | GPT-6 Luna | Max | 15 | 9/14 | 90.5% | 41.4% | 36.0% | 26.7% |
| 23 | Mistral Large 4 | High | 6 | 7/14 | 88.1% | 20.0% | 33.0% | 27.8% |
third-party · Models: 22 · Entries: 22 × Groups: 6 (benchmark coverage 6/14)GPQA Diamond, FrontierMath Tiers 1-3(v2), FrontierMath Tier 4(v2), OTIS Mock AIME 2024-2025, SimpleQA Verified, MirrorCode
- GPQA Diamondthird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, gpqa_diamond.csv) CC BY 4.0measured/reported by: Epoch AI (22)
- FrontierMath Tiers 1-3(v2)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, frontiermath_tiers_1_3_v2.csv) CC BY 4.0measured/reported by: Epoch AI (22)
- FrontierMath Tier 4(v2)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, frontiermath_tier_4_v2.csv) CC BY 4.0measured/reported by: Epoch AI (22)
- OTIS Mock AIME 2024-2025third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0measured/reported by: Epoch AI (22)
- SimpleQA Verifiedthird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, simpleqa_verified.csv) CC BY 4.0measured/reported by: Epoch AI (22)
- MirrorCodethird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, mirrorcode.csv) CC BY 4.0measured/reported by: Epoch AI (22)
| Rank | Model | effort | Mean percentile | Benchmarks with values | 1. GPQA Diamond | 2. FrontierMath Tiers 1-3(v2) | 3. FrontierMath Tier 4(v2) | 4. OTIS Mock AIME 2024-2025 | 5. SimpleQA Verified | 6. MirrorCode |
|---|---|---|---|---|---|---|---|---|---|---|
| 1 | GPT-6 Astra | Max | 90 | 9/14 | 95.8% | 93.7% | 97.6% | 100.0% | 75.6% | 60.0% |
| 2 | GPT-6.1 Sol | Max | 89 | 9/14 | 95.4% | 93.7% | 100.0% | 100.0% | 73.9% | 61.1% |
| 3 | Gemini 4 Argon | High | 82 | 9/14 | 96.4% | 86.0% | 78.1% | 100.0% | 76.0% | 58.9% |
| 4 | GPT-6.1 Terra | Max | 80 | 6/14 | 94.3% | 90.9% | 92.7% | 100.0% | 57.3% | 76.7% |
| 5 | Claude Opus 5.5 | Max | 78 | 7/14 | 90.6% | 91.2% | 95.0% | 100.0% | 72.2% | 95.6% |
| 6 | Claude Sonnet 5.5 | Max | 76 | 7/14 | 95.6% | 88.8% | 80.5% | 100.0% | 46.5% | 97.8% |
| 7 | GPT-6 Sol | Max | 72 | 7/14 | 94.3% | 89.8% | 90.0% | 100.0% | 60.7% | 48.9% |
| 8 | GPT-5.6 Terra | Max | 53 | 7/14 | 93.3% | 86.0% | 70.7% | 99.7% | 43.2% | 41.1% |
| 9 | Claude Opus 4.8 | Max | 51 | 8/14 | 91.0% | 80.0% | 56.1% | 98.3% | 53.0% | 65.6% |
| 10 | Gemini 3.8 Flash | High | 47 | 7/14 | 95.4% | 68.4% | 22.0% | 98.9% | 69.7% | 38.9% |
| 11 | Gemini 3.7 Flash | High | 46 | 7/14 | 94.8% | 71.6% | 36.6% | 97.2% | 69.2% | 41.1% |
| 12 | Qwen3.8-Max | Xhigh | 45 | 8/14 | 92.7% | 74.7% | 46.3% | 99.4% | 45.8% | 52.2% |
| 13 | Kimi K3 | Max | 41 | 8/14 | 93.1% | 72.2% | 39.0% | 97.2% | 50.6% | 54.4% |
| 14 | Grok 4.6 | Xhigh | 40 | 7/14 | 93.2% | 66.0% | 31.7% | 99.2% | 48.9% | 41.1% |
| 15 | GPT-6 Luna | Max | 32 | 9/14 | 90.5% | 79.0% | 56.1% | 98.9% | 41.4% | 26.7% |
| 16 | GPT-5.6 Luna | Max | 31 | 7/14 | 91.6% | 82.1% | 61.0% | 98.3% | 41.0% | 21.1% |
| 17 | DeepSeek-V4.1-Flash | not stated | 31 | 8/14 | 89.8% | 67.4% | 26.8% | 98.3% | 42.9% | 64.4% |
| 18 | DeepSeek-V4-Pro-0813 | Max | 30 | 7/14 | 91.7% | 64.6% | 26.8% | 98.6% | 52.9% | 33.3% |
| 19 | Grok 4.7 | Xhigh | 30 | 7/14 | 92.7% | 53.0% | 17.1% | 98.1% | 56.0% | 54.4% |
| 20 | Gemini 3.6 Flash | High | 27 | 7/14 | 94.1% | 59.0% | 22.0% | 94.2% | 66.2% | 23.3% |
| 21 | Claude Opus 4.6 | Max | 17 | 8/14 | 88.4% | 66.0% | 26.8% | 91.1% | 47.0% | 33.3% |
| 22 | Mistral Large 4 | High | 12 | 7/14 | 88.1% | 58.6% | 26.8% | 98.3% | 20.0% | 27.8% |
third-party · Models: 22 · Entries: 22 × Groups: 6 (benchmark coverage 6/14)GPQA Diamond, FrontierMath Tiers 1-3(v2), FrontierMath Tier 4(v2), OTIS Mock AIME 2024-2025, Chess Puzzles(Epoch AI), MirrorCode
- GPQA Diamondthird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, gpqa_diamond.csv) CC BY 4.0measured/reported by: Epoch AI (22)
- FrontierMath Tiers 1-3(v2)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, frontiermath_tiers_1_3_v2.csv) CC BY 4.0measured/reported by: Epoch AI (22)
- FrontierMath Tier 4(v2)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, frontiermath_tier_4_v2.csv) CC BY 4.0measured/reported by: Epoch AI (22)
- OTIS Mock AIME 2024-2025third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0measured/reported by: Epoch AI (22)
- Chess Puzzles(Epoch AI)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, chess_puzzles.csv) CC BY 4.0measured/reported by: Epoch AI (22)
- MirrorCodethird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, mirrorcode.csv) CC BY 4.0measured/reported by: Epoch AI (22)
| Rank | Model | effort | Mean percentile | Benchmarks with values | 1. GPQA Diamond | 2. FrontierMath Tiers 1-3(v2) | 3. FrontierMath Tier 4(v2) | 4. OTIS Mock AIME 2024-2025 | 5. Chess Puzzles(Epoch AI) | 6. MirrorCode |
|---|---|---|---|---|---|---|---|---|---|---|
| 1 | GPT-6 Astra | Max | 91 | 9/14 | 95.8% | 93.7% | 97.6% | 100.0% | 74.0% | 60.0% |
| 2 | GPT-6.1 Sol | Max | 90 | 9/14 | 95.4% | 93.7% | 100.0% | 100.0% | 68.0% | 61.1% |
| 3 | Claude Sonnet 5.5 | Max | 86 | 7/14 | 95.6% | 88.8% | 80.5% | 100.0% | 59.0% | 97.8% |
| 4 | Claude Opus 5.5 | Max | 80 | 7/14 | 90.6% | 91.2% | 95.0% | 100.0% | 67.0% | 95.6% |
| 5 | Gemini 4 Argon | High | 79 | 9/14 | 96.4% | 86.0% | 78.1% | 100.0% | 59.0% | 58.9% |
| 6 | GPT-6 Sol | Max | 73 | 7/14 | 94.3% | 89.8% | 90.0% | 100.0% | 56.0% | 48.9% |
| 7 | Claude Fable 5 | Max | 62 | 7/14 | 85.9% | 87.0% | 90.2% | 99.7% | 47.0% | 70.0% |
| 8 | GPT-5.6 Terra | Max | 59 | 7/14 | 93.3% | 86.0% | 70.7% | 99.7% | 47.0% | 41.1% |
| 9 | Claude Opus 4.8 | Max | 53 | 8/14 | 91.0% | 80.0% | 56.1% | 98.3% | 52.0% | 65.6% |
| 10 | Qwen3.8-Max | Xhigh | 49 | 8/14 | 92.7% | 74.7% | 46.3% | 99.4% | 46.0% | 52.2% |
| 11 | Gemini 3.7 Flash | High | 48 | 7/14 | 94.8% | 71.6% | 36.6% | 97.2% | 66.0% | 41.1% |
| 12 | Gemini 3.8 Flash | High | 46 | 7/14 | 95.4% | 68.4% | 22.0% | 98.9% | 58.0% | 38.9% |
| 13 | DeepSeek-V4.1-Flash | not stated | 39 | 8/14 | 89.8% | 67.4% | 26.8% | 98.3% | 48.0% | 64.4% |
| 14 | Kimi K3 | Max | 38 | 8/14 | 93.1% | 72.2% | 39.0% | 97.2% | 36.0% | 54.4% |
| 15 | GPT-5.6 Luna | Max | 37 | 7/14 | 91.6% | 82.1% | 61.0% | 98.3% | 43.0% | 21.1% |
| 16 | Grok 4.6 | Xhigh | 36 | 7/14 | 93.2% | 66.0% | 31.7% | 99.2% | 33.0% | 41.1% |
| 17 | GPT-6 Luna | Max | 34 | 9/14 | 90.5% | 79.0% | 56.1% | 98.9% | 36.0% | 26.7% |
| 18 | DeepSeek-V4-Pro-0813 | Max | 30 | 7/14 | 91.7% | 64.6% | 26.8% | 98.6% | 46.0% | 33.3% |
| 19 | Grok 4.7 | Xhigh | 26 | 7/14 | 92.7% | 53.0% | 17.1% | 98.1% | 41.0% | 54.4% |
| 20 | Gemini 3.6 Flash | High | 19 | 7/14 | 94.1% | 59.0% | 22.0% | 94.2% | 40.0% | 23.3% |
| 21 | Mistral Large 4 | High | 14 | 7/14 | 88.1% | 58.6% | 26.8% | 98.3% | 33.0% | 27.8% |
| 22 | Claude Opus 4.6 | Max | 12 | 8/14 | 88.4% | 66.0% | 26.8% | 91.1% | 21.0% | 33.3% |
third-party · Models: 22 · Entries: 22 × Groups: 6 (benchmark coverage 6/14)GPQA Diamond, FrontierMath Tiers 1-3(v2), FrontierMath Tier 4(v2), SimpleQA Verified, Chess Puzzles(Epoch AI), MirrorCode
- GPQA Diamondthird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, gpqa_diamond.csv) CC BY 4.0measured/reported by: Epoch AI (22)
- FrontierMath Tiers 1-3(v2)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, frontiermath_tiers_1_3_v2.csv) CC BY 4.0measured/reported by: Epoch AI (22)
- FrontierMath Tier 4(v2)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, frontiermath_tier_4_v2.csv) CC BY 4.0measured/reported by: Epoch AI (22)
- SimpleQA Verifiedthird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, simpleqa_verified.csv) CC BY 4.0measured/reported by: Epoch AI (22)
- Chess Puzzles(Epoch AI)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, chess_puzzles.csv) CC BY 4.0measured/reported by: Epoch AI (22)
- MirrorCodethird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, mirrorcode.csv) CC BY 4.0measured/reported by: Epoch AI (22)
| Rank | Model | effort | Mean percentile | Benchmarks with values | 1. GPQA Diamond | 2. FrontierMath Tiers 1-3(v2) | 3. FrontierMath Tier 4(v2) | 4. SimpleQA Verified | 5. Chess Puzzles(Epoch AI) | 6. MirrorCode |
|---|---|---|---|---|---|---|---|---|---|---|
| 1 | GPT-6 Astra | Max | 92 | 9/14 | 95.8% | 93.7% | 97.6% | 75.6% | 74.0% | 60.0% |
| 2 | GPT-6.1 Sol | Max | 90 | 9/14 | 95.4% | 93.7% | 100.0% | 73.9% | 68.0% | 61.1% |
| 3 | Gemini 4 Argon | High | 83 | 9/14 | 96.4% | 86.0% | 78.1% | 76.0% | 59.0% | 58.9% |
| 4 | Claude Opus 5.5 | Max | 79 | 7/14 | 90.6% | 91.2% | 95.0% | 72.2% | 67.0% | 95.6% |
| 5 | Claude Sonnet 5.5 | Max | 77 | 7/14 | 95.6% | 88.8% | 80.5% | 46.5% | 59.0% | 97.8% |
| 6 | GPT-6 Sol | Max | 71 | 7/14 | 94.3% | 89.8% | 90.0% | 60.7% | 56.0% | 48.9% |
| 7 | Gemini 3.7 Flash | High | 59 | 7/14 | 94.8% | 71.6% | 36.6% | 69.2% | 66.0% | 41.1% |
| 8 | Claude Opus 4.8 | Max | 58 | 8/14 | 91.0% | 80.0% | 56.1% | 53.0% | 52.0% | 65.6% |
| 9 | GPT-5.6 Terra | Max | 53 | 7/14 | 93.3% | 86.0% | 70.7% | 43.2% | 47.0% | 41.1% |
| 10 | Gemini 3.8 Flash | High | 51 | 7/14 | 95.4% | 68.4% | 22.0% | 69.7% | 58.0% | 38.9% |
| 11 | Claude Haiku 5.5 | Max | 45 | 6/14 | 89.7% | 75.1% | 46.3% | 23.8% | 55.0% | 78.9% |
| 12 | Kimi K3 | Max | 44 | 8/14 | 93.1% | 72.2% | 39.0% | 50.6% | 36.0% | 54.4% |
| 12 | Qwen3.8-Max | Xhigh | 44 | 8/14 | 92.7% | 74.7% | 46.3% | 45.8% | 46.0% | 52.2% |
| 14 | DeepSeek-V4.1-Flash | not stated | 36 | 8/14 | 89.8% | 67.4% | 26.8% | 42.9% | 48.0% | 64.4% |
| 15 | GPT-5.6 Luna | Max | 35 | 7/14 | 91.6% | 82.1% | 61.0% | 41.0% | 43.0% | 21.1% |
| 16 | Grok 4.6 | Xhigh | 33 | 7/14 | 93.2% | 66.0% | 31.7% | 48.9% | 33.0% | 41.1% |
| 17 | Grok 4.7 | Xhigh | 33 | 7/14 | 92.7% | 53.0% | 17.1% | 56.0% | 41.0% | 54.4% |
| 18 | DeepSeek-V4-Pro-0813 | Max | 31 | 7/14 | 91.7% | 64.6% | 26.8% | 52.9% | 46.0% | 33.3% |
| 19 | Gemini 3.6 Flash | High | 31 | 7/14 | 94.1% | 59.0% | 22.0% | 66.2% | 40.0% | 23.3% |
| 20 | GPT-6 Luna | Max | 29 | 9/14 | 90.5% | 79.0% | 56.1% | 41.4% | 36.0% | 26.7% |
| 21 | Claude Opus 4.6 | Max | 18 | 8/14 | 88.4% | 66.0% | 26.8% | 47.0% | 21.0% | 33.3% |
| 22 | Mistral Large 4 | High | 8 | 7/14 | 88.1% | 58.6% | 26.8% | 20.0% | 33.0% | 27.8% |
third-party · Models: 22 · Entries: 22 × Groups: 5 (benchmark coverage 5/14)GPQA Diamond, OTIS Mock AIME 2024-2025, SimpleQA Verified, Chess Puzzles(Epoch AI), MirrorCode
- GPQA Diamondthird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, gpqa_diamond.csv) CC BY 4.0measured/reported by: Epoch AI (22)
- OTIS Mock AIME 2024-2025third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0measured/reported by: Epoch AI (22)
- SimpleQA Verifiedthird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, simpleqa_verified.csv) CC BY 4.0measured/reported by: Epoch AI (22)
- Chess Puzzles(Epoch AI)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, chess_puzzles.csv) CC BY 4.0measured/reported by: Epoch AI (22)
- MirrorCodethird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, mirrorcode.csv) CC BY 4.0measured/reported by: Epoch AI (22)
| Rank | Model | effort | Mean percentile | Benchmarks with values | 1. GPQA Diamond | 2. OTIS Mock AIME 2024-2025 | 3. SimpleQA Verified | 4. Chess Puzzles(Epoch AI) | 5. MirrorCode |
|---|---|---|---|---|---|---|---|---|---|
| 1 | GPT-6 Astra | Max | 91 | 9/14 | 95.8% | 100.0% | 75.6% | 74.0% | 60.0% |
| 2 | Gemini 4 Argon | High | 88 | 9/14 | 96.4% | 100.0% | 76.0% | 59.0% | 58.9% |
| 2 | GPT-6.1 Sol | Max | 88 | 9/14 | 95.4% | 100.0% | 73.9% | 68.0% | 61.1% |
| 4 | Claude Sonnet 5.5 | Max | 77 | 7/14 | 95.6% | 100.0% | 46.5% | 59.0% | 97.8% |
| 5 | Claude Opus 5.5 | Max | 75 | 7/14 | 90.6% | 100.0% | 72.2% | 67.0% | 95.6% |
| 6 | GPT-6 Sol | Max | 67 | 7/14 | 94.3% | 100.0% | 60.7% | 56.0% | 48.9% |
| 7 | Gemini 3.8 Flash | High | 64 | 7/14 | 95.4% | 98.9% | 69.7% | 58.0% | 38.9% |
| 8 | Gemini 3.7 Flash | High | 59 | 7/14 | 94.8% | 97.2% | 69.2% | 66.0% | 41.1% |
| 9 | Claude Opus 4.8 | Max | 53 | 8/14 | 91.0% | 98.3% | 53.0% | 52.0% | 65.6% |
| 10 | GPT-5.6 Terra | Max | 48 | 7/14 | 93.3% | 99.7% | 43.2% | 47.0% | 41.1% |
| 11 | Qwen3.8-Max | Xhigh | 45 | 8/14 | 92.7% | 99.4% | 45.8% | 46.0% | 52.2% |
| 12 | Gemini 3.1 Pro Preview | High | 45 | 6/14 | 94.4% | 95.6% | 73.5% | 51.0% | 8.9% |
| 13 | Grok 4.7 | Xhigh | 43 | 7/14 | 92.7% | 98.1% | 56.0% | 41.0% | 54.4% |
| 14 | Grok 4.6 | Xhigh | 40 | 7/14 | 93.2% | 99.2% | 48.9% | 33.0% | 41.1% |
| 15 | DeepSeek-V4.1-Flash | not stated | 40 | 8/14 | 89.8% | 98.3% | 42.9% | 48.0% | 64.4% |
| 16 | DeepSeek-V4-Pro-0813 | Max | 39 | 7/14 | 91.7% | 98.6% | 52.9% | 46.0% | 33.3% |
| 17 | Kimi K3 | Max | 38 | 8/14 | 93.1% | 97.2% | 50.6% | 36.0% | 54.4% |
| 18 | Gemini 3.6 Flash | High | 33 | 7/14 | 94.1% | 94.2% | 66.2% | 40.0% | 23.3% |
| 19 | GPT-6 Luna | Max | 22 | 9/14 | 90.5% | 98.9% | 41.4% | 36.0% | 26.7% |
| 20 | GPT-5.6 Luna | Max | 21 | 7/14 | 91.6% | 98.3% | 41.0% | 43.0% | 21.1% |
| 21 | Claude Opus 4.6 | Max | 13 | 8/14 | 88.4% | 91.1% | 47.0% | 21.0% | 33.3% |
| 22 | Mistral Large 4 | High | 12 | 7/14 | 88.1% | 98.3% | 20.0% | 33.0% | 27.8% |
third-party · Models: 21 · Entries: 21 × Groups: 7 (benchmark coverage 7/14)GPQA Diamond, FrontierMath Tiers 1-3(v2), FrontierMath Tier 4(v2), OTIS Mock AIME 2024-2025, SimpleQA Verified, Chess Puzzles(Epoch AI), MirrorCode
- GPQA Diamondthird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, gpqa_diamond.csv) CC BY 4.0measured/reported by: Epoch AI (21)
- FrontierMath Tiers 1-3(v2)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, frontiermath_tiers_1_3_v2.csv) CC BY 4.0measured/reported by: Epoch AI (21)
- FrontierMath Tier 4(v2)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, frontiermath_tier_4_v2.csv) CC BY 4.0measured/reported by: Epoch AI (21)
- OTIS Mock AIME 2024-2025third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0measured/reported by: Epoch AI (21)
- SimpleQA Verifiedthird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, simpleqa_verified.csv) CC BY 4.0measured/reported by: Epoch AI (21)
- Chess Puzzles(Epoch AI)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, chess_puzzles.csv) CC BY 4.0measured/reported by: Epoch AI (21)
- MirrorCodethird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, mirrorcode.csv) CC BY 4.0measured/reported by: Epoch AI (21)
| Rank | Model | effort | Mean percentile | Benchmarks with values | 1. GPQA Diamond | 2. FrontierMath Tiers 1-3(v2) | 3. FrontierMath Tier 4(v2) | 4. OTIS Mock AIME 2024-2025 | 5. SimpleQA Verified | 6. Chess Puzzles(Epoch AI) | 7. MirrorCode |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | GPT-6 Astra | Max | 92 | 9/14 | 95.8% | 93.7% | 97.6% | 100.0% | 75.6% | 74.0% | 60.0% |
| 2 | GPT-6.1 Sol | Max | 90 | 9/14 | 95.4% | 93.7% | 100.0% | 100.0% | 73.9% | 68.0% | 61.1% |
| 3 | Gemini 4 Argon | High | 83 | 9/14 | 96.4% | 86.0% | 78.1% | 100.0% | 76.0% | 59.0% | 58.9% |
| 4 | Claude Opus 5.5 | Max | 80 | 7/14 | 90.6% | 91.2% | 95.0% | 100.0% | 72.2% | 67.0% | 95.6% |
| 5 | Claude Sonnet 5.5 | Max | 78 | 7/14 | 95.6% | 88.8% | 80.5% | 100.0% | 46.5% | 59.0% | 97.8% |
| 6 | GPT-6 Sol | Max | 73 | 7/14 | 94.3% | 89.8% | 90.0% | 100.0% | 60.7% | 56.0% | 48.9% |
| 7 | GPT-5.6 Terra | Max | 55 | 7/14 | 93.3% | 86.0% | 70.7% | 99.7% | 43.2% | 47.0% | 41.1% |
| 8 | Claude Opus 4.8 | Max | 54 | 8/14 | 91.0% | 80.0% | 56.1% | 98.3% | 53.0% | 52.0% | 65.6% |
| 9 | Gemini 3.7 Flash | High | 53 | 7/14 | 94.8% | 71.6% | 36.6% | 97.2% | 69.2% | 66.0% | 41.1% |
| 10 | Gemini 3.8 Flash | High | 51 | 7/14 | 95.4% | 68.4% | 22.0% | 98.9% | 69.7% | 58.0% | 38.9% |
| 11 | Qwen3.8-Max | Xhigh | 47 | 8/14 | 92.7% | 74.7% | 46.3% | 99.4% | 45.8% | 46.0% | 52.2% |
| 12 | Kimi K3 | Max | 40 | 8/14 | 93.1% | 72.2% | 39.0% | 97.2% | 50.6% | 36.0% | 54.4% |
| 13 | Grok 4.6 | Xhigh | 37 | 7/14 | 93.2% | 66.0% | 31.7% | 99.2% | 48.9% | 33.0% | 41.1% |
| 14 | DeepSeek-V4.1-Flash | not stated | 36 | 8/14 | 89.8% | 67.4% | 26.8% | 98.3% | 42.9% | 48.0% | 64.4% |
| 15 | GPT-5.6 Luna | Max | 33 | 7/14 | 91.6% | 82.1% | 61.0% | 98.3% | 41.0% | 43.0% | 21.1% |
| 15 | DeepSeek-V4-Pro-0813 | Max | 33 | 7/14 | 91.7% | 64.6% | 26.8% | 98.6% | 52.9% | 46.0% | 33.3% |
| 17 | GPT-6 Luna | Max | 31 | 9/14 | 90.5% | 79.0% | 56.1% | 98.9% | 41.4% | 36.0% | 26.7% |
| 17 | Grok 4.7 | Xhigh | 31 | 7/14 | 92.7% | 53.0% | 17.1% | 98.1% | 56.0% | 41.0% | 54.4% |
| 19 | Gemini 3.6 Flash | High | 27 | 7/14 | 94.1% | 59.0% | 22.0% | 94.2% | 66.2% | 40.0% | 23.3% |
| 20 | Claude Opus 4.6 | Max | 15 | 8/14 | 88.4% | 66.0% | 26.8% | 91.1% | 47.0% | 21.0% | 33.3% |
| 21 | Mistral Large 4 | High | 12 | 7/14 | 88.1% | 58.6% | 26.8% | 98.3% | 20.0% | 33.0% | 27.8% |
third-party · Models: 7 · Entries: 7 × Groups: 4 (benchmark coverage 4/14)GPQA Diamond, OTIS Mock AIME 2024-2025, Chess Puzzles(Epoch AI), τ²-bench banking_knowledge (v1.0.1)
- GPQA Diamondthird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, gpqa_diamond.csv) CC BY 4.0measured/reported by: Epoch AI (7)
- OTIS Mock AIME 2024-2025third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0measured/reported by: Epoch AI (7)
- Chess Puzzles(Epoch AI)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, chess_puzzles.csv) CC BY 4.0measured/reported by: Epoch AI (7)
- τ²-bench banking_knowledge (v1.0.1)third-party · τ²-bench v1.0.1 banking_knowledge (run by Sierra, search alltools)measured/reported by: Sierra (7)
| Rank | Model | effort | Mean percentile | Benchmarks with values | 1. GPQA Diamond | 2. OTIS Mock AIME 2024-2025 | 3. Chess Puzzles(Epoch AI) | 4. τ²-bench banking_knowledge (v1.0.1) |
|---|---|---|---|---|---|---|---|---|
| 1 | Qwen3.8-Max | Xhigh | 75 | 8/14 | 92.7% | 99.4% | 46.0% | 55.2% |
| 2 | Claude Opus 4.8 | Max | 69 | 8/14 | 91.0% | 98.3% | 52.0% | 39.7% |
| 3 | Claude Fable 5 | Max | 56 | 7/14 | 85.9% | 99.7% | 47.0% | 39.7% |
| 4 | Gemini 3.1 Pro Preview | High | 54 | 6/14 | 94.4% | 95.6% | 51.0% | 26.0% |
| 5 | Kimi K3 | Max | 46 | 8/14 | 93.1% | 97.2% | 36.0% | 37.1% |
| 6 | Claude Opus 4.7 | Max | 33 | 6/14 | 86.4% | 86.7% | 41.0% | 40.2% |
| 7 | Claude Opus 4.6 | Max | 17 | 8/14 | 88.4% | 91.1% | 21.0% | 27.3% |
third-party · Models: 7 · Entries: 7 × Groups: 4 (benchmark coverage 4/14)FrontierMath Tiers 1-3(v2), FrontierMath Tier 4(v2), Chess Puzzles(Epoch AI), τ²-bench banking_knowledge (v1.0.1)
- FrontierMath Tiers 1-3(v2)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, frontiermath_tiers_1_3_v2.csv) CC BY 4.0measured/reported by: Epoch AI (7)
- FrontierMath Tier 4(v2)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, frontiermath_tier_4_v2.csv) CC BY 4.0measured/reported by: Epoch AI (7)
- Chess Puzzles(Epoch AI)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, chess_puzzles.csv) CC BY 4.0measured/reported by: Epoch AI (7)
- τ²-bench banking_knowledge (v1.0.1)third-party · τ²-bench v1.0.1 banking_knowledge (run by Sierra, search alltools)measured/reported by: Sierra (7)
| Rank | Model | effort | Mean percentile | Benchmarks with values | 1. FrontierMath Tiers 1-3(v2) | 2. FrontierMath Tier 4(v2) | 3. Chess Puzzles(Epoch AI) | 4. τ²-bench banking_knowledge (v1.0.1) |
|---|---|---|---|---|---|---|---|---|
| 1 | GPT-5.5 | Xhigh | 88 | 6/14 | 85.3% | 72.5% | 63.0% | 44.6% |
| 2 | Claude Fable 5 | Max | 77 | 7/14 | 87.0% | 90.2% | 47.0% | 39.7% |
| 3 | Claude Opus 4.8 | Max | 65 | 8/14 | 80.0% | 56.1% | 52.0% | 39.7% |
| 4 | Qwen3.8-Max | Xhigh | 63 | 8/14 | 74.7% | 46.3% | 46.0% | 55.2% |
| 5 | Claude Opus 4.7 | Max | 33 | 6/14 | 70.2% | 31.7% | 41.0% | 40.2% |
| 6 | Kimi K3 | Max | 25 | 8/14 | 72.2% | 39.0% | 36.0% | 37.1% |
| 7 | Claude Opus 4.6 | Max | 0 | 8/14 | 66.0% | 26.8% | 21.0% | 27.3% |
third-party · Models: 7 · Entries: 8 × Groups: 3 (benchmark coverage 3/14)τ²-bench airline (v1.0.1), τ²-bench retail (v1.0.1), τ²-bench telecom (v1.0.1)
- τ²-bench airline (v1.0.1)third-party · τ²-bench v1.0.1 (run by Sierra)measured/reported by: Sierra (8)
- τ²-bench retail (v1.0.1)third-party · τ²-bench v1.0.1 (run by Sierra)measured/reported by: Sierra (8)
- τ²-bench telecom (v1.0.1)third-party · τ²-bench v1.0.1 (run by Sierra)measured/reported by: Sierra (8)
| Rank | Model | effort | Mean percentile | Benchmarks with values | 1. τ²-bench airline (v1.0.1) | 2. τ²-bench retail (v1.0.1) | 3. τ²-bench telecom (v1.0.1) |
|---|---|---|---|---|---|---|---|
| 1 | Claude Opus 4.5 | High | 86 | 4/14 | 84.0% | 79.6% | 92.3% |
| 2 | Qwen3.5-397B-A17B | not stated | 81 | 4/14 | 81.5% | 84.4% | 97.8% |
| 3 | GPT-5.2 | High | 71 | 4/14 | 83.0% | 81.6% | 89.7% |
| 4 | Gemini 3 Flash Preview | High | 64 | 8/14 | 82.5% | 76.8% | 91.2% |
| 5 | Gemini 3 Pro Preview | High | 43 | 4/14 | 80.5% | 75.9% | 91.0% |
| 6 | GLM-5 | not stated | 36 | 4/14 | 82.5% | 73.7% | 86.8% |
| 7 | Claude Sonnet 4.5 | not stated | 10 | 9/14 | 72.0% | 72.4% | 84.9% |
| 7 | GPT-5.2 | None | 10 | 4/14 | 52.5% | 75.0% | 57.2% |
third-party · Models: 7 · Entries: 7 × Groups: 3 (benchmark coverage 3/14)GPQA Diamond, OTIS Mock AIME 2024-2025, Epoch Capabilities Index(ECI)
- GPQA Diamondthird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, gpqa_diamond.csv) CC BY 4.0measured/reported by: Epoch AI (7)
- OTIS Mock AIME 2024-2025third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0measured/reported by: Epoch AI (7)
- Epoch Capabilities Index(ECI)third-party · Epoch AI, epoch_capabilities_index/eci_scores.csv (CC BY 4.0)measured/reported by: Epoch AI (7)
| Rank | Model | effort | Mean percentile | Benchmarks with values | 1. GPQA Diamond | 2. OTIS Mock AIME 2024-2025 | 3. Epoch Capabilities Index(ECI) |
|---|---|---|---|---|---|---|---|
| 1 | DeepSeek-V4.1-Flash | not stated | 89 | 8/14 | 89.8% | 98.3% | 156.7 |
| 2 | Gemini 3.1 Pro Preview | not stated | 83 | 5/14 | 94.1% | 95.6% | 156.18 |
| 3 | Claude Opus 4.8 | not stated | 67 | 4/14 | 85.4% | 84.4% | 159.94 |
| 4 | Gemini 3 Flash Preview | not stated | 50 | 5/14 | 83.2% | 92.8% | 152.32 |
| 5 | Gemini 2.5 Pro | not stated | 33 | 5/14 | 85.3% | 84.7% | 144.87 |
| 6 | Claude Opus 4.5 | not stated | 22 | 4/14 | 80.7% | 48.1% | 151.92 |
| 7 | Claude Sonnet 4.5 | not stated | 6 | 9/14 | 73.7% | 35.6% | 147.63 |
third-party · Models: 7 · Entries: 7 × Groups: 3 (benchmark coverage 3/14)Chess Puzzles(Epoch AI), MirrorCode, τ²-bench banking_knowledge (v1.0.1)
- Chess Puzzles(Epoch AI)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, chess_puzzles.csv) CC BY 4.0measured/reported by: Epoch AI (7)
- MirrorCodethird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, mirrorcode.csv) CC BY 4.0measured/reported by: Epoch AI (7)
- τ²-bench banking_knowledge (v1.0.1)third-party · τ²-bench v1.0.1 banking_knowledge (run by Sierra, search alltools)measured/reported by: Sierra (7)
| Rank | Model | effort | Mean percentile | Benchmarks with values | 1. Chess Puzzles(Epoch AI) | 2. MirrorCode | 3. τ²-bench banking_knowledge (v1.0.1) |
|---|---|---|---|---|---|---|---|
| 1 | Claude Opus 4.8 | Max | 75 | 8/14 | 52.0% | 65.6% | 39.7% |
| 2 | Claude Fable 5 | Max | 69 | 7/14 | 47.0% | 70.0% | 39.7% |
| 3 | GPT-5.5 | Xhigh | 67 | 6/14 | 63.0% | 23.3% | 44.6% |
| 4 | Qwen3.8-Max | Xhigh | 61 | 8/14 | 46.0% | 52.2% | 55.2% |
| 5 | Kimi K3 | Max | 39 | 8/14 | 36.0% | 54.4% | 37.1% |
| 6 | Gemini 3.1 Pro Preview | High | 22 | 6/14 | 51.0% | 8.9% | 26.0% |
| 7 | Claude Opus 4.6 | Max | 17 | 8/14 | 21.0% | 33.3% | 27.3% |
third-party · Models: 6 · Entries: 6 × Groups: 6 (benchmark coverage 6/14)GPQA Diamond, FrontierMath Tiers 1-3(v2), FrontierMath Tier 4(v2), OTIS Mock AIME 2024-2025, Chess Puzzles(Epoch AI), τ²-bench banking_knowledge (v1.0.1)
- GPQA Diamondthird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, gpqa_diamond.csv) CC BY 4.0measured/reported by: Epoch AI (6)
- FrontierMath Tiers 1-3(v2)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, frontiermath_tiers_1_3_v2.csv) CC BY 4.0measured/reported by: Epoch AI (6)
- FrontierMath Tier 4(v2)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, frontiermath_tier_4_v2.csv) CC BY 4.0measured/reported by: Epoch AI (6)
- OTIS Mock AIME 2024-2025third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0measured/reported by: Epoch AI (6)
- Chess Puzzles(Epoch AI)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, chess_puzzles.csv) CC BY 4.0measured/reported by: Epoch AI (6)
- τ²-bench banking_knowledge (v1.0.1)third-party · τ²-bench v1.0.1 banking_knowledge (run by Sierra, search alltools)measured/reported by: Sierra (6)
| Rank | Model | effort | Mean percentile | Benchmarks with values | 1. GPQA Diamond | 2. FrontierMath Tiers 1-3(v2) | 3. FrontierMath Tier 4(v2) | 4. OTIS Mock AIME 2024-2025 | 5. Chess Puzzles(Epoch AI) | 6. τ²-bench banking_knowledge (v1.0.1) |
|---|---|---|---|---|---|---|---|---|---|---|
| 1 | Qwen3.8-Max | Xhigh | 73 | 8/14 | 92.7% | 74.7% | 46.3% | 99.4% | 46.0% | 55.2% |
| 2 | Claude Fable 5 | Max | 72 | 7/14 | 85.9% | 87.0% | 90.2% | 99.7% | 47.0% | 39.7% |
| 2 | Claude Opus 4.8 | Max | 72 | 8/14 | 91.0% | 80.0% | 56.1% | 98.3% | 52.0% | 39.7% |
| 4 | Kimi K3 | Max | 43 | 8/14 | 93.1% | 72.2% | 39.0% | 97.2% | 36.0% | 37.1% |
| 5 | Claude Opus 4.7 | Max | 30 | 6/14 | 86.4% | 70.2% | 31.7% | 86.7% | 41.0% | 40.2% |
| 6 | Claude Opus 4.6 | Max | 10 | 8/14 | 88.4% | 66.0% | 26.8% | 91.1% | 21.0% | 27.3% |
third-party · Models: 6 · Entries: 6 × Groups: 5 (benchmark coverage 5/14)GPQA Diamond, OTIS Mock AIME 2024-2025, Chess Puzzles(Epoch AI), MirrorCode, τ²-bench banking_knowledge (v1.0.1)
- GPQA Diamondthird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, gpqa_diamond.csv) CC BY 4.0measured/reported by: Epoch AI (6)
- OTIS Mock AIME 2024-2025third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0measured/reported by: Epoch AI (6)
- Chess Puzzles(Epoch AI)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, chess_puzzles.csv) CC BY 4.0measured/reported by: Epoch AI (6)
- MirrorCodethird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, mirrorcode.csv) CC BY 4.0measured/reported by: Epoch AI (6)
- τ²-bench banking_knowledge (v1.0.1)third-party · τ²-bench v1.0.1 banking_knowledge (run by Sierra, search alltools)measured/reported by: Sierra (6)
| Rank | Model | effort | Mean percentile | Benchmarks with values | 1. GPQA Diamond | 2. OTIS Mock AIME 2024-2025 | 3. Chess Puzzles(Epoch AI) | 4. MirrorCode | 5. τ²-bench banking_knowledge (v1.0.1) |
|---|---|---|---|---|---|---|---|---|---|
| 1 | Claude Opus 4.8 | Max | 70 | 8/14 | 91.0% | 98.3% | 52.0% | 65.6% | 39.7% |
| 2 | Claude Fable 5 | Max | 66 | 7/14 | 85.9% | 99.7% | 47.0% | 70.0% | 39.7% |
| 3 | Qwen3.8-Max | Xhigh | 64 | 8/14 | 92.7% | 99.4% | 46.0% | 52.2% | 55.2% |
| 4 | Kimi K3 | Max | 48 | 8/14 | 93.1% | 97.2% | 36.0% | 54.4% | 37.1% |
| 5 | Gemini 3.1 Pro Preview | High | 40 | 6/14 | 94.4% | 95.6% | 51.0% | 8.9% | 26.0% |
| 6 | Claude Opus 4.6 | Max | 12 | 8/14 | 88.4% | 91.1% | 21.0% | 33.3% | 27.3% |
third-party · Models: 6 · Entries: 6 × Groups: 5 (benchmark coverage 5/14)FrontierMath Tiers 1-3(v2), FrontierMath Tier 4(v2), Chess Puzzles(Epoch AI), MirrorCode, τ²-bench banking_knowledge (v1.0.1)
- FrontierMath Tiers 1-3(v2)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, frontiermath_tiers_1_3_v2.csv) CC BY 4.0measured/reported by: Epoch AI (6)
- FrontierMath Tier 4(v2)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, frontiermath_tier_4_v2.csv) CC BY 4.0measured/reported by: Epoch AI (6)
- Chess Puzzles(Epoch AI)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, chess_puzzles.csv) CC BY 4.0measured/reported by: Epoch AI (6)
- MirrorCodethird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, mirrorcode.csv) CC BY 4.0measured/reported by: Epoch AI (6)
- τ²-bench banking_knowledge (v1.0.1)third-party · τ²-bench v1.0.1 banking_knowledge (run by Sierra, search alltools)measured/reported by: Sierra (6)
| Rank | Model | effort | Mean percentile | Benchmarks with values | 1. FrontierMath Tiers 1-3(v2) | 2. FrontierMath Tier 4(v2) | 3. Chess Puzzles(Epoch AI) | 4. MirrorCode | 5. τ²-bench banking_knowledge (v1.0.1) |
|---|---|---|---|---|---|---|---|---|---|
| 1 | Claude Fable 5 | Max | 82 | 7/14 | 87.0% | 90.2% | 47.0% | 70.0% | 39.7% |
| 2 | GPT-5.5 | Xhigh | 68 | 6/14 | 85.3% | 72.5% | 63.0% | 23.3% | 44.6% |
| 3 | Claude Opus 4.8 | Max | 66 | 8/14 | 80.0% | 56.1% | 52.0% | 65.6% | 39.7% |
| 4 | Qwen3.8-Max | Xhigh | 52 | 8/14 | 74.7% | 46.3% | 46.0% | 52.2% | 55.2% |
| 5 | Kimi K3 | Max | 28 | 8/14 | 72.2% | 39.0% | 36.0% | 54.4% | 37.1% |
| 6 | Claude Opus 4.6 | Max | 4 | 8/14 | 66.0% | 26.8% | 21.0% | 33.3% | 27.3% |
third-party · Models: 6 · Entries: 6 × Groups: 4 (benchmark coverage 4/14)SimpleQA Verified, Chess Puzzles(Epoch AI), MirrorCode, τ²-bench banking_knowledge (v1.0.1)
- SimpleQA Verifiedthird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, simpleqa_verified.csv) CC BY 4.0measured/reported by: Epoch AI (6)
- Chess Puzzles(Epoch AI)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, chess_puzzles.csv) CC BY 4.0measured/reported by: Epoch AI (6)
- MirrorCodethird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, mirrorcode.csv) CC BY 4.0measured/reported by: Epoch AI (6)
- τ²-bench banking_knowledge (v1.0.1)third-party · τ²-bench v1.0.1 banking_knowledge (run by Sierra, search alltools)measured/reported by: Sierra (6)
| Rank | Model | effort | Mean percentile | Benchmarks with values | 1. SimpleQA Verified | 2. Chess Puzzles(Epoch AI) | 3. MirrorCode | 4. τ²-bench banking_knowledge (v1.0.1) |
|---|---|---|---|---|---|---|---|---|
| 1 | Claude Opus 4.8 | Max | 75 | 8/14 | 53.0% | 52.0% | 65.6% | 39.7% |
| 2 | GPT-5.5 | Xhigh | 70 | 6/14 | 63.0% | 63.0% | 23.3% | 44.6% |
| 3 | Qwen3.8-Max | Xhigh | 50 | 8/14 | 45.8% | 46.0% | 52.2% | 55.2% |
| 4 | Kimi K3 | Max | 45 | 8/14 | 50.6% | 36.0% | 54.4% | 37.1% |
| 5 | Gemini 3.1 Pro Preview | High | 40 | 6/14 | 73.5% | 51.0% | 8.9% | 26.0% |
| 6 | Claude Opus 4.6 | Max | 20 | 8/14 | 47.0% | 21.0% | 33.3% | 27.3% |
third-party · Models: 6 · Entries: 6 × Groups: 3 (benchmark coverage 3/14)OTIS Mock AIME 2024-2025, LMArena Text (style control, overall), LMArena Text Coding (without style control)
- OTIS Mock AIME 2024-2025third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0measured/reported by: Epoch AI (6)
- LMArena Text (style control, overall)third-party · lmarena-ai/leaderboard-dataset text_style_control/latest-00000-of-00001.parquet (CC BY 4.0)measured/reported by: LMArena (6)
- LMArena Text Coding (without style control)third-party · lmarena-ai/leaderboard-dataset text/latest-00000-of-00001.parquet (CC BY 4.0)measured/reported by: LMArena (6)
| Rank | Model | effort | Mean percentile | Benchmarks with values | 1. OTIS Mock AIME 2024-2025 | 2. LMArena Text (style control, overall) | 3. LMArena Text Coding (without style control) |
|---|---|---|---|---|---|---|---|
| 1 | Gemini 4 Carbon | High | 90 | 8/14 | 100.0% | 1,526 | 1,563 |
| 2 | Gemini 4 Argon | High | 77 | 9/14 | 100.0% | 1,522 | 1,540 |
| 3 | GPT-6.1 Sol | Max | 57 | 9/14 | 100.0% | 1,482 | 1,486 |
| 3 | GPT-6 Astra | Max | 57 | 9/14 | 100.0% | 1,474 | 1,494 |
| 5 | GPT-6 Luna | Max | 13 | 9/14 | 98.9% | 1,441 | 1,432 |
| 6 | Claude Haiku 5.5 | High | 7 | 3/14 | 97.2% | 1,422 | 1,454 |
third-party · Models: 5 · Entries: 5 × Groups: 8 (benchmark coverage 8/14)GPQA Diamond, FrontierMath Tiers 1-3(v2), FrontierMath Tier 4(v2), OTIS Mock AIME 2024-2025, SimpleQA Verified, Chess Puzzles(Epoch AI), LMArena Text (style control, overall), LMArena Text Coding (without style control)
- GPQA Diamondthird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, gpqa_diamond.csv) CC BY 4.0measured/reported by: Epoch AI (5)
- FrontierMath Tiers 1-3(v2)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, frontiermath_tiers_1_3_v2.csv) CC BY 4.0measured/reported by: Epoch AI (5)
- FrontierMath Tier 4(v2)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, frontiermath_tier_4_v2.csv) CC BY 4.0measured/reported by: Epoch AI (5)
- OTIS Mock AIME 2024-2025third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0measured/reported by: Epoch AI (5)
- SimpleQA Verifiedthird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, simpleqa_verified.csv) CC BY 4.0measured/reported by: Epoch AI (5)
- Chess Puzzles(Epoch AI)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, chess_puzzles.csv) CC BY 4.0measured/reported by: Epoch AI (5)
- LMArena Text (style control, overall)third-party · lmarena-ai/leaderboard-dataset text_style_control/latest-00000-of-00001.parquet (CC BY 4.0)measured/reported by: LMArena (5)
- LMArena Text Coding (without style control)third-party · lmarena-ai/leaderboard-dataset text/latest-00000-of-00001.parquet (CC BY 4.0)measured/reported by: LMArena (5)
| Rank | Model | effort | Mean percentile | Benchmarks with values | 1. GPQA Diamond | 2. FrontierMath Tiers 1-3(v2) | 3. FrontierMath Tier 4(v2) | 4. OTIS Mock AIME 2024-2025 | 5. SimpleQA Verified | 6. Chess Puzzles(Epoch AI) | 7. LMArena Text (style control, overall) | 8. LMArena Text Coding (without style control) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | Gemini 4 Carbon | High | 83 | 8/14 | 96.5% | 90.5% | 95.1% | 100.0% | 77.7% | 78.0% | 1,526 | 1,563 |
| 2 | GPT-6 Astra | Max | 59 | 9/14 | 95.8% | 93.7% | 97.6% | 100.0% | 75.6% | 74.0% | 1,474 | 1,494 |
| 3 | Gemini 4 Argon | High | 55 | 9/14 | 96.4% | 86.0% | 78.1% | 100.0% | 76.0% | 59.0% | 1,522 | 1,540 |
| 4 | GPT-6.1 Sol | Max | 53 | 9/14 | 95.4% | 93.7% | 100.0% | 100.0% | 73.9% | 68.0% | 1,482 | 1,486 |
| 5 | GPT-6 Luna | Max | 0 | 9/14 | 90.5% | 79.0% | 56.1% | 98.9% | 41.4% | 36.0% | 1,441 | 1,432 |
third-party · Models: 5 · Entries: 5 × Groups: 7 (benchmark coverage 7/14)GPQA Diamond, FrontierMath Tiers 1-3(v2), FrontierMath Tier 4(v2), OTIS Mock AIME 2024-2025, Chess Puzzles(Epoch AI), MirrorCode, τ²-bench banking_knowledge (v1.0.1)
- GPQA Diamondthird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, gpqa_diamond.csv) CC BY 4.0measured/reported by: Epoch AI (5)
- FrontierMath Tiers 1-3(v2)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, frontiermath_tiers_1_3_v2.csv) CC BY 4.0measured/reported by: Epoch AI (5)
- FrontierMath Tier 4(v2)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, frontiermath_tier_4_v2.csv) CC BY 4.0measured/reported by: Epoch AI (5)
- OTIS Mock AIME 2024-2025third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0measured/reported by: Epoch AI (5)
- Chess Puzzles(Epoch AI)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, chess_puzzles.csv) CC BY 4.0measured/reported by: Epoch AI (5)
- MirrorCodethird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, mirrorcode.csv) CC BY 4.0measured/reported by: Epoch AI (5)
- τ²-bench banking_knowledge (v1.0.1)third-party · τ²-bench v1.0.1 banking_knowledge (run by Sierra, search alltools)measured/reported by: Sierra (5)
| Rank | Model | effort | Mean percentile | Benchmarks with values | 1. GPQA Diamond | 2. FrontierMath Tiers 1-3(v2) | 3. FrontierMath Tier 4(v2) | 4. OTIS Mock AIME 2024-2025 | 5. Chess Puzzles(Epoch AI) | 6. MirrorCode | 7. τ²-bench banking_knowledge (v1.0.1) |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | Claude Fable 5 | Max | 77 | 7/14 | 85.9% | 87.0% | 90.2% | 99.7% | 47.0% | 70.0% | 39.7% |
| 2 | Claude Opus 4.8 | Max | 70 | 8/14 | 91.0% | 80.0% | 56.1% | 98.3% | 52.0% | 65.6% | 39.7% |
| 3 | Qwen3.8-Max | Xhigh | 61 | 8/14 | 92.7% | 74.7% | 46.3% | 99.4% | 46.0% | 52.2% | 55.2% |
| 4 | Kimi K3 | Max | 39 | 8/14 | 93.1% | 72.2% | 39.0% | 97.2% | 36.0% | 54.4% | 37.1% |
| 5 | Claude Opus 4.6 | Max | 4 | 8/14 | 88.4% | 66.0% | 26.8% | 91.1% | 21.0% | 33.3% | 27.3% |
third-party · Models: 5 · Entries: 5 × Groups: 6 (benchmark coverage 6/14)GPQA Diamond, OTIS Mock AIME 2024-2025, SimpleQA Verified, Chess Puzzles(Epoch AI), MirrorCode, τ²-bench banking_knowledge (v1.0.1)
- GPQA Diamondthird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, gpqa_diamond.csv) CC BY 4.0measured/reported by: Epoch AI (5)
- OTIS Mock AIME 2024-2025third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0measured/reported by: Epoch AI (5)
- SimpleQA Verifiedthird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, simpleqa_verified.csv) CC BY 4.0measured/reported by: Epoch AI (5)
- Chess Puzzles(Epoch AI)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, chess_puzzles.csv) CC BY 4.0measured/reported by: Epoch AI (5)
- MirrorCodethird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, mirrorcode.csv) CC BY 4.0measured/reported by: Epoch AI (5)
- τ²-bench banking_knowledge (v1.0.1)third-party · τ²-bench v1.0.1 banking_knowledge (run by Sierra, search alltools)measured/reported by: Sierra (5)
| Rank | Model | effort | Mean percentile | Benchmarks with values | 1. GPQA Diamond | 2. OTIS Mock AIME 2024-2025 | 3. SimpleQA Verified | 4. Chess Puzzles(Epoch AI) | 5. MirrorCode | 6. τ²-bench banking_knowledge (v1.0.1) |
|---|---|---|---|---|---|---|---|---|---|---|
| 1 | Claude Opus 4.8 | Max | 75 | 8/14 | 91.0% | 98.3% | 53.0% | 52.0% | 65.6% | 39.7% |
| 2 | Qwen3.8-Max | Xhigh | 58 | 8/14 | 92.7% | 99.4% | 45.8% | 46.0% | 52.2% | 55.2% |
| 3 | Kimi K3 | Max | 54 | 8/14 | 93.1% | 97.2% | 50.6% | 36.0% | 54.4% | 37.1% |
| 4 | Gemini 3.1 Pro Preview | High | 50 | 6/14 | 94.4% | 95.6% | 73.5% | 51.0% | 8.9% | 26.0% |
| 5 | Claude Opus 4.6 | Max | 13 | 8/14 | 88.4% | 91.1% | 47.0% | 21.0% | 33.3% | 27.3% |
third-party · Models: 5 · Entries: 5 × Groups: 6 (benchmark coverage 6/14)FrontierMath Tiers 1-3(v2), FrontierMath Tier 4(v2), SimpleQA Verified, Chess Puzzles(Epoch AI), MirrorCode, τ²-bench banking_knowledge (v1.0.1)
- FrontierMath Tiers 1-3(v2)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, frontiermath_tiers_1_3_v2.csv) CC BY 4.0measured/reported by: Epoch AI (5)
- FrontierMath Tier 4(v2)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, frontiermath_tier_4_v2.csv) CC BY 4.0measured/reported by: Epoch AI (5)
- SimpleQA Verifiedthird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, simpleqa_verified.csv) CC BY 4.0measured/reported by: Epoch AI (5)
- Chess Puzzles(Epoch AI)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, chess_puzzles.csv) CC BY 4.0measured/reported by: Epoch AI (5)
- MirrorCodethird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, mirrorcode.csv) CC BY 4.0measured/reported by: Epoch AI (5)
- τ²-bench banking_knowledge (v1.0.1)third-party · τ²-bench v1.0.1 banking_knowledge (run by Sierra, search alltools)measured/reported by: Sierra (5)
| Rank | Model | effort | Mean percentile | Benchmarks with values | 1. FrontierMath Tiers 1-3(v2) | 2. FrontierMath Tier 4(v2) | 3. SimpleQA Verified | 4. Chess Puzzles(Epoch AI) | 5. MirrorCode | 6. τ²-bench banking_knowledge (v1.0.1) |
|---|---|---|---|---|---|---|---|---|---|---|
| 1 | GPT-5.5 | Xhigh | 79 | 6/14 | 85.3% | 72.5% | 63.0% | 63.0% | 23.3% | 44.6% |
| 2 | Claude Opus 4.8 | Max | 75 | 8/14 | 80.0% | 56.1% | 53.0% | 52.0% | 65.6% | 39.7% |
| 3 | Qwen3.8-Max | Xhigh | 50 | 8/14 | 74.7% | 46.3% | 45.8% | 46.0% | 52.2% | 55.2% |
| 4 | Kimi K3 | Max | 38 | 8/14 | 72.2% | 39.0% | 50.6% | 36.0% | 54.4% | 37.1% |
| 5 | Claude Opus 4.6 | Max | 8 | 8/14 | 66.0% | 26.8% | 47.0% | 21.0% | 33.3% | 27.3% |
third-party · Models: 5 · Entries: 5 × Groups: 3 (benchmark coverage 3/14)MirrorCode, LMArena Text (style control, overall), LMArena Text Coding (without style control)
- MirrorCodethird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, mirrorcode.csv) CC BY 4.0measured/reported by: Epoch AI (5)
- LMArena Text (style control, overall)third-party · lmarena-ai/leaderboard-dataset text_style_control/latest-00000-of-00001.parquet (CC BY 4.0)measured/reported by: LMArena (5)
- LMArena Text Coding (without style control)third-party · lmarena-ai/leaderboard-dataset text/latest-00000-of-00001.parquet (CC BY 4.0)measured/reported by: LMArena (5)
| Rank | Model | effort | Mean percentile | Benchmarks with values | 1. MirrorCode | 2. LMArena Text (style control, overall) | 3. LMArena Text Coding (without style control) |
|---|---|---|---|---|---|---|---|
| 1 | Claude Opus 5.5 | High | 92 | 3/14 | 78.9% | 1,506 | 1,542 |
| 2 | Gemini 4 Argon | High | 67 | 9/14 | 58.9% | 1,522 | 1,540 |
| 3 | GPT-6.1 Sol | Max | 50 | 9/14 | 61.1% | 1,482 | 1,486 |
| 4 | GPT-6 Astra | Max | 42 | 9/14 | 60.0% | 1,474 | 1,494 |
| 5 | GPT-6 Luna | Max | 0 | 9/14 | 26.7% | 1,441 | 1,432 |
third-party · Models: 4 · Entries: 4 × Groups: 9 (benchmark coverage 9/14)GPQA Diamond, FrontierMath Tiers 1-3(v2), FrontierMath Tier 4(v2), OTIS Mock AIME 2024-2025, SimpleQA Verified, Chess Puzzles(Epoch AI), MirrorCode, LMArena Text (style control, overall), LMArena Text Coding (without style control)
- GPQA Diamondthird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, gpqa_diamond.csv) CC BY 4.0measured/reported by: Epoch AI (4)
- FrontierMath Tiers 1-3(v2)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, frontiermath_tiers_1_3_v2.csv) CC BY 4.0measured/reported by: Epoch AI (4)
- FrontierMath Tier 4(v2)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, frontiermath_tier_4_v2.csv) CC BY 4.0measured/reported by: Epoch AI (4)
- OTIS Mock AIME 2024-2025third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0measured/reported by: Epoch AI (4)
- SimpleQA Verifiedthird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, simpleqa_verified.csv) CC BY 4.0measured/reported by: Epoch AI (4)
- Chess Puzzles(Epoch AI)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, chess_puzzles.csv) CC BY 4.0measured/reported by: Epoch AI (4)
- MirrorCodethird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, mirrorcode.csv) CC BY 4.0measured/reported by: Epoch AI (4)
- LMArena Text (style control, overall)third-party · lmarena-ai/leaderboard-dataset text_style_control/latest-00000-of-00001.parquet (CC BY 4.0)measured/reported by: LMArena (4)
- LMArena Text Coding (without style control)third-party · lmarena-ai/leaderboard-dataset text/latest-00000-of-00001.parquet (CC BY 4.0)measured/reported by: LMArena (4)
| Rank | Model | effort | Mean percentile | Benchmarks with values | 1. GPQA Diamond | 2. FrontierMath Tiers 1-3(v2) | 3. FrontierMath Tier 4(v2) | 4. OTIS Mock AIME 2024-2025 | 5. SimpleQA Verified | 6. Chess Puzzles(Epoch AI) | 7. MirrorCode | 8. LMArena Text (style control, overall) | 9. LMArena Text Coding (without style control) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | GPT-6 Astra | Max | 69 | 9/14 | 95.8% | 93.7% | 97.6% | 100.0% | 75.6% | 74.0% | 60.0% | 1,474 | 1,494 |
| 2 | Gemini 4 Argon | High | 67 | 9/14 | 96.4% | 86.0% | 78.1% | 100.0% | 76.0% | 59.0% | 58.9% | 1,522 | 1,540 |
| 3 | GPT-6.1 Sol | Max | 65 | 9/14 | 95.4% | 93.7% | 100.0% | 100.0% | 73.9% | 68.0% | 61.1% | 1,482 | 1,486 |
| 4 | GPT-6 Luna | Max | 0 | 9/14 | 90.5% | 79.0% | 56.1% | 98.9% | 41.4% | 36.0% | 26.7% | 1,441 | 1,432 |
third-party · Models: 4 · Entries: 4 × Groups: 8 (benchmark coverage 8/14)GPQA Diamond, FrontierMath Tiers 1-3(v2), FrontierMath Tier 4(v2), OTIS Mock AIME 2024-2025, SimpleQA Verified, Chess Puzzles(Epoch AI), MirrorCode, τ²-bench banking_knowledge (v1.0.1)
- GPQA Diamondthird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, gpqa_diamond.csv) CC BY 4.0measured/reported by: Epoch AI (4)
- FrontierMath Tiers 1-3(v2)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, frontiermath_tiers_1_3_v2.csv) CC BY 4.0measured/reported by: Epoch AI (4)
- FrontierMath Tier 4(v2)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, frontiermath_tier_4_v2.csv) CC BY 4.0measured/reported by: Epoch AI (4)
- OTIS Mock AIME 2024-2025third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0measured/reported by: Epoch AI (4)
- SimpleQA Verifiedthird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, simpleqa_verified.csv) CC BY 4.0measured/reported by: Epoch AI (4)
- Chess Puzzles(Epoch AI)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, chess_puzzles.csv) CC BY 4.0measured/reported by: Epoch AI (4)
- MirrorCodethird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, mirrorcode.csv) CC BY 4.0measured/reported by: Epoch AI (4)
- τ²-bench banking_knowledge (v1.0.1)third-party · τ²-bench v1.0.1 banking_knowledge (run by Sierra, search alltools)measured/reported by: Sierra (4)
| Rank | Model | effort | Mean percentile | Benchmarks with values | 1. GPQA Diamond | 2. FrontierMath Tiers 1-3(v2) | 3. FrontierMath Tier 4(v2) | 4. OTIS Mock AIME 2024-2025 | 5. SimpleQA Verified | 6. Chess Puzzles(Epoch AI) | 7. MirrorCode | 8. τ²-bench banking_knowledge (v1.0.1) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | Claude Opus 4.8 | Max | 83 | 8/14 | 91.0% | 80.0% | 56.1% | 98.3% | 53.0% | 52.0% | 65.6% | 39.7% |
| 2 | Qwen3.8-Max | Xhigh | 63 | 8/14 | 92.7% | 74.7% | 46.3% | 99.4% | 45.8% | 46.0% | 52.2% | 55.2% |
| 3 | Kimi K3 | Max | 50 | 8/14 | 93.1% | 72.2% | 39.0% | 97.2% | 50.6% | 36.0% | 54.4% | 37.1% |
| 4 | Claude Opus 4.6 | Max | 4 | 8/14 | 88.4% | 66.0% | 26.8% | 91.1% | 47.0% | 21.0% | 33.3% | 27.3% |
third-party · Models: 4 · Entries: 4 × Groups: 5 (benchmark coverage 5/14)GPQA Diamond, FrontierMath Tiers 1-3(v2), FrontierMath Tier 4(v2), OTIS Mock AIME 2024-2025, Epoch Capabilities Index(ECI)
- GPQA Diamondthird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, gpqa_diamond.csv) CC BY 4.0measured/reported by: Epoch AI (4)
- FrontierMath Tiers 1-3(v2)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, frontiermath_tiers_1_3_v2.csv) CC BY 4.0measured/reported by: Epoch AI (4)
- FrontierMath Tier 4(v2)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, frontiermath_tier_4_v2.csv) CC BY 4.0measured/reported by: Epoch AI (4)
- OTIS Mock AIME 2024-2025third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0measured/reported by: Epoch AI (4)
- Epoch Capabilities Index(ECI)third-party · Epoch AI, epoch_capabilities_index/eci_scores.csv (CC BY 4.0)measured/reported by: Epoch AI (4)
| Rank | Model | effort | Mean percentile | Benchmarks with values | 1. GPQA Diamond | 2. FrontierMath Tiers 1-3(v2) | 3. FrontierMath Tier 4(v2) | 4. OTIS Mock AIME 2024-2025 | 5. Epoch Capabilities Index(ECI) |
|---|---|---|---|---|---|---|---|---|---|
| 1 | DeepSeek-V4.1-Flash | not stated | 90 | 8/14 | 89.8% | 67.4% | 26.8% | 98.3% | 156.7 |
| 2 | Gemini 3.1 Pro Preview | not stated | 77 | 5/14 | 94.1% | 59.7% | 26.8% | 95.6% | 156.18 |
| 3 | Gemini 3 Flash Preview | not stated | 27 | 5/14 | 83.2% | 51.2% | 17.1% | 92.8% | 152.32 |
| 4 | Gemini 2.5 Pro | not stated | 7 | 5/14 | 85.3% | 24.6% | 0.0% | 84.7% | 144.87 |
third-party · Models: 4 · Entries: 4 × Groups: 4 (benchmark coverage 4/14)GPQA Diamond, OTIS Mock AIME 2024-2025, Chess Puzzles(Epoch AI), Epoch Capabilities Index(ECI)
- GPQA Diamondthird-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, gpqa_diamond.csv) CC BY 4.0measured/reported by: Epoch AI (4)
- OTIS Mock AIME 2024-2025third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0measured/reported by: Epoch AI (4)
- Chess Puzzles(Epoch AI)third-party · Epoch AI, Capabilities & benchmarking (benchmark_data.zip, chess_puzzles.csv) CC BY 4.0measured/reported by: Epoch AI (4)
- Epoch Capabilities Index(ECI)third-party · Epoch AI, epoch_capabilities_index/eci_scores.csv (CC BY 4.0)measured/reported by: Epoch AI (4)
| Rank | Model | effort | Mean percentile | Benchmarks with values | 1. GPQA Diamond | 2. OTIS Mock AIME 2024-2025 | 3. Chess Puzzles(Epoch AI) | 4. Epoch Capabilities Index(ECI) |
|---|---|---|---|---|---|---|---|---|
| 1 | DeepSeek-V4.1-Flash | not stated | 92 | 8/14 | 89.8% | 98.3% | 48.0% | 156.7 |
| 2 | Claude Opus 4.8 | not stated | 75 | 4/14 | 85.4% | 84.4% | 19.0% | 159.94 |
| 3 | Claude Opus 4.5 | not stated | 25 | 4/14 | 80.7% | 48.1% | 5.0% | 151.92 |
| 4 | Claude Sonnet 4.5 | not stated | 8 | 9/14 | 73.7% | 35.6% | 6.0% | 147.63 |
third-party · Models: 3 · Entries: 3 × Groups: 4 (benchmark coverage 4/14)τ²-bench airline (v1.0.1), τ²-bench retail (v1.0.1), τ²-bench telecom (v1.0.1), τ²-bench banking_knowledge (v1.0.1)
- τ²-bench airline (v1.0.1)third-party · τ²-bench v1.0.1 (run by Sierra)measured/reported by: Sierra (3)
- τ²-bench retail (v1.0.1)third-party · τ²-bench v1.0.1 (run by Sierra)measured/reported by: Sierra (3)
- τ²-bench telecom (v1.0.1)third-party · τ²-bench v1.0.1 (run by Sierra)measured/reported by: Sierra (3)
- τ²-bench banking_knowledge (v1.0.1)third-party · τ²-bench v1.0.1 banking_knowledge (run by Sierra, search terminal)measured/reported by: Sierra (3)
| Rank | Model | effort | Mean percentile | Benchmarks with values | 1. τ²-bench airline (v1.0.1) | 2. τ²-bench retail (v1.0.1) | 3. τ²-bench telecom (v1.0.1) | 4. τ²-bench banking_knowledge (v1.0.1) |
|---|---|---|---|---|---|---|---|---|
| 1 | Gemini 3 Flash Preview | High | 100 | 8/14 | 82.5% | 76.8% | 91.2% | 27.3% |
| 2 | Gemini 3 Pro Preview | High | 38 | 4/14 | 80.5% | 75.9% | 91.0% | 18.0% |
| 3 | Claude Sonnet 4.5 | not stated | 13 | 9/14 | 72.0% | 72.4% | 84.9% | 25.3% |
Cost balance
Source costs have different meanings across benchmarks, so they are considered only within comparison groups and never combined across benchmarks.
- Frontier: Points for which no better score exists at the same or lower cost (Pareto frontier), ordered by cost. Points without source costs are excluded.
- Effort changes: Score differences (in points for percentage metrics) and cost ratios between adjacent efforts with values for the same model. Missing intermediate efforts are noted. When measured/reported by differs at the two ends, it is shown as “before → after”.
Terminal-Bench 4.0
official results (Accuracy vs Cost figure announced by Anthropic (Opus 5.5 and Sonnet 5.5 pages))
| Models on the frontier | effort | Score | Source cost (metric) | measured/reported by |
|---|---|---|---|---|
| Claude Sonnet 5.5 | Low | 20.0% | $0.76Cost per attempt | Anthropic |
| Claude Sonnet 5.5 | Med | 28.8% | $0.83Cost per attempt | Anthropic |
| Claude Opus 5.5 | Low | 38.5% | $1.29Cost per attempt | Anthropic |
| Claude Sonnet 5.5 | High | 43.0% | $1.94Cost per attempt | Anthropic |
| Claude Opus 5.5 | Med | 57.6% | $2.94Cost per attempt | Anthropic |
| Claude Opus 5.5 | High | 64.2% | $3.88Cost per attempt | Anthropic |
| Claude Opus 5.5 | Xhigh | 66.4% | $7.35Cost per attempt | Anthropic |
| Claude Sonnet 5.5 | Max | 70.6% | $12.54Cost per attempt | Anthropic |
| Model | effort | Score difference | Cost ratio | measured/reported by |
|---|---|---|---|---|
| Claude Opus 5.5 | Low → Med | +19.1pt | ×2.28 | Anthropic |
| Claude Opus 5.5 | Med → High | +6.6pt | ×1.32 | Anthropic |
| Claude Opus 5.5 | High → Xhigh | +2.2pt | ×1.89 | Anthropic |
| Claude Opus 5.5 | Xhigh → Max | -1.6pt | ×1.53 | Anthropic |
| Claude Sonnet 5.5 | Low → Med | +8.8pt | ×1.09 | Anthropic |
| Claude Sonnet 5.5 | Med → High | +14.2pt | ×2.34 | Anthropic |
| Claude Sonnet 5.5 | High → Xhigh | +18.5pt | ×2.73 | Anthropic |
| Claude Sonnet 5.5 | Xhigh → Max | +9.1pt | ×2.37 | Anthropic |
| Claude Fable 5.1 | Low → Med | +3.2pt | ×1.37 | Anthropic |
| Claude Fable 5.1 | Med → High | +6.0pt | ×1.35 | Anthropic |
| Claude Fable 5.1 | High → Xhigh | +1.9pt | ×1.5 | Anthropic |
| Claude Fable 5.1 | Xhigh → Max | +4.5pt | ×1.23 | Anthropic |
| Claude Opus 5 | Low → Med | +12.7pt | ×1.65 | Anthropic |
| Claude Opus 5 | Med → High | +5.8pt | ×1.52 | Anthropic |
| Claude Opus 5 | High → Xhigh | +3.6pt | ×1.27 | Anthropic |
| Claude Opus 5 | Xhigh → Max | +1.7pt | ×1.17 | Anthropic |
| Claude Sonnet 5 | Low → Med | +1.2pt | ×1.28 | Anthropic |
| Claude Sonnet 5 | Med → High | +0.1pt | ×1.7 | Anthropic |
| Claude Sonnet 5 | High → Xhigh | +2.5pt | ×1.21 | Anthropic |
| Claude Sonnet 5 | Xhigh → Max | +3.3pt | ×1.17 | Anthropic |
| GPT-6 Astra | Low → Med | +4.2pt | ×1.24 | OpenAI |
| GPT-6 Astra | Med → High | +4.0pt | ×1.17 | OpenAI |
| GPT-6 Astra | High → Xhigh | -0.3pt | ×1.04 | OpenAI |
| GPT-6 Astra | Xhigh → Max | -0.9pt | ×1.38 | OpenAI |
| GPT-5.6 Sol | Low → Med | +13.0pt | ×1.84 | OpenAI |
| GPT-5.6 Sol | Med → High | +5.2pt | ×1.53 | OpenAI |
| GPT-5.6 Sol | High → Xhigh | +2.4pt | ×1.31 | OpenAI |
| GPT-5.6 Sol | Xhigh → Max | +8.8pt | ×1.46 | OpenAI |
official results (Introducing Claude Haiku 5.5)
| Models on the frontier | effort | Score | Source cost (metric) | measured/reported by |
|---|---|---|---|---|
| Claude Haiku 5.5 | Low | 12.7% | $0.42Cost per attempt | Anthropic |
| Claude Sonnet 5.5 | Low | 20.0% | $0.62Cost per attempt | Anthropic |
| Claude Sonnet 5.5 | Med | 28.8% | $0.68Cost per attempt | Anthropic |
| Claude Sonnet 5.5 | High | 43.0% | $1.46Cost per attempt | Anthropic |
| Claude Sonnet 5.5 | Xhigh | 61.5% | $4.34Cost per attempt | Anthropic |
| Claude Sonnet 5.5 | Max | 70.6% | $10.44Cost per attempt | Anthropic |
Points without source costs are excluded from the frontier (excluded: 1).
| Model | effort | Score difference | Cost ratio | measured/reported by |
|---|---|---|---|---|
| Claude Sonnet 5.5 | Low → Med | +8.8pt | ×1.1 | Anthropic |
| Claude Sonnet 5.5 | Med → High | +14.2pt | ×2.15 | Anthropic |
| Claude Sonnet 5.5 | High → Xhigh | +18.5pt | ×2.97 | Anthropic |
| Claude Sonnet 5.5 | Xhigh → Max | +9.1pt | ×2.41 | Anthropic |
| Claude Haiku 5.5 | Low → Med | +7.6pt | ×1.62 | Anthropic |
| Claude Haiku 5.5 | Med → High | +4.5pt | ×1.53 | Anthropic |
| Claude Haiku 5.5 | High → Xhigh | +6.7pt | ×1.68 | Anthropic |
| Claude Haiku 5.5 | Xhigh → Max | +7.7pt | ×1.51 | Anthropic |
FrontierCode v1.1(main set)
official results (Accuracy vs Cost figure announced by Anthropic (Opus 5.5 and Sonnet 5.5 pages))
| Models on the frontier | effort | Score | Source cost (metric) | measured/reported by |
|---|---|---|---|---|
| Claude Sonnet 5.5 | Low | 29.3% | $0.19Cost per task | Anthropic |
| Claude Sonnet 5.5 | Med | 36.5% | $0.24Cost per task | Anthropic |
| Claude Opus 5.5 | Low | 47.3% | $0.40Cost per task | Anthropic |
| Claude Sonnet 5.5 | High | 49.4% | $0.42Cost per task | Anthropic |
| Claude Opus 5.5 | Med | 54.6% | $0.80Cost per task | Anthropic |
| Model | effort | Score difference | Cost ratio | measured/reported by |
|---|---|---|---|---|
| Claude Opus 5.5 | Low → Med | +7.3pt | ×2.0 | Anthropic |
| Claude Opus 5.5 | Med → High | -0.6pt | ×1.36 | Anthropic |
| Claude Opus 5.5 | High → Xhigh | -2.6pt | ×2.06 | Anthropic |
| Claude Opus 5.5 | Xhigh → Max | +3.0pt | ×2.75 | Anthropic |
| Claude Sonnet 5.5 | Low → Med | +7.2pt | ×1.26 | Anthropic |
| Claude Sonnet 5.5 | Med → High | +12.9pt | ×1.75 | Anthropic |
| Claude Sonnet 5.5 | High → Xhigh | +2.7pt | ×3.79 | Anthropic |
| Claude Sonnet 5.5 | Xhigh → Max | -5.9pt | ×13.07 | Anthropic |
| Claude Fable 5.1 | Low → Med | -1.9pt | ×1.33 | Anthropic |
| Claude Fable 5.1 | Med → High | -0.6pt | ×1.61 | Anthropic |
| Claude Fable 5.1 | High → Xhigh | -1.6pt | ×1.76 | Anthropic |
| Claude Fable 5.1 | Xhigh → Max | +1.6pt | ×1.38 | Anthropic |
| Claude Opus 5 | Low → Med | +11.4pt | ×1.75 | Anthropic |
| Claude Opus 5 | Med → High | -5.4pt | ×1.65 | Anthropic |
| Claude Opus 5 | High → Xhigh | -4.4pt | ×1.18 | Anthropic |
| Claude Opus 5 | Xhigh → Max | +4.4pt | ×1.37 | Anthropic |
| Claude Sonnet 5 | Low → Med | +6.5pt | ×1.59 | Anthropic |
| Claude Sonnet 5 | Med → High | +4.2pt | ×1.6 | Anthropic |
| Claude Sonnet 5 | High → Xhigh | +3.3pt | ×1.65 | Anthropic |
| Claude Sonnet 5 | Xhigh → Max | -0.3pt | ×1.7 | Anthropic |
| GPT-6 Astra | Low → Med | +3.5pt | ×1.43 | not stated |
| GPT-6 Astra | Med → High | +2.1pt | ×1.25 | not stated |
| GPT-6 Astra | High → Xhigh | -0.3pt | ×1.09 | not stated |
| GPT-6 Astra | Xhigh → Max | +2.7pt | ×1.41 | not stated |
| GPT-6 Sol | Low → Med | +8.6pt | ×1.79 | not stated |
| GPT-6 Sol | Med → High | +1.8pt | ×1.35 | not stated |
| GPT-6 Sol | High → Xhigh | +0.7pt | ×1.27 | not stated |
| GPT-6 Sol | Xhigh → Max | +0.9pt | ×1.57 | not stated |
| GPT-5.6 Sol | Low → Med | +4.5pt | ×1.43 | not stated |
| GPT-5.6 Sol | Med → High | +5.2pt | ×1.3 | not stated |
| GPT-5.6 Sol | High → Xhigh | +1.7pt | ×1.19 | not stated |
| GPT-5.6 Sol | Xhigh → Max | +0.7pt | ×1.25 | not stated |
FrontierCode 2.0
official results (Introducing Claude Fable 5.2 and Claude Mythos 5.2)
| Models on the frontier | effort | Score | Source cost (metric) | measured/reported by |
|---|---|---|---|---|
| Claude Opus 5.5 | Low | 14.7% | $0.75Cost per task | Anthropic |
| Claude Opus 5.5 | Med | 30.1% | $1.70Cost per task | Anthropic |
| Claude Fable 5.2 | Low | 34.2% | $2.10Cost per task | Anthropic |
| Claude Opus 5.5 | High | 34.5% | $2.42Cost per task | Anthropic |
| Claude Fable 5.2 | Med | 37.6% | $2.48Cost per task | Anthropic |
| GPT-6 Astra | High | 38.3% | $4.28Cost per task | not stated |
| GPT-6 Astra | Xhigh | 41.4% | $4.54Cost per task | not stated |
| Claude Opus 5.5 | Xhigh | 43.4% | $5.18Cost per task | Anthropic |
| Claude Fable 5.2 | High | 45.3% | $6.72Cost per task | Anthropic |
| Claude Opus 5.5 | Max | 45.5% | $12.82Cost per task | Anthropic |
| Claude Fable 5.2 | Xhigh | 49.1% | $14.19Cost per task | Anthropic |
| Model | effort | Score difference | Cost ratio | measured/reported by |
|---|---|---|---|---|
| Claude Fable 5.2 | Low → Med | +3.4pt | ×1.18 | Anthropic |
| Claude Fable 5.2 | Med → High | +7.7pt | ×2.71 | Anthropic |
| Claude Fable 5.2 | High → Xhigh | +3.8pt | ×2.11 | Anthropic |
| Claude Fable 5.2 | Xhigh → Max | -3.5pt | ×3.15 | Anthropic |
| Claude Opus 5.5 | Low → Med | +15.4pt | ×2.27 | Anthropic |
| Claude Opus 5.5 | Med → High | +4.4pt | ×1.42 | Anthropic |
| Claude Opus 5.5 | High → Xhigh | +8.9pt | ×2.14 | Anthropic |
| Claude Opus 5.5 | Xhigh → Max | +2.1pt | ×2.47 | Anthropic |
| Claude Fable 5.1 | Low → Med | +0.9pt | ×1.29 | Anthropic |
| Claude Fable 5.1 | Med → High | +4.2pt | ×1.39 | Anthropic |
| Claude Fable 5.1 | High → Xhigh | +6.7pt | ×1.61 | Anthropic |
| Claude Fable 5.1 | Xhigh → Max | -0.7pt | ×1.32 | Anthropic |
| GPT-6 Astra | Low → Med | +5.7pt | ×1.43 | not stated |
| GPT-6 Astra | Med → High | +1.5pt | ×1.23 | not stated |
| GPT-6 Astra | High → Xhigh | +3.1pt | ×1.06 | not stated |
| GPT-6 Astra | Xhigh → Max | +1.4pt | ×1.47 | not stated |
| Gemini 4 Carbon | Low → Med | +8.7pt | ×0.94 | not stated |
| Gemini 4 Carbon | Med → High | -2.7pt | ×1.13 | not stated |
CursorBench 4.0
official results (Introducing Claude Opus 5.5)
| Models on the frontier | effort | Score | Source cost (metric) | measured/reported by |
|---|---|---|---|---|
| GPT-5.6 Sol | Low | 24.6% | $0.87Cost per task | not stated |
| Claude Opus 5.5 | Low | 43.7% | $1.18Cost per task | Anthropic |
| Claude Opus 5.5 | Med | 52.5% | $2.90Cost per task | Anthropic |
| Claude Opus 5.5 | High | 56.0% | $3.97Cost per task | Anthropic |
| Claude Opus 5.5 | Xhigh | 56.0% | $6.99Cost per task | Anthropic |
| Claude Opus 5.5 | Max | 57.8% | $13.43Cost per task | Anthropic |
| Model | effort | Score difference | Cost ratio | measured/reported by |
|---|---|---|---|---|
| Claude Opus 5.5 | Low → Med | +8.8pt | ×2.46 | Anthropic |
| Claude Opus 5.5 | Med → High | +3.5pt | ×1.37 | Anthropic |
| Claude Opus 5.5 | High → Xhigh | ±0.0pt | ×1.76 | Anthropic |
| Claude Opus 5.5 | Xhigh → Max | +1.8pt | ×1.92 | Anthropic |
| Claude Fable 5.1 | Low → Med | +1.7pt | ×1.3 | Anthropic |
| Claude Fable 5.1 | Med → High | +2.4pt | ×1.29 | Anthropic |
| Claude Fable 5.1 | High → Xhigh | +2.4pt | ×1.43 | Anthropic |
| Claude Fable 5.1 | Xhigh → Max | +0.2pt | ×1.33 | Anthropic |
| Claude Opus 5 | Low → Med | +2.6pt | ×1.43 | Anthropic |
| Claude Opus 5 | Med → High | +1.4pt | ×1.3 | Anthropic |
| Claude Opus 5 | High → Xhigh | +1.4pt | ×1.27 | Anthropic |
| Claude Opus 5 | Xhigh → Max | +0.5pt | ×1.05 | Anthropic |
| GPT-5.6 Sol | Low → Med | +6.5pt | ×2.03 | not stated |
| GPT-5.6 Sol | Med → High | +4.6pt | ×1.61 | not stated |
| GPT-5.6 Sol | High → Xhigh | +2.0pt | ×1.54 | not stated |
| GPT-5.6 Sol | Xhigh → Max | +4.0pt | ×1.87 | not stated |
official results (Introducing Claude Sonnet 5.5)
| Models on the frontier | effort | Score | Source cost (metric) | measured/reported by |
|---|---|---|---|---|
| Claude Sonnet 5.5 | Low | 35.8% | $0.50Cost per task | Anthropic |
| Claude Sonnet 5.5 | Med | 39.2% | $0.70Cost per task | Anthropic |
| Claude Opus 5.5 | Low | 43.7% | $1.17Cost per task | Anthropic |
| Claude Sonnet 5.5 | High | 47.8% | $1.67Cost per task | Anthropic |
| Claude Opus 5.5 | Med | 52.5% | $2.91Cost per task | Anthropic |
| Claude Sonnet 5.5 | Xhigh | 53.1% | $3.88Cost per task | Anthropic |
| Claude Opus 5.5 | High | 56.0% | $3.97Cost per task | Anthropic |
| Claude Opus 5.5 | Xhigh | 56.0% | $6.98Cost per task | Anthropic |
| Claude Opus 5.5 | Max | 57.8% | $13.43Cost per task | Anthropic |
| Model | effort | Score difference | Cost ratio | measured/reported by |
|---|---|---|---|---|
| Claude Opus 5.5 | Low → Med | +8.8pt | ×2.49 | Anthropic |
| Claude Opus 5.5 | Med → High | +3.5pt | ×1.36 | Anthropic |
| Claude Opus 5.5 | High → Xhigh | ±0.0pt | ×1.76 | Anthropic |
| Claude Opus 5.5 | Xhigh → Max | +1.8pt | ×1.92 | Anthropic |
| Claude Sonnet 5.5 | Low → Med | +3.4pt | ×1.4 | Anthropic |
| Claude Sonnet 5.5 | Med → High | +8.6pt | ×2.39 | Anthropic |
| Claude Sonnet 5.5 | High → Xhigh | +5.3pt | ×2.32 | Anthropic |
| Claude Sonnet 5.5 | Xhigh → Max | +2.4pt | ×2.49 | Anthropic |
| Claude Sonnet 5 | Low → Med | +3.9pt | ×1.66 | Anthropic |
| Claude Sonnet 5 | Med → High | +2.8pt | ×1.51 | Anthropic |
| Claude Sonnet 5 | High → Xhigh | +1.2pt | ×1.31 | Anthropic |
| Claude Sonnet 5 | Xhigh → Max | +2.1pt | ×1.58 | Anthropic |
| GPT-5.6 Sol | Low → Med | +6.5pt | ×2.03 | not stated |
| GPT-5.6 Sol | Med → High | +4.6pt | ×1.61 | not stated |
| GPT-5.6 Sol | High → Xhigh | +2.0pt | ×1.54 | not stated |
| GPT-5.6 Sol | Xhigh → Max | +4.0pt | ×1.87 | not stated |
official results (Claude Fable 5.2 and Claude Mythos 5.2 System Card)
No frontier can be shown because no points have source costs.
| Model | effort | Score difference | Cost ratio | measured/reported by |
|---|---|---|---|---|
| Claude Fable 5.2 | Low → Med | +1.6pt | — | Cursor |
| Claude Fable 5.2 | Med → High | +2.1pt | — | Cursor |
| Claude Fable 5.2 | High → Xhigh | -0.6pt | — | Cursor |
| Claude Fable 5.2 | Xhigh → Max | +3.7pt | — | Cursor |
GDPval-AA v2.1
official results (Introducing Claude Opus 5.5)
| Models on the frontier | effort | Score | Source cost (metric) | measured/reported by |
|---|---|---|---|---|
| Claude Opus 5.5 | Low | 1,224 | $0.21Estimated cost per task | Anthropic |
| GPT-5.6 Sol | Low | 1,289 | $0.27Estimated cost per task | not stated |
| Claude Opus 5 | Low | 1,294 | $0.58Estimated cost per task | Anthropic |
| GPT-5.6 Sol | Med | 1,403 | $0.60Estimated cost per task | not stated |
| Claude Opus 5.5 | Med | 1,576 | $0.86Estimated cost per task | Anthropic |
| Claude Opus 5.5 | High | 1,692 | $1.54Estimated cost per task | Anthropic |
| Claude Opus 5.5 | Xhigh | 1,820 | $4.21Estimated cost per task | Anthropic |
| Claude Opus 5.5 | Max | 1,846 | $8.92Estimated cost per task | Anthropic |
| Model | effort | Score difference | Cost ratio | measured/reported by |
|---|---|---|---|---|
| Claude Opus 5.5 | Low → Med | +352 | ×4.1 | Anthropic |
| Claude Opus 5.5 | Med → High | +116 | ×1.79 | Anthropic |
| Claude Opus 5.5 | High → Xhigh | +128 | ×2.73 | Anthropic |
| Claude Opus 5.5 | Xhigh → Max | +26 | ×2.12 | Anthropic |
| Claude Fable 5.1 | Low → Med | +86 | ×1.54 | Anthropic |
| Claude Fable 5.1 | Med → High | +81 | ×1.58 | Anthropic |
| Claude Fable 5.1 | High → Xhigh | +104 | ×2.07 | Anthropic |
| Claude Fable 5.1 | Xhigh → Max | +14 | ×1.35 | Anthropic |
| Claude Opus 5 | Low → Med | +182 | ×2.34 | Anthropic |
| Claude Opus 5 | Med → High | +105 | ×2.23 | Anthropic |
| Claude Opus 5 | High → Xhigh | +95 | ×1.64 | Anthropic |
| Claude Opus 5 | Xhigh → Max | +32 | ×1.36 | Anthropic |
| GPT-6 Astra | Low → Med | +102 | ×2.14 | not stated |
| GPT-6 Astra | Med → High | +17 | ×1.34 | not stated |
| GPT-6 Astra | High → Xhigh | +31 | ×1.25 | not stated |
| GPT-6 Astra | Xhigh → Max | +26 | ×1.49 | not stated |
| GPT-5.6 Sol | Low → Med | +114 | ×2.22 | not stated |
| GPT-5.6 Sol | Med → High | +77 | ×1.85 | not stated |
| GPT-5.6 Sol | High → Xhigh | +68 | ×1.53 | not stated |
| GPT-5.6 Sol | Xhigh → Max | +40 | ×1.65 | not stated |
official results (Introducing Claude Haiku 5.5)
| Models on the frontier | effort | Score | Source cost (metric) | measured/reported by |
|---|---|---|---|---|
| GPT-6 Luna | Low | 1,036 | $0.004Cost per task | not stated |
| Claude Haiku 5.5 | Low | 1,125 | $0.01Cost per task | Anthropic |
| GPT-6 Luna | Med | 1,262 | $0.02Cost per task | not stated |
| GPT-6 Luna | High | 1,344 | $0.03Cost per task | not stated |
| GPT-6 Luna | Xhigh | 1,364 | $0.05Cost per task | not stated |
| GPT-6 Luna | Max | 1,437 | $0.09Cost per task | not stated |
| Claude Haiku 5.5 | Xhigh | 1,513 | $0.27Cost per task | Anthropic |
| Claude Sonnet 5.5 | High | 1,551 | $0.62Cost per task | Anthropic |
| Claude Haiku 5.5 | Max | 1,620 | $0.87Cost per task | Anthropic |
| Claude Sonnet 5.5 | Xhigh | 1,731 | $1.88Cost per task | Anthropic |
| Claude Sonnet 5.5 | Max | 1,840 | $6.78Cost per task | Anthropic |
| Model | effort | Score difference | Cost ratio | measured/reported by |
|---|---|---|---|---|
| Claude Sonnet 5.5 | Low → Med | +145 | ×1.23 | Anthropic |
| Claude Sonnet 5.5 | Med → High | +227 | ×2.3 | Anthropic |
| Claude Sonnet 5.5 | High → Xhigh | +180 | ×3.03 | Anthropic |
| Claude Sonnet 5.5 | Xhigh → Max | +109 | ×3.61 | Anthropic |
| Claude Haiku 5.5 | Low → Med | +152 | ×3.0 | Anthropic |
| Claude Haiku 5.5 | Med → High | +143 | ×3.0 | Anthropic |
| Claude Haiku 5.5 | High → Xhigh | +93 | ×3.0 | Anthropic |
| Claude Haiku 5.5 | Xhigh → Max | +107 | ×3.22 | Anthropic |
| GPT-6 Luna | Low → Med | +226 | ×5.0 | not stated |
| GPT-6 Luna | Med → High | +82 | ×1.5 | not stated |
| GPT-6 Luna | High → Xhigh | +20 | ×1.67 | not stated |
| GPT-6 Luna | Xhigh → Max | +73 | ×1.8 | not stated |
official results (Claude Fable 5.2 and Claude Mythos 5.2 System Card)
No frontier can be shown because no points have source costs.
| Model | effort | Score difference | Cost ratio | measured/reported by |
|---|---|---|---|---|
| Claude Fable 5.2 | Xhigh → Max | +80 | — | Artificial Analysis |
AA-Briefcase v1.1(Elo)
official results (Introducing Claude Sonnet 5.5)
| Models on the frontier | effort | Score | Source cost (metric) | measured/reported by |
|---|---|---|---|---|
| GPT-6 Sol | Low | 905 | $0.12Cost per task | not stated |
| GPT-6 Sol | Med | 1,142 | $0.34Cost per task | not stated |
| GPT-6 Sol | High | 1,289 | $0.63Cost per task | not stated |
| GPT-6 Sol | Xhigh | 1,364 | $1.19Cost per task | not stated |
| Claude Sonnet 5.5 | Med | 1,461 | $1.64Cost per task | Anthropic |
| GPT-6 Sol | Max | 1,483 | $2.67Cost per task | not stated |
| Claude Sonnet 5.5 | High | 1,634 | $3.95Cost per task | Anthropic |
| Claude Opus 5.5 | Med | 1,642 | $4.40Cost per task | Anthropic |
| Claude Opus 5.5 | High | 1,705 | $6.27Cost per task | Anthropic |
| Claude Sonnet 5.5 | Xhigh | 1,746 | $9.63Cost per task | Anthropic |
| Claude Opus 5.5 | Xhigh | 1,780 | $12.27Cost per task | Anthropic |
| Claude Opus 5.5 | Max | 1,822 | $21.05Cost per task | Anthropic |
| Model | effort | Score difference | Cost ratio | measured/reported by |
|---|---|---|---|---|
| Claude Opus 5.5 | Low → Med | +357 | ×3.83 | Anthropic |
| Claude Opus 5.5 | Med → High | +63 | ×1.42 | Anthropic |
| Claude Opus 5.5 | High → Xhigh | +75 | ×1.96 | Anthropic |
| Claude Opus 5.5 | Xhigh → Max | +42 | ×1.72 | Anthropic |
| Claude Sonnet 5.5 | Low → Med | +197 | ×1.89 | Anthropic |
| Claude Sonnet 5.5 | Med → High | +173 | ×2.41 | Anthropic |
| Claude Sonnet 5.5 | High → Xhigh | +112 | ×2.44 | Anthropic |
| Claude Sonnet 5.5 | Xhigh → Max | +65 | ×3.03 | Anthropic |
| Claude Sonnet 5 | Low → Med | +133 | ×2.11 | Anthropic |
| Claude Sonnet 5 | Med → High | +121 | ×2.17 | Anthropic |
| Claude Sonnet 5 | High → Xhigh | +97 | ×2.01 | Anthropic |
| Claude Sonnet 5 | Xhigh → Max | +85 | ×1.91 | Anthropic |
| GPT-6 Sol | Low → Med | +237 | ×2.83 | not stated |
| GPT-6 Sol | Med → High | +147 | ×1.85 | not stated |
| GPT-6 Sol | High → Xhigh | +75 | ×1.89 | not stated |
| GPT-6 Sol | Xhigh → Max | +119 | ×2.24 | not stated |
AutomationBench
official results (Introducing Claude Opus 5.5)
| Models on the frontier | effort | Score | Source cost (metric) | measured/reported by |
|---|---|---|---|---|
| GPT-5.6 Sol | Low | 11.7% | $0.42Cost per task | Zapier |
| Claude Opus 5.5 | Low | 23.3% | $0.50Cost per task | Zapier |
| Claude Opus 5.5 | Med | 28.6% | $0.64Cost per task | Zapier |
| Claude Opus 5.5 | High | 32.0% | $0.70Cost per task | Zapier |
| Claude Opus 5.5 | Xhigh | 34.4% | $0.86Cost per task | Zapier |
| Claude Opus 5.5 | Max | 40.0% | $1.37Cost per task | Zapier |
| GPT-6 Astra | Max | 41.4% | $1.77Cost per task | Zapier |
Points without source costs are excluded from the frontier (excluded: 1).
| Model | effort | Score difference | Cost ratio | measured/reported by |
|---|---|---|---|---|
| Claude Opus 5.5 | Low → Med | +5.3pt | ×1.28 | Zapier |
| Claude Opus 5.5 | Med → High | +3.4pt | ×1.09 | Zapier |
| Claude Opus 5.5 | High → Xhigh | +2.4pt | ×1.23 | Zapier |
| Claude Opus 5.5 | Xhigh → Max | +5.6pt | ×1.59 | Zapier |
| Claude Opus 5 | Low → Med | +3.5pt | ×1.19 | Zapier |
| Claude Opus 5 | Med → High | -3.3pt | ×1.16 | Zapier |
| Claude Opus 5 | High → Xhigh | +4.7pt | ×1.12 | Zapier |
| Claude Opus 5 | Xhigh → Max | +1.6pt | ×1.1 | Zapier |
| GPT-6 Astra | Low → Med | +3.8pt | ×1.19 | Zapier |
| GPT-6 Astra | Med → High | +3.0pt | ×1.13 | Zapier |
| GPT-6 Astra | High → Xhigh | +1.9pt | ×1.06 | Zapier |
| GPT-6 Astra | Xhigh → Max | +2.4pt | ×1.16 | Zapier |
| GPT-5.6 Sol | Low → Med | +7.9pt | ×1.36 | Zapier |
| GPT-5.6 Sol | Med → High | +5.2pt | ×1.12 | Zapier |
| GPT-5.6 Sol | High → Xhigh | +1.5pt | ×1.16 | Zapier |
| GPT-5.6 Sol | Xhigh → Max | +2.5pt | ×1.23 | Zapier |
Humanity's Last Exam (with tools)
official results (Introducing Claude Fable 5.2 and Claude Mythos 5.2)
| Models on the frontier | effort | Score | Source cost (metric) | measured/reported by |
|---|---|---|---|---|
| Claude Opus 5.5 | Low | 57.4% | $0.08Cost per attempt | Anthropic |
| Claude Opus 5.5 | Med | 63.0% | $0.17Cost per attempt | Anthropic |
| Claude Fable 5.2 | Low | 68.4% | $0.19Cost per attempt | Anthropic |
| Claude Fable 5.2 | Med | 68.6% | $0.21Cost per attempt | Anthropic |
| Claude Fable 5.2 | High | 71.5% | $0.63Cost per attempt | Anthropic |
| Claude Fable 5.2 | Xhigh | 73.4% | $1.48Cost per attempt | Anthropic |
| Claude Fable 5.2 | Max | 74.2% | $4.51Cost per attempt | Anthropic |
Points without source costs are excluded from the frontier (excluded: 1).
| Model | effort | Score difference | Cost ratio | measured/reported by |
|---|---|---|---|---|
| Claude Fable 5.2 | Low → Med | +0.2pt | ×1.11 | Anthropic |
| Claude Fable 5.2 | Med → High | +2.9pt | ×3.0 | Anthropic |
| Claude Fable 5.2 | High → Xhigh | +1.9pt | ×2.35 | Anthropic |
| Claude Fable 5.2 | Xhigh → Max | +0.8pt | ×3.05 | Anthropic |
| Claude Opus 5.5 | Low → Med | +5.6pt | ×2.13 | Anthropic |
| Claude Opus 5.5 | Med → High | +0.9pt | ×1.71 | Anthropic |
| Claude Opus 5.5 | High → Xhigh | +2.5pt | ×2.38 | Anthropic |
| Claude Opus 5.5 | Xhigh → Max | +1.3pt | ×3.16 | Anthropic |
| Claude Fable 5.1 | Low → Med | +2.2pt | ×1.44 | Anthropic |
| Claude Fable 5.1 | Med → High | +0.7pt | ×1.61 | Anthropic |
| Claude Fable 5.1 | High → Xhigh | +2.1pt | ×1.97 | Anthropic |
| Claude Fable 5.1 | Xhigh → Max | +0.4pt | ×1.4 | Anthropic |
Humanity's Last Exam (without tools)
official results (Introducing Claude Haiku 5.5)
| Models on the frontier | effort | Score | Source cost (metric) | measured/reported by |
|---|---|---|---|---|
| Claude Haiku 5.5 | Low | 30.9% | $0.003Cost per attempt | Anthropic |
| Claude Haiku 5.5 | Med | 35.5% | $0.007Cost per attempt | Anthropic |
| Claude Haiku 5.5 | High | 39.7% | $0.02Cost per attempt | Anthropic |
| Claude Sonnet 5.5 | Med | 43.2% | $0.03Cost per attempt | Anthropic |
| Claude Haiku 5.5 | Xhigh | 44.1% | $0.06Cost per attempt | Anthropic |
| Claude Sonnet 5.5 | High | 47.9% | $0.08Cost per attempt | Anthropic |
| Claude Sonnet 5.5 | Xhigh | 53.0% | $0.20Cost per attempt | Anthropic |
| Claude Sonnet 5.5 | Max | 56.9% | $0.89Cost per attempt | Anthropic |
| Model | effort | Score difference | Cost ratio | measured/reported by |
|---|---|---|---|---|
| Claude Sonnet 5.5 | Low → Med | +1.8pt | ×1.0 | Anthropic |
| Claude Sonnet 5.5 | Med → High | +4.7pt | ×2.67 | Anthropic |
| Claude Sonnet 5.5 | High → Xhigh | +5.1pt | ×2.5 | Anthropic |
| Claude Sonnet 5.5 | Xhigh → Max | +3.9pt | ×4.45 | Anthropic |
| Claude Haiku 5.5 | Low → Med | +4.6pt | ×2.33 | Anthropic |
| Claude Haiku 5.5 | Med → High | +4.2pt | ×2.86 | Anthropic |
| Claude Haiku 5.5 | High → Xhigh | +4.4pt | ×3.0 | Anthropic |
| Claude Haiku 5.5 | Xhigh → Max | +1.8pt | ×3.5 | Anthropic |
OSWorld 2.1(offline subset)
official results (Introducing Claude Haiku 5.5)
| Models on the frontier | effort | Score | Source cost (metric) | measured/reported by |
|---|---|---|---|---|
| GPT-6 Luna | Low | 19.2% | $0.04Cost per attempt | not stated |
| Claude Haiku 5.5 | Low | 42.0% | $0.07Cost per attempt | Anthropic |
| Claude Haiku 5.5 | Med | 53.3% | $0.13Cost per attempt | Anthropic |
| Claude Haiku 5.5 | High | 61.3% | $0.18Cost per attempt | Anthropic |
| Claude Haiku 5.5 | Xhigh | 67.6% | $0.28Cost per attempt | Anthropic |
| Claude Haiku 5.5 | Max | 72.4% | $0.61Cost per attempt | Anthropic |
| Claude Sonnet 5.5 | High | 73.2% | $1.38Cost per attempt | Anthropic |
| Claude Sonnet 5.5 | Xhigh | 81.1% | $2.22Cost per attempt | Anthropic |
| Claude Sonnet 5.5 | Max | 83.9% | $5.73Cost per attempt | Anthropic |
| Model | effort | Score difference | Cost ratio | measured/reported by |
|---|---|---|---|---|
| Claude Sonnet 5.5 | Low → Med | +8.1pt | ×1.37 | Anthropic |
| Claude Sonnet 5.5 | Med → High | +7.2pt | ×1.48 | Anthropic |
| Claude Sonnet 5.5 | High → Xhigh | +7.9pt | ×1.61 | Anthropic |
| Claude Sonnet 5.5 | Xhigh → Max | +2.8pt | ×2.58 | Anthropic |
| Claude Haiku 5.5 | Low → Med | +11.3pt | ×1.86 | Anthropic |
| Claude Haiku 5.5 | Med → High | +8.0pt | ×1.38 | Anthropic |
| Claude Haiku 5.5 | High → Xhigh | +6.3pt | ×1.56 | Anthropic |
| Claude Haiku 5.5 | Xhigh → Max | +4.8pt | ×2.18 | Anthropic |
| GPT-6 Luna | Low → Med | +18.3pt | ×3.25 | not stated |
| GPT-6 Luna | Med → High | +4.8pt | ×1.08 | not stated |
| GPT-6 Luna | High → Xhigh | +2.5pt | ×1.21 | not stated |
| GPT-6 Luna | Xhigh → Max | +4.1pt | ×1.18 | not stated |
Chartography (without tools)
official results (Introducing Claude Fable 5.2 and Claude Mythos 5.2)
| Models on the frontier | effort | Score | Source cost (metric) | measured/reported by |
|---|---|---|---|---|
| Claude Opus 5.5 | Low | 57.0% | $0.02Cost per attempt | Anthropic |
| Claude Fable 5.2 | Med | 66.4% | $0.03Cost per attempt | Anthropic |
| Claude Fable 5.2 | High | 69.8% | $0.07Cost per attempt | Anthropic |
| Claude Fable 5.2 | Xhigh | 71.4% | $0.15Cost per attempt | Anthropic |
| Claude Fable 5.2 | Max | 73.4% | $0.75Cost per attempt | Anthropic |
Points without source costs are excluded from the frontier (excluded: 2).
| Model | effort | Score difference | Cost ratio | measured/reported by |
|---|---|---|---|---|
| Claude Fable 5.2 | Low → Med | +2.2pt | ×1.0 | Anthropic |
| Claude Fable 5.2 | Med → High | +3.4pt | ×2.33 | Anthropic |
| Claude Fable 5.2 | High → Xhigh | +1.6pt | ×2.14 | Anthropic |
| Claude Fable 5.2 | Xhigh → Max | +2.0pt | ×5.0 | Anthropic |
| Claude Opus 5.5 | Low → Med | +2.0pt | ×1.5 | Anthropic |
| Claude Opus 5.5 | Med → High | +0.2pt | ×1.0 | Anthropic |
| Claude Opus 5.5 | High → Xhigh | +3.2pt | ×2.67 | Anthropic |
| Claude Opus 5.5 | Xhigh → Max | +2.0pt | ×6.13 | Anthropic |
| Claude Fable 5.1 | Low → Med | +5.6pt | ×1.67 | Anthropic |
| Claude Fable 5.1 | Med → High | +3.8pt | ×1.6 | Anthropic |
| Claude Fable 5.1 | High → Xhigh | +1.0pt | ×4.75 | Anthropic |
| Claude Fable 5.1 | Xhigh → Max | -2.2pt | ×2.29 | Anthropic |
WANDR
official results (Introducing Claude Opus 5.5)
| Models on the frontier | effort | Score | Source cost (metric) | measured/reported by |
|---|---|---|---|---|
| Claude Opus 5.5 | Low | 31.2% | $1.20Cost per attempt | Anthropic |
| Claude Opus 5 | Low | 50.5% | $10.65Cost per attempt | Anthropic |
| Claude Opus 5.5 | Med | 62.8% | $11.20Cost per attempt | Anthropic |
| Claude Opus 5.5 | High | 67.3% | $16.88Cost per attempt | Anthropic |
| Claude Opus 5.5 | Xhigh | 71.3% | $29.06Cost per attempt | Anthropic |
| Claude Opus 5.5 | Max | 72.3% | $37.92Cost per attempt | Anthropic |
| Model | effort | Score difference | Cost ratio | measured/reported by |
|---|---|---|---|---|
| Claude Opus 5.5 | Low → Med | +31.6pt | ×9.33 | Anthropic |
| Claude Opus 5.5 | Med → High | +4.5pt | ×1.51 | Anthropic |
| Claude Opus 5.5 | High → Xhigh | +4.0pt | ×1.72 | Anthropic |
| Claude Opus 5.5 | Xhigh → Max | +1.0pt | ×1.3 | Anthropic |
| Claude Fable 5.1 | Low → Med | +1.2pt | ×1.18 | Anthropic |
| Claude Fable 5.1 | Med → High | +2.2pt | ×1.23 | Anthropic |
| Claude Fable 5.1 | High → Xhigh | +1.0pt | ×1.27 | Anthropic |
| Claude Fable 5.1 | Xhigh → Max | +1.0pt | ×1.15 | Anthropic |
| Claude Opus 5 | Low → Med | +7.6pt | ×2.25 | Anthropic |
| Claude Opus 5 | Med → High | +6.5pt | ×1.81 | Anthropic |
| Claude Opus 5 | High → Xhigh | +2.4pt | ×1.24 | Anthropic |
| Claude Opus 5 | Xhigh → Max | +0.2pt | ×1.14 | Anthropic |
WANDR(relaxed score)
official results (Introducing Claude Fable 5.2 and Claude Mythos 5.2)
| Models on the frontier | effort | Score | Source cost (metric) | measured/reported by |
|---|---|---|---|---|
| Claude Opus 5.5 | Low | 31.2% | $1.25Cost per attempt | Anthropic |
| Claude Opus 5.5 | Med | 62.8% | $11.40Cost per attempt | Anthropic |
| Claude Fable 5.2 | Low | 71.4% | $12.66Cost per attempt | Anthropic |
| Claude Fable 5.2 | Med | 73.6% | $15.71Cost per attempt | Anthropic |
| Claude Fable 5.2 | High | 79.3% | $29.20Cost per attempt | Anthropic |
| Claude Fable 5.2 | Xhigh | 80.6% | $39.84Cost per attempt | Anthropic |
| Claude Fable 5.2 | Max | 82.1% | $53.11Cost per attempt | Anthropic |
| Model | effort | Score difference | Cost ratio | measured/reported by |
|---|---|---|---|---|
| Claude Fable 5.2 | Low → Med | +2.2pt | ×1.24 | Anthropic |
| Claude Fable 5.2 | Med → High | +5.7pt | ×1.86 | Anthropic |
| Claude Fable 5.2 | High → Xhigh | +1.3pt | ×1.36 | Anthropic |
| Claude Fable 5.2 | Xhigh → Max | +1.5pt | ×1.33 | Anthropic |
| Claude Opus 5.5 | Low → Med | +31.6pt | ×9.12 | Anthropic |
| Claude Opus 5.5 | Med → High | +4.5pt | ×1.52 | Anthropic |
| Claude Opus 5.5 | High → Xhigh | +4.0pt | ×1.69 | Anthropic |
| Claude Opus 5.5 | Xhigh → Max | +1.0pt | ×1.32 | Anthropic |
| Claude Fable 5.1 | Low → Med | +1.2pt | ×1.18 | Anthropic |
| Claude Fable 5.1 | Med → High | +2.2pt | ×1.31 | Anthropic |
| Claude Fable 5.1 | High → Xhigh | +1.0pt | ×1.18 | Anthropic |
| Claude Fable 5.1 | Xhigh → Max | +1.0pt | ×1.15 | Anthropic |
BenchCAD 2.1 (with tools)
official results (Introducing Claude Fable 5.2 and Claude Mythos 5.2)
| Models on the frontier | effort | Score | Source cost (metric) | measured/reported by |
|---|---|---|---|---|
| Claude Opus 5.5 | Low | 6.3 | $0.03Cost per attempt | Anthropic |
| Claude Opus 5.5 | Med | 19.1 | $1.11Cost per attempt | Anthropic |
| Claude Fable 5.2 | Low | 26.9 | $1.89Cost per attempt | Anthropic |
| Claude Opus 5.5 | High | 29.7 | $2.57Cost per attempt | Anthropic |
| Claude Fable 5.2 | Med | 32.4 | $2.66Cost per attempt | Anthropic |
| Claude Opus 5.5 | Xhigh | 54.5 | $8.91Cost per attempt | Anthropic |
| Claude Fable 5.2 | High | 61 | $9.99Cost per attempt | Anthropic |
| Claude Opus 5.5 | Max | 65.3 | $19.86Cost per attempt | Anthropic |
| Claude Fable 5.2 | Xhigh | 74.1 | $23.23Cost per attempt | Anthropic |
| Claude Fable 5.2 | Max | 83.5 | $50.77Cost per attempt | Anthropic |
| Model | effort | Score difference | Cost ratio | measured/reported by |
|---|---|---|---|---|
| Claude Fable 5.2 | Low → Med | +5.5 | ×1.41 | Anthropic |
| Claude Fable 5.2 | Med → High | +28.6 | ×3.76 | Anthropic |
| Claude Fable 5.2 | High → Xhigh | +13.1 | ×2.33 | Anthropic |
| Claude Fable 5.2 | Xhigh → Max | +9.4 | ×2.19 | Anthropic |
| Claude Opus 5.5 | Low → Med | +12.8 | ×37.0 | Anthropic |
| Claude Opus 5.5 | Med → High | +10.6 | ×2.32 | Anthropic |
| Claude Opus 5.5 | High → Xhigh | +24.8 | ×3.47 | Anthropic |
| Claude Opus 5.5 | Xhigh → Max | +10.8 | ×2.23 | Anthropic |
| Claude Fable 5.1 | Low → Med | +3.6 | ×1.93 | Anthropic |
| Claude Fable 5.1 | Med → High | +10.1 | ×2.18 | Anthropic |
| Claude Fable 5.1 | High → Xhigh | +14.7 | ×2.2 | Anthropic |
| Claude Fable 5.1 | Xhigh → Max | +3.9 | ×1.27 | Anthropic |
CVE Rediscovery
official results (Introducing Claude Fable 5.2 and Claude Mythos 5.2)
| Models on the frontier | effort | Score | Source cost (metric) | measured/reported by |
|---|---|---|---|---|
| Claude Opus 5.5 | Low | 59.4% | $0.11Cost per attempt | Anthropic |
| Claude Opus 5.5 | Med | 66.1% | $0.28Cost per attempt | Anthropic |
| Claude Mythos 5.2 | Low | 72.4% | $0.34Cost per attempt | Anthropic |
| Claude Mythos 5.2 | Med | 73.3% | $0.40Cost per attempt | Anthropic |
| Claude Mythos 5.2 | High | 83.2% | $1.01Cost per attempt | Anthropic |
| Claude Mythos 5.2 | Xhigh | 88.1% | $2.40Cost per attempt | Anthropic |
| Claude Mythos 5.2 | Max | 89.6% | $9.13Cost per attempt | Anthropic |
| Model | effort | Score difference | Cost ratio | measured/reported by |
|---|---|---|---|---|
| Claude Mythos 5.2 | Low → Med | +0.9pt | ×1.18 | Anthropic |
| Claude Mythos 5.2 | Med → High | +9.9pt | ×2.53 | Anthropic |
| Claude Mythos 5.2 | High → Xhigh | +4.9pt | ×2.38 | Anthropic |
| Claude Mythos 5.2 | Xhigh → Max | +1.5pt | ×3.8 | Anthropic |
| Claude Opus 5.5 | Low → Med | +6.7pt | ×2.55 | Anthropic |
| Claude Opus 5.5 | Med → High | +3.9pt | ×1.54 | Anthropic |
| Claude Opus 5.5 | High → Xhigh | +6.4pt | ×2.44 | Anthropic |
| Claude Opus 5.5 | Xhigh → Max | +4.9pt | ×2.84 | Anthropic |
| Claude Mythos 5.1 | Low → Med | +3.9pt | ×1.51 | Anthropic |
| Claude Mythos 5.1 | Med → High | +4.0pt | ×1.56 | Anthropic |
| Claude Mythos 5.1 | High → Xhigh | +4.0pt | ×2.02 | Anthropic |
| Claude Mythos 5.1 | Xhigh → Max | +0.5pt | ×1.47 | Anthropic |
GPQA Diamond
third-party results (Epoch AI, Capabilities & benchmarking (benchmark_data.zip, gpqa_diamond.csv) CC BY 4.0)
No frontier can be shown because no points have source costs.
| Model | effort | Score difference | Cost ratio | measured/reported by |
|---|---|---|---|---|
| Claude Fable 5 | Low → HighNo value for Med | +4.5pt | — | Epoch AI |
| Claude Fable 5 | High → MaxNo value for Xhigh | +2.5pt | — | Epoch AI |
| Claude Opus 4.8 | Low → MaxNo value for Med · High · Xhigh | +2.7pt | — | Epoch AI |
| Claude Opus 4.7 | Xhigh → Max | -3.8pt | — | Epoch AI |
| Claude Sonnet 4.6 | Med → High | ±0.0pt | — | Epoch AI |
| Claude Sonnet 4.6 | High → Max | -4.5pt | — | Epoch AI |
| GPT-5.6 Sol | None → Low | +7.1pt | — | Epoch AI |
| GPT-5.6 Sol | Low → MaxNo value for Med · High · Xhigh | +3.6pt | — | Epoch AI |
| GPT-5.6 Terra | None → Low | +10.1pt | — | Epoch AI |
| GPT-5.6 Terra | Low → MaxNo value for Med · High · Xhigh | +5.9pt | — | Epoch AI |
| GPT-5.6 Luna | None → Low | +18.7pt | — | Epoch AI |
| GPT-5.6 Luna | Low → MaxNo value for Med · High · Xhigh | +9.3pt | — | Epoch AI |
| GPT-5.5 | None → Low | +13.4pt | — | Epoch AI |
| Gemini 3.6 Flash | Minimal → Low | +0.5pt | — | Epoch AI |
| Gemini 3.6 Flash | Low → HighNo value for Med | +7.8pt | — | Epoch AI |
| Gemini 3.5 Flash-Lite | Minimal → Low | +1.5pt | — | Epoch AI |
| Gemini 3.5 Flash-Lite | Low → HighNo value for Med | +7.6pt | — | Epoch AI |
| Gemini 3.1 Flash-Lite | Minimal → Low | +0.5pt | — | Epoch AI |
| Gemini 3.1 Flash-Lite | Low → HighNo value for Med | +7.6pt | — | Epoch AI |
| Grok 4.6 | High → Xhigh | -0.8pt | — | Epoch AI |
| Kimi K3 | Low → High | +7.1pt | — | Epoch AI |
| Kimi K3 | High → Max | +1.2pt | — | Epoch AI |
FrontierMath Tiers 1-3(v2)
third-party results (Epoch AI, Capabilities & benchmarking (benchmark_data.zip, frontiermath_tiers_1_3_v2.csv) CC BY 4.0)
No frontier can be shown because no points have source costs.
| Model | effort | Score difference | Cost ratio | measured/reported by |
|---|---|---|---|---|
| GPT-5.6 Luna | None → Low | +1.8pt | — | Epoch AI |
| GPT-5.6 Luna | Low → MaxNo value for Med · High · Xhigh | +40.7pt | — | Epoch AI |
| Gemini 3.1 Flash-Lite | Minimal → Low | +1.1pt | — | Epoch AI |
| Gemini 3.1 Flash-Lite | Low → HighNo value for Med | +5.3pt | — | Epoch AI |
FrontierMath Tier 4(v2)
third-party results (Epoch AI, Capabilities & benchmarking (benchmark_data.zip, frontiermath_tier_4_v2.csv) CC BY 4.0)
No frontier can be shown because no points have source costs.
| Model | effort | Score difference | Cost ratio | measured/reported by |
|---|---|---|---|---|
| GPT-6 Astra | Low → Med | +9.8pt | — | Epoch AI |
| GPT-6 Astra | Med → High | ±0.0pt | — | Epoch AI |
| GPT-6 Astra | High → Xhigh | ±0.0pt | — | Epoch AI |
| GPT-6 Astra | Xhigh → Max | ±0.0pt | — | Epoch AI |
OTIS Mock AIME 2024-2025
third-party results (Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0)
No frontier can be shown because no points have source costs.
| Model | effort | Score difference | Cost ratio | measured/reported by |
|---|---|---|---|---|
| Claude Haiku 5.5 | High → Xhigh | +1.7pt | — | Epoch AI |
| Claude Fable 5 | Low → HighNo value for Med | +2.2pt | — | Epoch AI |
| Claude Fable 5 | High → MaxNo value for Xhigh | -0.3pt | — | Epoch AI |
| Claude Opus 4.8 | Low → MaxNo value for Med · High · Xhigh | +0.5pt | — | Epoch AI |
| Claude Opus 4.7 | Xhigh → Max | -11.1pt | — | Epoch AI |
| Claude Sonnet 4.6 | Med → High | -6.7pt | — | Epoch AI |
| Claude Sonnet 4.6 | High → Max | -4.5pt | — | Epoch AI |
| GPT-5.6 Sol | None → Low | +26.7pt | — | Epoch AI |
| GPT-5.6 Sol | Low → MaxNo value for Med · High · Xhigh | +4.4pt | — | Epoch AI |
| GPT-5.6 Terra | None → Low | +35.6pt | — | Epoch AI |
| GPT-5.6 Terra | Low → MaxNo value for Med · High · Xhigh | +10.8pt | — | Epoch AI |
| GPT-5.6 Luna | None → Low | +26.7pt | — | Epoch AI |
| GPT-5.6 Luna | Low → MaxNo value for Med · High · Xhigh | +31.7pt | — | Epoch AI |
| GPT-5.5 | None → Low | +26.7pt | — | Epoch AI |
| Gemini 3.6 Flash | Minimal → Low | +2.2pt | — | Epoch AI |
| Gemini 3.6 Flash | Low → HighNo value for Med | +12.0pt | — | Epoch AI |
| Gemini 3.5 Flash-Lite | Minimal → Low | +8.9pt | — | Epoch AI |
| Gemini 3.5 Flash-Lite | Low → HighNo value for Med | +11.1pt | — | Epoch AI |
| Gemini 3.1 Flash-Lite | Minimal → Low | +6.7pt | — | Epoch AI |
| Gemini 3.1 Flash-Lite | Low → HighNo value for Med | +35.6pt | — | Epoch AI |
| Grok 4.6 | High → Xhigh | +1.4pt | — | Epoch AI |
| Kimi K3 | Low → High | +24.4pt | — | Epoch AI |
| Kimi K3 | High → Max | +3.9pt | — | Epoch AI |
SimpleQA Verified
third-party results (Epoch AI, Capabilities & benchmarking (benchmark_data.zip, simpleqa_verified.csv) CC BY 4.0)
No frontier can be shown because no points have source costs.
| Model | effort | Score difference | Cost ratio | measured/reported by |
|---|---|---|---|---|
| Claude Sonnet 4.6 | High → Max | -2.7pt | — | Epoch AI |
| Grok 4.6 | High → Xhigh | -0.4pt | — | Epoch AI |
Chess Puzzles(Epoch AI)
third-party results (Epoch AI, Capabilities & benchmarking (benchmark_data.zip, chess_puzzles.csv) CC BY 4.0)
No frontier can be shown because no points have source costs.
| Model | effort | Score difference | Cost ratio | measured/reported by |
|---|---|---|---|---|
| Claude Haiku 5.5 | Low → MaxNo value for Med · High · Xhigh | +31.0pt | — | Epoch AI |
| Claude Fable 5 | Low → HighNo value for Med | +14.0pt | — | Epoch AI |
| Claude Fable 5 | High → MaxNo value for Xhigh | +2.0pt | — | Epoch AI |
| Claude Opus 4.8 | Low → MaxNo value for Med · High · Xhigh | +23.0pt | — | Epoch AI |
| Claude Opus 4.7 | Low → XhighNo value for Med · High | +10.0pt | — | Epoch AI |
| Claude Opus 4.7 | Xhigh → Max | +5.0pt | — | Epoch AI |
| Claude Sonnet 4.6 | Med → High | +3.0pt | — | Epoch AI |
| Claude Sonnet 4.6 | High → Max | +4.0pt | — | Epoch AI |
| GPT-5.6 Sol | None → Low | +24.0pt | — | Epoch AI |
| GPT-5.6 Sol | Low → MaxNo value for Med · High · Xhigh | +28.0pt | — | Epoch AI |
| GPT-5.6 Terra | None → Low | +22.0pt | — | Epoch AI |
| GPT-5.6 Terra | Low → MaxNo value for Med · High · Xhigh | +20.0pt | — | Epoch AI |
| GPT-5.6 Luna | None → Low | +14.0pt | — | Epoch AI |
| GPT-5.6 Luna | Low → MaxNo value for Med · High · Xhigh | +26.0pt | — | Epoch AI |
| GPT-5.5 | None → Low | +21.0pt | — | Epoch AI |
| GPT-5.5 | Low → XhighNo value for Med · High | +33.0pt | — | Epoch AI |
| Gemini 3.6 Flash | Minimal → Low | +13.0pt | — | Epoch AI |
| Gemini 3.6 Flash | Low → HighNo value for Med | -2.0pt | — | Epoch AI |
| Gemini 3.5 Flash-Lite | Minimal → Low | +14.0pt | — | Epoch AI |
| Gemini 3.5 Flash-Lite | Low → HighNo value for Med | -5.0pt | — | Epoch AI |
| Grok 4.6 | High → Xhigh | ±0.0pt | — | Epoch AI |
| Kimi K3 | Low → High | +19.0pt | — | Epoch AI |
| Kimi K3 | High → Max | -1.0pt | — | Epoch AI |
MirrorCode
third-party results (Epoch AI, Capabilities & benchmarking (benchmark_data.zip, mirrorcode.csv) CC BY 4.0)
No frontier can be shown because no points have source costs.
| Model | effort | Score difference | Cost ratio | measured/reported by |
|---|---|---|---|---|
| Claude Opus 5.5 | High → MaxNo value for Xhigh | +16.7pt | — | Epoch AI |
| Claude Fable 5 | High → MaxNo value for Xhigh | +6.1pt | — | Epoch AI |
| GPT-6 Astra | High → MaxNo value for Xhigh | +13.3pt | — | Epoch AI |
| GPT-5.5 | High → Xhigh | +13.3pt | — | Epoch AI |
τ²-bench airline (v1.0.1)
third-party results (τ²-bench v1.0.1 (run by Sierra))
| Models on the frontier | effort | Score | Source cost (metric) | measured/reported by |
|---|---|---|---|---|
| GPT-5.2 | None | 52.5% | $0.054Average cost in USD to run one trajectory in this domain | Sierra |
| GPT-5.2 | High | 83.0% | $0.11Average cost in USD to run one trajectory in this domain | Sierra |
| Claude Opus 4.5 | High | 84.0% | $0.40Average cost in USD to run one trajectory in this domain | Sierra |
Points without source costs are excluded from the frontier (excluded: 4).
| Model | effort | Score difference | Cost ratio | measured/reported by |
|---|---|---|---|---|
| GPT-5.2 | None → HighNo value for Low · Med | +30.5pt | ×2.12 | Sierra |
τ²-bench banking_knowledge (v1.0.1)
third-party results (τ²-bench v1.0.1 banking_knowledge (run by Sierra, search alltools))
| Models on the frontier | effort | Score | Source cost (metric) | measured/reported by |
|---|---|---|---|---|
| GPT-5.4 | Xhigh | 39.4% | $1.84Average cost in USD to run one trajectory in this domain | Sierra |
Points without source costs are excluded from the frontier (excluded: 15).
third-party results (τ²-bench v1.0.1 banking_knowledge (run by Sierra, search terminal))
| Models on the frontier | effort | Score | Source cost (metric) | measured/reported by |
|---|---|---|---|---|
| Claude Sonnet 4.5 | not stated | 25.3% | $4.05Average cost in USD to run one trajectory in this domain | Sierra |
Points without source costs are excluded from the frontier (excluded: 2).
third-party results (τ²-bench v1.0.1 banking_knowledge (run by Sierra, search qwen_embeddings))
| Models on the frontier | effort | Score | Source cost (metric) | measured/reported by |
|---|---|---|---|---|
| GPT-5.2 | None | 12.6% | $0.24Average cost in USD to run one trajectory in this domain | Sierra |
τ²-bench retail (v1.0.1)
third-party results (τ²-bench v1.0.1 (run by Sierra))
| Models on the frontier | effort | Score | Source cost (metric) | measured/reported by |
|---|---|---|---|---|
| GPT-5.2 | None | 75.0% | $0.048Average cost in USD to run one trajectory in this domain | Sierra |
| GPT-5.2 | High | 81.6% | $0.072Average cost in USD to run one trajectory in this domain | Sierra |
Points without source costs are excluded from the frontier (excluded: 4).
| Model | effort | Score difference | Cost ratio | measured/reported by |
|---|---|---|---|---|
| GPT-5.2 | None → HighNo value for Low · Med | +6.6pt | ×1.49 | Sierra |
τ²-bench telecom (v1.0.1)
third-party results (τ²-bench v1.0.1 (run by Sierra))
| Models on the frontier | effort | Score | Source cost (metric) | measured/reported by |
|---|---|---|---|---|
| GPT-5.2 | None | 57.2% | $0.049Average cost in USD to run one trajectory in this domain | Sierra |
| GPT-5.2 | High | 89.7% | $0.093Average cost in USD to run one trajectory in this domain | Sierra |
| Claude Opus 4.5 | High | 92.3% | $0.72Average cost in USD to run one trajectory in this domain | Sierra |
Points without source costs are excluded from the frontier (excluded: 4).
| Model | effort | Score difference | Cost ratio | measured/reported by |
|---|---|---|---|---|
| GPT-5.2 | None → HighNo value for Low · Med | +32.4pt | ×1.88 | Sierra |
By metric
Benchmarks are ordered by category and ranked within each comparison group (the same source, or sources verified to use the same measurement conditions). “measured/reported by” identifies the party stated in the source as measuring or reporting the value. Even in official announcements, the publisher may have measured values for other companies’ models.
Agentic codingTerminal-Bench 4.0
official results (Accuracy vs Cost figure announced by Anthropic (Opus 5.5 and Sonnet 5.5 pages))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Claude Sonnet 5.5 | Max | 70.6% | $12.54Cost per attempt | frontier | Anthropic | Introducing Claude Sonnet 5.5Retrieved 2026-11-12 |
| 2 | Claude Opus 5.5 | Xhigh | 66.4% | $7.35Cost per attempt | frontier | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 3 | Claude Opus 5.5 | Max | 64.8% | $11.24Cost per attempt | — | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 4 | Claude Opus 5.5 | High | 64.2% | $3.88Cost per attempt | frontier | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 5 | Claude Sonnet 5.5 | Xhigh | 61.5% | $5.30Cost per attempt | — | Anthropic | Introducing Claude Sonnet 5.5Retrieved 2026-11-12 |
| 6 | GPT-6 Astra | High | 57.9% | $7.21Cost per attempt | — | OpenAI | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 7 | Claude Opus 5.5 | Med | 57.6% | $2.94Cost per attempt | frontier | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 7 | GPT-6 Astra | Xhigh | 57.6% | $7.48Cost per attempt | — | OpenAI | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 9 | GPT-6 Astra | Max | 56.7% | $10.35Cost per attempt | — | OpenAI | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 10 | Claude Fable 5.1 | Max | 55.8% | $19.50Cost per attempt | — | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 11 | GPT-6 Astra | Med | 53.9% | $6.15Cost per attempt | — | OpenAI | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 12 | Claude Opus 5 | Max | 52.3% | $15.83Cost per attempt | — | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 13 | Claude Fable 5.1 | Xhigh | 51.3% | $15.80Cost per attempt | — | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 14 | Claude Opus 5 | Xhigh | 50.6% | $13.48Cost per attempt | — | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 15 | GPT-6 Astra | Low | 49.7% | $4.95Cost per attempt | — | OpenAI | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 16 | Claude Fable 5.1 | High | 49.4% | $10.50Cost per attempt | — | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 17 | Claude Opus 5 | High | 47.0% | $10.64Cost per attempt | — | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 18 | Claude Fable 5.1 | Med | 43.4% | $7.80Cost per attempt | — | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 19 | Claude Sonnet 5.5 | High | 43.0% | $1.94Cost per attempt | frontier | Anthropic | Introducing Claude Sonnet 5.5Retrieved 2026-11-12 |
| 20 | Claude Opus 5 | Med | 41.2% | $7.00Cost per attempt | — | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 21 | Claude Fable 5.1 | Low | 40.2% | $5.70Cost per attempt | — | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 22 | Claude Opus 5.5 | Low | 38.5% | $1.29Cost per attempt | frontier | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 23 | GPT-5.6 Sol | Max | 37.3% | $7.89Cost per attempt | — | OpenAI | Introducing Claude Sonnet 5.5Retrieved 2026-11-12 |
| 24 | Claude Sonnet 5.5 | Med | 28.8% | $0.83Cost per attempt | frontier | Anthropic | Introducing Claude Sonnet 5.5Retrieved 2026-11-12 |
| 25 | Claude Opus 5 | Low | 28.5% | $4.25Cost per attempt | — | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 25 | GPT-5.6 Sol | Xhigh | 28.5% | $5.39Cost per attempt | — | OpenAI | Introducing Claude Sonnet 5.5Retrieved 2026-11-12 |
| 27 | GPT-5.6 Sol | High | 26.1% | $4.12Cost per attempt | — | OpenAI | Introducing Claude Sonnet 5.5Retrieved 2026-11-12 |
| 28 | GPT-5.6 Sol | Med | 20.9% | $2.69Cost per attempt | — | OpenAI | Introducing Claude Sonnet 5.5Retrieved 2026-11-12 |
| 29 | Claude Sonnet 5.5 | Low | 20.0% | $0.76Cost per attempt | frontier | Anthropic | Introducing Claude Sonnet 5.5Retrieved 2026-11-12 |
| 30 | Claude Sonnet 5 | Max | 10.3% | $11.62Cost per attempt | — | Anthropic | Introducing Claude Sonnet 5.5Retrieved 2026-11-12 |
| 31 | GPT-5.6 Sol | Low | 7.9% | $1.46Cost per attempt | — | OpenAI | Introducing Claude Sonnet 5.5Retrieved 2026-11-12 |
| 32 | Claude Sonnet 5 | Xhigh | 7.0% | $9.95Cost per attempt | — | Anthropic | Introducing Claude Sonnet 5.5Retrieved 2026-11-12 |
| 33 | Claude Sonnet 5 | High | 4.5% | $8.20Cost per attempt | — | Anthropic | Introducing Claude Sonnet 5.5Retrieved 2026-11-12 |
| 34 | Claude Sonnet 5 | Med | 4.4% | $4.83Cost per attempt | — | Anthropic | Introducing Claude Sonnet 5.5Retrieved 2026-11-12 |
| 35 | Claude Sonnet 5 | Low | 3.2% | $3.78Cost per attempt | — | Anthropic | Introducing Claude Sonnet 5.5Retrieved 2026-11-12 |
official results (Introducing Claude Haiku 5.5)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Claude Sonnet 5.5 | Max | 70.6% | $10.44Cost per attempt | frontier | Anthropic | Introducing Claude Haiku 5.5Retrieved 2026-11-12 |
| 2 | Claude Sonnet 5.5 | Xhigh | 61.5% | $4.34Cost per attempt | frontier | Anthropic | Introducing Claude Haiku 5.5Retrieved 2026-11-12 |
| 3 | Claude Sonnet 5.5 | High | 43.0% | $1.46Cost per attempt | frontier | Anthropic | Introducing Claude Haiku 5.5Retrieved 2026-11-12 |
| 4 | Claude Haiku 5.5 | Max | 39.2% | $2.64Cost per attempt | — | Anthropic | Introducing Claude Haiku 5.5Retrieved 2026-11-12 |
| 5 | Claude Haiku 5.5 | Xhigh | 31.5% | $1.75Cost per attempt | — | Anthropic | Introducing Claude Haiku 5.5Retrieved 2026-11-12 |
| 6 | Claude Sonnet 5.5 | Med | 28.8% | $0.68Cost per attempt | frontier | Anthropic | Introducing Claude Haiku 5.5Retrieved 2026-11-12 |
| 7 | Claude Haiku 5.5 | High | 24.8% | $1.04Cost per attempt | — | Anthropic | Introducing Claude Haiku 5.5Retrieved 2026-11-12 |
| 8 | Claude Haiku 5.5 | Med | 20.3% | $0.68Cost per attempt | — | Anthropic | Introducing Claude Haiku 5.5Retrieved 2026-11-12 |
| 9 | Claude Sonnet 5.5 | Low | 20.0% | $0.62Cost per attempt | frontier | Anthropic | Introducing Claude Haiku 5.5Retrieved 2026-11-12 |
| 10 | GPT-6 Luna | not stated | 16.4% | — | — | not stated | Introducing Claude Haiku 5.5Retrieved 2026-11-12 |
| 11 | Claude Haiku 5.5 | Low | 12.7% | $0.42Cost per attempt | frontier | Anthropic | Introducing Claude Haiku 5.5Retrieved 2026-11-12 |
| 12 | Claude Haiku 4.5 | not stated | 0.0% | $0.79Cost per attempt | — | Anthropic | Introducing Claude Haiku 5.5Retrieved 2026-11-12 |
official results (Gemini 4 Carbon (Google DeepMind models page, benchmark table))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Gemini 4 Carbon | High | 65.2% | — | — | Artificial Analysis | Gemini 4 Carbon (Google DeepMind models page, benchmark table)Retrieved 2026-11-12 |
| 2 | Gemini 4 Argon | High | 57.1% | — | — | Artificial Analysis | Gemini 4 Carbon (Google DeepMind models page, benchmark table)Retrieved 2026-11-12 |
official results (Gemini 4 Argon model card)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Gemini 4 Argon | not stated | 57.4% | — | — | Gemini 4 Argon model cardRetrieved 2026-11-12 |
official results (Gemini 3.8 Flash model card)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Gemini 3.8 Flash | not stated | 19.1% | — | — | Gemini 3.8 Flash model cardRetrieved 2026-11-12 |
official results (Grok 4.7 (xAI news))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Grok 4.7 | Xhigh | 37.6% | — | — | xAI | Grok 4.7 (xAI news)Retrieved 2026-11-12 |
| 2 | Grok 4.6 | High | 20.3% | — | — | xAI | Grok 4.7 (xAI news)Retrieved 2026-11-12 |
official results (Introducing Kimi K3.1 Pro)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Kimi K3.1 Pro | Max | 65.7% | — | — | Moonshot AI | Introducing Kimi K3.1 ProRetrieved 2026-11-12 |
official results (Introducing Mistral Large 4)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Mistral Large 4 | not stated | 28.3% | — | — | Mistral AI | Introducing Mistral Large 4Retrieved 2026-11-12 |
official results (DeepSeek-V4.1-Flash (Hugging Face model card, deepseek-ai))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | DeepSeek-V4.1-Flash | not stated | 31.2% | — | — | DeepSeek | DeepSeek-V4.1-Flash (Hugging Face model card, deepseek-ai)Retrieved 2026-11-13 |
official results (DeepSeek-V4.1-Flash-1017 (Hugging Face model card, deepseek-ai))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | DeepSeek-V4.1-Flash-1017 | not stated | 42.3% | — | — | DeepSeek | DeepSeek-V4.1-Flash-1017 (Hugging Face model card, deepseek-ai)Retrieved 2026-11-13 |
official results (Claude Fable 5.2 and Claude Mythos 5.2 System Card)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Claude Fable 5.2 | Max | 67.6% | — | — | Anthropic | Claude Fable 5.2 and Claude Mythos 5.2 System CardRetrieved 2026-11-14 |
Agentic codingTerminal-Bench 3.0
official results (GLM-5.3 (Z.ai docs))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | GLM-5.3 | not stated | 28.3% | — | — | Z.ai | GLM-5.3 (Z.ai docs)Retrieved 2026-11-12 |
official results (Gemini 3.7 Flash model card)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Gemini 3.7 Flash | not stated | 14.9% | — | — | Gemini 3.7 Flash model cardRetrieved 2026-11-13 | |
| 2 | Gemini 3.6 Flash | not stated | 5.4% | — | — | Gemini 3.7 Flash model cardRetrieved 2026-11-13 |
official results (DeepSeek-V4.1-Flash (Hugging Face model card, deepseek-ai))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | DeepSeek-V4.1-Flash | not stated | 30.0% | — | — | DeepSeek | DeepSeek-V4.1-Flash (Hugging Face model card, deepseek-ai)Retrieved 2026-11-13 |
Agentic codingTerminal-Bench 2.1
official results (Gemini 3.8 Flash model card)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Gemini 3.8 Flash | not stated | 89.4% | — | — | Gemini 3.8 Flash model cardRetrieved 2026-11-12 |
official results (Kimi K3 (Hugging Face model card, moonshotai))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Kimi K3 | Max | 88.3% | — | — | Moonshot AI | Kimi K3 (Hugging Face model card, moonshotai)Retrieved 2026-11-13 |
official results (GLM-5.3 (Hugging Face model card, zai-org))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | GLM-5.3 | not stated | 88.2% | — | — | Z.ai | GLM-5.3 (Hugging Face model card, zai-org)Retrieved 2026-11-13 |
official results (Gemini 3.7 Flash model card)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Gemini 3.7 Flash | not stated | 85.8% | — | — | Gemini 3.7 Flash model cardRetrieved 2026-11-13 | |
| 2 | Gemini 3.6 Flash | not stated | 78.0% | — | — | Gemini 3.7 Flash model cardRetrieved 2026-11-13 |
official results (Gemini 3.5 Flash model card)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Gemini 3.5 Flash | not stated | 76.2% | — | — | Gemini 3.5 Flash model cardRetrieved 2026-11-13 | |
| 2 | Gemini 3 Flash Preview | not stated | 58.0% | — | — | Gemini 3.5 Flash model cardRetrieved 2026-11-13 |
official results (Qwen3.8-2.4T-A95B (Hugging Face model card, Qwen))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Qwen3.8-Max | not stated | 86.6% | — | — | Alibaba (Qwen) | Qwen3.8-2.4T-A95B (Hugging Face model card, Qwen)Retrieved 2026-11-13 |
official results (DeepSeek-V4.1-Flash (Hugging Face model card, deepseek-ai))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | DeepSeek-V4.1-Flash | not stated | 90.6% | — | — | DeepSeek | DeepSeek-V4.1-Flash (Hugging Face model card, deepseek-ai)Retrieved 2026-11-13 |
Agentic codingTerminal-Bench-Science 0.1
official results (Introducing Claude Opus 5.5)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | GPT-6 Astra | not stated | 64.6% | — | — | OpenAI | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 2 | Claude Opus 5.5 | Max | 58.7% | — | — | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 3 | Claude Fable 5.1 | not stated | 52.6% | — | — | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 4 | Claude Opus 5 | not stated | 29.0% | — | — | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 5 | GPT-5.6 Sol | not stated | 22.4% | — | — | not stated | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
official results (Introducing Claude Fable 5.2 and Claude Mythos 5.2)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Claude Mythos 5.2 | not stated | 75.3% | — | — | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 2 | Claude Fable 5.2 | not stated | 74.6% | — | — | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 3 | GPT-6 Astra | not stated | 68.1% | — | — | not stated | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 4 | Gemini 4 Carbon | not stated | 64.8% | — | — | not stated | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 5 | Claude Opus 5.5 | not stated | 58.7% | — | — | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 6 | Claude Fable 5.1 | not stated | 52.6% | — | — | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
official results (Gemini 4 Carbon (Google DeepMind models page, benchmark table))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Gemini 4 Carbon | High | 64.8% | — | — | Artificial Analysis | Gemini 4 Carbon (Google DeepMind models page, benchmark table)Retrieved 2026-11-12 |
| 2 | Gemini 4 Argon | High | 52.9% | — | — | Artificial Analysis | Gemini 4 Carbon (Google DeepMind models page, benchmark table)Retrieved 2026-11-12 |
official results (Kimi K3.1 Pro (Hugging Face model card, moonshotai))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Kimi K3.1 Pro | Max | 38.1% | — | — | Moonshot AI | Kimi K3.1 Pro (Hugging Face model card, moonshotai)Retrieved 2026-11-13 |
official results (Claude Sonnet 5.5 System Card)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Claude Sonnet 5.5 | Max | 59.9% | — | — | Anthropic | Claude Sonnet 5.5 System CardRetrieved 2026-11-14 |
official results (Claude Haiku 5.5 System Card)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | GPT-6.1 Sol | Max | 57.0% | — | — | OpenAI | Claude Haiku 5.5 System CardRetrieved 2026-11-14 |
| 2 | Claude Haiku 5.5 | Max | 20.6% | — | — | Anthropic | Claude Haiku 5.5 System CardRetrieved 2026-11-14 |
Agentic codingFrontierCode v1.1(main set)
official results (Accuracy vs Cost figure announced by Anthropic (Opus 5.5 and Sonnet 5.5 pages))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Claude Opus 5.5 | Med | 54.6% | $0.80Cost per task | frontier | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 2 | Claude Opus 5.5 | Max | 54.4% | $6.19Cost per task | — | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 3 | Claude Opus 5.5 | High | 54.0% | $1.09Cost per task | — | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 4 | Claude Opus 5 | Med | 53.4% | $4.61Cost per task | — | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 5 | GPT-6 Astra | Max | 53.3% | $4.36Cost per task | — | not stated | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 6 | Claude Fable 5.1 | Low | 52.8% | $2.47Cost per task | — | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 7 | Claude Sonnet 5.5 | Xhigh | 52.1% | $1.59Cost per task | — | Anthropic | Introducing Claude Sonnet 5.5Retrieved 2026-11-12 |
| 8 | Claude Opus 5.5 | Xhigh | 51.4% | $2.25Cost per task | — | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 9 | Claude Fable 5.1 | Med | 50.9% | $3.28Cost per task | — | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 9 | GPT-6 Astra | High | 50.9% | $2.85Cost per task | — | not stated | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 11 | GPT-6 Astra | Xhigh | 50.6% | $3.10Cost per task | — | not stated | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 12 | Claude Fable 5.1 | High | 50.3% | $5.27Cost per task | — | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 12 | Claude Fable 5.1 | Max | 50.3% | $12.82Cost per task | — | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 14 | Claude Sonnet 5.5 | High | 49.4% | $0.42Cost per task | frontier | Anthropic | Introducing Claude Sonnet 5.5Retrieved 2026-11-12 |
| 15 | GPT-6 Sol | Max | 49.3% | $2.07Cost per task | — | not stated | Introducing Claude Sonnet 5.5Retrieved 2026-11-12 |
| 16 | GPT-6 Astra | Med | 48.8% | $2.28Cost per task | — | not stated | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 17 | Claude Fable 5.1 | Xhigh | 48.7% | $9.27Cost per task | — | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 18 | GPT-6 Sol | Xhigh | 48.4% | $1.32Cost per task | — | not stated | Introducing Claude Sonnet 5.5Retrieved 2026-11-12 |
| 19 | Claude Opus 5 | High | 48.0% | $7.62Cost per task | — | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 19 | Claude Opus 5 | Max | 48.0% | $12.28Cost per task | — | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 21 | GPT-6 Sol | High | 47.7% | $1.04Cost per task | — | not stated | Introducing Claude Sonnet 5.5Retrieved 2026-11-12 |
| 22 | GPT-5.6 Sol | Max | 47.5% | $4.85Cost per task | — | not stated | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 23 | Claude Opus 5.5 | Low | 47.3% | $0.40Cost per task | frontier | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 24 | GPT-5.6 Sol | Xhigh | 46.8% | $3.88Cost per task | — | not stated | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 25 | Claude Sonnet 5.5 | Max | 46.2% | $20.78Cost per task | — | Anthropic | Introducing Claude Sonnet 5.5Retrieved 2026-11-12 |
| 26 | GPT-6 Sol | Med | 45.9% | $0.77Cost per task | — | not stated | Introducing Claude Sonnet 5.5Retrieved 2026-11-12 |
| 27 | GPT-6 Astra | Low | 45.3% | $1.59Cost per task | — | not stated | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 28 | GPT-5.6 Sol | High | 45.1% | $3.25Cost per task | — | not stated | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 29 | Claude Opus 5 | Xhigh | 43.6% | $8.99Cost per task | — | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 30 | Claude Sonnet 5 | Xhigh | 42.7% | $10.07Cost per task | — | Anthropic | Introducing Claude Sonnet 5.5Retrieved 2026-11-12 |
| 31 | Claude Sonnet 5 | Max | 42.4% | $17.12Cost per task | — | Anthropic | Introducing Claude Sonnet 5.5Retrieved 2026-11-12 |
| 32 | Claude Opus 5 | Low | 42.0% | $2.64Cost per task | — | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 33 | GPT-5.6 Sol | Med | 39.9% | $2.50Cost per task | — | not stated | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 34 | Claude Sonnet 5 | High | 39.4% | $6.10Cost per task | — | Anthropic | Introducing Claude Sonnet 5.5Retrieved 2026-11-12 |
| 35 | GPT-6 Sol | Low | 37.3% | $0.43Cost per task | — | not stated | Introducing Claude Sonnet 5.5Retrieved 2026-11-12 |
| 36 | Claude Sonnet 5.5 | Med | 36.5% | $0.24Cost per task | frontier | Anthropic | Introducing Claude Sonnet 5.5Retrieved 2026-11-12 |
| 37 | GPT-5.6 Sol | Low | 35.4% | $1.75Cost per task | — | not stated | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 38 | Claude Sonnet 5 | Med | 35.2% | $3.81Cost per task | — | Anthropic | Introducing Claude Sonnet 5.5Retrieved 2026-11-12 |
| 39 | Claude Sonnet 5.5 | Low | 29.3% | $0.19Cost per task | frontier | Anthropic | Introducing Claude Sonnet 5.5Retrieved 2026-11-12 |
| 40 | Claude Sonnet 5 | Low | 28.7% | $2.39Cost per task | — | Anthropic | Introducing Claude Sonnet 5.5Retrieved 2026-11-12 |
official results (Introducing Claude Haiku 5.5)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Claude Sonnet 5.5 | Xhigh | 52.1% | — | — | Anthropic | Introducing Claude Haiku 5.5Retrieved 2026-11-12 |
| 2 | Claude Haiku 5.5 | not stated | 46.4% | — | — | Anthropic | Introducing Claude Haiku 5.5Retrieved 2026-11-12 |
| 3 | GPT-6 Luna | not stated | 42.4% | — | — | not stated | Introducing Claude Haiku 5.5Retrieved 2026-11-12 |
Agentic codingFrontierCode 2.0
official results (Introducing Claude Fable 5.2 and Claude Mythos 5.2)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Claude Fable 5.2 | Xhigh | 49.1% | $14.19Cost per task | frontier | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 2 | Claude Fable 5.2 | Max | 45.6% | $44.70Cost per task | — | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 3 | Claude Opus 5.5 | Max | 45.5% | $12.82Cost per task | frontier | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 4 | Claude Fable 5.2 | High | 45.3% | $6.72Cost per task | frontier | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 5 | Claude Opus 5.5 | Xhigh | 43.4% | $5.18Cost per task | frontier | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 6 | GPT-6 Astra | Max | 42.8% | $6.68Cost per task | — | not stated | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 7 | GPT-6 Astra | Xhigh | 41.4% | $4.54Cost per task | frontier | not stated | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 8 | Gemini 4 Carbon | Med | 41.1% | $5.16Cost per task | — | not stated | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 9 | Gemini 4 Carbon | High | 38.4% | $5.83Cost per task | — | not stated | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 10 | GPT-6 Astra | High | 38.3% | $4.28Cost per task | frontier | not stated | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 11 | Claude Fable 5.2 | Med | 37.6% | $2.48Cost per task | frontier | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 12 | GPT-6 Astra | Med | 36.8% | $3.49Cost per task | — | not stated | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 13 | Claude Opus 5.5 | High | 34.5% | $2.42Cost per task | frontier | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 14 | Claude Fable 5.2 | Low | 34.2% | $2.10Cost per task | frontier | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 15 | Claude Fable 5.1 | Xhigh | 32.4% | $8.68Cost per task | — | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 15 | Gemini 4 Carbon | Low | 32.4% | $5.49Cost per task | — | not stated | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 17 | Claude Fable 5.1 | Max | 31.7% | $11.46Cost per task | — | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 18 | GPT-6 Astra | Low | 31.1% | $2.44Cost per task | — | not stated | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 19 | Claude Opus 5.5 | Med | 30.1% | $1.70Cost per task | frontier | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 20 | Claude Fable 5.1 | High | 25.7% | $5.39Cost per task | — | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 21 | Claude Fable 5.1 | Med | 21.5% | $3.89Cost per task | — | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 22 | Claude Fable 5.1 | Low | 20.6% | $3.01Cost per task | — | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 23 | Claude Opus 5.5 | Low | 14.7% | $0.75Cost per task | frontier | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
Agentic codingFrontierSWE 2
official results (Introducing Claude Fable 5.2 and Claude Mythos 5.2)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Claude Fable 5.2 | not stated | 69.9% | — | — | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 2 | GPT-6 Astra | not stated | 65.5% | — | — | not stated | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 3 | Claude Opus 5.5 | not stated | 62.3% | — | — | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 4 | Gemini 4 Carbon | not stated | 61.2% | — | — | not stated | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 5 | Claude Fable 5.1 | not stated | 56.3% | — | — | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
official results (Gemini 4 Carbon (Google DeepMind models page, benchmark table))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Gemini 4 Carbon | High | 61.2% | — | — | Proximal | Gemini 4 Carbon (Google DeepMind models page, benchmark table)Retrieved 2026-11-12 |
| 2 | Gemini 4 Argon | High | 55.0% | — | — | Proximal | Gemini 4 Carbon (Google DeepMind models page, benchmark table)Retrieved 2026-11-12 |
official results (Claude Sonnet 5.5 System Card)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Claude Sonnet 5.5 | Max | 61.9% | — | — | Proximal | Claude Sonnet 5.5 System CardRetrieved 2026-11-14 |
official results (Claude Haiku 5.5 System Card)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Claude Haiku 5.5 | Max | 43.8% | — | — | Proximal | Claude Haiku 5.5 System CardRetrieved 2026-11-14 |
Agentic codingCursorBench 4.0
official results (Introducing Claude Opus 5.5)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Claude Opus 5.5 | Max | 57.8% | $13.43Cost per task | frontier | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 2 | Claude Opus 5.5 | High | 56.0% | $3.97Cost per task | frontier | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 2 | Claude Opus 5.5 | Xhigh | 56.0% | $6.99Cost per task | frontier | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 4 | Claude Opus 5.5 | Med | 52.5% | $2.90Cost per task | frontier | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 5 | Claude Fable 5.1 | Max | 51.8% | $17.28Cost per task | — | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 6 | Claude Fable 5.1 | Xhigh | 51.6% | $13.01Cost per task | — | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 7 | Claude Fable 5.1 | High | 49.2% | $9.08Cost per task | — | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 8 | Claude Fable 5.1 | Med | 46.8% | $7.05Cost per task | — | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 9 | Claude Opus 5 | Max | 46.6% | $11.95Cost per task | — | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 10 | Claude Opus 5 | Xhigh | 46.1% | $11.43Cost per task | — | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 11 | Claude Fable 5.1 | Low | 45.1% | $5.44Cost per task | — | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 12 | Claude Opus 5 | High | 44.7% | $9.00Cost per task | — | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 13 | Claude Opus 5.5 | Low | 43.7% | $1.18Cost per task | frontier | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 14 | Claude Opus 5 | Med | 43.3% | $6.94Cost per task | — | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 15 | GPT-5.6 Sol | Max | 41.7% | $8.23Cost per task | — | not stated | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 16 | Claude Opus 5 | Low | 40.7% | $4.87Cost per task | — | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 17 | GPT-5.6 Sol | Xhigh | 37.7% | $4.40Cost per task | — | not stated | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 18 | GPT-5.6 Sol | High | 35.7% | $2.85Cost per task | — | not stated | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 19 | GPT-5.6 Sol | Med | 31.1% | $1.77Cost per task | — | not stated | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 20 | GPT-5.6 Sol | Low | 24.6% | $0.87Cost per task | frontier | not stated | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
official results (Introducing Claude Sonnet 5.5)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Claude Opus 5.5 | Max | 57.8% | $13.43Cost per task | frontier | Anthropic | Introducing Claude Sonnet 5.5Retrieved 2026-11-12 |
| 2 | Claude Opus 5.5 | High | 56.0% | $3.97Cost per task | frontier | Anthropic | Introducing Claude Sonnet 5.5Retrieved 2026-11-12 |
| 2 | Claude Opus 5.5 | Xhigh | 56.0% | $6.98Cost per task | frontier | Anthropic | Introducing Claude Sonnet 5.5Retrieved 2026-11-12 |
| 4 | Claude Sonnet 5.5 | Max | 55.5% | $9.67Cost per task | — | Anthropic | Introducing Claude Sonnet 5.5Retrieved 2026-11-12 |
| 5 | Claude Sonnet 5.5 | Xhigh | 53.1% | $3.88Cost per task | frontier | Anthropic | Introducing Claude Sonnet 5.5Retrieved 2026-11-12 |
| 6 | Claude Opus 5.5 | Med | 52.5% | $2.91Cost per task | frontier | Anthropic | Introducing Claude Sonnet 5.5Retrieved 2026-11-12 |
| 7 | Claude Sonnet 5.5 | High | 47.8% | $1.67Cost per task | frontier | Anthropic | Introducing Claude Sonnet 5.5Retrieved 2026-11-12 |
| 8 | Claude Opus 5.5 | Low | 43.7% | $1.17Cost per task | frontier | Anthropic | Introducing Claude Sonnet 5.5Retrieved 2026-11-12 |
| 9 | GPT-5.6 Sol | Max | 41.7% | $8.23Cost per task | — | not stated | Introducing Claude Sonnet 5.5Retrieved 2026-11-12 |
| 10 | Claude Sonnet 5.5 | Med | 39.2% | $0.70Cost per task | frontier | Anthropic | Introducing Claude Sonnet 5.5Retrieved 2026-11-12 |
| 11 | GPT-5.6 Sol | Xhigh | 37.7% | $4.40Cost per task | — | not stated | Introducing Claude Sonnet 5.5Retrieved 2026-11-12 |
| 12 | Claude Sonnet 5.5 | Low | 35.8% | $0.50Cost per task | frontier | Anthropic | Introducing Claude Sonnet 5.5Retrieved 2026-11-12 |
| 13 | GPT-5.6 Sol | High | 35.7% | $2.85Cost per task | — | not stated | Introducing Claude Sonnet 5.5Retrieved 2026-11-12 |
| 14 | Claude Sonnet 5 | Max | 34.1% | $7.17Cost per task | — | Anthropic | Introducing Claude Sonnet 5.5Retrieved 2026-11-12 |
| 15 | Claude Sonnet 5 | Xhigh | 32.0% | $4.55Cost per task | — | Anthropic | Introducing Claude Sonnet 5.5Retrieved 2026-11-12 |
| 16 | GPT-5.6 Sol | Med | 31.1% | $1.77Cost per task | — | not stated | Introducing Claude Sonnet 5.5Retrieved 2026-11-12 |
| 17 | Claude Sonnet 5 | High | 30.8% | $3.48Cost per task | — | Anthropic | Introducing Claude Sonnet 5.5Retrieved 2026-11-12 |
| 18 | Claude Sonnet 5 | Med | 28.0% | $2.31Cost per task | — | Anthropic | Introducing Claude Sonnet 5.5Retrieved 2026-11-12 |
| 19 | GPT-5.6 Sol | Low | 24.6% | $0.87Cost per task | — | not stated | Introducing Claude Sonnet 5.5Retrieved 2026-11-12 |
| 20 | Claude Sonnet 5 | Low | 24.1% | $1.39Cost per task | — | Anthropic | Introducing Claude Sonnet 5.5Retrieved 2026-11-12 |
official results (Grok 4.7 (xAI news))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Grok 4.7 | Xhigh | 46.3% | — | — | xAI | Grok 4.7 (xAI news)Retrieved 2026-11-12 |
| 2 | Grok 4.6 | High | 40.4% | — | — | xAI | Grok 4.7 (xAI news)Retrieved 2026-11-12 |
official results (Claude Fable 5.2 and Claude Mythos 5.2 System Card)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Claude Fable 5.2 | Max | 60.0% | — | — | Cursor | Claude Fable 5.2 and Claude Mythos 5.2 System CardRetrieved 2026-11-14 |
| 2 | Claude Fable 5.2 | High | 56.9% | — | — | Cursor | Claude Fable 5.2 and Claude Mythos 5.2 System CardRetrieved 2026-11-14 |
| 3 | Claude Fable 5.2 | Xhigh | 56.3% | — | — | Cursor | Claude Fable 5.2 and Claude Mythos 5.2 System CardRetrieved 2026-11-14 |
| 4 | Claude Fable 5.2 | Med | 54.8% | — | — | Cursor | Claude Fable 5.2 and Claude Mythos 5.2 System CardRetrieved 2026-11-14 |
| 5 | Claude Fable 5.2 | Low | 53.2% | — | — | Cursor | Claude Fable 5.2 and Claude Mythos 5.2 System CardRetrieved 2026-11-14 |
Agentic codingDeepSWE v1.1
official results (Gemini 4 Argon model card)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Gemini 4 Argon | not stated | 77.9% | — | — | Gemini 4 Argon model cardRetrieved 2026-11-12 |
official results (Gemini 3.8 Flash model card)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Gemini 3.8 Flash | not stated | 73.7% | — | — | Gemini 3.8 Flash model cardRetrieved 2026-11-12 |
official results (Grok 4.7 (xAI news))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Grok 4.7 | High | 71.0% | — | — | xAI | Grok 4.7 (xAI news)Retrieved 2026-11-12 |
| 2 | Grok 4.6 | High | 65.2% | — | — | xAI | Grok 4.7 (xAI news)Retrieved 2026-11-12 |
official results (GLM-5.3 (Z.ai docs))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | GLM-5.3 | not stated | 66.9% | — | — | Z.ai | GLM-5.3 (Z.ai docs)Retrieved 2026-11-12 |
official results (Introducing Mistral Large 4)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Mistral Large 4 | not stated | 61.7% | — | — | Mistral AI | Introducing Mistral Large 4Retrieved 2026-11-12 |
official results (Kimi K3.1 Pro (Hugging Face model card, moonshotai))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Kimi K3.1 Pro | Max | 75.5% | — | — | Moonshot AI | Kimi K3.1 Pro (Hugging Face model card, moonshotai)Retrieved 2026-11-13 |
official results (Kimi K3 (Hugging Face model card, moonshotai))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Kimi K3 | Max | 67.5% | — | — | Moonshot AI | Kimi K3 (Hugging Face model card, moonshotai)Retrieved 2026-11-13 |
official results (GLM-5.3-Flash (Z.ai docs))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | GLM-5.3-Flash | not stated | 63.4% | — | — | Z.ai | GLM-5.3-Flash (Z.ai docs)Retrieved 2026-11-13 |
official results (Gemini 3.7 Flash model card)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Gemini 3.7 Flash | High | 65.3% | — | — | Gemini 3.7 Flash model cardRetrieved 2026-11-13 |
official results (Qwen3.8-2.4T-A95B (Hugging Face model card, Qwen))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Qwen3.8-Max | not stated | 56.6% | — | — | Alibaba (Qwen) | Qwen3.8-2.4T-A95B (Hugging Face model card, Qwen)Retrieved 2026-11-13 |
official results (Qwen3.8-27B (Hugging Face model card, Qwen))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Qwen3.8-27B | not stated | 42.2% | — | — | Alibaba (Qwen) | Qwen3.8-27B (Hugging Face model card, Qwen)Retrieved 2026-11-13 |
official results (DeepSeek-V4.1-Flash (Hugging Face model card, deepseek-ai))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | DeepSeek-V4.1-Flash | not stated | 74.2% | — | — | DeepSeek | DeepSeek-V4.1-Flash (Hugging Face model card, deepseek-ai)Retrieved 2026-11-13 |
official results (DeepSeek-V4.1-Flash-1017 (Hugging Face model card, deepseek-ai))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | DeepSeek-V4.1-Flash-1017 | not stated | 77.0% | — | — | DeepSeek | DeepSeek-V4.1-Flash-1017 (Hugging Face model card, deepseek-ai)Retrieved 2026-11-13 |
Agentic codingSWE-sweep
official results (Gemini 4 Carbon (Google DeepMind models page, benchmark table))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Gemini 4 Carbon | High | 9.6% | — | — | Gemini 4 Carbon (Google DeepMind models page, benchmark table)Retrieved 2026-11-12 |
Agentic codingSWE-Bench Pro
official results (Gemini 3.5 Flash model card)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Gemini 3.5 Flash | not stated | 55.1% | — | — | Gemini 3.5 Flash model cardRetrieved 2026-11-13 | |
| 2 | Gemini 3.1 Pro Preview | not stated | 54.2% | — | — | Gemini 3.5 Flash model cardRetrieved 2026-11-13 | |
| 3 | Gemini 3 Flash Preview | not stated | 49.6% | — | — | Gemini 3.5 Flash model cardRetrieved 2026-11-13 |
official results (Qwen3.8-2.4T-A95B (Hugging Face model card, Qwen))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Qwen3.8-Max | not stated | 67.7% | — | — | Alibaba (Qwen) | Qwen3.8-2.4T-A95B (Hugging Face model card, Qwen)Retrieved 2026-11-13 |
official results (Qwen3.8-27B (Hugging Face model card, Qwen))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Qwen3.8-27B | not stated | 61.7% | — | — | Alibaba (Qwen) | Qwen3.8-27B (Hugging Face model card, Qwen)Retrieved 2026-11-13 |
Agentic codingSWE-Marathon v1.1
official results (Kimi K3.1 Pro (Hugging Face model card, moonshotai))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Kimi K3.1 Pro | Max | 51.7% | — | — | Moonshot AI | Kimi K3.1 Pro (Hugging Face model card, moonshotai)Retrieved 2026-11-13 |
official results (GLM-5.3 (Hugging Face model card, zai-org))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | GLM-5.3 | Max | 42.5% | — | — | Z.ai | GLM-5.3 (Hugging Face model card, zai-org)Retrieved 2026-11-13 |
Agentic codingSWE-Marathon (version not stated)
official results (Kimi K3 (Hugging Face model card, moonshotai))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Kimi K3 | Max | 42.0% | — | — | Moonshot AI | Kimi K3 (Hugging Face model card, moonshotai)Retrieved 2026-11-13 |
Agentic codingProgramBench(Almost Solved)
official results (Kimi K3.1 Pro (Hugging Face model card, moonshotai))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Kimi K3.1 Pro | Max | 31.5% | — | — | Moonshot AI | Kimi K3.1 Pro (Hugging Face model card, moonshotai)Retrieved 2026-11-13 |
official results (DeepSeek-V4.1-Flash (Hugging Face model card, deepseek-ai))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | DeepSeek-V4.1-Flash | not stated | 20.3% | — | — | DeepSeek | DeepSeek-V4.1-Flash (Hugging Face model card, deepseek-ai)Retrieved 2026-11-13 |
Agentic codingNL2Repo-Bench
official results (GLM-5.3 (Hugging Face model card, zai-org))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | GLM-5.3 | not stated | 58 | — | — | Z.ai | GLM-5.3 (Hugging Face model card, zai-org)Retrieved 2026-11-13 |
official results (Qwen3.8-2.4T-A95B (Hugging Face model card, Qwen))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Qwen3.8-Max | not stated | 55.9 | — | — | Alibaba (Qwen) | Qwen3.8-2.4T-A95B (Hugging Face model card, Qwen)Retrieved 2026-11-13 |
official results (Qwen3.8-27B (Hugging Face model card, Qwen))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Qwen3.8-27B | not stated | 42.3 | — | — | Alibaba (Qwen) | Qwen3.8-27B (Hugging Face model card, Qwen)Retrieved 2026-11-13 |
official results (DeepSeek-V4.1-Flash (Hugging Face model card, deepseek-ai))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | DeepSeek-V4.1-Flash | not stated | 64 | — | — | DeepSeek | DeepSeek-V4.1-Flash (Hugging Face model card, deepseek-ai)Retrieved 2026-11-13 |
Agentic codingMLS-Bench-Lite
official results (Kimi K3 (Hugging Face model card, moonshotai))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Kimi K3 | Max | 48.3% | — | — | Moonshot AI | Kimi K3 (Hugging Face model card, moonshotai)Retrieved 2026-11-13 |
official results (Qwen3.8-2.4T-A95B (Hugging Face model card, Qwen))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Qwen3.8-Max | not stated | 41.0% | — | — | Alibaba (Qwen) | Qwen3.8-2.4T-A95B (Hugging Face model card, Qwen)Retrieved 2026-11-13 |
CodingCWE-bench v1
official results (Gemini 4 Carbon (Google DeepMind models page, benchmark table))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Gemini 4 Carbon | High | 71.0% | — | — | Collinear AI / Artificial Analysis | Gemini 4 Carbon (Google DeepMind models page, benchmark table)Retrieved 2026-11-12 |
| 2 | Gemini 4 Argon | High | 68.0% | — | — | Collinear AI / Artificial Analysis | Gemini 4 Carbon (Google DeepMind models page, benchmark table)Retrieved 2026-11-12 |
CodingVibe Code Bench
official results (Gemini 4 Argon model card)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Gemini 4 Argon | not stated | 91.9% | — | — | Gemini 4 Argon model cardRetrieved 2026-11-12 |
CodingSWE-Atlas-QnA
official results (Introducing Mistral Large 4)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Mistral Large 4 | not stated | 59.4% | — | — | Mistral AI | Introducing Mistral Large 4Retrieved 2026-11-12 |
CodingMirrorCode
third-party results (Epoch AI, Capabilities & benchmarking (benchmark_data.zip, mirrorcode.csv) CC BY 4.0)
CodingLMArena Text Coding (without style control)
third-party results (lmarena-ai/leaderboard-dataset text/latest-00000-of-00001.parquet (CC BY 4.0))
ReasoningHumanity's Last Exam (with tools)
official results (Introducing Claude Fable 5.2 and Claude Mythos 5.2)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Claude Fable 5.2 | Max | 74.2% | $4.51Cost per attempt | frontier | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 2 | Claude Fable 5.2 | Xhigh | 73.4% | $1.48Cost per attempt | frontier | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 3 | Claude Fable 5.2 | High | 71.5% | $0.63Cost per attempt | frontier | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 4 | Claude Fable 5.2 | Med | 68.6% | $0.21Cost per attempt | frontier | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 5 | Claude Fable 5.2 | Low | 68.4% | $0.19Cost per attempt | frontier | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 6 | Claude Opus 5.5 | Max | 67.7% | $2.18Cost per attempt | — | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 7 | Claude Opus 5.5 | Xhigh | 66.4% | $0.69Cost per attempt | — | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 8 | Claude Fable 5.1 | Max | 65.6% | $3.77Cost per attempt | — | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 9 | Claude Fable 5.1 | Xhigh | 65.2% | $2.70Cost per attempt | — | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 10 | Claude Opus 5.5 | High | 63.9% | $0.29Cost per attempt | — | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 11 | Claude Fable 5.1 | High | 63.1% | $1.37Cost per attempt | — | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 12 | Claude Opus 5.5 | Med | 63.0% | $0.17Cost per attempt | frontier | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 13 | Claude Fable 5.1 | Med | 62.4% | $0.85Cost per attempt | — | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 14 | Claude Fable 5.1 | Low | 60.2% | $0.59Cost per attempt | — | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 15 | Claude Opus 5.5 | Low | 57.4% | $0.08Cost per attempt | frontier | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 16 | GPT-6 Astra | not stated | 57.2% | — | — | not stated | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
official results (Introducing Claude Opus 5.5)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Claude Opus 5.5 | Max | 67.7% | — | — | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 2 | Claude Fable 5.1 | not stated | 65.6% | — | — | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 3 | Claude Opus 5 | not stated | 63.6% | — | — | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 4 | GPT-6 Astra | not stated | 57.2% | — | — | not stated | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
official results (Introducing Claude Sonnet 5.5)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Claude Opus 5.5 | not stated | 67.7% | — | — | Anthropic | Introducing Claude Sonnet 5.5Retrieved 2026-11-12 |
| 2 | Claude Sonnet 5.5 | not stated | 64.5% | — | — | Anthropic | Introducing Claude Sonnet 5.5Retrieved 2026-11-12 |
| 3 | Claude Sonnet 5 | not stated | 54.9% | — | — | Anthropic | Introducing Claude Sonnet 5.5Retrieved 2026-11-12 |
official results (Introducing Claude Haiku 5.5)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Claude Sonnet 5.5 | not stated | 64.5% | — | — | Anthropic | Introducing Claude Haiku 5.5Retrieved 2026-11-12 |
| 2 | Claude Haiku 5.5 | not stated | 57.4% | — | — | Anthropic | Introducing Claude Haiku 5.5Retrieved 2026-11-12 |
| 3 | Claude Haiku 4.5 | not stated | 18.7% | — | — | Anthropic | Introducing Claude Haiku 5.5Retrieved 2026-11-12 |
official results (GLM-5.3 (Hugging Face model card, zai-org))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | GLM-5.3 | not stated | 62.5% | — | — | Z.ai | GLM-5.3 (Hugging Face model card, zai-org)Retrieved 2026-11-13 |
official results (Introducing Claude Opus 4.6)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Claude Opus 4.6 | Max | 53.0% | — | — | Anthropic | Introducing Claude Opus 4.6Retrieved 2026-11-13 |
ReasoningHumanity's Last Exam (without tools)
official results (Introducing Claude Haiku 5.5)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Claude Sonnet 5.5 | Max | 56.9% | $0.89Cost per attempt | frontier | Anthropic | Introducing Claude Haiku 5.5Retrieved 2026-11-12 |
| 2 | Claude Sonnet 5.5 | Xhigh | 53.0% | $0.20Cost per attempt | frontier | Anthropic | Introducing Claude Haiku 5.5Retrieved 2026-11-12 |
| 3 | Claude Sonnet 5.5 | High | 47.9% | $0.08Cost per attempt | frontier | Anthropic | Introducing Claude Haiku 5.5Retrieved 2026-11-12 |
| 4 | Claude Haiku 5.5 | Max | 45.9% | $0.21Cost per attempt | — | Anthropic | Introducing Claude Haiku 5.5Retrieved 2026-11-12 |
| 5 | Claude Haiku 5.5 | Xhigh | 44.1% | $0.06Cost per attempt | frontier | Anthropic | Introducing Claude Haiku 5.5Retrieved 2026-11-12 |
| 6 | Claude Sonnet 5.5 | Med | 43.2% | $0.03Cost per attempt | frontier | Anthropic | Introducing Claude Haiku 5.5Retrieved 2026-11-12 |
| 7 | Claude Sonnet 5.5 | Low | 41.4% | $0.03Cost per attempt | — | Anthropic | Introducing Claude Haiku 5.5Retrieved 2026-11-12 |
| 8 | Claude Haiku 5.5 | High | 39.7% | $0.02Cost per attempt | frontier | Anthropic | Introducing Claude Haiku 5.5Retrieved 2026-11-12 |
| 9 | Claude Haiku 5.5 | Med | 35.5% | $0.007Cost per attempt | frontier | Anthropic | Introducing Claude Haiku 5.5Retrieved 2026-11-12 |
| 10 | Claude Haiku 5.5 | Low | 30.9% | $0.003Cost per attempt | frontier | Anthropic | Introducing Claude Haiku 5.5Retrieved 2026-11-12 |
| 11 | Claude Haiku 4.5 | not stated | 10.2% | $0.09Cost per attempt | — | Anthropic | Introducing Claude Haiku 5.5Retrieved 2026-11-12 |
official results (Claude Fable 5.2 and Claude Mythos 5.2 System Card)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Claude Fable 5.2 | Max | 73.2% | — | — | Anthropic | Claude Fable 5.2 and Claude Mythos 5.2 System CardRetrieved 2026-11-14 |
official results (Claude Opus 5.5 System Card)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Claude Opus 5.5 | Max | 64.4% | — | — | Anthropic | Claude Opus 5.5 System CardRetrieved 2026-11-14 |
ReasoningHLE-Verified
official results (Gemini 3.8 Flash model card)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Gemini 3.8 Flash | not stated | 54.9% | — | — | Gemini 3.8 Flash model cardRetrieved 2026-11-12 |
official results (Gemini 3.7 Flash model card)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Gemini 3.7 Flash | not stated | 53.6% | — | — | Gemini 3.7 Flash model cardRetrieved 2026-11-13 | |
| 2 | Gemini 3.6 Flash | not stated | 51.2% | — | — | Gemini 3.7 Flash model cardRetrieved 2026-11-13 |
ReasoningGPQA Diamond
third-party results (Epoch AI, Capabilities & benchmarking (benchmark_data.zip, gpqa_diamond.csv) CC BY 4.0)
ReasoningChess Puzzles(Epoch AI)
third-party results (Epoch AI, Capabilities & benchmarking (benchmark_data.zip, chess_puzzles.csv) CC BY 4.0)
MathRiemannBench
official results (Gemini 4 Carbon (Google DeepMind models page, benchmark table))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Gemini 4 Argon | High | 76.0% | — | — | Surge AI | Gemini 4 Carbon (Google DeepMind models page, benchmark table)Retrieved 2026-11-12 |
| 2 | Gemini 4 Carbon | High | 72.0% | — | — | Surge AI | Gemini 4 Carbon (Google DeepMind models page, benchmark table)Retrieved 2026-11-12 |
MathFrontierMath Tiers 1-3(v2)
third-party results (Epoch AI, Capabilities & benchmarking (benchmark_data.zip, frontiermath_tiers_1_3_v2.csv) CC BY 4.0)
MathFrontierMath Tier 4(v2)
third-party results (Epoch AI, Capabilities & benchmarking (benchmark_data.zip, frontiermath_tier_4_v2.csv) CC BY 4.0)
MathOTIS Mock AIME 2024-2025
third-party results (Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0)
MathMathVision
official results (Kimi K3 (Hugging Face model card, moonshotai))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Kimi K3 | Max | 94.3% | — | — | Moonshot AI | Kimi K3 (Hugging Face model card, moonshotai)Retrieved 2026-11-13 |
official results (Qwen3.8-27B (Hugging Face model card, Qwen))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Qwen3.8-27B | not stated | 90.0% | — | — | Alibaba (Qwen) | Qwen3.8-27B (Hugging Face model card, Qwen)Retrieved 2026-11-13 |
MathMathVision (with tools)
official results (Kimi K3 (Hugging Face model card, moonshotai))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Kimi K3 | Max | 97.8% | — | — | Moonshot AI | Kimi K3 (Hugging Face model card, moonshotai)Retrieved 2026-11-13 |
official results (Qwen3.8-27B (Hugging Face model card, Qwen))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Qwen3.8-27B | not stated | 94.6% | — | — | Alibaba (Qwen) | Qwen3.8-27B (Hugging Face model card, Qwen)Retrieved 2026-11-13 |
KnowledgeGDPval-AA v2.1
official results (Introducing Claude Opus 5.5)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Claude Opus 5.5 | Max | 1,846 | $8.92Estimated cost per task | frontier | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 2 | Claude Opus 5.5 | Xhigh | 1,820 | $4.21Estimated cost per task | frontier | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 3 | Claude Fable 5.1 | Max | 1,735 | $9.59Estimated cost per task | — | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 4 | Claude Fable 5.1 | Xhigh | 1,721 | $7.09Estimated cost per task | — | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 5 | Claude Opus 5 | Max | 1,708 | $6.76Estimated cost per task | — | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 6 | Claude Opus 5.5 | High | 1,692 | $1.54Estimated cost per task | frontier | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 7 | Claude Opus 5 | Xhigh | 1,676 | $4.96Estimated cost per task | — | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 8 | Claude Fable 5.1 | High | 1,617 | $3.43Estimated cost per task | — | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 9 | GPT-5.6 Sol | Max | 1,588 | $2.81Estimated cost per task | — | not stated | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 10 | Claude Opus 5 | High | 1,581 | $3.03Estimated cost per task | — | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 11 | Claude Opus 5.5 | Med | 1,576 | $0.86Estimated cost per task | frontier | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 12 | GPT-5.6 Sol | Xhigh | 1,548 | $1.70Estimated cost per task | — | not stated | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 13 | GPT-6 Astra | Max | 1,542 | $4.53Estimated cost per task | — | not stated | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 14 | Claude Fable 5.1 | Med | 1,536 | $2.17Estimated cost per task | — | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 15 | GPT-6 Astra | Xhigh | 1,516 | $3.04Estimated cost per task | — | not stated | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 16 | GPT-6 Astra | High | 1,485 | $2.43Estimated cost per task | — | not stated | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 17 | GPT-5.6 Sol | High | 1,480 | $1.11Estimated cost per task | — | not stated | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 18 | Claude Opus 5 | Med | 1,476 | $1.36Estimated cost per task | — | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 19 | GPT-6 Astra | Med | 1,468 | $1.82Estimated cost per task | — | not stated | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 20 | Claude Fable 5.1 | Low | 1,450 | $1.41Estimated cost per task | — | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 21 | GPT-5.6 Sol | Med | 1,403 | $0.60Estimated cost per task | frontier | not stated | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 22 | GPT-6 Astra | Low | 1,366 | $0.85Estimated cost per task | — | not stated | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 23 | Claude Opus 5 | Low | 1,294 | $0.58Estimated cost per task | frontier | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 24 | GPT-5.6 Sol | Low | 1,289 | $0.27Estimated cost per task | frontier | not stated | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 25 | Claude Opus 5.5 | Low | 1,224 | $0.21Estimated cost per task | frontier | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
official results (Introducing Claude Haiku 5.5)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Claude Sonnet 5.5 | Max | 1,840 | $6.78Cost per task | frontier | Anthropic | Introducing Claude Haiku 5.5Retrieved 2026-11-12 |
| 2 | Claude Sonnet 5.5 | Xhigh | 1,731 | $1.88Cost per task | frontier | Anthropic | Introducing Claude Haiku 5.5Retrieved 2026-11-12 |
| 3 | Claude Haiku 5.5 | Max | 1,620 | $0.87Cost per task | frontier | Anthropic | Introducing Claude Haiku 5.5Retrieved 2026-11-12 |
| 4 | Claude Sonnet 5.5 | High | 1,551 | $0.62Cost per task | frontier | Anthropic | Introducing Claude Haiku 5.5Retrieved 2026-11-12 |
| 5 | Claude Haiku 5.5 | Xhigh | 1,513 | $0.27Cost per task | frontier | Anthropic | Introducing Claude Haiku 5.5Retrieved 2026-11-12 |
| 6 | GPT-6 Luna | Max | 1,437 | $0.09Cost per task | frontier | not stated | Introducing Claude Haiku 5.5Retrieved 2026-11-12 |
| 7 | Claude Haiku 5.5 | High | 1,420 | $0.09Cost per task | — | Anthropic | Introducing Claude Haiku 5.5Retrieved 2026-11-12 |
| 8 | GPT-6 Luna | Xhigh | 1,364 | $0.05Cost per task | frontier | not stated | Introducing Claude Haiku 5.5Retrieved 2026-11-12 |
| 9 | GPT-6 Luna | High | 1,344 | $0.03Cost per task | frontier | not stated | Introducing Claude Haiku 5.5Retrieved 2026-11-12 |
| 10 | Claude Sonnet 5.5 | Med | 1,324 | $0.27Cost per task | — | Anthropic | Introducing Claude Haiku 5.5Retrieved 2026-11-12 |
| 11 | Claude Haiku 5.5 | Med | 1,277 | $0.03Cost per task | — | Anthropic | Introducing Claude Haiku 5.5Retrieved 2026-11-12 |
| 12 | GPT-6 Luna | Med | 1,262 | $0.02Cost per task | frontier | not stated | Introducing Claude Haiku 5.5Retrieved 2026-11-12 |
| 13 | Claude Sonnet 5.5 | Low | 1,179 | $0.22Cost per task | — | Anthropic | Introducing Claude Haiku 5.5Retrieved 2026-11-12 |
| 14 | Claude Haiku 5.5 | Low | 1,125 | $0.01Cost per task | frontier | Anthropic | Introducing Claude Haiku 5.5Retrieved 2026-11-12 |
| 15 | GPT-6 Luna | Low | 1,036 | $0.004Cost per task | frontier | not stated | Introducing Claude Haiku 5.5Retrieved 2026-11-12 |
| 16 | Claude Haiku 4.5 | not stated | 735 | $0.24Cost per task | — | Anthropic | Introducing Claude Haiku 5.5Retrieved 2026-11-12 |
official results (Introducing Claude Sonnet 5.5)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Claude Opus 5.5 | not stated | 1,846 | — | — | Artificial Analysis | Introducing Claude Sonnet 5.5Retrieved 2026-11-12 |
| 2 | Claude Sonnet 5.5 | not stated | 1,844 | — | — | Artificial Analysis | Introducing Claude Sonnet 5.5Retrieved 2026-11-12 |
| 3 | GPT-6 Sol | not stated | 1,487 | — | — | Artificial Analysis | Introducing Claude Sonnet 5.5Retrieved 2026-11-12 |
| 4 | Claude Sonnet 5 | not stated | 1,449 | — | — | Artificial Analysis | Introducing Claude Sonnet 5.5Retrieved 2026-11-12 |
official results (Claude Fable 5.2 and Claude Mythos 5.2 System Card)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Claude Fable 5.2 | Max | 1,831 | — | — | Artificial Analysis | Claude Fable 5.2 and Claude Mythos 5.2 System CardRetrieved 2026-11-14 |
| 2 | Claude Fable 5.2 | Xhigh | 1,751 | — | — | Artificial Analysis | Claude Fable 5.2 and Claude Mythos 5.2 System CardRetrieved 2026-11-14 |
| 3 | GPT-6.1 Sol | not stated | 1,609 | — | — | Artificial Analysis | Claude Fable 5.2 and Claude Mythos 5.2 System CardRetrieved 2026-11-14 |
KnowledgeGDPval-AA v2
official results (Gemini 3.8 Flash model card)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Gemini 3.8 Flash | not stated | 1,545 | — | — | Gemini 3.8 Flash model cardRetrieved 2026-11-12 |
KnowledgeAA-Briefcase v1.1(Elo)
official results (Introducing Claude Sonnet 5.5)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Claude Opus 5.5 | Max | 1,822 | $21.05Cost per task | frontier | Anthropic | Introducing Claude Sonnet 5.5Retrieved 2026-11-12 |
| 2 | Claude Sonnet 5.5 | Max | 1,811 | $29.19Cost per task | — | Anthropic | Introducing Claude Sonnet 5.5Retrieved 2026-11-12 |
| 3 | Claude Opus 5.5 | Xhigh | 1,780 | $12.27Cost per task | frontier | Anthropic | Introducing Claude Sonnet 5.5Retrieved 2026-11-12 |
| 4 | Claude Sonnet 5.5 | Xhigh | 1,746 | $9.63Cost per task | frontier | Anthropic | Introducing Claude Sonnet 5.5Retrieved 2026-11-12 |
| 5 | Claude Opus 5.5 | High | 1,705 | $6.27Cost per task | frontier | Anthropic | Introducing Claude Sonnet 5.5Retrieved 2026-11-12 |
| 6 | Claude Opus 5.5 | Med | 1,642 | $4.40Cost per task | frontier | Anthropic | Introducing Claude Sonnet 5.5Retrieved 2026-11-12 |
| 7 | Claude Sonnet 5.5 | High | 1,634 | $3.95Cost per task | frontier | Anthropic | Introducing Claude Sonnet 5.5Retrieved 2026-11-12 |
| 8 | GPT-6 Sol | Max | 1,483 | $2.67Cost per task | frontier | not stated | Introducing Claude Sonnet 5.5Retrieved 2026-11-12 |
| 9 | Claude Sonnet 5.5 | Med | 1,461 | $1.64Cost per task | frontier | Anthropic | Introducing Claude Sonnet 5.5Retrieved 2026-11-12 |
| 10 | GPT-6 Sol | Xhigh | 1,364 | $1.19Cost per task | frontier | not stated | Introducing Claude Sonnet 5.5Retrieved 2026-11-12 |
| 11 | Claude Sonnet 5 | Max | 1,359 | $14.43Cost per task | — | Anthropic | Introducing Claude Sonnet 5.5Retrieved 2026-11-12 |
| 12 | GPT-6 Sol | High | 1,289 | $0.63Cost per task | frontier | not stated | Introducing Claude Sonnet 5.5Retrieved 2026-11-12 |
| 13 | Claude Opus 5.5 | Low | 1,285 | $1.15Cost per task | — | Anthropic | Introducing Claude Sonnet 5.5Retrieved 2026-11-12 |
| 14 | Claude Sonnet 5 | Xhigh | 1,274 | $7.56Cost per task | — | Anthropic | Introducing Claude Sonnet 5.5Retrieved 2026-11-12 |
| 15 | Claude Sonnet 5.5 | Low | 1,264 | $0.87Cost per task | — | Anthropic | Introducing Claude Sonnet 5.5Retrieved 2026-11-12 |
| 16 | Claude Sonnet 5 | High | 1,177 | $3.76Cost per task | — | Anthropic | Introducing Claude Sonnet 5.5Retrieved 2026-11-12 |
| 17 | GPT-6 Sol | Med | 1,142 | $0.34Cost per task | frontier | not stated | Introducing Claude Sonnet 5.5Retrieved 2026-11-12 |
| 18 | Claude Sonnet 5 | Med | 1,056 | $1.73Cost per task | — | Anthropic | Introducing Claude Sonnet 5.5Retrieved 2026-11-12 |
| 19 | Claude Sonnet 5 | Low | 923 | $0.82Cost per task | — | Anthropic | Introducing Claude Sonnet 5.5Retrieved 2026-11-12 |
| 20 | GPT-6 Sol | Low | 905 | $0.12Cost per task | frontier | not stated | Introducing Claude Sonnet 5.5Retrieved 2026-11-12 |
official results (Introducing Claude Haiku 5.5)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Claude Sonnet 5.5 | not stated | 1,824 | — | — | Anthropic | Introducing Claude Haiku 5.5Retrieved 2026-11-12 |
| 2 | Claude Haiku 5.5 | not stated | 1,578 | — | — | Anthropic | Introducing Claude Haiku 5.5Retrieved 2026-11-12 |
| 3 | GPT-6 Luna | not stated | 1,336 | — | — | not stated | Introducing Claude Haiku 5.5Retrieved 2026-11-12 |
| 4 | Claude Haiku 4.5 | not stated | 614 | — | — | Anthropic | Introducing Claude Haiku 5.5Retrieved 2026-11-12 |
official results (Introducing Claude Fable 5.2 and Claude Mythos 5.2)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Claude Opus 5.5 | not stated | 1,800 | — | — | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 2 | Claude Fable 5.2 | not stated | 1,785 | — | — | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 3 | Claude Fable 5.1 | not stated | 1,669 | — | — | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 4 | Gemini 4 Carbon | not stated | 1,611 | — | — | not stated | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 5 | GPT-6 Astra | not stated | 1,569 | — | — | not stated | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
official results (Grok 4.7 (xAI news))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Grok 4.7 | Xhigh | 1,657 | — | — | xAI | Grok 4.7 (xAI news)Retrieved 2026-11-12 |
| 2 | Grok 4.6 | High | 1,546 | — | — | xAI | Grok 4.7 (xAI news)Retrieved 2026-11-12 |
official results (Claude Fable 5.2 and Claude Mythos 5.2 System Card)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | GPT-6.1 Sol | not stated | 1,555 | — | — | Artificial Analysis | Claude Fable 5.2 and Claude Mythos 5.2 System CardRetrieved 2026-11-14 |
KnowledgeAA-Briefcase (Elo, version not stated)
official results (Introducing Mistral Large 4)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Mistral Large 4 | not stated | 1,393 | — | — | Mistral AI | Introducing Mistral Large 4Retrieved 2026-11-12 |
KnowledgeAA-Briefcase v1.1(rubric)
official results (Introducing Claude Fable 5.2 and Claude Mythos 5.2)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Claude Fable 5.2 | not stated | 66.1% | — | — | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 2 | Gemini 4 Carbon | not stated | 61.8% | — | — | not stated | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 3 | Claude Fable 5.1 | not stated | 61.5% | — | — | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 4 | Claude Opus 5.5 | not stated | 61.2% | — | — | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 5 | GPT-6 Astra | not stated | 52.0% | — | — | not stated | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
KnowledgeRemote Labor Index(RLI)
official results (Introducing Claude Fable 5.2 and Claude Mythos 5.2)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Claude Fable 5.2 | High | 41.3% | — | — | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 2 | Claude Opus 5.5 | not stated | 21.3% | — | — | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 3 | GPT-6 Astra | not stated | 20.8% | — | — | not stated | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 4 | Claude Fable 5.1 | not stated | 17.9% | — | — | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
KnowledgeHealthBench Professional (length-adjusted score)
official results (GPT-6 Astra System Card (Table 6. HealthBench Evaluations))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | GPT-6 Astra | not stated | 64.7 | — | — | OpenAI | GPT-6 Astra System Card (Table 6. HealthBench Evaluations)Retrieved 2026-11-12 |
| 2 | GPT-5.6 Sol | not stated | 60.5 | — | — | OpenAI | GPT-6 Astra System Card (Table 6. HealthBench Evaluations)Retrieved 2026-11-12 |
| 3 | GPT-5.6 Terra | not stated | 57.7 | — | — | OpenAI | GPT-6 Astra System Card (Table 6. HealthBench Evaluations)Retrieved 2026-11-12 |
| 4 | GPT-5.6 Luna | not stated | 55.7 | — | — | OpenAI | GPT-6 Astra System Card (Table 6. HealthBench Evaluations)Retrieved 2026-11-12 |
official results (Claude Fable 5.2 and Claude Mythos 5.2 System Card)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Claude Fable 5.2 | Max | 67.7 | — | — | Artificial Analysis | Claude Fable 5.2 and Claude Mythos 5.2 System CardRetrieved 2026-11-14 |
| 2 | Claude Sonnet 5.5 | Max | 63 | — | — | Artificial Analysis | Claude Fable 5.2 and Claude Mythos 5.2 System CardRetrieved 2026-11-14 |
| 3 | Claude Opus 5.5 | Max | 59.9 | — | — | Artificial Analysis | Claude Fable 5.2 and Claude Mythos 5.2 System CardRetrieved 2026-11-14 |
| 4 | Claude Haiku 5.5 | Max | 58.4 | — | — | Artificial Analysis | Claude Fable 5.2 and Claude Mythos 5.2 System CardRetrieved 2026-11-14 |
official results (Addendum to GPT-6 Astra System Card: GPT-6.1 Terra)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | GPT-6.1 Sol | not stated | 64.2 | — | — | OpenAI | Addendum to GPT-6 Astra System Card: GPT-6.1 TerraRetrieved 2026-11-14 |
| 2 | GPT-6.1 Terra | not stated | 62.7 | — | — | OpenAI | Addendum to GPT-6 Astra System Card: GPT-6.1 TerraRetrieved 2026-11-14 |
| 3 | GPT-6 Luna | not stated | 60.8 | — | — | OpenAI | Addendum to GPT-6 Astra System Card: GPT-6.1 TerraRetrieved 2026-11-14 |
KnowledgeHealthBench Hard (length-adjusted score)
official results (GPT-6 Astra System Card (Table 6. HealthBench Evaluations))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | GPT-6 Astra | not stated | 36.6 | — | — | OpenAI | GPT-6 Astra System Card (Table 6. HealthBench Evaluations)Retrieved 2026-11-12 |
| 2 | GPT-5.6 Sol | not stated | 33.1 | — | — | OpenAI | GPT-6 Astra System Card (Table 6. HealthBench Evaluations)Retrieved 2026-11-12 |
| 3 | GPT-5.6 Terra | not stated | 32.7 | — | — | OpenAI | GPT-6 Astra System Card (Table 6. HealthBench Evaluations)Retrieved 2026-11-12 |
| 4 | GPT-5.6 Luna | not stated | 32 | — | — | OpenAI | GPT-6 Astra System Card (Table 6. HealthBench Evaluations)Retrieved 2026-11-12 |
official results (Addendum to GPT-6 Astra System Card: GPT-6.1 Terra)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | GPT-6.1 Sol | not stated | 36.2 | — | — | OpenAI | Addendum to GPT-6 Astra System Card: GPT-6.1 TerraRetrieved 2026-11-14 |
| 2 | GPT-6.1 Terra | not stated | 31.4 | — | — | OpenAI | Addendum to GPT-6 Astra System Card: GPT-6.1 TerraRetrieved 2026-11-14 |
| 2 | GPT-6 Luna | not stated | 31.4 | — | — | OpenAI | Addendum to GPT-6 Astra System Card: GPT-6.1 TerraRetrieved 2026-11-14 |
KnowledgeHealthBench Professional
official results (Grok 4.7 (xAI news))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Grok 4.7 | Xhigh | 56.7% | — | — | xAI | Grok 4.7 (xAI news)Retrieved 2026-11-12 |
| 2 | Grok 4.6 | High | 48.5% | — | — | xAI | Grok 4.7 (xAI news)Retrieved 2026-11-12 |
KnowledgeLABBench 2
official results (Gemini 4 Argon model card)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Gemini 4 Argon | not stated | 88.8% | — | — | Gemini 4 Argon model cardRetrieved 2026-11-12 |
official results (Gemini 3.8 Flash model card)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Gemini 3.8 Flash | not stated | 86.2% | — | — | Gemini 3.8 Flash model cardRetrieved 2026-11-12 |
official results (Gemini 3.7 Flash model card)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Gemini 3.7 Flash | not stated | 82.1% | — | — | Gemini 3.7 Flash model cardRetrieved 2026-11-13 | |
| 2 | Gemini 3.6 Flash | not stated | 76.1% | — | — | Gemini 3.7 Flash model cardRetrieved 2026-11-13 |
KnowledgeGDP.PDF
official results (Gemini 3.8 Flash model card)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Gemini 3.8 Flash | not stated | 35.0% | — | — | Gemini 3.8 Flash model cardRetrieved 2026-11-12 |
official results (Kimi K3.1 Pro (Hugging Face model card, moonshotai))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Kimi K3.1 Pro | Max | 34.6% | — | — | Moonshot AI | Kimi K3.1 Pro (Hugging Face model card, moonshotai)Retrieved 2026-11-13 |
official results (Gemini 3.7 Flash model card)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Gemini 3.7 Flash | not stated | 34.0% | — | — | Gemini 3.7 Flash model cardRetrieved 2026-11-13 | |
| 2 | Gemini 3.6 Flash | not stated | 22.0% | — | — | Gemini 3.7 Flash model cardRetrieved 2026-11-13 |
KnowledgeBioMysteryBench(Human Solvable)
official results (Gemini 3.8 Flash model card)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Gemini 3.8 Flash | not stated | 88.8% | — | — | Gemini 3.8 Flash model cardRetrieved 2026-11-12 |
official results (Gemini 3.7 Flash model card)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Gemini 3.7 Flash | not stated | 87.1% | — | — | Gemini 3.7 Flash model cardRetrieved 2026-11-13 | |
| 2 | Gemini 3.6 Flash | not stated | 80.6% | — | — | Gemini 3.7 Flash model cardRetrieved 2026-11-13 |
KnowledgeBioMysteryBench(Human Difficult)
official results (Gemini 3.8 Flash model card)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Gemini 3.8 Flash | not stated | 56.5% | — | — | Gemini 3.8 Flash model cardRetrieved 2026-11-12 |
official results (Gemini 3.7 Flash model card)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Gemini 3.7 Flash | not stated | 43.5% | — | — | Gemini 3.7 Flash model cardRetrieved 2026-11-13 | |
| 2 | Gemini 3.6 Flash | not stated | 41.2% | — | — | Gemini 3.7 Flash model cardRetrieved 2026-11-13 |
KnowledgeOfficeQA Pro v2
official results (Introducing Kimi K3.1 Pro)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Kimi K3.1 Pro | Max | 64.4 | — | — | Moonshot AI | Introducing Kimi K3.1 ProRetrieved 2026-11-12 |
KnowledgeFrontier Finance
official results (Introducing Kimi K3.1 Pro)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Kimi K3.1 Pro | Max | 58.6 | — | — | Moonshot AI | Introducing Kimi K3.1 ProRetrieved 2026-11-12 |
KnowledgeSimpleQA Verified
third-party results (Epoch AI, Capabilities & benchmarking (benchmark_data.zip, simpleqa_verified.csv) CC BY 4.0)
KnowledgeOfficeQA Pro
official results (Kimi K3.1 Pro (Hugging Face model card, moonshotai))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Kimi K3.1 Pro | Max | 67.4 | — | — | Moonshot AI | Kimi K3.1 Pro (Hugging Face model card, moonshotai)Retrieved 2026-11-13 |
official results (Kimi K3 (Hugging Face model card, moonshotai))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Kimi K3 | Max | 63.3 | — | — | Moonshot AI | Kimi K3 (Hugging Face model card, moonshotai)Retrieved 2026-11-13 |
MultimodalChartography (with tools)
official results (Introducing Claude Opus 5.5)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Claude Opus 5.5 | Max | 89.0% | — | — | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 2 | Claude Fable 5.1 | not stated | 88.4% | — | — | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 3 | Claude Opus 5 | not stated | 83.4% | — | — | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
official results (Kimi K3.1 Pro (Hugging Face model card, moonshotai))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Kimi K3.1 Pro | Max | 90.2% | — | — | Moonshot AI | Kimi K3.1 Pro (Hugging Face model card, moonshotai)Retrieved 2026-11-13 |
official results (DeepSeek-V4.1-Flash (Hugging Face model card, deepseek-ai))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | DeepSeek-V4.1-Flash | not stated | 78.9% | — | — | DeepSeek | DeepSeek-V4.1-Flash (Hugging Face model card, deepseek-ai)Retrieved 2026-11-13 |
official results (Claude Fable 5.2 and Claude Mythos 5.2 System Card)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Claude Fable 5.2 | Max | 91.6% | — | — | Anthropic | Claude Fable 5.2 and Claude Mythos 5.2 System CardRetrieved 2026-11-14 |
official results (Claude Sonnet 5.5 System Card)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Claude Sonnet 5.5 | Max | 90.2% | — | — | Anthropic | Claude Sonnet 5.5 System CardRetrieved 2026-11-14 |
official results (Claude Haiku 5.5 System Card)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Claude Haiku 5.5 | Max | 86.2% | — | — | Anthropic | Claude Haiku 5.5 System CardRetrieved 2026-11-14 |
MultimodalChartography (without tools)
official results (Introducing Claude Fable 5.2 and Claude Mythos 5.2)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Claude Fable 5.2 | Max | 73.4% | $0.75Cost per attempt | frontier | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 2 | Gemini 4 Carbon | not stated | 72.5% | — | — | not stated | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 3 | Claude Fable 5.2 | Xhigh | 71.4% | $0.15Cost per attempt | frontier | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 4 | GPT-6 Astra | not stated | 71.0% | — | — | not stated | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 5 | Claude Fable 5.2 | High | 69.8% | $0.07Cost per attempt | frontier | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 6 | Claude Fable 5.2 | Med | 66.4% | $0.03Cost per attempt | frontier | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 7 | Claude Opus 5.5 | Max | 64.4% | $0.49Cost per attempt | — | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 8 | Claude Fable 5.2 | Low | 64.2% | $0.03Cost per attempt | — | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 9 | Claude Opus 5.5 | Xhigh | 62.4% | $0.08Cost per attempt | — | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 10 | Claude Opus 5.5 | High | 59.2% | $0.03Cost per attempt | — | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 11 | Claude Opus 5.5 | Med | 59.0% | $0.03Cost per attempt | — | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 12 | Claude Opus 5.5 | Low | 57.0% | $0.02Cost per attempt | frontier | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 13 | Claude Fable 5.1 | Xhigh | 47.0% | $0.38Cost per attempt | — | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 14 | Claude Fable 5.1 | High | 46.0% | $0.08Cost per attempt | — | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 15 | Claude Fable 5.1 | Max | 44.8% | $0.87Cost per attempt | — | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 16 | Claude Fable 5.1 | Med | 42.2% | $0.05Cost per attempt | — | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 17 | Claude Fable 5.1 | Low | 36.6% | $0.03Cost per attempt | — | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
official results (Introducing Claude Sonnet 5.5)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Claude Opus 5.5 | not stated | 64.4% | — | — | Anthropic | Introducing Claude Sonnet 5.5Retrieved 2026-11-12 |
| 2 | Claude Sonnet 5.5 | not stated | 61.6% | — | — | Anthropic | Introducing Claude Sonnet 5.5Retrieved 2026-11-12 |
| 3 | GPT-6 Sol | not stated | 53.6% | — | — | not stated | Introducing Claude Sonnet 5.5Retrieved 2026-11-12 |
| 4 | Claude Sonnet 5 | not stated | 15.6% | — | — | Anthropic | Introducing Claude Sonnet 5.5Retrieved 2026-11-12 |
official results (Introducing Claude Haiku 5.5)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Claude Sonnet 5.5 | not stated | 61.6% | — | — | Anthropic | Introducing Claude Haiku 5.5Retrieved 2026-11-12 |
| 2 | Claude Haiku 5.5 | not stated | 46.4% | — | — | Anthropic | Introducing Claude Haiku 5.5Retrieved 2026-11-12 |
| 3 | GPT-6 Luna | not stated | 29.1% | — | — | not stated | Introducing Claude Haiku 5.5Retrieved 2026-11-12 |
| 4 | Claude Haiku 4.5 | not stated | 6.4% | — | — | Anthropic | Introducing Claude Haiku 5.5Retrieved 2026-11-12 |
official results (Gemini 4 Carbon (Google DeepMind models page, benchmark table))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Gemini 4 Carbon | High | 72.5% | — | — | not stated | Gemini 4 Carbon (Google DeepMind models page, benchmark table)Retrieved 2026-11-12 |
| 2 | Gemini 4 Argon | High | 71.6% | — | — | not stated | Gemini 4 Carbon (Google DeepMind models page, benchmark table)Retrieved 2026-11-12 |
MultimodalBlueprint-Bench 2
official results (Gemini 4 Carbon (Google DeepMind models page, benchmark table))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Gemini 4 Carbon | High | 58.6% | — | — | Andon Labs | Gemini 4 Carbon (Google DeepMind models page, benchmark table)Retrieved 2026-11-12 |
| 2 | Gemini 4 Argon | High | 54.4% | — | — | Andon Labs | Gemini 4 Carbon (Google DeepMind models page, benchmark table)Retrieved 2026-11-12 |
MultimodalLVBench
official results (Gemini 4 Argon model card)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Gemini 4 Argon | not stated | 91.7% | — | — | Gemini 4 Argon model cardRetrieved 2026-11-12 |
official results (Gemini 3.8 Flash model card)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Gemini 3.8 Flash | not stated | 87.8% | — | — | Gemini 3.8 Flash model cardRetrieved 2026-11-12 |
official results (Gemini 3.7 Flash model card)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Gemini 3.7 Flash | not stated | 85.4% | — | — | Gemini 3.7 Flash model cardRetrieved 2026-11-13 | |
| 2 | Gemini 3.6 Flash | not stated | 84.2% | — | — | Gemini 3.7 Flash model cardRetrieved 2026-11-13 |
MultimodalCharXiv Reasoning
official results (Gemini 3.8 Flash model card)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Gemini 3.8 Flash | not stated | 86.2% | — | — | Gemini 3.8 Flash model cardRetrieved 2026-11-12 |
official results (Kimi K3.1 Pro (Hugging Face model card, moonshotai))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Kimi K3.1 Pro | Max | 89.0% | — | — | Moonshot AI | Kimi K3.1 Pro (Hugging Face model card, moonshotai)Retrieved 2026-11-13 |
official results (Kimi K3 (Hugging Face model card, moonshotai))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Kimi K3 | Max | 84.8% | — | — | Moonshot AI | Kimi K3 (Hugging Face model card, moonshotai)Retrieved 2026-11-13 |
official results (Gemini 3.7 Flash model card)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Gemini 3.6 Flash | not stated | 85.2% | — | — | Gemini 3.7 Flash model cardRetrieved 2026-11-13 | |
| 2 | Gemini 3.7 Flash | not stated | 84.5% | — | — | Gemini 3.7 Flash model cardRetrieved 2026-11-13 |
official results (Gemini 3.5 Flash model card)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Gemini 3.5 Flash | not stated | 84.2% | — | — | Gemini 3.5 Flash model cardRetrieved 2026-11-13 | |
| 2 | Gemini 3.1 Pro Preview | not stated | 83.3% | — | — | Gemini 3.5 Flash model cardRetrieved 2026-11-13 | |
| 3 | Gemini 3 Flash Preview | not stated | 80.3% | — | — | Gemini 3.5 Flash model cardRetrieved 2026-11-13 |
MultimodalCharXiv Reasoning (with tools)
official results (Kimi K3.1 Pro (Hugging Face model card, moonshotai))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Kimi K3.1 Pro | Max | 93.4% | — | — | Moonshot AI | Kimi K3.1 Pro (Hugging Face model card, moonshotai)Retrieved 2026-11-13 |
official results (Kimi K3 (Hugging Face model card, moonshotai))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Kimi K3 | Max | 91.3% | — | — | Moonshot AI | Kimi K3 (Hugging Face model card, moonshotai)Retrieved 2026-11-13 |
official results (Gemini 3.7 Flash model card)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Gemini 3.6 Flash | not stated | 89.4% | — | — | Gemini 3.7 Flash model cardRetrieved 2026-11-13 | |
| 2 | Gemini 3.7 Flash | not stated | 88.7% | — | — | Gemini 3.7 Flash model cardRetrieved 2026-11-13 |
MultimodalMMMU-Pro
official results (Kimi K3.1 Pro (Hugging Face model card, moonshotai))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Kimi K3.1 Pro | Max | 84.1% | — | — | Moonshot AI | Kimi K3.1 Pro (Hugging Face model card, moonshotai)Retrieved 2026-11-13 |
official results (Kimi K3 (Hugging Face model card, moonshotai))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Kimi K3 | Max | 81.6% | — | — | Moonshot AI | Kimi K3 (Hugging Face model card, moonshotai)Retrieved 2026-11-13 |
official results (Gemini 3.5 Flash model card)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Gemini 3.5 Flash | not stated | 83.6% | — | — | Gemini 3.5 Flash model cardRetrieved 2026-11-13 | |
| 2 | Gemini 3 Flash Preview | not stated | 81.2% | — | — | Gemini 3.5 Flash model cardRetrieved 2026-11-13 | |
| 3 | Gemini 3.1 Pro Preview | not stated | 80.5% | — | — | Gemini 3.5 Flash model cardRetrieved 2026-11-13 |
MultimodalMMMU-Pro (with tools)
official results (Kimi K3.1 Pro (Hugging Face model card, moonshotai))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Kimi K3.1 Pro | Max | 90.0% | — | — | Moonshot AI | Kimi K3.1 Pro (Hugging Face model card, moonshotai)Retrieved 2026-11-13 |
official results (Kimi K3 (Hugging Face model card, moonshotai))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Kimi K3 | Max | 83.4% | — | — | Moonshot AI | Kimi K3 (Hugging Face model card, moonshotai)Retrieved 2026-11-13 |
MultimodalBabyVision (with tools)
official results (Kimi K3.1 Pro (Hugging Face model card, moonshotai))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Kimi K3.1 Pro | Max | 95.4% | — | — | Moonshot AI | Kimi K3.1 Pro (Hugging Face model card, moonshotai)Retrieved 2026-11-13 |
official results (Kimi K3 (Hugging Face model card, moonshotai))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Kimi K3 | Max | 85.7% | — | — | Moonshot AI | Kimi K3 (Hugging Face model card, moonshotai)Retrieved 2026-11-13 |
official results (DeepSeek-V4.1-Flash (Hugging Face model card, deepseek-ai))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | DeepSeek-V4.1-Flash | not stated | 89.6% | — | — | DeepSeek | DeepSeek-V4.1-Flash (Hugging Face model card, deepseek-ai)Retrieved 2026-11-13 |
MultimodalZeroBench main (with tools, pass@5)
official results (Kimi K3.1 Pro (Hugging Face model card, moonshotai))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Kimi K3.1 Pro | Max | 63.0% | — | — | Moonshot AI | Kimi K3.1 Pro (Hugging Face model card, moonshotai)Retrieved 2026-11-13 |
official results (Kimi K3 (Hugging Face model card, moonshotai))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Kimi K3 | Max | 41.0% | — | — | Moonshot AI | Kimi K3 (Hugging Face model card, moonshotai)Retrieved 2026-11-13 |
official results (DeepSeek-V4.1-Flash (Hugging Face model card, deepseek-ai))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | DeepSeek-V4.1-Flash | not stated | 49.0% | — | — | DeepSeek | DeepSeek-V4.1-Flash (Hugging Face model card, deepseek-ai)Retrieved 2026-11-13 |
Long contextGraphWalks BFS (256k to 1M)
official results (Gemini 4 Carbon (Google DeepMind models page, benchmark table))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Gemini 4 Carbon | High | 96.2% | — | — | Gemini 4 Carbon (Google DeepMind models page, benchmark table)Retrieved 2026-11-12 | |
| 2 | Gemini 4 Argon | High | 84.2% | — | — | Gemini 4 Carbon (Google DeepMind models page, benchmark table)Retrieved 2026-11-12 |
Long contextGraphWalks BFS (128k or less)
official results (Gemini 4 Argon model card)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Gemini 4 Argon | not stated | 99.7% | — | — | Gemini 4 Argon model cardRetrieved 2026-11-12 |
Long contextMRCR v2 (8-needle, 128k)
official results (Gemini 3.7 Flash model card)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Gemini 3.7 Flash | not stated | 97.0% | — | — | Gemini 3.7 Flash model cardRetrieved 2026-11-13 | |
| 2 | Gemini 3.6 Flash | not stated | 91.8% | — | — | Gemini 3.7 Flash model cardRetrieved 2026-11-13 |
official results (Gemini 3.5 Flash model card)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Gemini 3.1 Pro Preview | not stated | 84.9% | — | — | Gemini 3.5 Flash model cardRetrieved 2026-11-13 | |
| 2 | Gemini 3.5 Flash | not stated | 77.3% | — | — | Gemini 3.5 Flash model cardRetrieved 2026-11-13 | |
| 3 | Gemini 3 Flash Preview | not stated | 67.2% | — | — | Gemini 3.5 Flash model cardRetrieved 2026-11-13 |
Long contextMRCR v2 (8-needle, 1M)
official results (Gemini 3.5 Flash model card)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Gemini 3.5 Flash | not stated | 26.6% | — | — | Gemini 3.5 Flash model cardRetrieved 2026-11-13 | |
| 2 | Gemini 3.1 Pro Preview | not stated | 26.3% | — | — | Gemini 3.5 Flash model cardRetrieved 2026-11-13 | |
| 3 | Gemini 3 Flash Preview | not stated | 22.1% | — | — | Gemini 3.5 Flash model cardRetrieved 2026-11-13 |
Tool useAutomationBench
official results (Introducing Claude Opus 5.5)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | GPT-6 Astra | Max | 41.4% | $1.77Cost per task | frontier | Zapier | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 2 | Claude Opus 5.5 | Max | 40.0% | $1.37Cost per task | frontier | Zapier | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 3 | GPT-6 Astra | Xhigh | 39.0% | $1.53Cost per task | — | Zapier | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 4 | GPT-6 Astra | High | 37.1% | $1.45Cost per task | — | Zapier | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 5 | Claude Opus 5.5 | Xhigh | 34.4% | $0.86Cost per task | frontier | Zapier | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 6 | GPT-6 Astra | Med | 34.1% | $1.28Cost per task | — | Zapier | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 7 | Claude Opus 5.5 | High | 32.0% | $0.70Cost per task | frontier | Zapier | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 8 | Claude Fable 5.1 | not stated | 31.4% | — | — | Zapier | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 9 | GPT-6 Astra | Low | 30.3% | $1.08Cost per task | — | Zapier | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 10 | GPT-5.6 Sol | Max | 28.8% | $0.91Cost per task | — | Zapier | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 11 | Claude Opus 5.5 | Med | 28.6% | $0.64Cost per task | frontier | Zapier | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 12 | Claude Opus 5 | Max | 26.9% | $1.27Cost per task | — | Zapier | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 13 | GPT-5.6 Sol | Xhigh | 26.3% | $0.74Cost per task | — | Zapier | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 14 | Claude Opus 5 | Xhigh | 25.3% | $1.15Cost per task | — | Zapier | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 15 | GPT-5.6 Sol | High | 24.8% | $0.64Cost per task | — | Zapier | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 16 | Claude Opus 5 | Med | 23.9% | $0.89Cost per task | — | Zapier | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 17 | Claude Opus 5.5 | Low | 23.3% | $0.50Cost per task | frontier | Zapier | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 18 | Claude Opus 5 | High | 20.6% | $1.03Cost per task | — | Zapier | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 19 | Claude Opus 5 | Low | 20.4% | $0.75Cost per task | — | Zapier | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 20 | GPT-5.6 Sol | Med | 19.6% | $0.57Cost per task | — | Zapier | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 21 | GPT-5.6 Sol | Low | 11.7% | $0.42Cost per task | frontier | Zapier | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
official results (Introducing Claude Fable 5.2 and Claude Mythos 5.2)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Gemini 4 Carbon | not stated | 53.9% | — | — | not stated | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 2 | Claude Fable 5.2 | not stated | 51.4% | — | — | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 3 | Claude Opus 5.5 | not stated | 42.5% | — | — | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 4 | GPT-6 Astra | not stated | 41.4% | — | — | not stated | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 5 | Claude Fable 5.1 | not stated | 31.4% | — | — | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
official results (Gemini 4 Carbon (Google DeepMind models page, benchmark table))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Gemini 4 Carbon | High | 53.9% | — | — | Zapier | Gemini 4 Carbon (Google DeepMind models page, benchmark table)Retrieved 2026-11-12 |
| 2 | Gemini 4 Argon | High | 51.3% | — | — | Zapier | Gemini 4 Carbon (Google DeepMind models page, benchmark table)Retrieved 2026-11-12 |
official results (Introducing Mistral Large 4)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Mistral Large 4 | not stated | 59.9% | — | — | Mistral AI | Introducing Mistral Large 4Retrieved 2026-11-12 |
Tool useWANDR
official results (Introducing Claude Opus 5.5)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Claude Opus 5.5 | Max | 72.3% | $37.92Cost per attempt | frontier | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 2 | Claude Opus 5.5 | Xhigh | 71.3% | $29.06Cost per attempt | frontier | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 3 | Claude Fable 5.1 | Max | 68.7% | $49.02Cost per attempt | — | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 4 | Claude Fable 5.1 | Xhigh | 67.7% | $42.81Cost per attempt | — | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 5 | Claude Opus 5.5 | High | 67.3% | $16.88Cost per attempt | frontier | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 6 | Claude Opus 5 | Max | 67.2% | $61.62Cost per attempt | — | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 7 | Claude Opus 5 | Xhigh | 67.0% | $53.84Cost per attempt | — | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 8 | Claude Fable 5.1 | High | 66.7% | $33.62Cost per attempt | — | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 9 | Claude Opus 5 | High | 64.6% | $43.49Cost per attempt | — | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 10 | Claude Fable 5.1 | Med | 64.5% | $27.41Cost per attempt | — | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 11 | Claude Fable 5.1 | Low | 63.3% | $23.25Cost per attempt | — | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 12 | Claude Opus 5.5 | Med | 62.8% | $11.20Cost per attempt | frontier | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 13 | Claude Opus 5 | Med | 58.1% | $23.97Cost per attempt | — | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 14 | Claude Opus 5 | Low | 50.5% | $10.65Cost per attempt | frontier | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 15 | Claude Opus 5.5 | Low | 31.2% | $1.20Cost per attempt | frontier | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
Tool useWANDR(relaxed score)
official results (Introducing Claude Fable 5.2 and Claude Mythos 5.2)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Claude Fable 5.2 | Max | 82.1% | $53.11Cost per attempt | frontier | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 2 | Claude Fable 5.2 | Xhigh | 80.6% | $39.84Cost per attempt | frontier | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 3 | Claude Fable 5.2 | High | 79.3% | $29.20Cost per attempt | frontier | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 4 | Claude Fable 5.2 | Med | 73.6% | $15.71Cost per attempt | frontier | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 5 | Claude Opus 5.5 | Max | 72.3% | $38.68Cost per attempt | — | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 6 | Claude Fable 5.2 | Low | 71.4% | $12.66Cost per attempt | frontier | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 7 | Claude Opus 5.5 | Xhigh | 71.3% | $29.25Cost per attempt | — | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 8 | Claude Fable 5.1 | Max | 68.7% | $51.10Cost per attempt | — | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 9 | Claude Fable 5.1 | Xhigh | 67.7% | $44.50Cost per attempt | — | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 10 | Claude Opus 5.5 | High | 67.3% | $17.31Cost per attempt | — | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 11 | Claude Fable 5.1 | High | 66.7% | $37.60Cost per attempt | — | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 12 | Claude Fable 5.1 | Med | 64.5% | $28.78Cost per attempt | — | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 13 | Claude Fable 5.1 | Low | 63.3% | $24.41Cost per attempt | — | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 14 | Claude Opus 5.5 | Med | 62.8% | $11.40Cost per attempt | frontier | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 15 | Claude Opus 5.5 | Low | 31.2% | $1.25Cost per attempt | frontier | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
Tool useVals Finance Agent v2
official results (Gemini 4 Argon model card)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Gemini 4 Argon | not stated | 65.4% | — | — | Gemini 4 Argon model cardRetrieved 2026-11-12 |
official results (Gemini 3.8 Flash model card)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Gemini 3.8 Flash | not stated | 61.4% | — | — | Gemini 3.8 Flash model cardRetrieved 2026-11-12 |
Tool useHarvey Legal Agent Benchmark
official results (Gemini 4 Argon model card)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Gemini 4 Argon | not stated | 19.6% | — | — | Gemini 4 Argon model cardRetrieved 2026-11-12 |
official results (Gemini 3.8 Flash model card)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Gemini 3.8 Flash | not stated | 10.0% | — | — | Gemini 3.8 Flash model cardRetrieved 2026-11-12 |
official results (Grok 4.7 (xAI news))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Grok 4.7 | Xhigh | 19.6% | — | — | xAI | Grok 4.7 (xAI news)Retrieved 2026-11-12 |
| 2 | Grok 4.6 | High | 15.8% | — | — | xAI | Grok 4.7 (xAI news)Retrieved 2026-11-12 |
Tool useAutomationBench v1.0.6
official results (GLM-5.3 (Hugging Face model card, zai-org))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | GLM-5.3 | not stated | 48.2% | — | — | Z.ai | GLM-5.3 (Hugging Face model card, zai-org)Retrieved 2026-11-13 |
official results (GLM-5.3-Flash (Z.ai docs))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | GLM-5.3-Flash | not stated | 48.8% | — | — | Z.ai | GLM-5.3-Flash (Z.ai docs)Retrieved 2026-11-13 |
official results (DeepSeek-V4.1-Flash (Hugging Face model card, deepseek-ai))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | DeepSeek-V4.1-Flash | not stated | 54.8% | — | — | DeepSeek | DeepSeek-V4.1-Flash (Hugging Face model card, deepseek-ai)Retrieved 2026-11-13 |
Tool useAutomationBench (600 public tasks)
official results (Kimi K3 (Hugging Face model card, moonshotai))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Kimi K3 | Max | 30.8% | — | — | Moonshot AI | Kimi K3 (Hugging Face model card, moonshotai)Retrieved 2026-11-13 |
official results (Qwen3.8-2.4T-A95B (Hugging Face model card, Qwen))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Qwen3.8-Max | not stated | 27.3% | — | — | Alibaba (Qwen) | Qwen3.8-2.4T-A95B (Hugging Face model card, Qwen)Retrieved 2026-11-13 |
Tool useBrowseComp
official results (Kimi K3.1 Pro (Hugging Face model card, moonshotai))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Kimi K3.1 Pro | Max | 92.1% | — | — | Moonshot AI | Kimi K3.1 Pro (Hugging Face model card, moonshotai)Retrieved 2026-11-13 |
official results (Kimi K3 (Hugging Face model card, moonshotai))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Kimi K3 | Max | 91.2% | — | — | Moonshot AI | Kimi K3 (Hugging Face model card, moonshotai)Retrieved 2026-11-13 |
Tool useMCP Atlas
official results (Kimi K3 (Hugging Face model card, moonshotai))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Kimi K3 | Max | 84.2% | — | — | Moonshot AI | Kimi K3 (Hugging Face model card, moonshotai)Retrieved 2026-11-13 |
Tool useτ²-bench airline (v1.0.1)
third-party results (τ²-bench v1.0.1 (run by Sierra))
Tool useτ²-bench banking_knowledge (v1.0.1)
third-party results (τ²-bench v1.0.1 banking_knowledge (run by Sierra, search alltools))
third-party results (τ²-bench v1.0.1 banking_knowledge (run by Sierra, search terminal))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Gemini 3 Flash Preview | High | 27.3% | — | — | Sierra | τ²-bench leaderboard submission gemini-3-flash_sierra_2026-03-02 (sierra-research/tau2-bench, MIT)Retrieved 2026-11-13 |
| 2 | Claude Sonnet 4.5 | not stated | 25.3% | $4.05Average cost in USD to run one trajectory in this domain | frontier | Sierra | τ²-bench leaderboard submission claude-sonnet-4-5_sierra_2026-02-26 (sierra-research/tau2-bench, MIT)Retrieved 2026-11-13 |
| 3 | Gemini 3 Pro Preview | High | 18.0% | — | — | Sierra | τ²-bench leaderboard submission gemini-3-pro_sierra_2026-03-02 (sierra-research/tau2-bench, MIT)Retrieved 2026-11-13 |
third-party results (τ²-bench v1.0.1 banking_knowledge (run by Sierra, search text-emb-3-large))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | GLM-5 | not stated | 9.8% | — | — | Sierra | τ²-bench leaderboard submission glm-5-think_sierra_2026-03-02 (sierra-research/tau2-bench, MIT)Retrieved 2026-11-13 |
| 1 | Qwen3.5-397B-A17B | not stated | 9.8% | — | — | Sierra | τ²-bench leaderboard submission qwen3.5-397b-a17b-think_sierra_2026-03-02 (sierra-research/tau2-bench, MIT)Retrieved 2026-11-13 |
third-party results (τ²-bench v1.0.1 banking_knowledge (run by Sierra, search qwen_embeddings))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | GPT-5.2 | None | 12.6% | $0.24Average cost in USD to run one trajectory in this domain | frontier | Sierra | τ²-bench leaderboard submission gpt-5-2-none_sierra_2026-02-26 (sierra-research/tau2-bench, MIT)Retrieved 2026-11-13 |
Tool useτ²-bench retail (v1.0.1)
third-party results (τ²-bench v1.0.1 (run by Sierra))
Tool useτ²-bench telecom (v1.0.1)
third-party results (τ²-bench v1.0.1 (run by Sierra))
Computer useOSWorld 2.1(partial)
official results (Introducing Claude Opus 5.5)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Claude Opus 5.5 | Max | 81.8% | — | — | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 2 | Claude Fable 5.1 | not stated | 80.7% | — | — | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
| 3 | Claude Opus 5 | not stated | 74.0% | — | — | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 |
official results (Introducing Claude Sonnet 5.5)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Claude Opus 5.5 | not stated | 81.8% | — | — | Anthropic | Introducing Claude Sonnet 5.5Retrieved 2026-11-12 |
| 2 | Claude Sonnet 5.5 | not stated | 80.1% | — | — | Anthropic | Introducing Claude Sonnet 5.5Retrieved 2026-11-12 |
| 3 | Claude Sonnet 5 | not stated | 57.0% | — | — | Anthropic | Introducing Claude Sonnet 5.5Retrieved 2026-11-12 |
Computer useOSWorld 2.1(offline subset)
official results (Introducing Claude Haiku 5.5)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Claude Sonnet 5.5 | Max | 83.9% | $5.73Cost per attempt | frontier | Anthropic | Introducing Claude Haiku 5.5Retrieved 2026-11-12 |
| 2 | Claude Sonnet 5.5 | Xhigh | 81.1% | $2.22Cost per attempt | frontier | Anthropic | Introducing Claude Haiku 5.5Retrieved 2026-11-12 |
| 3 | Claude Sonnet 5.5 | High | 73.2% | $1.38Cost per attempt | frontier | Anthropic | Introducing Claude Haiku 5.5Retrieved 2026-11-12 |
| 4 | Claude Haiku 5.5 | Max | 72.4% | $0.61Cost per attempt | frontier | Anthropic | Introducing Claude Haiku 5.5Retrieved 2026-11-12 |
| 5 | Claude Haiku 5.5 | Xhigh | 67.6% | $0.28Cost per attempt | frontier | Anthropic | Introducing Claude Haiku 5.5Retrieved 2026-11-12 |
| 6 | Claude Sonnet 5.5 | Med | 66.0% | $0.93Cost per attempt | — | Anthropic | Introducing Claude Haiku 5.5Retrieved 2026-11-12 |
| 7 | Claude Haiku 5.5 | High | 61.3% | $0.18Cost per attempt | frontier | Anthropic | Introducing Claude Haiku 5.5Retrieved 2026-11-12 |
| 8 | Claude Sonnet 5.5 | Low | 57.9% | $0.68Cost per attempt | — | Anthropic | Introducing Claude Haiku 5.5Retrieved 2026-11-12 |
| 9 | Claude Haiku 5.5 | Med | 53.3% | $0.13Cost per attempt | frontier | Anthropic | Introducing Claude Haiku 5.5Retrieved 2026-11-12 |
| 10 | GPT-6 Luna | Max | 48.9% | $0.20Cost per attempt | — | not stated | Introducing Claude Haiku 5.5Retrieved 2026-11-12 |
| 11 | GPT-6 Luna | Xhigh | 44.8% | $0.17Cost per attempt | — | not stated | Introducing Claude Haiku 5.5Retrieved 2026-11-12 |
| 12 | GPT-6 Luna | High | 42.3% | $0.14Cost per attempt | — | not stated | Introducing Claude Haiku 5.5Retrieved 2026-11-12 |
| 13 | Claude Haiku 5.5 | Low | 42.0% | $0.07Cost per attempt | frontier | Anthropic | Introducing Claude Haiku 5.5Retrieved 2026-11-12 |
| 14 | GPT-6 Luna | Med | 37.5% | $0.13Cost per attempt | — | not stated | Introducing Claude Haiku 5.5Retrieved 2026-11-12 |
| 15 | GPT-6 Luna | Low | 19.2% | $0.04Cost per attempt | frontier | not stated | Introducing Claude Haiku 5.5Retrieved 2026-11-12 |
| 16 | Claude Haiku 4.5 | not stated | 15.7% | $1.45Cost per attempt | — | Anthropic | Introducing Claude Haiku 5.5Retrieved 2026-11-12 |
Computer useOSWorld 2.1 (offline set, partial)
official results (Introducing Claude Fable 5.2 and Claude Mythos 5.2)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Claude Fable 5.2 | not stated | 89.3% | — | — | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 2 | Claude Opus 5.5 | not stated | 87.2% | — | — | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 3 | Claude Fable 5.1 | not stated | 83.5% | — | — | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 4 | Gemini 4 Carbon | not stated | 81.5% | — | — | not stated | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 5 | GPT-6 Astra | not stated | 79.5% | — | — | not stated | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
official results (Gemini 4 Carbon (Google DeepMind models page, benchmark table))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Gemini 4 Carbon | High | 81.5% | — | — | Gemini 4 Carbon (Google DeepMind models page, benchmark table)Retrieved 2026-11-12 | |
| 2 | Gemini 4 Argon | High | 74.7% | — | — | Gemini 4 Carbon (Google DeepMind models page, benchmark table)Retrieved 2026-11-12 |
Computer useOSWorld 2.0
official results (Gemini 4 Argon model card)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Gemini 4 Argon | not stated | 69.2% | — | — | Gemini 4 Argon model cardRetrieved 2026-11-12 |
official results (Gemini 3.8 Flash model card)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Gemini 3.8 Flash | not stated | 59.0% | — | — | Gemini 3.8 Flash model cardRetrieved 2026-11-12 |
official results (Gemini 3.7 Flash model card)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Gemini 3.7 Flash | not stated | 47.9% | — | — | Gemini 3.7 Flash model cardRetrieved 2026-11-13 | |
| 2 | Gemini 3.6 Flash | not stated | 33.8% | — | — | Gemini 3.7 Flash model cardRetrieved 2026-11-13 |
Computer useAgents' Last Exam v1.1(rubric score)
official results (Gemini 4 Carbon (Google DeepMind models page, benchmark table))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Gemini 4 Carbon | High | 77.4% | — | — | Gemini 4 Carbon (Google DeepMind models page, benchmark table)Retrieved 2026-11-12 | |
| 2 | Gemini 4 Argon | High | 69.0% | — | — | Gemini 4 Carbon (Google DeepMind models page, benchmark table)Retrieved 2026-11-12 |
Computer useAgents' Last Exam (pass rate, version not stated)
official results (Gemini 4 Argon model card)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Gemini 4 Argon | not stated | 39.5% | — | — | Gemini 4 Argon model cardRetrieved 2026-11-12 |
official results (Gemini 3.7 Flash model card)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Gemini 3.7 Flash | not stated | 26.3% | — | — | Gemini 3.7 Flash model cardRetrieved 2026-11-13 | |
| 2 | Gemini 3.6 Flash | not stated | 24.2% | — | — | Gemini 3.7 Flash model cardRetrieved 2026-11-13 |
official results (DeepSeek-V4.1-Flash (Hugging Face model card, deepseek-ai))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | DeepSeek-V4.1-Flash | not stated | 31.8% | — | — | DeepSeek | DeepSeek-V4.1-Flash (Hugging Face model card, deepseek-ai)Retrieved 2026-11-13 |
Computer useAgents' Last Exam (version and metric not stated)
official results (GLM-5.3 (Z.ai docs))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | GLM-5.3 | not stated | 28.5% | — | — | Z.ai | GLM-5.3 (Z.ai docs)Retrieved 2026-11-12 |
Computer useOSWorld-Verified
official results (Gemini 3.5 Flash model card)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Gemini 3.5 Flash | not stated | 78.4% | — | — | Gemini 3.5 Flash model cardRetrieved 2026-11-13 | |
| 2 | Gemini 3.1 Pro Preview | not stated | 76.2% | — | — | Gemini 3.5 Flash model cardRetrieved 2026-11-13 | |
| 3 | Gemini 3 Flash Preview | not stated | 65.1% | — | — | Gemini 3.5 Flash model cardRetrieved 2026-11-13 |
official results (Introducing Claude Opus 4.8)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Claude Opus 4.7 | not stated | 82.3% | — | — | Anthropic | Introducing Claude Opus 4.8Retrieved 2026-11-13 |
WritingLMArena Text (style control, overall)
third-party results (lmarena-ai/leaderboard-dataset text_style_control/latest-00000-of-00001.parquet (CC BY 4.0))
OtherBenchCAD 2.1 (with tools)
official results (Introducing Claude Fable 5.2 and Claude Mythos 5.2)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Claude Fable 5.2 | Max | 83.5 | $50.77Cost per attempt | frontier | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 2 | Claude Fable 5.2 | Xhigh | 74.1 | $23.23Cost per attempt | frontier | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 3 | Claude Opus 5.5 | Max | 65.3 | $19.86Cost per attempt | frontier | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 4 | Claude Fable 5.2 | High | 61 | $9.99Cost per attempt | frontier | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 5 | Claude Opus 5.5 | Xhigh | 54.5 | $8.91Cost per attempt | frontier | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 6 | Claude Fable 5.1 | Max | 44.4 | $34.69Cost per attempt | — | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 7 | Claude Fable 5.1 | Xhigh | 40.5 | $27.38Cost per attempt | — | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 8 | Claude Fable 5.2 | Med | 32.4 | $2.66Cost per attempt | frontier | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 9 | Claude Opus 5.5 | High | 29.7 | $2.57Cost per attempt | frontier | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 10 | Claude Fable 5.2 | Low | 26.9 | $1.89Cost per attempt | frontier | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 11 | Claude Fable 5.1 | High | 25.8 | $12.44Cost per attempt | — | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 12 | Claude Opus 5.5 | Med | 19.1 | $1.11Cost per attempt | frontier | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 13 | Claude Fable 5.1 | Med | 15.7 | $5.70Cost per attempt | — | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 14 | Claude Fable 5.1 | Low | 12.1 | $2.95Cost per attempt | — | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 15 | Claude Opus 5.5 | Low | 6.3 | $0.03Cost per attempt | frontier | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
OtherBenchCAD (with tools, version not stated)
official results (Introducing Kimi K3.1 Pro)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Kimi K3.1 Pro | Max | 92.7 | — | — | Moonshot AI | Introducing Kimi K3.1 ProRetrieved 2026-11-12 |
OtherBenchCAD (without tools, version not stated)
official results (Introducing Kimi K3.1 Pro)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Kimi K3.1 Pro | Max | 72.8 | — | — | Moonshot AI | Introducing Kimi K3.1 ProRetrieved 2026-11-12 |
OtherCVE Rediscovery
official results (Introducing Claude Fable 5.2 and Claude Mythos 5.2)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Claude Mythos 5.2 | Max | 89.6% | $9.13Cost per attempt | frontier | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 2 | Claude Mythos 5.2 | Xhigh | 88.1% | $2.40Cost per attempt | frontier | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 3 | Claude Mythos 5.2 | High | 83.2% | $1.01Cost per attempt | frontier | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 4 | Claude Opus 5.5 | Max | 81.3% | $2.98Cost per attempt | — | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 5 | Claude Mythos 5.1 | Max | 78.3% | $4.14Cost per attempt | — | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 6 | Claude Mythos 5.1 | Xhigh | 77.8% | $2.81Cost per attempt | — | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 7 | Claude Opus 5.5 | Xhigh | 76.4% | $1.05Cost per attempt | — | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 8 | Claude Mythos 5.1 | High | 73.8% | $1.39Cost per attempt | — | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 9 | Claude Mythos 5.2 | Med | 73.3% | $0.40Cost per attempt | frontier | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 10 | Claude Mythos 5.2 | Low | 72.4% | $0.34Cost per attempt | frontier | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 11 | Claude Opus 5.5 | High | 70.0% | $0.43Cost per attempt | — | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 12 | Claude Mythos 5.1 | Med | 69.8% | $0.89Cost per attempt | — | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 13 | Claude Opus 5.5 | Med | 66.1% | $0.28Cost per attempt | frontier | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 14 | Claude Mythos 5.1 | Low | 65.9% | $0.59Cost per attempt | — | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 15 | Claude Opus 5.5 | Low | 59.4% | $0.11Cost per attempt | frontier | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
OtherCyScenarioBench
official results (Introducing Claude Fable 5.2 and Claude Mythos 5.2)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Claude Mythos 5.2 | not stated | 76.3% | — | — | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 2 | Claude Opus 5.5 | not stated | 67.6% | — | — | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 3 | Claude Mythos 5.1 | not stated | 61.7% | — | — | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
OtherDe novo binder design (24 hours)
official results (Introducing Claude Fable 5.2 and Claude Mythos 5.2)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Claude Mythos 5.2 | not stated | 85.1 | — | — | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 2 | Claude Opus 5.5 | not stated | 82.6 | — | — | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 3 | Claude Mythos 5.1 | not stated | 79.1 | — | — | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
OtherRoboDojo Sim
official results (Introducing Claude Fable 5.2 and Claude Mythos 5.2)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Claude Fable 5.2 | not stated | 64.9% | — | — | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 2 | Claude Opus 5.5 | not stated | 39.4% | — | — | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 3 | GPT-6 Astra | not stated | 29.8% | — | — | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
| 4 | Claude Fable 5.1 | not stated | 27.6% | — | — | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 |
OtherArtificial Analysis Intelligence Index v5.0
official results (Gemini 4 Carbon (Google DeepMind models page, benchmark table))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Gemini 4 Carbon | High | 59 | — | — | Artificial Analysis | Gemini 4 Carbon (Google DeepMind models page, benchmark table)Retrieved 2026-11-12 |
| 2 | Gemini 4 Argon | High | 50 | — | — | Artificial Analysis | Gemini 4 Carbon (Google DeepMind models page, benchmark table)Retrieved 2026-11-12 |
OtherVals Index v2.2
official results (Gemini 4 Carbon (Google DeepMind models page, benchmark table))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Gemini 4 Carbon | High | 70.4% | — | — | Vals AI | Gemini 4 Carbon (Google DeepMind models page, benchmark table)Retrieved 2026-11-12 |
| 2 | Gemini 4 Argon | High | 69.2% | — | — | Vals AI | Gemini 4 Carbon (Google DeepMind models page, benchmark table)Retrieved 2026-11-12 |
OtherVending-Bench 2
official results (Gemini 4 Carbon (Google DeepMind models page, benchmark table))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Gemini 4 Carbon | High | 16,484 | — | — | Andon Labs | Gemini 4 Carbon (Google DeepMind models page, benchmark table)Retrieved 2026-11-12 |
| 2 | Gemini 4 Argon | High | 13,718 | — | — | Andon Labs | Gemini 4 Carbon (Google DeepMind models page, benchmark table)Retrieved 2026-11-12 |
OtherExploitGym
official results (Gemini 4 Carbon (Google DeepMind models page, benchmark table))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Gemini 4 Carbon | High | 54.8% | — | — | Gemini 4 Carbon (Google DeepMind models page, benchmark table)Retrieved 2026-11-12 | |
| 2 | Gemini 4 Argon | High | 42.0% | — | — | Gemini 4 Carbon (Google DeepMind models page, benchmark table)Retrieved 2026-11-12 |
official results (Kimi K3.1 Pro (Hugging Face model card, moonshotai))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Kimi K3.1 Pro | Max | 20.7% | — | — | Moonshot AI | Kimi K3.1 Pro (Hugging Face model card, moonshotai)Retrieved 2026-11-13 |
OtherSEC-bench Pro
official results (Gemini 4 Carbon (Google DeepMind models page, benchmark table))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Gemini 4 Carbon | High | 88.2% | — | — | Gemini 4 Carbon (Google DeepMind models page, benchmark table)Retrieved 2026-11-12 | |
| 2 | Gemini 4 Argon | High | 83.4% | — | — | Gemini 4 Carbon (Google DeepMind models page, benchmark table)Retrieved 2026-11-12 |
official results (DeepSeek-V4.1-Flash (Hugging Face model card, deepseek-ai))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | DeepSeek-V4.1-Flash | not stated | 62.8% | — | — | DeepSeek | DeepSeek-V4.1-Flash (Hugging Face model card, deepseek-ai)Retrieved 2026-11-13 |
OtherPostTrainBench
official results (Gemini 4 Argon model card)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Gemini 4 Argon | not stated | 45.3% | — | — | Gemini 4 Argon model cardRetrieved 2026-11-12 |
official results (Kimi K3 (Hugging Face model card, moonshotai))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Kimi K3 | Max | 36.6% | — | — | Moonshot AI | Kimi K3 (Hugging Face model card, moonshotai)Retrieved 2026-11-13 |
official results (GLM-5.3 (Hugging Face model card, zai-org))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | GLM-5.3 | Max | 39.8% | — | — | Z.ai | GLM-5.3 (Hugging Face model card, zai-org)Retrieved 2026-11-13 |
OtherEEBench
official results (Grok 4.7 (xAI news))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Grok 4.7 | Xhigh | 64.0% | — | — | xAI | Grok 4.7 (xAI news)Retrieved 2026-11-12 |
| 2 | Grok 4.6 | High | 53.0% | — | — | xAI | Grok 4.7 (xAI news)Retrieved 2026-11-12 |
OtherCyberGym
official results (GLM-5.3 (Z.ai docs))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | GLM-5.3 | not stated | 84.5% | — | — | Z.ai | GLM-5.3 (Z.ai docs)Retrieved 2026-11-12 |
official results (Introducing Claude Opus 4.7)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Claude Opus 4.6 | not stated | 73.8% | — | — | Anthropic | Introducing Claude Opus 4.7Retrieved 2026-11-13 |
OtherExploitBench
official results (GLM-5.3 (Z.ai docs))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | GLM-5.3 | not stated | 54.4% | — | — | Z.ai | GLM-5.3 (Z.ai docs)Retrieved 2026-11-12 |
OtherCybench
official results (Introducing Mistral Large 4)
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Mistral Large 4 | not stated | 93.0% | — | — | Mistral AI | Introducing Mistral Large 4Retrieved 2026-11-12 |
OtherEpoch Capabilities Index(ECI)
third-party results (Epoch AI, epoch_capabilities_index/eci_scores.csv (CC BY 4.0))
OtherPostTrainBench v1.2
official results (Kimi K3.1 Pro (Hugging Face model card, moonshotai))
| Rank | Model | effort | Score | Source cost (metric) | frontier | measured/reported by | source |
|---|---|---|---|---|---|---|---|
| 1 | Kimi K3.1 Pro | Max | 36.5% | — | — | Moonshot AI | Kimi K3.1 Pro (Hugging Face model card, moonshotai)Retrieved 2026-11-13 |