OTIS Mock AIME 2024-2025
AIME-style mathematics competition problems (run by Epoch AI).
- Category
- Math
- Unit
- Percent (%)
- Higher is better
- Comparison groups
- 1total
- Currently available model rankings · Ranked results: 9
- All models (records): Groups: 1 · Entries: 67
Rankings
Values with different source types (official, third-party, paper) or measurement conditions cannot be compared, so rankings are separated by comparison group (the same source, or sources verified to use the same conditions).
Rankings include only models currently available in Claude Code / Codex, excluding legacy models.
- View on rankings page (by metric)
- Cost balance (frontier and effort changes)
- Overall ranking (performance first)
third-party results (Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0)
| Rank | Model | effort | Score | Source cost (metric) | measured/reported by | source |
|---|---|---|---|---|---|---|
| 1 | GPT-6.1 Terra | Max | 100.0% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "GPT-6.1 Terra (max)". effort is indicated by the suffix _max in Epoch's Model version. Data retrieved 2026-11-12.Source excerptotis_mock_aime_2024_2025.csv: Model version=gpt-6.1-terra_max, Best score (across scorers)=1.0, stderr=0.0, Started at=2026-10-26T20:00:37.000Z |
| 1 | GPT-6.1 Sol | Max | 100.0% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "GPT-6.1 Sol (max)". effort is indicated by the suffix _max in Epoch's Model version. Data retrieved 2026-11-12.Source excerptotis_mock_aime_2024_2025.csv: Model version=gpt-6.1-sol_max, Best score (across scorers)=1.0, stderr=0.0, Started at=2026-09-29T22:00:34.000Z |
| 1 | Claude Sonnet 5.5 | Max | 100.0% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "Claude Sonnet 5.5 (max)". effort is indicated by the suffix _max in Epoch's Model version. Data retrieved 2026-11-12.Source excerptotis_mock_aime_2024_2025.csv: Model version=claude-sonnet-5-5_max, Best score (across scorers)=1.0, stderr=0.0, Started at=2026-09-29T15:15:14.000Z |
| 1 | Claude Opus 5.5 | Max | 100.0% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "Claude Opus 5.5 (max)". effort is indicated by the suffix _max in Epoch's Model version. Data retrieved 2026-11-12.Source excerptotis_mock_aime_2024_2025.csv: Model version=claude-opus-5-5_max, Best score (across scorers)=1.0, stderr=0.0, Started at=2026-09-22T18:27:39.000Z |
| 1 | GPT-6 Astra | Max | 100.0% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "GPT-6 Astra (max)". effort is indicated by the suffix _max in Epoch's Model version. Data retrieved 2026-11-12.Source excerptotis_mock_aime_2024_2025.csv: Model version=gpt-6-astra_max, Best score (across scorers)=1.0, stderr=0.0, Started at=2026-08-30T14:57:43.000Z |
| 6 | Claude Fable 5.2 | Max | 99.7% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "Claude Fable 5.2 (max)". effort is indicated by the suffix _max in Epoch's Model version. Data retrieved 2026-11-12.Source excerptotis_mock_aime_2024_2025.csv: Model version=claude-fable-5-2_max, Best score (across scorers)=0.9972222222222222, stderr=0.002777777777777778, Started at=2026-11-06T01:14:45.000Z |
| 7 | Claude Haiku 5.5 | Xhigh | 98.9% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "Claude Haiku 5.5 (xhigh)". effort is indicated by the suffix _xhigh in Epoch's Model version. Data retrieved 2026-11-12.Source excerptotis_mock_aime_2024_2025.csv: Model version=claude-haiku-5-5_xhigh, Best score (across scorers)=0.9888888888888889, stderr=0.005362818328685681, Started at=2026-10-07T21:16:36.000Z |
| 7 | GPT-6 Luna | Max | 98.9% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "GPT-6 Luna (max)". effort is indicated by the suffix _max in Epoch's Model version. Data retrieved 2026-11-12.Source excerptotis_mock_aime_2024_2025.csv: Model version=gpt-6-luna_max, Best score (across scorers)=0.9888888888888889, stderr=0.006674027922425511, Started at=2026-09-22T22:14:05.000Z |
| 9 | Claude Haiku 5.5 | High | 97.2% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "Claude Haiku 5.5 (high)". effort is indicated by the suffix _high in Epoch's Model version. Data retrieved 2026-11-12.Source excerptotis_mock_aime_2024_2025.csv: Model version=claude-haiku-5-5_high, Best score (across scorers)=0.9722222222222222, stderr=0.013109441770118595, Started at=2026-10-07T21:16:25.000Z |
All model rankings (records)
These rankings are recalculated by comparison group including legacy models and models unavailable in Claude Code / Codex. They differ from the rankings above because the set of models differs.
- View all model rankings (by metric)
- Cost balance (all models, records)
- Overall ranking (all models, records)
Show rankings including legacy and unavailable models (entries: 67)
third-party results (Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0)
| Rank | Model | effort | Score | Source cost (metric) | measured/reported by | source |
|---|---|---|---|---|---|---|
| 1 | Gemini 4 Carbon | High | 100.0% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "Gemini 4 Carbon (high)". effort is indicated by the suffix _high in Epoch's Model version. Data retrieved 2026-11-12.Source excerptotis_mock_aime_2024_2025.csv: Model version=gemini-4-carbon_high, Best score (across scorers)=1.0, stderr=0.0, Started at=2026-11-02T23:17:42.000Z |
| 1 | GPT-6.1 Terra | Max | 100.0% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "GPT-6.1 Terra (max)". effort is indicated by the suffix _max in Epoch's Model version. Data retrieved 2026-11-12.Source excerptotis_mock_aime_2024_2025.csv: Model version=gpt-6.1-terra_max, Best score (across scorers)=1.0, stderr=0.0, Started at=2026-10-26T20:00:37.000Z |
| 1 | Gemini 4 Argon | High | 100.0% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "Gemini 4 Argon (high)". effort is indicated by the suffix _high in Epoch's Model version. Data retrieved 2026-11-12.Source excerptotis_mock_aime_2024_2025.csv: Model version=gemini-4-argon_high, Best score (across scorers)=1.0, stderr=0.0, Started at=2026-10-13T17:14:57.000Z |
| 1 | GPT-6.1 Sol | Max | 100.0% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "GPT-6.1 Sol (max)". effort is indicated by the suffix _max in Epoch's Model version. Data retrieved 2026-11-12.Source excerptotis_mock_aime_2024_2025.csv: Model version=gpt-6.1-sol_max, Best score (across scorers)=1.0, stderr=0.0, Started at=2026-09-29T22:00:34.000Z |
| 1 | Claude Sonnet 5.5 | Max | 100.0% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "Claude Sonnet 5.5 (max)". effort is indicated by the suffix _max in Epoch's Model version. Data retrieved 2026-11-12.Source excerptotis_mock_aime_2024_2025.csv: Model version=claude-sonnet-5-5_max, Best score (across scorers)=1.0, stderr=0.0, Started at=2026-09-29T15:15:14.000Z |
| 1 | Claude Opus 5.5 | Max | 100.0% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "Claude Opus 5.5 (max)". effort is indicated by the suffix _max in Epoch's Model version. Data retrieved 2026-11-12.Source excerptotis_mock_aime_2024_2025.csv: Model version=claude-opus-5-5_max, Best score (across scorers)=1.0, stderr=0.0, Started at=2026-09-22T18:27:39.000Z |
| 1 | GPT-6 Astra | Max | 100.0% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "GPT-6 Astra (max)". effort is indicated by the suffix _max in Epoch's Model version. Data retrieved 2026-11-12.Source excerptotis_mock_aime_2024_2025.csv: Model version=gpt-6-astra_max, Best score (across scorers)=1.0, stderr=0.0, Started at=2026-08-30T14:57:43.000Z |
| 1 | Kimi K3.1 Pro | Max | 100.0% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "Kimi K3.1 Pro (Together, max)". effort is indicated by the suffix _max in Epoch's Model version. Data retrieved 2026-11-12. Epoch ran it via Together's inference API (Model version kimi-k3.1-pro-together).Source excerptotis_mock_aime_2024_2025.csv: Model version=kimi-k3.1-pro-together_max, Best score (across scorers)=1.0, stderr=0.0, Started at=2026-11-06T00:59:23.000ZLocation otis_mock_aime_2024_2025.csv, data row 2 (file line 3), id=8Qe2qf4Eg77PMfp2Eh7V6K, Model version=kimi-k3.1-pro-together_max |
| 1 | GPT-6 Sol | Max | 100.0% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "GPT-6 Sol (max)". effort is indicated by the suffix _max in Epoch's Model version. Data retrieved 2026-11-12.Source excerptotis_mock_aime_2024_2025.csv: Model version=gpt-6-sol_max, Best score (across scorers)=1.0, stderr=0.0, Started at=2026-09-22T19:59:24.000ZLocation otis_mock_aime_2024_2025.csv, data row 24 (file line 48), id=VZAUcNkzeMSMm9paxBhj6B, Model version=gpt-6-sol_max |
| 1 | Claude Fable 5 | High | 100.0% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "Claude Fable 5 (high)". effort is indicated by the suffix _high in Epoch's Model version. Data retrieved 2026-11-12.Source excerptotis_mock_aime_2024_2025.csv: Model version=claude-fable-5_high, Best score (across scorers)=1.0, stderr=0.0, Started at=2026-08-06T22:18:19.000ZLocation otis_mock_aime_2024_2025.csv, data row 149 (file line 214), id=hwrnvtEbDAocFtjcsHqsQh, Model version=claude-fable-5_high |
| 1 | GPT-5.6 Sol | Max | 100.0% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "GPT-5.6 Sol (max)". effort is indicated by the suffix _max in Epoch's Model version. Data retrieved 2026-11-12.Source excerptotis_mock_aime_2024_2025.csv: Model version=gpt-5.6-sol_max, Best score (across scorers)=1.0, stderr=0.0, Started at=2026-07-09T02:07:13.000ZLocation otis_mock_aime_2024_2025.csv, data row 178 (file line 253), id=7RjsANKwwsePSstcZeSMhX, Model version=gpt-5.6-sol_max |
| 12 | Claude Fable 5.2 | Max | 99.7% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "Claude Fable 5.2 (max)". effort is indicated by the suffix _max in Epoch's Model version. Data retrieved 2026-11-12.Source excerptotis_mock_aime_2024_2025.csv: Model version=claude-fable-5-2_max, Best score (across scorers)=0.9972222222222222, stderr=0.002777777777777778, Started at=2026-11-06T01:14:45.000Z |
| 12 | GPT-5.6 Terra | Max | 99.7% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "GPT-5.6 Terra (max)". effort is indicated by the suffix _max in Epoch's Model version. Data retrieved 2026-11-12.Source excerptotis_mock_aime_2024_2025.csv: Model version=gpt-5.6-terra_max, Best score (across scorers)=0.9972222222222222, stderr=0.002777777777777778, Started at=2026-07-09T17:07:54.000ZLocation otis_mock_aime_2024_2025.csv, data row 176 (file line 251), id=jUmmCNezoW7jWiK72SVCaP, Model version=gpt-5.6-terra_max |
| 12 | Claude Fable 5 | Max | 99.7% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "Claude Fable 5 (max)". effort is indicated by the suffix _max in Epoch's Model version. Data retrieved 2026-11-12.Source excerptotis_mock_aime_2024_2025.csv: Model version=claude-fable-5_max, Best score (across scorers)=0.9972222222222222, stderr=0.002777777777777778, Started at=2026-06-10T20:03:17.000ZLocation otis_mock_aime_2024_2025.csv, data row 184 (file line 259), id=eiLEpRUvK48wUKDh4sYkzV, Model version=claude-fable-5_max |
| 15 | Qwen3.8-Max | Xhigh | 99.4% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "Qwen3.8 Max (xhigh)". effort is indicated by the suffix _xhigh in Epoch's Model version. Data retrieved 2026-11-12.Source excerptotis_mock_aime_2024_2025.csv: Model version=qwen3.8-max_xhigh, Best score (across scorers)=0.9944444444444445, stderr=0.0038834738634418433, Started at=2026-08-04T16:28:46.000ZLocation otis_mock_aime_2024_2025.csv, data row 152 (file line 217), id=aGRKHaq9uSawfYVpC565zU, Model version=qwen3.8-max_xhigh |
| 15 | DeepSeek-V4.1-Flash-1017 | Max | 99.4% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "DeepSeek V4.1 Flash 1017 (max)". effort is indicated by the suffix _max in Epoch's Model version. Data retrieved 2026-11-12.Source excerptotis_mock_aime_2024_2025.csv: Model version=deepseek-v4.1-flash-1017_max, Best score (across scorers)=0.9944444444444445, stderr=0.003883473863441843, Started at=2026-10-19T22:39:00.000ZLocation otis_mock_aime_2024_2025.csv, data row 12 (file line 24), id=chayEvHvpt2Wcif7pg64hF, Model version=deepseek-v4.1-flash-1017_max |
| 17 | Grok 4.6 | Xhigh | 99.2% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "Grok 4.6 (xhigh)". effort is indicated by the suffix _xhigh in Epoch's Model version. Data retrieved 2026-11-12.Source excerptotis_mock_aime_2024_2025.csv: Model version=grok-4.6_xhigh, Best score (across scorers)=0.9916666666666667, stderr=0.004700633956814717, Started at=2026-08-14T13:47:42.000ZLocation otis_mock_aime_2024_2025.csv, data row 82 (file line 142), id=7crLvnfQc987dC6mr3iGbf, Model version=grok-4.6_xhigh |
| 18 | Claude Haiku 5.5 | Xhigh | 98.9% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "Claude Haiku 5.5 (xhigh)". effort is indicated by the suffix _xhigh in Epoch's Model version. Data retrieved 2026-11-12.Source excerptotis_mock_aime_2024_2025.csv: Model version=claude-haiku-5-5_xhigh, Best score (across scorers)=0.9888888888888889, stderr=0.005362818328685681, Started at=2026-10-07T21:16:36.000Z |
| 18 | GPT-6 Luna | Max | 98.9% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "GPT-6 Luna (max)". effort is indicated by the suffix _max in Epoch's Model version. Data retrieved 2026-11-12.Source excerptotis_mock_aime_2024_2025.csv: Model version=gpt-6-luna_max, Best score (across scorers)=0.9888888888888889, stderr=0.006674027922425511, Started at=2026-09-22T22:14:05.000Z |
| 18 | Gemini 3.8 Flash | High | 98.9% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "Gemini 3.8 Flash (high)". effort is indicated by the suffix _high in Epoch's Model version. Data retrieved 2026-11-12.Source excerptotis_mock_aime_2024_2025.csv: Model version=gemini-3.8-flash_high, Best score (across scorers)=0.9888888888888889, stderr=0.008724007409153888, Started at=2026-09-02T17:15:36.000ZLocation otis_mock_aime_2024_2025.csv, data row 28 (file line 52), id=NeYuqgQyWtFWAHEjQYPm5Z, Model version=gemini-3.8-flash_high |
| 21 | DeepSeek-V4-Pro-0813 | Max | 98.6% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "DeepSeek V4 Pro 0813 (max)". effort is indicated by the suffix _max in Epoch's Model version. Data retrieved 2026-11-12.Source excerptotis_mock_aime_2024_2025.csv: Model version=deepseek-v4-pro-0813_max, Best score (across scorers)=0.9861111111111112, stderr=0.00990980650017605, Started at=2026-08-18T21:03:23.000ZLocation otis_mock_aime_2024_2025.csv, data row 79 (file line 139), id=4HVvM9ZWKKPLQPcC8WYYMT, Model version=deepseek-v4-pro-0813_max |
| 22 | Mistral Large 4 | High | 98.3% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "Mistral Large 4 Preview (high)". effort is indicated by the suffix _high in Epoch's Model version. Data retrieved 2026-11-12. Epoch's label is Mistral Large 4 Preview (2026-10-06 version; Mistral Large 4 from 2026-10-29 has a different Model version, mistral-large-4-0-oss, and is not included here).Source excerptotis_mock_aime_2024_2025.csv: Model version=mistral-large-4-0_high, Best score (across scorers)=0.9833333333333333, stderr=0.006405884289881728, Started at=2026-10-12T21:38:35.000ZLocation otis_mock_aime_2024_2025.csv, data row 15 (file lines 27-38; multi-line record), id=Pby5WuP8Muu6Jzs2v5gB7F, Model version=mistral-large-4-0_high |
| 22 | GPT-5.6 Luna | Max | 98.3% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "GPT-5.6 Luna (max)". effort is indicated by the suffix _max in Epoch's Model version. Data retrieved 2026-11-12.Source excerptotis_mock_aime_2024_2025.csv: Model version=gpt-5.6-luna_max, Best score (across scorers)=0.9833333333333333, stderr=0.011650421590325524, Started at=2026-07-09T13:36:49.000ZLocation otis_mock_aime_2024_2025.csv, data row 177 (file line 252), id=AhtQ5ggs4pTfLDqHFjueGy, Model version=gpt-5.6-luna_max |
| 22 | Claude Opus 4.8 | Max | 98.3% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "Claude Opus 4.8". effort is indicated by the suffix _max in Epoch's Model version. Data retrieved 2026-11-12.Source excerptotis_mock_aime_2024_2025.csv: Model version=claude-opus-4-8_max, Best score (across scorers)=0.9833333333333333, stderr=0.014101901870444154, Started at=2026-06-07T23:44:54.000ZLocation otis_mock_aime_2024_2025.csv, data row 185 (file line 260), id=37pJqPpXTE6ia9ikmmGPJ4, Model version=claude-opus-4-8_max |
| 22 | DeepSeek-V4.1-Flash | not stated | 98.3% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "DeepSeek V4.1 Flash (max)". effort is indicated by the suffix _max in Epoch's Model version. Data retrieved 2026-11-12. No effort for the source effort label "max" because the official effort list has not been collected.Source excerptotis_mock_aime_2024_2025.csv: Model version=deepseek-v4.1-flash_max, Best score (across scorers)=0.9833333333333333, stderr=0.006405884289881728, Started at=2026-10-08T19:02:32.000ZLocation otis_mock_aime_2024_2025.csv, data row 17 (file lines 40-41; multi-line record), id=7eHNiVQCzLcAyj5LxmeeSU, Model version=deepseek-v4.1-flash_max |
| 26 | Grok 4.7 | Xhigh | 98.1% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "Grok 4.7 (xhigh)". effort is indicated by the suffix _xhigh in Epoch's Model version. Data retrieved 2026-11-12.Source excerptotis_mock_aime_2024_2025.csv: Model version=grok-4.7_xhigh, Best score (across scorers)=0.9805555555555555, stderr=0.014299486306926387, Started at=2026-09-22T21:49:22.000ZLocation otis_mock_aime_2024_2025.csv, data row 23 (file line 47), id=Jhgwm7QpspGcvm7DUVqUDg, Model version=grok-4.7_xhigh |
| 27 | Claude Opus 4.7 | Xhigh | 97.8% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "Claude Opus 4.7 (xhigh)". effort is indicated by the suffix _xhigh in Epoch's Model version. Data retrieved 2026-11-12.Source excerptotis_mock_aime_2024_2025.csv: Model version=claude-opus-4-7_xhigh, Best score (across scorers)=0.978, stderr=0.022, Started at=2026-04-17T18:26:45.000ZLocation otis_mock_aime_2024_2025.csv, data row 190 (file line 265), id=jyCZnpXG2bh9G43X8pT75m, Model version=claude-opus-4-7_xhigh |
| 28 | Grok 4.6 | High | 97.8% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "Grok 4.6 (high)". effort is indicated by the suffix _high in Epoch's Model version. Data retrieved 2026-11-12.Source excerptotis_mock_aime_2024_2025.csv: Model version=grok-4.6_high, Best score (across scorers)=0.9777777777777777, stderr=0.00996275330534286, Started at=2026-08-12T19:39:52.000ZLocation otis_mock_aime_2024_2025.csv, data row 83 (file line 143), id=jtfdCURnwfZzak4BSiBKzx, Model version=grok-4.6_high |
| 28 | Claude Opus 4.8 | Low | 97.8% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "Claude Opus 4.8". effort is indicated by the suffix _low in Epoch's Model version. Data retrieved 2026-11-12.Source excerptotis_mock_aime_2024_2025.csv: Model version=claude-opus-4-8_low, Best score (across scorers)=0.9777777777777777, stderr=0.022222222222222223, Started at=2026-08-06T22:20:24.000ZLocation otis_mock_aime_2024_2025.csv, data row 146 (file line 211), id=MEhV5sh6nbM8cKJGQ6HzcS, Model version=claude-opus-4-8_low |
| 28 | Claude Fable 5 | Low | 97.8% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "Claude Fable 5 (low)". effort is indicated by the suffix _low in Epoch's Model version. Data retrieved 2026-11-12.Source excerptotis_mock_aime_2024_2025.csv: Model version=claude-fable-5_low, Best score (across scorers)=0.9777777777777777, stderr=0.022222222222222223, Started at=2026-08-06T22:18:24.000ZLocation otis_mock_aime_2024_2025.csv, data row 148 (file line 213), id=6E989Xxeuw925T7cCV3XGB, Model version=claude-fable-5_low |
| 31 | Claude Haiku 5.5 | High | 97.2% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "Claude Haiku 5.5 (high)". effort is indicated by the suffix _high in Epoch's Model version. Data retrieved 2026-11-12.Source excerptotis_mock_aime_2024_2025.csv: Model version=claude-haiku-5-5_high, Best score (across scorers)=0.9722222222222222, stderr=0.013109441770118595, Started at=2026-10-07T21:16:25.000Z |
| 31 | Gemini 3.7 Flash | High | 97.2% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "Gemini 3.7 Flash (high)". effort is indicated by the suffix _high in Epoch's Model version. Data retrieved 2026-11-12.Source excerptotis_mock_aime_2024_2025.csv: Model version=gemini-3.7-flash_high, Best score (across scorers)=0.9722222222222222, stderr=0.018586784946171872, Started at=2026-08-14T18:29:12.000ZLocation otis_mock_aime_2024_2025.csv, data row 80 (file line 140), id=kQjrYCaa7yYyMXofeR23Dd, Model version=gemini-3.7-flash_high |
| 31 | Kimi K3 | Max | 97.2% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "Kimi K3 (max)". effort is indicated by the suffix _max in Epoch's Model version. Data retrieved 2026-11-12.Source excerptotis_mock_aime_2024_2025.csv: Model version=kimi-k3_max, Best score (across scorers)=0.9722222222222222, stderr=0.011158359139033182, Started at=2026-07-16T21:59:40.000ZLocation otis_mock_aime_2024_2025.csv, data row 159 (file lines 228-230; multi-line record), id=A33om9MTYMC7xYeLv9X7Ha, Model version=kimi-k3_max |
| 34 | Gemini 3.1 Pro Preview | not stated | 95.6% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "Gemini 3.1 Pro Preview". No effort suffix in Model version. Data retrieved 2026-11-12. No effort because effort is not stated in the source.Source excerptotis_mock_aime_2024_2025.csv: Model version=gemini-3.1-pro-preview, Best score (across scorers)=0.956, stderr=0.031, Started at=2026-02-20T04:39:36.998ZLocation otis_mock_aime_2024_2025.csv, data row 196 (file line 271), id=bXgZZgzRE3wXoSRUgrbsVf, Model version=gemini-3.1-pro-preview |
| 35 | GPT-5.6 Sol | Low | 95.6% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "GPT-5.6 Sol (low)". effort is indicated by the suffix _low in Epoch's Model version. Data retrieved 2026-11-12.Source excerptotis_mock_aime_2024_2025.csv: Model version=gpt-5.6-sol_low, Best score (across scorers)=0.9555555555555556, stderr=0.031067790907534736, Started at=2026-08-07T00:42:21.000ZLocation otis_mock_aime_2024_2025.csv, data row 98 (file line 162), id=Mifz96Jm442bn43Qnxj5rT, Model version=gpt-5.6-sol_low |
| 35 | Gemini 3.1 Pro Preview | High | 95.6% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "Gemini 3.1 Pro Preview". effort is indicated by the suffix _high in Epoch's Model version. Data retrieved 2026-11-12.Source excerptotis_mock_aime_2024_2025.csv: Model version=gemini-3.1-pro-preview_high, Best score (across scorers)=0.9555555555555556, stderr=0.031067790907534736, Started at=2026-08-06T23:59:11.000ZLocation otis_mock_aime_2024_2025.csv, data row 129 (file line 194), id=nhim2QCRgpir5cZdqrmUz3, Model version=gemini-3.1-pro-preview_high |
| 35 | Gemini 3 Flash Preview | High | 95.6% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "Gemini 3 Flash Preview". effort is indicated by the suffix _high in Epoch's Model version. Data retrieved 2026-11-12.Source excerptotis_mock_aime_2024_2025.csv: Model version=gemini-3-flash-preview_high, Best score (across scorers)=0.9555555555555556, stderr=0.031067790907534743, Started at=2026-08-06T22:25:57.000ZLocation otis_mock_aime_2024_2025.csv, data row 140 (file line 205), id=a4uE6KgMhGSUjcnemRnbkA, Model version=gemini-3-flash-preview_high |
| 38 | Gemini 3.6 Flash | High | 94.2% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "Gemini 3.6 Flash". effort is indicated by the suffix _high in Epoch's Model version. Data retrieved 2026-11-12.Source excerptotis_mock_aime_2024_2025.csv: Model version=gemini-3.6-flash_high, Best score (across scorers)=0.9416666666666667, stderr=0.030926354359015452, Started at=2026-08-02T00:21:27.000ZLocation otis_mock_aime_2024_2025.csv, data row 155 (file line 222), id=ajQFZGMjvyMTH2ULDZhwtz, Model version=gemini-3.6-flash_high |
| 39 | GLM-5.3-Flash | Max | 93.9% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "GLM-5.3-Flash (max)". effort is indicated by the suffix _max in Epoch's Model version. Data retrieved 2026-11-12.Source excerptotis_mock_aime_2024_2025.csv: Model version=glm-5.3-flash_max, Best score (across scorers)=0.9388888888888889, stderr=0.021622267380853318, Started at=2026-08-26T22:26:55.000ZLocation otis_mock_aime_2024_2025.csv, data row 77 (file lines 135-137; multi-line record), id=NCikvVWc5SEyMEbCqvVNb2, Model version=glm-5.3-flash_max |
| 40 | Kimi K3 | High | 93.3% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "Kimi K3 (high)". effort is indicated by the suffix _high in Epoch's Model version. Data retrieved 2026-11-12.Source excerptotis_mock_aime_2024_2025.csv: Model version=kimi-k3_high, Best score (across scorers)=0.9333333333333333, stderr=0.03760507165451775, Started at=2026-08-07T00:46:05.000ZLocation otis_mock_aime_2024_2025.csv, data row 94 (file lines 156-158; multi-line record), id=4E5xGsuRMwMBhAfC4F2G4N, Model version=kimi-k3_high |
| 41 | Gemini 3 Flash Preview | not stated | 92.8% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "Gemini 3 Flash Preview". No effort suffix in Model version. Data retrieved 2026-11-12. No effort because effort is not stated in the source.Source excerptotis_mock_aime_2024_2025.csv: Model version=gemini-3-flash-preview, Best score (across scorers)=0.9277777777777778, stderr=0.035196259111492545, Started at=2025-12-17T22:40:30.896ZLocation otis_mock_aime_2024_2025.csv, data row 202 (file line 289), id=H3E7EjjXfGCPnm9ooLdmAo, Model version=gemini-3-flash-preview |
| 42 | GLM-5.3 | Max | 91.1% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "GLM-5.3 (max)". effort is indicated by the suffix _max in Epoch's Model version. Data retrieved 2026-11-12.Source excerptotis_mock_aime_2024_2025.csv: Model version=glm-5.3_max, Best score (across scorers)=0.9111111111111111, stderr=0.03454754040013868, Started at=2026-08-24T19:44:18.000ZLocation otis_mock_aime_2024_2025.csv, data row 78 (file line 138), id=goZR5yGxZic9TZ23ezCL3w, Model version=glm-5.3_max |
| 42 | Claude Opus 4.6 | Max | 91.1% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "Claude Opus 4.6 (max)". effort is indicated by the suffix _max in Epoch's Model version. Data retrieved 2026-11-12.Source excerptotis_mock_aime_2024_2025.csv: Model version=claude-opus-4-6_max, Best score (across scorers)=0.9111111111111111, stderr=0.04290254662948546, Started at=2026-08-06T23:58:17.000ZLocation otis_mock_aime_2024_2025.csv, data row 132 (file line 197), id=coxYk5vK5cBhQ7iZohMtA5, Model version=claude-opus-4-6_max |
| 44 | GPT-5.6 Terra | Low | 88.9% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "GPT-5.6 Terra (low)". effort is indicated by the suffix _low in Epoch's Model version. Data retrieved 2026-11-12.Source excerptotis_mock_aime_2024_2025.csv: Model version=gpt-5.6-terra_low, Best score (across scorers)=0.8888888888888888, stderr=0.04737793696791343, Started at=2026-08-07T00:20:21.000ZLocation otis_mock_aime_2024_2025.csv, data row 104 (file line 169), id=Z6S3nMC2tnkpKUixCPLiDj, Model version=gpt-5.6-terra_low |
| 45 | Claude Opus 4.7 | Max | 86.7% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "Claude Opus 4.7 (max)". effort is indicated by the suffix _max in Epoch's Model version. Data retrieved 2026-11-12.Source excerptotis_mock_aime_2024_2025.csv: Model version=claude-opus-4-7_max, Best score (across scorers)=0.8666666666666667, stderr=0.051247074319053836, Started at=2026-08-06T22:19:46.000ZLocation otis_mock_aime_2024_2025.csv, data row 147 (file line 212), id=T4y4iGHMqA8tXCdBKr5GjS, Model version=claude-opus-4-7_max |
| 46 | Gemini 2.5 Pro | not stated | 84.7% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "gemini-2.5-pro". No effort suffix in Model version. Data retrieved 2026-11-12. No effort because effort is not stated in the source.Source excerptotis_mock_aime_2024_2025.csv: Model version=gemini-2.5-pro, Best score (across scorers)=0.8472222222222222, stderr=0.048143196054711346, Started at=2025-11-16T19:00:06.794ZLocation otis_mock_aime_2024_2025.csv, data row 216 (file line 313), id=iJp7PgYZa3PiXAV8UWd7MC, Model version=gemini-2.5-pro |
| 47 | GPT-5.5 | Low | 84.4% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "GPT-5.5 (low)". effort is indicated by the suffix _low in Epoch's Model version. Data retrieved 2026-11-12.Source excerptotis_mock_aime_2024_2025.csv: Model version=gpt-5.5_low, Best score (across scorers)=0.8444444444444444, stderr=0.05463890236888291, Started at=2026-08-07T00:39:05.000ZLocation otis_mock_aime_2024_2025.csv, data row 99 (file line 163), id=CdmtwZ8NR8YeTXWiLVTRDJ, Model version=gpt-5.5_low |
| 47 | Claude Opus 4.8 | not stated | 84.4% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "Claude Opus 4.8". effort is indicated by the suffix _none in Epoch's Model version. Data retrieved 2026-11-12. No effort because the source effort label "none" is not in the official effort list.Source excerptotis_mock_aime_2024_2025.csv: Model version=claude-opus-4-8_none, Best score (across scorers)=0.8444444444444444, stderr=0.05463890236888291, Started at=2026-08-06T22:21:06.000ZLocation otis_mock_aime_2024_2025.csv, data row 145 (file line 210), id=adGE8rXdNuUFs96ioppbc5, Model version=claude-opus-4-8_none |
| 49 | Gemini 3.6 Flash | Low | 82.2% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "Gemini 3.6 Flash". effort is indicated by the suffix _low in Epoch's Model version. Data retrieved 2026-11-12.Source excerptotis_mock_aime_2024_2025.csv: Model version=gemini-3.6-flash_low, Best score (across scorers)=0.8222222222222222, stderr=0.057637747950250916, Started at=2026-08-07T00:07:20.000ZLocation otis_mock_aime_2024_2025.csv, data row 114 (file line 179), id=6iEkMa88YjLDgcde4bLm8B, Model version=gemini-3.6-flash_low |
| 49 | Claude Sonnet 4.6 | Med | 82.2% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "Claude Sonnet 4.6 (medium)". effort is indicated by the suffix _medium in Epoch's Model version. Data retrieved 2026-11-12.Source excerptotis_mock_aime_2024_2025.csv: Model version=claude-sonnet-4-6_medium, Best score (across scorers)=0.8222222222222222, stderr=0.057637747950250916, Started at=2026-07-13T17:28:00.000ZLocation otis_mock_aime_2024_2025.csv, data row 173 (file line 248), id=kDNNbUYXhrAH5uDBGykFaf, Model version=claude-sonnet-4-6_medium |
| 51 | Gemini 3.6 Flash | Minimal | 80.0% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "Gemini 3.6 Flash". effort is indicated by the suffix _minimal in Epoch's Model version. Data retrieved 2026-11-12.Source excerptotis_mock_aime_2024_2025.csv: Model version=gemini-3.6-flash_minimal, Best score (across scorers)=0.8, stderr=0.06030226891555273, Started at=2026-08-07T00:07:32.000ZLocation otis_mock_aime_2024_2025.csv, data row 113 (file line 178), id=cF97JqHbFxxp7ePjYVK9HV, Model version=gemini-3.6-flash_minimal |
| 51 | Gemini 3.1 Flash-Lite | High | 80.0% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "Gemini 3.1 Flash-Lite". effort is indicated by the suffix _high in Epoch's Model version. Data retrieved 2026-11-12.Source excerptotis_mock_aime_2024_2025.csv: Model version=gemini-3.1-flash-lite_high, Best score (across scorers)=0.8, stderr=0.06030226891555273, Started at=2026-08-06T22:29:57.000ZLocation otis_mock_aime_2024_2025.csv, data row 139 (file line 204), id=NMdaAXyqqPRmWfuxZUfHFF, Model version=gemini-3.1-flash-lite_high |
| 53 | Claude Sonnet 4.6 | High | 75.6% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "Claude Sonnet 4.6 (high)". effort is indicated by the suffix _high in Epoch's Model version. Data retrieved 2026-11-12.Source excerptotis_mock_aime_2024_2025.csv: Model version=claude-sonnet-4-6_high, Best score (across scorers)=0.7555555555555555, stderr=0.06478835438717, Started at=2026-07-13T17:28:15.000ZLocation otis_mock_aime_2024_2025.csv, data row 170 (file line 245), id=oXZ7ZVZtADrwFPKAe6zTvL, Model version=claude-sonnet-4-6_high |
| 54 | Gemini 3.5 Flash-Lite | High | 71.1% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "Gemini 3.5 Flash-Lite". effort is indicated by the suffix _high in Epoch's Model version. Data retrieved 2026-11-12.Source excerptotis_mock_aime_2024_2025.csv: Model version=gemini-3.5-flash-lite_high, Best score (across scorers)=0.7111111111111111, stderr=0.0683294324254051, Started at=2026-08-06T22:35:36.000ZLocation otis_mock_aime_2024_2025.csv, data row 135 (file line 200), id=mqsc4vFuKtAHXxbYyX9WwK, Model version=gemini-3.5-flash-lite_high |
| 54 | Claude Sonnet 4.6 | Max | 71.1% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "Claude Sonnet 4.6 (max)". effort is indicated by the suffix _max in Epoch's Model version. Data retrieved 2026-11-12.Source excerptotis_mock_aime_2024_2025.csv: Model version=claude-sonnet-4-6_max, Best score (across scorers)=0.7111111111111111, stderr=0.06832943242540508, Started at=2026-08-06T22:21:16.000ZLocation otis_mock_aime_2024_2025.csv, data row 143 (file line 208), id=XnQVH5qSfxVjpzoPwUEmnh, Model version=claude-sonnet-4-6_max |
| 56 | GPT-5.6 Sol | None | 68.9% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "GPT-5.6 Sol (none)". effort is indicated by the suffix _none in Epoch's Model version. Data retrieved 2026-11-12.Source excerptotis_mock_aime_2024_2025.csv: Model version=gpt-5.6-sol_none, Best score (across scorers)=0.6888888888888889, stderr=0.0697920592732311, Started at=2026-08-07T00:54:12.000ZLocation otis_mock_aime_2024_2025.csv, data row 91 (file line 151), id=g7czyLHudHegzP9yRD939X, Model version=gpt-5.6-sol_none |
| 56 | Kimi K3 | Low | 68.9% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "Kimi K3 (low)". effort is indicated by the suffix _low in Epoch's Model version. Data retrieved 2026-11-12.Source excerptotis_mock_aime_2024_2025.csv: Model version=kimi-k3_low, Best score (across scorers)=0.6888888888888889, stderr=0.06979205927323111, Started at=2026-08-07T00:46:06.000ZLocation otis_mock_aime_2024_2025.csv, data row 93 (file lines 153-155; multi-line record), id=frszScNxpHXXjSHawCt9Zb, Model version=kimi-k3_low |
| 58 | GPT-5.6 Luna | Low | 66.7% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "GPT-5.6 Luna (low)". effort is indicated by the suffix _low in Epoch's Model version. Data retrieved 2026-11-12.Source excerptotis_mock_aime_2024_2025.csv: Model version=gpt-5.6-luna_low, Best score (across scorers)=0.6666666666666666, stderr=0.07106690545187014, Started at=2026-08-07T00:02:15.000ZLocation otis_mock_aime_2024_2025.csv, data row 121 (file line 186), id=ZgU8BagFmaj7BE94y3JDZA, Model version=gpt-5.6-luna_low |
| 59 | Gemini 3.5 Flash-Lite | Low | 60.0% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "Gemini 3.5 Flash-Lite". effort is indicated by the suffix _low in Epoch's Model version. Data retrieved 2026-11-12.Source excerptotis_mock_aime_2024_2025.csv: Model version=gemini-3.5-flash-lite_low, Best score (across scorers)=0.6, stderr=0.07385489458759963, Started at=2026-08-06T22:36:02.000ZLocation otis_mock_aime_2024_2025.csv, data row 133 (file line 198), id=B3Kno3buYWRYbZxU33STiX, Model version=gemini-3.5-flash-lite_low |
| 60 | GPT-5.5 | None | 57.8% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "GPT-5.5 (no thinking)". effort is indicated by the suffix _none in Epoch's Model version. Data retrieved 2026-11-12.Source excerptotis_mock_aime_2024_2025.csv: Model version=gpt-5.5_none, Best score (across scorers)=0.5777777777777777, stderr=0.07446027270295805, Started at=2026-08-07T00:53:58.000ZLocation otis_mock_aime_2024_2025.csv, data row 92 (file line 152), id=QuqPmMUTbiEsUvoFYZx5Mk, Model version=gpt-5.5_none |
| 61 | GPT-5.6 Terra | None | 53.3% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "GPT-5.6 Terra (none)". effort is indicated by the suffix _none in Epoch's Model version. Data retrieved 2026-11-12.Source excerptotis_mock_aime_2024_2025.csv: Model version=gpt-5.6-terra_none, Best score (across scorers)=0.5333333333333333, stderr=0.07521014330903548, Started at=2026-08-07T00:42:38.000ZLocation otis_mock_aime_2024_2025.csv, data row 95 (file line 159), id=cjUVqF2qQJ4qhFKBU5BQpW, Model version=gpt-5.6-terra_none |
| 62 | Gemini 3.5 Flash-Lite | Minimal | 51.1% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "Gemini 3.5 Flash-Lite". effort is indicated by the suffix _minimal in Epoch's Model version. Data retrieved 2026-11-12.Source excerptotis_mock_aime_2024_2025.csv: Model version=gemini-3.5-flash-lite_minimal, Best score (across scorers)=0.5111111111111111, stderr=0.07535922203472521, Started at=2026-08-06T22:35:57.000ZLocation otis_mock_aime_2024_2025.csv, data row 134 (file line 199), id=ivr2LfmdZoCMfLME7jh2L8, Model version=gemini-3.5-flash-lite_minimal |
| 63 | Claude Opus 4.5 | not stated | 48.1% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "Claude Opus 4.5 (no thinking)". No effort suffix in Model version. Data retrieved 2026-11-12. No effort because effort is not stated in the source.Source excerptotis_mock_aime_2024_2025.csv: Model version=claude-opus-4-5-20251101, Best score (across scorers)=0.48055555555555557, stderr=0.06405336755004033, Started at=2025-11-24T20:47:49.364ZLocation otis_mock_aime_2024_2025.csv, data row 211 (file line 307), id=nSb2focDRMyPJXErk3MV94, Model version=claude-opus-4-5-20251101 |
| 64 | Gemini 3.1 Flash-Lite | Low | 44.4% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "Gemini 3.1 Flash-Lite". effort is indicated by the suffix _low in Epoch's Model version. Data retrieved 2026-11-12.Source excerptotis_mock_aime_2024_2025.csv: Model version=gemini-3.1-flash-lite_low, Best score (across scorers)=0.4444444444444444, stderr=0.07491109582924914, Started at=2026-08-06T22:30:07.000ZLocation otis_mock_aime_2024_2025.csv, data row 138 (file line 203), id=CFBCC2QrkCAFBAvFj5pqPF, Model version=gemini-3.1-flash-lite_low |
| 65 | GPT-5.6 Luna | None | 40.0% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "GPT-5.6 Luna (none)". effort is indicated by the suffix _none in Epoch's Model version. Data retrieved 2026-11-12.Source excerptotis_mock_aime_2024_2025.csv: Model version=gpt-5.6-luna_none, Best score (across scorers)=0.4, stderr=0.07385489458759964, Started at=2026-08-07T00:20:15.000ZLocation otis_mock_aime_2024_2025.csv, data row 105 (file line 170), id=QYZ6wRuT7WMkaYPKc9dRPy, Model version=gpt-5.6-luna_none |
| 66 | Gemini 3.1 Flash-Lite | Minimal | 37.8% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "Gemini 3.1 Flash-Lite". effort is indicated by the suffix _minimal in Epoch's Model version. Data retrieved 2026-11-12.Source excerptotis_mock_aime_2024_2025.csv: Model version=gemini-3.1-flash-lite_minimal, Best score (across scorers)=0.37777777777777777, stderr=0.07309112127323451, Started at=2026-08-06T22:30:17.000ZLocation otis_mock_aime_2024_2025.csv, data row 137 (file line 202), id=EoSFpgFhJB7iDPLcMNuLH4, Model version=gemini-3.1-flash-lite_minimal |
| 67 | Claude Sonnet 4.5 | not stated | 35.6% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, otis_mock_aime_2024_2025.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "Claude Sonnet 4.5 (no thinking)". No effort suffix in Model version. Data retrieved 2026-11-12. No effort because effort is not stated in the source.Source excerptotis_mock_aime_2024_2025.csv: Model version=claude-sonnet-4-5-20250929, Best score (across scorers)=0.35555555555555557, stderr=0.0721639236343101, Started at=2025-09-29T18:39:45.629ZLocation otis_mock_aime_2024_2025.csv, data row 230 (file line 340), id=dEVgttSw8u6Tt2RsPWTc3C, Model version=claude-sonnet-4-5-20250929 |