SimpleQA Verified
Short-answer factual knowledge questions (verified version, run by Epoch AI).
- Category
- Knowledge
- Unit
- Percent (%)
- Higher is better
- Comparison groups
- 1total
- Currently available model rankings · Ranked results: 8
- All models (records): Groups: 1 · Entries: 38
Rankings
Values with different source types (official, third-party, paper) or measurement conditions cannot be compared, so rankings are separated by comparison group (the same source, or sources verified to use the same conditions).
Rankings include only models currently available in Claude Code / Codex, excluding legacy models.
third-party results (Epoch AI, Capabilities & benchmarking (benchmark_data.zip, simpleqa_verified.csv) CC BY 4.0)
| Rank | Model | effort | Score | Source cost (metric) | measured/reported by | source |
|---|---|---|---|---|---|---|
| 1 | Claude Fable 5.2 | Max | 94.0% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, simpleqa_verified.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "Claude Fable 5.2 (max)". effort is indicated by the suffix _max in Epoch's Model version. Data retrieved 2026-11-12.Source excerptsimpleqa_verified.csv: Model version=claude-fable-5-2_max, Best score (across scorers)=0.94, stderr=0.007513751157474972, Started at=2026-11-06T01:14:45.000Z |
| 2 | GPT-6 Astra | Max | 75.6% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, simpleqa_verified.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "GPT-6 Astra (max)". effort is indicated by the suffix _max in Epoch's Model version. Data retrieved 2026-11-12.Source excerptsimpleqa_verified.csv: Model version=gpt-6-astra_max, Best score (across scorers)=0.756, stderr=0.013588548437881386, Started at=2026-08-30T14:57:43.000Z |
| 3 | GPT-6.1 Sol | Max | 73.9% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, simpleqa_verified.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "GPT-6.1 Sol (max)". effort is indicated by the suffix _max in Epoch's Model version. Data retrieved 2026-11-12.Source excerptsimpleqa_verified.csv: Model version=gpt-6.1-sol_max, Best score (across scorers)=0.739, stderr=0.013895037677965185, Started at=2026-09-29T22:00:34.000Z |
| 4 | Claude Opus 5.5 | Max | 72.2% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, simpleqa_verified.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "Claude Opus 5.5 (max)". effort is indicated by the suffix _max in Epoch's Model version. Data retrieved 2026-11-12.Source excerptsimpleqa_verified.csv: Model version=claude-opus-5-5_max, Best score (across scorers)=0.722, stderr=0.014174516461485273, Started at=2026-09-22T18:27:39.000Z |
| 5 | GPT-6.1 Terra | Max | 57.3% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, simpleqa_verified.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "GPT-6.1 Terra (max)". effort is indicated by the suffix _max in Epoch's Model version. Data retrieved 2026-11-12.Source excerptsimpleqa_verified.csv: Model version=gpt-6.1-terra_max, Best score (across scorers)=0.573, stderr=0.01564978964446219, Started at=2026-10-26T20:00:37.000Z |
| 6 | Claude Sonnet 5.5 | Max | 46.5% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, simpleqa_verified.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "Claude Sonnet 5.5 (max)". effort is indicated by the suffix _max in Epoch's Model version. Data retrieved 2026-11-12.Source excerptsimpleqa_verified.csv: Model version=claude-sonnet-5-5_max, Best score (across scorers)=0.465, stderr=0.015780495050030086, Started at=2026-09-29T15:15:14.000Z |
| 7 | GPT-6 Luna | Max | 41.4% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, simpleqa_verified.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "GPT-6 Luna (max)". effort is indicated by the suffix _max in Epoch's Model version. Data retrieved 2026-11-12.Source excerptsimpleqa_verified.csv: Model version=gpt-6-luna_max, Best score (across scorers)=0.414, stderr=0.015583544104177553, Started at=2026-09-22T22:14:05.000Z |
| 8 | Claude Haiku 5.5 | Max | 23.8% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, simpleqa_verified.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "Claude Haiku 5.5 (max)". effort is indicated by the suffix _max in Epoch's Model version. Data retrieved 2026-11-12.Source excerptsimpleqa_verified.csv: Model version=claude-haiku-5-5_max, Best score (across scorers)=0.238, stderr=0.013473586661967093, Started at=2026-10-09T14:36:06.000Z |
All model rankings (records)
These rankings are recalculated by comparison group including legacy models and models unavailable in Claude Code / Codex. They differ from the rankings above because the set of models differs.
- View all model rankings (by metric)
- Cost balance (all models, records)
- Overall ranking (all models, records)
Show rankings including legacy and unavailable models (entries: 38)
third-party results (Epoch AI, Capabilities & benchmarking (benchmark_data.zip, simpleqa_verified.csv) CC BY 4.0)
| Rank | Model | effort | Score | Source cost (metric) | measured/reported by | source |
|---|---|---|---|---|---|---|
| 1 | Claude Fable 5.2 | Max | 94.0% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, simpleqa_verified.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "Claude Fable 5.2 (max)". effort is indicated by the suffix _max in Epoch's Model version. Data retrieved 2026-11-12.Source excerptsimpleqa_verified.csv: Model version=claude-fable-5-2_max, Best score (across scorers)=0.94, stderr=0.007513751157474972, Started at=2026-11-06T01:14:45.000Z |
| 2 | Gemini 4 Carbon | High | 77.7% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, simpleqa_verified.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "Gemini 4 Carbon (high)". effort is indicated by the suffix _high in Epoch's Model version. Data retrieved 2026-11-12.Source excerptsimpleqa_verified.csv: Model version=gemini-4-carbon_high, Best score (across scorers)=0.777, stderr=0.013169830843425608, Started at=2026-11-02T23:17:42.000Z |
| 3 | Gemini 4 Argon | High | 76.0% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, simpleqa_verified.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "Gemini 4 Argon (high)". effort is indicated by the suffix _high in Epoch's Model version. Data retrieved 2026-11-12.Source excerptsimpleqa_verified.csv: Model version=gemini-4-argon_high, Best score (across scorers)=0.76, stderr=0.013512312258920845, Started at=2026-10-13T17:15:14.000Z |
| 4 | GPT-6 Astra | Max | 75.6% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, simpleqa_verified.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "GPT-6 Astra (max)". effort is indicated by the suffix _max in Epoch's Model version. Data retrieved 2026-11-12.Source excerptsimpleqa_verified.csv: Model version=gpt-6-astra_max, Best score (across scorers)=0.756, stderr=0.013588548437881386, Started at=2026-08-30T14:57:43.000Z |
| 5 | GPT-6.1 Sol | Max | 73.9% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, simpleqa_verified.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "GPT-6.1 Sol (max)". effort is indicated by the suffix _max in Epoch's Model version. Data retrieved 2026-11-12.Source excerptsimpleqa_verified.csv: Model version=gpt-6.1-sol_max, Best score (across scorers)=0.739, stderr=0.013895037677965185, Started at=2026-09-29T22:00:34.000Z |
| 6 | Gemini 3.1 Pro Preview | High | 73.5% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, simpleqa_verified.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "Gemini 3.1 Pro Preview". effort is indicated by the suffix _high in Epoch's Model version. Data retrieved 2026-11-12.Source excerptsimpleqa_verified.csv: Model version=gemini-3.1-pro-preview_high, Best score (across scorers)=0.735, stderr=0.013963164754810065, Started at=2026-08-10T21:35:39.000ZLocation simpleqa_verified.csv, data row 99 (file line 149), id=VGeF8Xac4WcSAGgSXVdEBi, Model version=gemini-3.1-pro-preview_high |
| 7 | Claude Opus 5.5 | Max | 72.2% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, simpleqa_verified.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "Claude Opus 5.5 (max)". effort is indicated by the suffix _max in Epoch's Model version. Data retrieved 2026-11-12.Source excerptsimpleqa_verified.csv: Model version=claude-opus-5-5_max, Best score (across scorers)=0.722, stderr=0.014174516461485273, Started at=2026-09-22T18:27:39.000Z |
| 8 | Claude Fable 5 | Xhigh | 70.7% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, simpleqa_verified.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "Claude Fable 5 (xhigh)". effort is indicated by the suffix _xhigh in Epoch's Model version. Data retrieved 2026-11-12.Source excerptsimpleqa_verified.csv: Model version=claude-fable-5_xhigh, Best score (across scorers)=0.707, stderr=0.014399942998441292, Started at=2026-08-10T23:44:03.000ZLocation simpleqa_verified.csv, data row 96 (file line 146), id=SEnvgtunRa5GwAtU2sHa87, Model version=claude-fable-5_xhigh |
| 9 | Gemini 3.8 Flash | High | 69.7% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, simpleqa_verified.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "Gemini 3.8 Flash (high)". effort is indicated by the suffix _high in Epoch's Model version. Data retrieved 2026-11-12.Source excerptsimpleqa_verified.csv: Model version=gemini-3.8-flash_high, Best score (across scorers)=0.697, stderr=0.014539683710535193, Started at=2026-09-02T17:15:36.000ZLocation simpleqa_verified.csv, data row 25 (file line 49), id=gZ4bwozTKtpmZTqnqX3a33, Model version=gemini-3.8-flash_high |
| 9 | GPT-5.6 Sol | Max | 69.7% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, simpleqa_verified.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "GPT-5.6 Sol (max)". effort is indicated by the suffix _max in Epoch's Model version. Data retrieved 2026-11-12.Source excerptsimpleqa_verified.csv: Model version=gpt-5.6-sol_max, Best score (across scorers)=0.697, stderr=0.014539683710535193, Started at=2026-08-10T23:44:30.000ZLocation simpleqa_verified.csv, data row 92 (file line 142), id=UnRUSuZzm2X27TUbpeMPva, Model version=gpt-5.6-sol_max |
| 11 | Gemini 3.7 Flash | High | 69.2% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, simpleqa_verified.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "Gemini 3.7 Flash (high)". effort is indicated by the suffix _high in Epoch's Model version. Data retrieved 2026-11-12.Source excerptsimpleqa_verified.csv: Model version=gemini-3.7-flash_high, Best score (across scorers)=0.692, stderr=0.01460648312734278, Started at=2026-08-27T19:30:25.000ZLocation simpleqa_verified.csv, data row 58 (file line 103), id=84ARuGyhQsjgh8umRmBvas, Model version=gemini-3.7-flash_high |
| 12 | Gemini 3 Flash Preview | High | 66.8% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, simpleqa_verified.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "Gemini 3 Flash Preview". effort is indicated by the suffix _high in Epoch's Model version. Data retrieved 2026-11-12.Source excerptsimpleqa_verified.csv: Model version=gemini-3-flash-preview_high, Best score (across scorers)=0.668, stderr=0.014899597242811567, Started at=2026-08-27T19:30:06.000ZLocation simpleqa_verified.csv, data row 63 (file line 110), id=7cc6cDCXjsPjtKt36bbeMR, Model version=gemini-3-flash-preview_high |
| 13 | Gemini 3.6 Flash | High | 66.2% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, simpleqa_verified.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "Gemini 3.6 Flash". effort is indicated by the suffix _high in Epoch's Model version. Data retrieved 2026-11-12.Source excerptsimpleqa_verified.csv: Model version=gemini-3.6-flash_high, Best score (across scorers)=0.662, stderr=0.014965960710224383, Started at=2026-08-27T19:30:09.000ZLocation simpleqa_verified.csv, data row 61 (file line 108), id=F83pa7Gfsb4UNVttTnS5Hk, Model version=gemini-3.6-flash_high |
| 14 | GPT-5.5 | Xhigh | 63.0% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, simpleqa_verified.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "GPT-5.5 (xhigh)". effort is indicated by the suffix _xhigh in Epoch's Model version. Data retrieved 2026-11-12.Source excerptsimpleqa_verified.csv: Model version=gpt-5.5_xhigh, Best score (across scorers)=0.63, stderr=0.015275252316519465, Started at=2026-08-27T19:28:25.000ZLocation simpleqa_verified.csv, data row 78 (file line 127), id=TijCafTTJUFrJ6jPGvBbgq, Model version=gpt-5.5_xhigh |
| 15 | Kimi K3.1 Pro | Max | 62.2% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, simpleqa_verified.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "Kimi K3.1 Pro (Together, max)". effort is indicated by the suffix _max in Epoch's Model version. Data retrieved 2026-11-12. Epoch ran it via Together's inference API (Model version kimi-k3.1-pro-together).Source excerptsimpleqa_verified.csv: Model version=kimi-k3.1-pro-together_max, Best score (across scorers)=0.622, stderr=0.015341165254026547, Started at=2026-11-06T00:59:23.000ZLocation simpleqa_verified.csv, data row 2 (file line 3), id=HumfsXVh5BZWuTo6Z8Gqnf, Model version=kimi-k3.1-pro-together_max |
| 16 | GPT-6 Sol | Max | 60.7% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, simpleqa_verified.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "GPT-6 Sol (max)". effort is indicated by the suffix _max in Epoch's Model version. Data retrieved 2026-11-12.Source excerptsimpleqa_verified.csv: Model version=gpt-6-sol_max, Best score (across scorers)=0.607, stderr=0.015452824654081525, Started at=2026-09-22T19:59:24.000ZLocation simpleqa_verified.csv, data row 22 (file line 46), id=donMeEKBZZdqpQsojKWJEm, Model version=gpt-6-sol_max |
| 17 | GPT-6.1 Terra | Max | 57.3% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, simpleqa_verified.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "GPT-6.1 Terra (max)". effort is indicated by the suffix _max in Epoch's Model version. Data retrieved 2026-11-12.Source excerptsimpleqa_verified.csv: Model version=gpt-6.1-terra_max, Best score (across scorers)=0.573, stderr=0.01564978964446219, Started at=2026-10-26T20:00:37.000Z |
| 18 | Grok 4.7 | Xhigh | 56.0% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, simpleqa_verified.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "Grok 4.7 (xhigh)". effort is indicated by the suffix _xhigh in Epoch's Model version. Data retrieved 2026-11-12.Source excerptsimpleqa_verified.csv: Model version=grok-4.7_xhigh, Best score (across scorers)=0.56, stderr=0.015704987954361718, Started at=2026-09-22T21:49:22.000ZLocation simpleqa_verified.csv, data row 21 (file line 45), id=6izDCw5ygCLZJEKPS8jV5T, Model version=grok-4.7_xhigh |
| 19 | Claude Opus 4.8 | Max | 53.0% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, simpleqa_verified.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "Claude Opus 4.8". effort is indicated by the suffix _max in Epoch's Model version. Data retrieved 2026-11-12.Source excerptsimpleqa_verified.csv: Model version=claude-opus-4-8_max, Best score (across scorers)=0.53, stderr=0.015790799515836722, Started at=2026-08-27T19:27:24.000ZLocation simpleqa_verified.csv, data row 91 (file line 141), id=ViC6pxQZUVS3E9M9iEubF7, Model version=claude-opus-4-8_max |
| 20 | DeepSeek-V4-Pro-0813 | Max | 52.9% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, simpleqa_verified.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "DeepSeek V4 Pro 0813 (max)". effort is indicated by the suffix _max in Epoch's Model version. Data retrieved 2026-11-12.Source excerptsimpleqa_verified.csv: Model version=deepseek-v4-pro-0813_max, Best score (across scorers)=0.5290581162324649, stderr=0.015808394665706597, Started at=2026-08-27T19:30:20.000ZLocation simpleqa_verified.csv, data row 59 (file line 104), id=2xfFf6KZAp2gUWYhHGkFgs, Model version=deepseek-v4-pro-0813_max |
| 21 | Claude Opus 4.7 | Xhigh | 51.7% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, simpleqa_verified.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "Claude Opus 4.7 (xhigh)". effort is indicated by the suffix _xhigh in Epoch's Model version. Data retrieved 2026-11-12.Source excerptsimpleqa_verified.csv: Model version=claude-opus-4-7_xhigh, Best score (across scorers)=0.517, stderr=0.015810153729833274, Started at=2026-08-27T19:27:26.000ZLocation simpleqa_verified.csv, data row 90 (file line 140), id=2c4rvmf5N7QXTugrFLMZ83, Model version=claude-opus-4-7_xhigh |
| 22 | Kimi K3 | Max | 50.6% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, simpleqa_verified.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "Kimi K3 (max)". effort is indicated by the suffix _max in Epoch's Model version. Data retrieved 2026-11-12.Source excerptsimpleqa_verified.csv: Model version=kimi-k3_max, Best score (across scorers)=0.506, stderr=0.015818160898606836, Started at=2026-08-27T19:36:12.000ZLocation simpleqa_verified.csv, data row 44 (file lines 73-75; multi-line record), id=mZLWt2JSYvJyGhMvji7CT8, Model version=kimi-k3_max |
| 23 | Grok 4.6 | High | 49.3% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, simpleqa_verified.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "Grok 4.6 (high)". effort is indicated by the suffix _high in Epoch's Model version. Data retrieved 2026-11-12.Source excerptsimpleqa_verified.csv: Model version=grok-4.6_high, Best score (across scorers)=0.493, stderr=0.01581774956184353, Started at=2026-08-27T19:29:53.000ZLocation simpleqa_verified.csv, data row 64 (file line 111), id=TpHYtuPG6kYZFvMK7Z6Hqj, Model version=grok-4.6_high |
| 24 | Grok 4.6 | Xhigh | 48.9% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, simpleqa_verified.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "Grok 4.6 (xhigh)". effort is indicated by the suffix _xhigh in Epoch's Model version. Data retrieved 2026-11-12.Source excerptsimpleqa_verified.csv: Model version=grok-4.6_xhigh, Best score (across scorers)=0.489, stderr=0.015815471195292575, Started at=2026-08-27T19:29:19.000ZLocation simpleqa_verified.csv, data row 68 (file line 115), id=dKCWFaoGCWJTnD5CPWQGru, Model version=grok-4.6_xhigh |
| 25 | Claude Opus 4.6 | Max | 47.0% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, simpleqa_verified.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "Claude Opus 4.6 (max)". effort is indicated by the suffix _max in Epoch's Model version. Data retrieved 2026-11-12.Source excerptsimpleqa_verified.csv: Model version=claude-opus-4-6_max, Best score (across scorers)=0.47, stderr=0.015790799515836722, Started at=2026-08-27T19:27:45.000ZLocation simpleqa_verified.csv, data row 87 (file line 137), id=85fhMxLaebWtLoi6oLpTbi, Model version=claude-opus-4-6_max |
| 26 | Claude Sonnet 5.5 | Max | 46.5% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, simpleqa_verified.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "Claude Sonnet 5.5 (max)". effort is indicated by the suffix _max in Epoch's Model version. Data retrieved 2026-11-12.Source excerptsimpleqa_verified.csv: Model version=claude-sonnet-5-5_max, Best score (across scorers)=0.465, stderr=0.015780495050030086, Started at=2026-09-29T15:15:14.000Z |
| 27 | Qwen3.8-Max | Xhigh | 45.8% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, simpleqa_verified.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "Qwen3.8 Max (xhigh)". effort is indicated by the suffix _xhigh in Epoch's Model version. Data retrieved 2026-11-12.Source excerptsimpleqa_verified.csv: Model version=qwen3.8-max_xhigh, Best score (across scorers)=0.458, stderr=0.015763390640483554, Started at=2026-08-27T19:30:55.000ZLocation simpleqa_verified.csv, data row 55 (file line 100), id=bfftd5EzUK9yAekafEofvK, Model version=qwen3.8-max_xhigh |
| 28 | GPT-5.6 Terra | Max | 43.2% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, simpleqa_verified.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "GPT-5.6 Terra (max)". effort is indicated by the suffix _max in Epoch's Model version. Data retrieved 2026-11-12.Source excerptsimpleqa_verified.csv: Model version=gpt-5.6-terra_max, Best score (across scorers)=0.432, stderr=0.015672320237336324, Started at=2026-08-10T23:44:21.000ZLocation simpleqa_verified.csv, data row 93 (file line 143), id=DienatPbgfgn3EzM7Gvzxr, Model version=gpt-5.6-terra_max |
| 29 | DeepSeek-V4.1-Flash | not stated | 42.9% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, simpleqa_verified.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "DeepSeek V4.1 Flash (max)". effort is indicated by the suffix _max in Epoch's Model version. Data retrieved 2026-11-12. No effort for the source effort label "max" because the official effort list has not been collected.Source excerptsimpleqa_verified.csv: Model version=deepseek-v4.1-flash_max, Best score (across scorers)=0.429, stderr=0.015658997547870177, Started at=2026-10-13T12:59:33.000ZLocation simpleqa_verified.csv, data row 15 (file lines 27-28; multi-line record), id=3JJ4ChNxMo8CGAh3HrKNrw, Model version=deepseek-v4.1-flash_max |
| 30 | DeepSeek-V4.1-Flash-1017 | Max | 42.5% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, simpleqa_verified.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "DeepSeek V4.1 Flash 1017 (max)". effort is indicated by the suffix _max in Epoch's Model version. Data retrieved 2026-11-12.Source excerptsimpleqa_verified.csv: Model version=deepseek-v4.1-flash-1017_max, Best score (across scorers)=0.425, stderr=0.01564032031704017, Started at=2026-10-19T22:38:54.000ZLocation simpleqa_verified.csv, data row 12 (file line 24), id=Ei4XWaCH5drYUnDRo2MYCW, Model version=deepseek-v4.1-flash-1017_max |
| 31 | GPT-6 Luna | Max | 41.4% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, simpleqa_verified.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "GPT-6 Luna (max)". effort is indicated by the suffix _max in Epoch's Model version. Data retrieved 2026-11-12.Source excerptsimpleqa_verified.csv: Model version=gpt-6-luna_max, Best score (across scorers)=0.414, stderr=0.015583544104177553, Started at=2026-09-22T22:14:05.000Z |
| 32 | GLM-5.3 | Max | 41.0% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, simpleqa_verified.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "GLM-5.3 (max)". effort is indicated by the suffix _max in Epoch's Model version. Data retrieved 2026-11-12.Source excerptsimpleqa_verified.csv: Model version=glm-5.3_max, Best score (across scorers)=0.41, stderr=0.015560917136921659, Started at=2026-08-28T17:00:06.000ZLocation simpleqa_verified.csv, data row 37 (file line 66), id=TY8R8FV7zCwByEipkqyg2P, Model version=glm-5.3_max |
| 32 | GPT-5.6 Luna | Max | 41.0% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, simpleqa_verified.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "GPT-5.6 Luna (max)". effort is indicated by the suffix _max in Epoch's Model version. Data retrieved 2026-11-12.Source excerptsimpleqa_verified.csv: Model version=gpt-5.6-luna_max, Best score (across scorers)=0.41, stderr=0.015560917136921657, Started at=2026-08-10T23:44:20.000ZLocation simpleqa_verified.csv, data row 94 (file line 144), id=CeDghdBk8CqtiFSbUzxNH5, Model version=gpt-5.6-luna_max |
| 34 | Claude Sonnet 4.6 | High | 35.5% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, simpleqa_verified.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "Claude Sonnet 4.6 (high)". effort is indicated by the suffix _high in Epoch's Model version. Data retrieved 2026-11-12.Source excerptsimpleqa_verified.csv: Model version=claude-sonnet-4-6_high, Best score (across scorers)=0.355, stderr=0.015139491543780595, Started at=2026-08-10T22:42:23.000ZLocation simpleqa_verified.csv, data row 97 (file line 147), id=LJyKf8Xy4dymxNHbGLwNdA, Model version=claude-sonnet-4-6_high |
| 35 | Claude Sonnet 4.6 | Max | 32.8% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, simpleqa_verified.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "Claude Sonnet 4.6 (max)". effort is indicated by the suffix _max in Epoch's Model version. Data retrieved 2026-11-12.Source excerptsimpleqa_verified.csv: Model version=claude-sonnet-4-6_max, Best score (across scorers)=0.328, stderr=0.014853842487270312, Started at=2026-08-27T19:27:37.000ZLocation simpleqa_verified.csv, data row 89 (file line 139), id=Zs5fCvNzaz8LhqgwDSAztU, Model version=claude-sonnet-4-6_max |
| 36 | Claude Haiku 5.5 | Max | 23.8% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, simpleqa_verified.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "Claude Haiku 5.5 (max)". effort is indicated by the suffix _max in Epoch's Model version. Data retrieved 2026-11-12.Source excerptsimpleqa_verified.csv: Model version=claude-haiku-5-5_max, Best score (across scorers)=0.238, stderr=0.013473586661967093, Started at=2026-10-09T14:36:06.000Z |
| 37 | Claude Sonnet 4.5 | not stated | 23.7% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, simpleqa_verified.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "Claude Sonnet 4.5 (no thinking)". No effort suffix in Model version. Data retrieved 2026-11-12. No effort because effort is not stated in the source.Source excerptsimpleqa_verified.csv: Model version=claude-sonnet-4-5-20250929, Best score (across scorers)=0.237, stderr=0.013454070462577935, Started at=2026-08-10T22:42:10.000ZLocation simpleqa_verified.csv, data row 98 (file line 148), id=MhqeuRVx4JcRfxPQDYPq7Z, Model version=claude-sonnet-4-5-20250929 |
| 38 | Mistral Large 4 | High | 20.0% | — | Epoch AI | Epoch AI, Capabilities & benchmarking (benchmark_data.zip, simpleqa_verified.csv) CC BY 4.0Retrieved 2026-11-12 · Run by Epoch AI (third-party measurement). Source label "Mistral Large 4 Preview (high)". effort is indicated by the suffix _high in Epoch's Model version. Data retrieved 2026-11-12. Epoch's label is Mistral Large 4 Preview (2026-10-06 version; Mistral Large 4 from 2026-10-29 has a different Model version, mistral-large-4-0-oss, and is not included here).Source excerptsimpleqa_verified.csv: Model version=mistral-large-4-0_high, Best score (across scorers)=0.2, stderr=0.012655439943366657, Started at=2026-10-09T14:12:09.000ZLocation simpleqa_verified.csv, data row 17 (file lines 30-41; multi-line record), id=hvNNKdnzLvr9X8zVxWKcfk, Model version=mistral-large-4-0_high |