Terminal-Bench-Science 0.1
An evaluation of agentic scientific research tasks performed in a terminal (v0.1).
- Category
- Agentic coding
- Unit
- Percent (%)
- Higher is better
- Comparison groups
- 4total
- Currently available model rankings · Ranked results: 8
- All models (records): Groups: 6 · Entries: 17
Rankings
Values with different source types (official, third-party, paper) or measurement conditions cannot be compared, so rankings are separated by comparison group (the same source, or sources verified to use the same conditions).
Rankings include only models currently available in Claude Code / Codex, excluding legacy models.
official results (Introducing Claude Opus 5.5)
| Rank | Model | effort | Score | Source cost (metric) | measured/reported by | source |
|---|---|---|---|---|---|---|
| 1 | GPT-6 Astra | not stated | 64.6% | — | OpenAI | Introducing Claude Opus 5.5Retrieved 2026-11-12 · Vendor-reported value. table note: Unless otherwise noted, all Claude Opus 5.5 results use adaptive thinking at max effort. production safeguards enabled. Values for other vendor models (as listed by Anthropic; null where effort was not stated). SE ±3.5–5pt. The GPT-6 Astra figure is a reported value from OpenAI.Source excerpt64.6%Location Table, Terminal-Bench-Science 0.1 row: 64.6 |
| 2 | Claude Opus 5.5 | Max | 58.7% | — | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 · Vendor-reported value. table note: Unless otherwise noted, all Claude Opus 5.5 results use adaptive thinking at max effort. production safeguards enabled. SE ±3.5–5pt. The GPT-6 Astra figure is a reported value from OpenAI.Source excerpt58.7%Location Table, Terminal-Bench-Science 0.1 row: 58.7 |
official results (Introducing Claude Fable 5.2 and Claude Mythos 5.2)
| Rank | Model | effort | Score | Source cost (metric) | measured/reported by | source |
|---|---|---|---|---|---|---|
| 1 | Claude Fable 5.2 | not stated | 74.6% | — | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 · Vendor-reported value. table note: Scores for Fable 5.2, Fable 5.1, and Opus 5.5 show results with safeguards on unless otherwise noted. All scores show each model’s highest-available effort unless otherwise noted.Source excerpt74.6%Location Table, Terminal-Bench-Science 0.1 row: 74.6 |
| 2 | GPT-6 Astra | not stated | 68.1% | — | not stated | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 · Vendor-reported value. table note: Scores for Fable 5.2, Fable 5.1, and Opus 5.5 show results with safeguards on unless otherwise noted. All scores show each model’s highest-available effort unless otherwise noted. Values for other vendor models (as listed by Anthropic).Source excerpt68.1%Location Table, Terminal-Bench-Science 0.1 row: 68.1 |
| 3 | Claude Opus 5.5 | not stated | 58.7% | — | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 · Vendor-reported value. table note: Scores for Fable 5.2, Fable 5.1, and Opus 5.5 show results with safeguards on unless otherwise noted. All scores show each model’s highest-available effort unless otherwise noted.Source excerpt58.7%Location Table, Terminal-Bench-Science 0.1 row: 58.7 |
official results (Claude Sonnet 5.5 System Card)
| Rank | Model | effort | Score | Source cost (metric) | measured/reported by | source |
|---|---|---|---|---|---|---|
| 1 | Claude Sonnet 5.5 | Max | 59.9% | — | Anthropic | Claude Sonnet 5.5 System CardRetrieved 2026-11-14 · Vendor-reported value. Claude Code --bare, max, 70 tasks. No internet; safeguards enabled. No fallback was triggered. Cost not stated.Source excerpt59.9%Location §8.6 “Terminal-Bench-Science 0.1”, body text, own model score |
official results (Claude Haiku 5.5 System Card)
| Rank | Model | effort | Score | Source cost (metric) | measured/reported by | source |
|---|---|---|---|---|---|---|
| 1 | GPT-6.1 Sol | Max | 57.0% | — | OpenAI | Claude Haiku 5.5 System CardRetrieved 2026-11-14 · Vendor-reported value. Anthropic’s body text explicitly states OpenAI reported and max thinking effort. Distinguished from Anthropic’s internal runs and public leaderboard values. Cost not stated.Source excerpt57.0%Location §8.5 “Terminal-Bench-Science 0.1”, body text, GPT-6.1 Sol under OpenAI reported |
| 2 | Claude Haiku 5.5 | Max | 20.6% | — | Anthropic | Claude Haiku 5.5 System CardRetrieved 2026-11-14 · Vendor-reported value. Claude Code --bare, max, 70 tasks. No internet; safeguards enabled. No fallback; safeguards-triggered exits count as failures (2.3% of 700 trials). Cost not stated.Source excerpt20.6%Location §8.5 “Terminal-Bench-Science 0.1”, body text, own model score |
All model rankings (records)
These rankings are recalculated by comparison group including legacy models and models unavailable in Claude Code / Codex. They differ from the rankings above because the set of models differs.
Show rankings including legacy and unavailable models (entries: 17)
official results (Introducing Claude Opus 5.5)
| Rank | Model | effort | Score | Source cost (metric) | measured/reported by | source |
|---|---|---|---|---|---|---|
| 1 | GPT-6 Astra | not stated | 64.6% | — | OpenAI | Introducing Claude Opus 5.5Retrieved 2026-11-12 · Vendor-reported value. table note: Unless otherwise noted, all Claude Opus 5.5 results use adaptive thinking at max effort. production safeguards enabled. Values for other vendor models (as listed by Anthropic; null where effort was not stated). SE ±3.5–5pt. The GPT-6 Astra figure is a reported value from OpenAI.Source excerpt64.6%Location Table, Terminal-Bench-Science 0.1 row: 64.6 |
| 2 | Claude Opus 5.5 | Max | 58.7% | — | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 · Vendor-reported value. table note: Unless otherwise noted, all Claude Opus 5.5 results use adaptive thinking at max effort. production safeguards enabled. SE ±3.5–5pt. The GPT-6 Astra figure is a reported value from OpenAI.Source excerpt58.7%Location Table, Terminal-Bench-Science 0.1 row: 58.7 |
| 3 | Claude Fable 5.1 | not stated | 52.6% | — | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 · Vendor-reported value. table note: Unless otherwise noted, all Claude Opus 5.5 results use adaptive thinking at max effort. production safeguards enabled. SE ±3.5–5pt. The GPT-6 Astra figure is a reported value from OpenAI.Source excerpt52.6%Location Table, Terminal-Bench-Science 0.1 row: 52.6 |
| 4 | Claude Opus 5 | not stated | 29.0% | — | Anthropic | Introducing Claude Opus 5.5Retrieved 2026-11-12 · Vendor-reported value. table note: Unless otherwise noted, all Claude Opus 5.5 results use adaptive thinking at max effort. production safeguards enabled. SE ±3.5–5pt. The GPT-6 Astra figure is a reported value from OpenAI.Source excerpt29.0%Location Table, Terminal-Bench-Science 0.1 row: 29.0 |
| 5 | GPT-5.6 Sol | not stated | 22.4% | — | not stated | Introducing Claude Opus 5.5Retrieved 2026-11-12 · Vendor-reported value. table note: Unless otherwise noted, all Claude Opus 5.5 results use adaptive thinking at max effort. production safeguards enabled. Values for other vendor models (as listed by Anthropic; null where effort was not stated). SE ±3.5–5pt. The GPT-6 Astra figure is a reported value from OpenAI.Source excerpt22.4%Location Table, Terminal-Bench-Science 0.1 row: 22.4 |
official results (Introducing Claude Fable 5.2 and Claude Mythos 5.2)
| Rank | Model | effort | Score | Source cost (metric) | measured/reported by | source |
|---|---|---|---|---|---|---|
| 1 | Claude Mythos 5.2 | not stated | 75.3% | — | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 · Vendor-reported value. table note: Scores for Fable 5.2, Fable 5.1, and Opus 5.5 show results with safeguards on unless otherwise noted. All scores show each model’s highest-available effort unless otherwise noted.Source excerpt75.3%Location Table, Terminal-Bench-Science 0.1 row: 74.6% 75.3% (Mythos 5.2) |
| 2 | Claude Fable 5.2 | not stated | 74.6% | — | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 · Vendor-reported value. table note: Scores for Fable 5.2, Fable 5.1, and Opus 5.5 show results with safeguards on unless otherwise noted. All scores show each model’s highest-available effort unless otherwise noted.Source excerpt74.6%Location Table, Terminal-Bench-Science 0.1 row: 74.6 |
| 3 | GPT-6 Astra | not stated | 68.1% | — | not stated | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 · Vendor-reported value. table note: Scores for Fable 5.2, Fable 5.1, and Opus 5.5 show results with safeguards on unless otherwise noted. All scores show each model’s highest-available effort unless otherwise noted. Values for other vendor models (as listed by Anthropic).Source excerpt68.1%Location Table, Terminal-Bench-Science 0.1 row: 68.1 |
| 4 | Gemini 4 Carbon | not stated | 64.8% | — | not stated | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 · Vendor-reported value. table note: Scores for Fable 5.2, Fable 5.1, and Opus 5.5 show results with safeguards on unless otherwise noted. All scores show each model’s highest-available effort unless otherwise noted. Values for other vendor models (as listed by Anthropic).Source excerpt64.8%Location Table, Terminal-Bench-Science 0.1 row: 64.8 |
| 5 | Claude Opus 5.5 | not stated | 58.7% | — | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 · Vendor-reported value. table note: Scores for Fable 5.2, Fable 5.1, and Opus 5.5 show results with safeguards on unless otherwise noted. All scores show each model’s highest-available effort unless otherwise noted.Source excerpt58.7%Location Table, Terminal-Bench-Science 0.1 row: 58.7 |
| 6 | Claude Fable 5.1 | not stated | 52.6% | — | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 · Vendor-reported value. table note: Scores for Fable 5.2, Fable 5.1, and Opus 5.5 show results with safeguards on unless otherwise noted. All scores show each model’s highest-available effort unless otherwise noted.Source excerpt52.6%Location Table, Terminal-Bench-Science 0.1 row: 52.6 |
official results (Gemini 4 Carbon (Google DeepMind models page, benchmark table))
| Rank | Model | effort | Score | Source cost (metric) | measured/reported by | source |
|---|---|---|---|---|---|---|
| 1 | Gemini 4 Carbon | High | 64.8% | — | Artificial Analysis | Gemini 4 Carbon (Google DeepMind models page, benchmark table)Retrieved 2026-11-12 · Vendor-reported value. Google published table. methodology PDF: Gemini 4 Carbon/Argon used the highest Gemini API thinking setting, pass@1, and a single trial. Row-specific source: a measured value from Artificial AnalysisSource excerptTerminal-Bench Science 0.1: Gemini 4 Carbon 64.8 |
| 2 | Gemini 4 Argon | High | 52.9% | — | Artificial Analysis | Gemini 4 Carbon (Google DeepMind models page, benchmark table)Retrieved 2026-11-12 · Vendor-reported value. Google published table. methodology PDF: Gemini 4 Carbon/Argon used the highest Gemini API thinking setting, pass@1, and a single trial. Row-specific source: a measured value from Artificial AnalysisSource excerptTerminal-Bench Science 0.1: Gemini 4 Argon 52.9 |
official results (Kimi K3.1 Pro (Hugging Face model card, moonshotai))
| Rank | Model | effort | Score | Source cost (metric) | measured/reported by | source |
|---|---|---|---|---|---|---|
| 1 | Kimi K3.1 Pro | Max | 38.1% | — | Moonshot AI | Kimi K3.1 Pro (Hugging Face model card, moonshotai)Retrieved 2026-11-13 · Vendor-reported value (self-reported by Moonshot AI). Table column heading Kimi K3.1 Pro (max), table note: all Kimi K3.1 Pro results use reasoning effort max, temperature 1.0, top-p 0.95. Evaluated with Claude Code.Source excerptTerminal-Bench-Science 0.1 | 38.1Location HF model card "3. Evaluation Results" table, Terminal-Bench-Science 0.1 row, Kimi K3.1 Pro (max) column (leftmost numeric column). Supporting footnote: Coding 2 |
official results (Claude Sonnet 5.5 System Card)
| Rank | Model | effort | Score | Source cost (metric) | measured/reported by | source |
|---|---|---|---|---|---|---|
| 1 | Claude Sonnet 5.5 | Max | 59.9% | — | Anthropic | Claude Sonnet 5.5 System CardRetrieved 2026-11-14 · Vendor-reported value. Claude Code --bare, max, 70 tasks. No internet; safeguards enabled. No fallback was triggered. Cost not stated.Source excerpt59.9%Location §8.6 “Terminal-Bench-Science 0.1”, body text, own model score |
official results (Claude Haiku 5.5 System Card)
| Rank | Model | effort | Score | Source cost (metric) | measured/reported by | source |
|---|---|---|---|---|---|---|
| 1 | GPT-6.1 Sol | Max | 57.0% | — | OpenAI | Claude Haiku 5.5 System CardRetrieved 2026-11-14 · Vendor-reported value. Anthropic’s body text explicitly states OpenAI reported and max thinking effort. Distinguished from Anthropic’s internal runs and public leaderboard values. Cost not stated.Source excerpt57.0%Location §8.5 “Terminal-Bench-Science 0.1”, body text, GPT-6.1 Sol under OpenAI reported |
| 2 | Claude Haiku 5.5 | Max | 20.6% | — | Anthropic | Claude Haiku 5.5 System CardRetrieved 2026-11-14 · Vendor-reported value. Claude Code --bare, max, 70 tasks. No internet; safeguards enabled. No fallback; safeguards-triggered exits count as failures (2.3% of 700 trials). Cost not stated.Source excerpt20.6%Location §8.5 “Terminal-Bench-Science 0.1”, body text, own model score |