Terminal-Bench 3.0
Version 3.0 of Terminal-Bench.
- Category
- Agentic coding
- Unit
- Percent (%)
- Higher is better
- Comparison groups
- 3total
- All model rankings (records) · Ranked results: 4
All model rankings (records)
These rankings are recalculated by comparison group including legacy models and models unavailable in Claude Code / Codex. They differ from the rankings above because the set of models differs.
Show rankings including legacy and unavailable models (entries: 4)
official results (GLM-5.3 (Z.ai docs))
| Rank | Model | effort | Score | Source cost (metric) | measured/reported by | source |
|---|---|---|---|---|---|---|
| 1 | GLM-5.3 | not stated | 28.3% | — | Z.ai | GLM-5.3 (Z.ai docs)Retrieved 2026-11-12 · Vendor-reported value. Self-reported by Z.ai. effort not stated (default is max)Source excerptTerminal-Bench 3.0 28.3 |
official results (Gemini 3.7 Flash model card)
| Rank | Model | effort | Score | Source cost (metric) | measured/reported by | source |
|---|---|---|---|---|---|---|
| 1 | Gemini 3.7 Flash | not stated | 14.9% | — | Gemini 3.7 Flash model cardRetrieved 2026-11-13 · Vendor-reported value. Table in the Google model card. methodology PDF (deepmind.com/models/evals-methodology/gemini-3-7-flash): All of the results are run with the Gemini API ... with default sampling settings unless indicated otherwise. Gemini computed in-house; mini swe agent harness (LiteLLM 1.96). Minor changes to 2 tasks for compatibility. thinking level not statedSource excerptTerminal-bench 3.0 General agent capabilities | 14.9%Location Gemini 3.7 Flash model card Evaluation table, Terminal-bench 3.0 row, Gemini 3.7 Flash column. In-house computation basis: methodology PDF, Coding / Terminal-Bench 3 | |
| 2 | Gemini 3.6 Flash | not stated | 5.4% | — | Gemini 3.7 Flash model cardRetrieved 2026-11-13 · Vendor-reported value. Table in the Google model card. methodology PDF (deepmind.com/models/evals-methodology/gemini-3-7-flash): All of the results are run with the Gemini API ... with default sampling settings unless indicated otherwise. Gemini computed in-house; mini swe agent harness (LiteLLM 1.96). Minor changes to 2 tasks for compatibility. thinking level not statedSource excerptTerminal-bench 3.0 General agent capabilities | 5.4%Location Gemini 3.7 Flash model card Evaluation table, Terminal-bench 3.0 row, Gemini 3.6 Flash column. In-house computation basis: methodology PDF, Coding / Terminal-Bench 3 |
official results (DeepSeek-V4.1-Flash (Hugging Face model card, deepseek-ai))
| Rank | Model | effort | Score | Source cost (metric) | measured/reported by | source |
|---|---|---|---|---|---|---|
| 1 | DeepSeek-V4.1-Flash | not stated | 30.0% | — | DeepSeek | DeepSeek-V4.1-Flash (Hugging Face model card, deepseek-ai)Retrieved 2026-11-13 · Vendor-reported value (self-reported by DeepSeek). Card "Instruct Model": All instruct results below use the maximum effort setting (reasoning_effort=100), temperature=1.0, top_p=0.95. The correspondence of reasoning_effort=100 to the API effort names (low/high/max) has no official statement, so effort is null. Evaluated in Minimal mode of DeepSeek Harness with 1M context.Source excerptTerminal-Bench 3.0 (Pass@1) | 30.0Location HF model card "Comparison with frontier models (Max reasoning effort)" table, Terminal-Bench 3.0 (Pass@1) row, DS-V4.1-Flash column (rightmost) |