Agents' Last Exam (pass rate, version not stated)
Agents' Last Exam pass rate. Kept separate from v1.1 (rubric score) because the version is not stated in the source.
- Category
- Computer use
- Unit
- Percent (%)
- Higher is better
- Comparison groups
- 3total
- All model rankings (records) · Ranked results: 4
All model rankings (records)
These rankings are recalculated by comparison group including legacy models and models unavailable in Claude Code / Codex. They differ from the rankings above because the set of models differs.
Show rankings including legacy and unavailable models (entries: 4)
official results (Gemini 4 Argon model card)
| Rank | Model | effort | Score | Source cost (metric) | measured/reported by | source |
|---|---|---|---|---|---|---|
| 1 | Gemini 4 Argon | not stated | 39.5% | — | Gemini 4 Argon model cardRetrieved 2026-11-12 · Vendor-reported value. Self-reported by Google (model card, as of 2026-10). thinking setting not stated. The metric is Pass rate (distinct from rubric Score 69.0% in the Carbon comparison table); version not statedSource excerptAgent's Last Exam (Pass rate) | 39.5% |
official results (Gemini 3.7 Flash model card)
| Rank | Model | effort | Score | Source cost (metric) | measured/reported by | source |
|---|---|---|---|---|---|---|
| 1 | Gemini 3.7 Flash | not stated | 26.3% | — | Gemini 3.7 Flash model cardRetrieved 2026-11-13 · Vendor-reported value. Table in the Google model card. methodology PDF (deepmind.com/models/evals-methodology/gemini-3-7-flash): All of the results are run with the Gemini API ... with default sampling settings unless indicated otherwise. Computed in-house; default ALE-Claw harness. version not statedSource excerptAgent's Last Exam Multimodal desktop and OS agent tasks | Pass rate | 26.3%Location Gemini 3.7 Flash model card Evaluation table, Agent's Last Exam(Pass rate) row, Gemini 3.7 Flash column. In-house computation basis: methodology PDF, Computer use / Agent's Last Exam | |
| 2 | Gemini 3.6 Flash | not stated | 24.2% | — | Gemini 3.7 Flash model cardRetrieved 2026-11-13 · Vendor-reported value. Table in the Google model card. methodology PDF (deepmind.com/models/evals-methodology/gemini-3-7-flash): All of the results are run with the Gemini API ... with default sampling settings unless indicated otherwise. Computed in-house; default ALE-Claw harness. version not statedSource excerptAgent's Last Exam Multimodal desktop and OS agent tasks | Pass rate | 24.2%Location Gemini 3.7 Flash model card Evaluation table, Agent's Last Exam(Pass rate) row, Gemini 3.6 Flash column. In-house computation basis: methodology PDF, Computer use / Agent's Last Exam |
official results (DeepSeek-V4.1-Flash (Hugging Face model card, deepseek-ai))
| Rank | Model | effort | Score | Source cost (metric) | measured/reported by | source |
|---|---|---|---|---|---|---|
| 1 | DeepSeek-V4.1-Flash | not stated | 31.8% | — | DeepSeek | DeepSeek-V4.1-Flash (Hugging Face model card, deepseek-ai)Retrieved 2026-11-13 · Vendor-reported value (self-reported by DeepSeek). Card "Instruct Model": All instruct results below use the maximum effort setting (reasoning_effort=100), temperature=1.0, top_p=0.95. The correspondence of reasoning_effort=100 to the API effort names (low/high/max) is not stated officially, so effort is null. Evaluated using the official scaffold. Version not stated.Source excerptAgent's Last Exam (Pass@1) | 31.8Location HF model card "Comparison with frontier models (Max reasoning effort)" table, Agent's Last Exam (Pass@1) row, DS-V4.1-Flash column (rightmost) |