OSWorld-Verified
Real computer use tasks (verified version). Execution methods differ by source (stated in conditions).
- Category
- Computer use
- Unit
- Percent (%)
- Higher is better
- Comparison groups
- 2total
- All model rankings (records) · Ranked results: 4
All model rankings (records)
These rankings are recalculated by comparison group including legacy models and models unavailable in Claude Code / Codex. They differ from the rankings above because the set of models differs.
Show rankings including legacy and unavailable models (entries: 4)
official results (Gemini 3.5 Flash model card)
| Rank | Model | effort | Score | Source cost (metric) | measured/reported by | source |
|---|---|---|---|---|---|---|
| 1 | Gemini 3.5 Flash | not stated | 78.4% | — | Gemini 3.5 Flash model cardRetrieved 2026-11-13 · Vendor-reported value. Table in the Google model card. methodology PDF (deepmind.google/models/evals-methodology/gemini-3-5-flash): All of the results are all run with the Gemini API ... with default sampling settings unless indicated otherwise. Gemini computed in-house; average of 5 runs (1 trial each); up to 100 steps. thinking level not statedSource excerptUI Control | OSWorld-Verified Agentic computer use | 78.4%Location Gemini 3.5 Flash model card Evaluation table, OSWorld-Verified row, Gemini 3.5 Flash column. In-house computation basis: methodology PDF, UI Control / OSWorld-Verified | |
| 2 | Gemini 3.1 Pro Preview | not stated | 76.2% | — | Gemini 3.5 Flash model cardRetrieved 2026-11-13 · Vendor-reported value. Table in the Google model card. methodology PDF (deepmind.google/models/evals-methodology/gemini-3-5-flash): All of the results are all run with the Gemini API ... with default sampling settings unless indicated otherwise. Gemini computed in-house; average of 5 runs (1 trial each); up to 100 steps. Column heading is Gemini 3.1 Pro (gemini-3.1-pro-preview is the only match in the Gemini API model list). thinking level not statedSource excerptUI Control | OSWorld-Verified Agentic computer use | 76.2%Location Gemini 3.5 Flash model card Evaluation table, OSWorld-Verified row, Gemini 3.1 Pro column. In-house computation basis: methodology PDF, UI Control / OSWorld-Verified | |
| 3 | Gemini 3 Flash Preview | not stated | 65.1% | — | Gemini 3.5 Flash model cardRetrieved 2026-11-13 · Vendor-reported value. Table in the Google model card. methodology PDF (deepmind.google/models/evals-methodology/gemini-3-5-flash): All of the results are all run with the Gemini API ... with default sampling settings unless indicated otherwise. Gemini computed in-house; average of 5 runs (1 trial each); up to 100 steps. Column heading is Gemini 3 Flash (gemini-3-flash-preview is the only match in the Gemini API model list). thinking level not statedSource excerptUI Control | OSWorld-Verified Agentic computer use | 65.1%Location Gemini 3.5 Flash model card Evaluation table, OSWorld-Verified row, Gemini 3 Flash column. In-house computation basis: methodology PDF, UI Control / OSWorld-Verified |
official results (Introducing Claude Opus 4.8)
| Rank | Model | effort | Score | Source cost (metric) | measured/reported by | source |
|---|---|---|---|---|---|---|
| 1 | Claude Opus 4.7 | not stated | 82.3% | — | Anthropic | Introducing Claude Opus 4.8Retrieved 2026-11-13 · Vendor-reported value (announced by Anthropic). Text values in the main text and the footnote (the table and figure were not used because they are images). Value after changing the execution method. Effort not stated.Source excerptOSWorld-Verified: We made changes to how we run the OSWorld-Verified evaluation in order to more accurately reflect the model’s performance in the real world, and have updated the Opus 4.7 score to 82.3%.Location Opus 4.8 announcement page, OSWorld-Verified entry in Footnotes |