MMMU-Pro
Understanding and reasoning about image-based questions across multiple disciplines (without tools).
- Category
- Multimodal
- Unit
- Percent (%)
- Higher is better
- Comparison groups
- 3total
- All model rankings (records) · Ranked results: 5
All model rankings (records)
These rankings are recalculated by comparison group including legacy models and models unavailable in Claude Code / Codex. They differ from the rankings above because the set of models differs.
Show rankings including legacy and unavailable models (entries: 5)
official results (Kimi K3.1 Pro (Hugging Face model card, moonshotai))
| Rank | Model | effort | Score | Source cost (metric) | measured/reported by | source |
|---|---|---|---|---|---|---|
| 1 | Kimi K3.1 Pro | Max | 84.1% | — | Moonshot AI | Kimi K3.1 Pro (Hugging Face model card, moonshotai)Retrieved 2026-11-13 · Vendor-reported value (self-reported by Moonshot AI). Table column heading Kimi K3.1 Pro (max), table note: all Kimi K3.1 Pro results use reasoning effort max, temperature 1.0, top-p 0.95. Official protocol (preserving input order and placing images first).Source excerptMMMU-Pro | 84.1Location HF model card "3. Evaluation Results" table, MMMU-Pro row, Kimi K3.1 Pro (max) column (leftmost numeric column). Supporting footnote: Reasoning 4 |
official results (Kimi K3 (Hugging Face model card, moonshotai))
| Rank | Model | effort | Score | Source cost (metric) | measured/reported by | source |
|---|---|---|---|---|---|---|
| 1 | Kimi K3 | Max | 81.6% | — | Moonshot AI | Kimi K3 (Hugging Face model card, moonshotai)Retrieved 2026-11-13 · Vendor-reported value (self-reported by Moonshot AI). Table note: All Kimi K3 results are obtained with reasoning effort set to 'max' and temperature = 1.0Source excerptMMMU-Pro | 81.6 / 83.4Location HF model card "3. Evaluation Results" table, MMMU-Pro (left-hand value in the without tools / with tools pair) row, Kimi K3 (max) column (leftmost numeric column). Supporting footnote: Multimodal (average of 3 runs, official protocol) |
official results (Gemini 3.5 Flash model card)
| Rank | Model | effort | Score | Source cost (metric) | measured/reported by | source |
|---|---|---|---|---|---|---|
| 1 | Gemini 3.5 Flash | not stated | 83.6% | — | Gemini 3.5 Flash model cardRetrieved 2026-11-13 · Vendor-reported value. Table in the Google model card. methodology PDF (deepmind.google/models/evals-methodology/gemini-3-5-flash): All of the results are all run with the Gemini API ... with default sampling settings unless indicated otherwise. Gemini computed in-house; average of Standard (10 choices) and Vision; without tools. thinking level not statedSource excerptMMMU-Pro Multimodal understanding and reasoning | No tools | 83.6%Location Gemini 3.5 Flash model card Evaluation table, MMMU-Pro(No tools) row, Gemini 3.5 Flash column. In-house computation basis: methodology PDF, Image / MMMU-Pro | |
| 2 | Gemini 3 Flash Preview | not stated | 81.2% | — | Gemini 3.5 Flash model cardRetrieved 2026-11-13 · Vendor-reported value. Table in the Google model card. methodology PDF (deepmind.google/models/evals-methodology/gemini-3-5-flash): All of the results are all run with the Gemini API ... with default sampling settings unless indicated otherwise. Gemini computed in-house; average of Standard (10 choices) and Vision; without tools. Column heading is Gemini 3 Flash (gemini-3-flash-preview is the only match in the Gemini API model list). thinking level not statedSource excerptMMMU-Pro Multimodal understanding and reasoning | No tools | 81.2%Location Gemini 3.5 Flash model card Evaluation table, MMMU-Pro(No tools) row, Gemini 3 Flash column. In-house computation basis: methodology PDF, Image / MMMU-Pro | |
| 3 | Gemini 3.1 Pro Preview | not stated | 80.5% | — | Gemini 3.5 Flash model cardRetrieved 2026-11-13 · Vendor-reported value. Table in the Google model card. methodology PDF (deepmind.google/models/evals-methodology/gemini-3-5-flash): All of the results are all run with the Gemini API ... with default sampling settings unless indicated otherwise. Gemini computed in-house; average of Standard (10 choices) and Vision; without tools. Column heading is Gemini 3.1 Pro (gemini-3.1-pro-preview is the only match in the Gemini API model list). thinking level not statedSource excerptMMMU-Pro Multimodal understanding and reasoning | No tools | 80.5%Location Gemini 3.5 Flash model card Evaluation table, MMMU-Pro(No tools) row, Gemini 3.1 Pro column. In-house computation basis: methodology PDF, Image / MMMU-Pro |