MMMU-Pro (with tools)

MMMU-Pro solved with tools such as Python.

Category: Multimodal
Unit: Percent (%); Higher is better
Comparison groups: 2total; All model rankings (records) · Ranked results: 2

All model rankings (records)

These rankings are recalculated by comparison group including legacy models and models unavailable in Claude Code / Codex. They differ from the rankings above because the set of models differs.

Show rankings including legacy and unavailable models (entries: 2)

official results (Kimi K3.1 Pro (Hugging Face model card, moonshotai))

Rank	Model	effort	Score	Source cost (metric)	measured/reported by	source
1	Kimi K3.1 Pro	Max	90.0%	—	Moonshot AI	Kimi K3.1 Pro (Hugging Face model card, moonshotai)Retrieved 2026-11-13 · Vendor-reported value (self-reported by Moonshot AI). Table column heading Kimi K3.1 Pro (max), table note: all Kimi K3.1 Pro results use reasoning effort max, temperature 1.0, top-p 0.95. With tools (Python); average of 4 runs. Source excerpt MMMU-Pro w/ python \| 90.0Location HF model card "3. Evaluation Results" table, MMMU-Pro w/ python row, Kimi K3.1 Pro (max) column (leftmost numeric column). Supporting footnote: Agentic 11

official results (Kimi K3 (Hugging Face model card, moonshotai))

Rank	Model	effort	Score	Source cost (metric)	measured/reported by	source
1	Kimi K3	Max	83.4%	—	Moonshot AI	Kimi K3 (Hugging Face model card, moonshotai)Retrieved 2026-11-13 · Vendor-reported value (self-reported by Moonshot AI). Table note: All Kimi K3 results are obtained with reasoning effort set to 'max' and temperature = 1.0 Source excerpt MMMU-Pro \| 81.6 / 83.4Location HF model card "3. Evaluation Results" table, MMMU-Pro (right-hand value in the without tools / with tools pair) row, Kimi K3 (max) column (leftmost numeric column). Supporting footnote: table note (vision tools use Python)