modelbench日本語

CharXiv Reasoning

Reasoning about paper figures and tables (without tools).

Category
Multimodal
Unit
Percent (%)
Higher is better
Comparison groups
5total
All model rankings (records) · Ranked results: 8

All model rankings (records)

These rankings are recalculated by comparison group including legacy models and models unavailable in Claude Code / Codex. They differ from the rankings above because the set of models differs.

Show rankings including legacy and unavailable models (entries: 8)

official results (Gemini 3.8 Flash model card)

RankModeleffortScoreSource cost (metric)measured/reported bysource
1Gemini 3.8 Flashnot stated86.2%—GoogleGemini 3.8 Flash model cardRetrieved 2026-11-12 · Vendor-reported value. Self-reported by Google (model card, as of 2026-09). thinking setting not stated
Source excerptCharXiv Reasoning (no tools) | 86.2%

official results (Kimi K3.1 Pro (Hugging Face model card, moonshotai))

RankModeleffortScoreSource cost (metric)measured/reported bysource
1Kimi K3.1 ProMax89.0%—Moonshot AIKimi K3.1 Pro (Hugging Face model card, moonshotai)Retrieved 2026-11-13 · Vendor-reported value (self-reported by Moonshot AI). Table column heading Kimi K3.1 Pro (max), table note: all Kimi K3.1 Pro results use reasoning effort max, temperature 1.0, top-p 0.95
Source excerptCharXiv (RQ) | 89.0Location HF model card "3. Evaluation Results" table, CharXiv (RQ) row, Kimi K3.1 Pro (max) column (leftmost numeric column). Supporting footnote: Reasoning 3 (average of 4 runs)

official results (Kimi K3 (Hugging Face model card, moonshotai))

RankModeleffortScoreSource cost (metric)measured/reported bysource
1Kimi K3Max84.8%—Moonshot AIKimi K3 (Hugging Face model card, moonshotai)Retrieved 2026-11-13 · Vendor-reported value (self-reported by Moonshot AI). Table note: All Kimi K3 results are obtained with reasoning effort set to 'max' and temperature = 1.0
Source excerptCharXiv (RQ) | 84.8 / 91.3Location HF model card "3. Evaluation Results" table, CharXiv (RQ) (left-hand value in the without tools / with tools pair) row, Kimi K3 (max) column (leftmost numeric column). Supporting footnote: Multimodal (average of 3 runs)

official results (Gemini 3.7 Flash model card)

RankModeleffortScoreSource cost (metric)measured/reported bysource
1Gemini 3.6 Flashnot stated85.2%—GoogleGemini 3.7 Flash model cardRetrieved 2026-11-13 · Vendor-reported value. Table in the Google model card. methodology PDF (deepmind.com/models/evals-methodology/gemini-3-7-flash): All of the results are run with the Gemini API ... with default sampling settings unless indicated otherwise. Gemini computed in-house; without tools
Source excerptCharXiv Reasoning Information synthesis from complex charts | No tools | 85.2%Location Gemini 3.7 Flash model card Evaluation table, CharXiv Reasoning(No tools) row, Gemini 3.6 Flash column. In-house computation basis: methodology PDF, Multimodal / CharXiv Reasoning
2Gemini 3.7 Flashnot stated84.5%—GoogleGemini 3.7 Flash model cardRetrieved 2026-11-13 · Vendor-reported value. Table in the Google model card. methodology PDF (deepmind.com/models/evals-methodology/gemini-3-7-flash): All of the results are run with the Gemini API ... with default sampling settings unless indicated otherwise. Gemini computed in-house; without tools
Source excerptCharXiv Reasoning Information synthesis from complex charts | No tools | 84.5%Location Gemini 3.7 Flash model card Evaluation table, CharXiv Reasoning(No tools) row, Gemini 3.7 Flash column. In-house computation basis: methodology PDF, Multimodal / CharXiv Reasoning

official results (Gemini 3.5 Flash model card)

RankModeleffortScoreSource cost (metric)measured/reported bysource
1Gemini 3.5 Flashnot stated84.2%—GoogleGemini 3.5 Flash model cardRetrieved 2026-11-13 · Vendor-reported value. Table in the Google model card. methodology PDF (deepmind.google/models/evals-methodology/gemini-3-5-flash): All of the results are all run with the Gemini API ... with default sampling settings unless indicated otherwise. Gemini computed in-house; without tools. thinking level not stated
Source excerptMultimodal | CharXiv Reasoning Information synthesis from complex charts | No tools | 84.2%Location Gemini 3.5 Flash model card Evaluation table, CharXiv Reasoning(No tools) row, Gemini 3.5 Flash column. In-house computation basis: methodology PDF, Image / CharXiv Reasoning
2Gemini 3.1 Pro Previewnot stated83.3%—GoogleGemini 3.5 Flash model cardRetrieved 2026-11-13 · Vendor-reported value. Table in the Google model card. methodology PDF (deepmind.google/models/evals-methodology/gemini-3-5-flash): All of the results are all run with the Gemini API ... with default sampling settings unless indicated otherwise. Gemini computed in-house; without tools. Column heading is Gemini 3.1 Pro (gemini-3.1-pro-preview is the only match in the Gemini API model list). thinking level not stated
Source excerptMultimodal | CharXiv Reasoning Information synthesis from complex charts | No tools | 83.3%Location Gemini 3.5 Flash model card Evaluation table, CharXiv Reasoning(No tools) row, Gemini 3.1 Pro column. In-house computation basis: methodology PDF, Image / CharXiv Reasoning
3Gemini 3 Flash Previewnot stated80.3%—GoogleGemini 3.5 Flash model cardRetrieved 2026-11-13 · Vendor-reported value. Table in the Google model card. methodology PDF (deepmind.google/models/evals-methodology/gemini-3-5-flash): All of the results are all run with the Gemini API ... with default sampling settings unless indicated otherwise. Gemini computed in-house; without tools. Column heading is Gemini 3 Flash (gemini-3-flash-preview is the only match in the Gemini API model list). thinking level not stated
Source excerptMultimodal | CharXiv Reasoning Information synthesis from complex charts | No tools | 80.3%Location Gemini 3.5 Flash model card Evaluation table, CharXiv Reasoning(No tools) row, Gemini 3 Flash column. In-house computation basis: methodology PDF, Image / CharXiv Reasoning