AutomationBench
Created by Zapier. Measures whether business workflows can be carried out across numerous integrated apps.
- Category
- Tool use
- Unit
- Percent (%)
- Higher is better
- Comparison groups
- 2total
- Currently available model rankings · Ranked results: 13
- All models (records): Groups: 4 · Entries: 29
Rankings
Values with different source types (official, third-party, paper) or measurement conditions cannot be compared, so rankings are separated by comparison group (the same source, or sources verified to use the same conditions).
Rankings include only models currently available in Claude Code / Codex, excluding legacy models.
- View on rankings page (by metric)
- Cost balance (frontier and effort changes)
- Overall ranking (performance first)
official results (Introducing Claude Opus 5.5)
| Rank | Model | effort | Score | Source cost (metric) | measured/reported by | source |
|---|---|---|---|---|---|---|
| 1 | GPT-6 Astra | Max | 41.4% | $1.77Cost per task | Zapier | Introducing Claude Opus 5.5Retrieved 2026-11-12 · Vendor-reported value. A point in the figure "AutomationBench / Accuracy vs Cost" on the Anthropic announcement page. values were taken from each point's aria-label (DOM); the page shows them only in the figure; measured by: Zapier (Opus 5.5 is from a Zapier early access evaluation; the others are from the Zapier public leaderboard); the figure's effort label "Max" was mapped to max in the OpenAI API reasoning effortSource excerptAutomationBench — Accuracy vs Cost: "GPT-6 Astra · Max: 41.4% $1.77" |
| 2 | Claude Opus 5.5 | Max | 40.0% | $1.37Cost per task | Zapier | Introducing Claude Opus 5.5Retrieved 2026-11-12 · Vendor-reported value. A point in the figure "AutomationBench / Accuracy vs Cost" on the Anthropic announcement page. values were taken from each point's aria-label (DOM); the page shows them only in the figure; measured by: Zapier (Opus 5.5 is from a Zapier early access evaluation; the others are from the Zapier public leaderboard)Source excerptAutomationBench — Accuracy vs Cost: "Opus 5.5 · Max: 40.0% $1.37" |
| 3 | GPT-6 Astra | Xhigh | 39.0% | $1.53Cost per task | Zapier | Introducing Claude Opus 5.5Retrieved 2026-11-12 · Vendor-reported value. A point in the figure "AutomationBench / Accuracy vs Cost" on the Anthropic announcement page. values were taken from each point's aria-label (DOM); the page shows them only in the figure; measured by: Zapier (Opus 5.5 is from a Zapier early access evaluation; the others are from the Zapier public leaderboard); the figure's effort label "Xhigh" was mapped to xhigh in the OpenAI API reasoning effortSource excerptAutomationBench — Accuracy vs Cost: "GPT-6 Astra · Xhigh: 39.0% $1.53" |
| 4 | GPT-6 Astra | High | 37.1% | $1.45Cost per task | Zapier | Introducing Claude Opus 5.5Retrieved 2026-11-12 · Vendor-reported value. A point in the figure "AutomationBench / Accuracy vs Cost" on the Anthropic announcement page. values were taken from each point's aria-label (DOM); the page shows them only in the figure; measured by: Zapier (Opus 5.5 is from a Zapier early access evaluation; the others are from the Zapier public leaderboard); the figure's effort label "High" was mapped to high in the OpenAI API reasoning effortSource excerptAutomationBench — Accuracy vs Cost: "GPT-6 Astra · High: 37.1% $1.45" |
| 5 | Claude Opus 5.5 | Xhigh | 34.4% | $0.86Cost per task | Zapier | Introducing Claude Opus 5.5Retrieved 2026-11-12 · Vendor-reported value. A point in the figure "AutomationBench / Accuracy vs Cost" on the Anthropic announcement page. values were taken from each point's aria-label (DOM); the page shows them only in the figure; measured by: Zapier (Opus 5.5 is from a Zapier early access evaluation; the others are from the Zapier public leaderboard)Source excerptAutomationBench — Accuracy vs Cost: "Opus 5.5 · Xhigh: 34.4% $0.86" |
| 6 | GPT-6 Astra | Med | 34.1% | $1.28Cost per task | Zapier | Introducing Claude Opus 5.5Retrieved 2026-11-12 · Vendor-reported value. A point in the figure "AutomationBench / Accuracy vs Cost" on the Anthropic announcement page. values were taken from each point's aria-label (DOM); the page shows them only in the figure; measured by: Zapier (Opus 5.5 is from a Zapier early access evaluation; the others are from the Zapier public leaderboard); the figure's effort label "Med" was mapped to medium in the OpenAI API reasoning effortSource excerptAutomationBench — Accuracy vs Cost: "GPT-6 Astra · Med: 34.1% $1.28" |
| 7 | Claude Opus 5.5 | High | 32.0% | $0.70Cost per task | Zapier | Introducing Claude Opus 5.5Retrieved 2026-11-12 · Vendor-reported value. A point in the figure "AutomationBench / Accuracy vs Cost" on the Anthropic announcement page. values were taken from each point's aria-label (DOM); the page shows them only in the figure; measured by: Zapier (Opus 5.5 is from a Zapier early access evaluation; the others are from the Zapier public leaderboard)Source excerptAutomationBench — Accuracy vs Cost: "Opus 5.5 · High: 32.0% $0.70" |
| 8 | GPT-6 Astra | Low | 30.3% | $1.08Cost per task | Zapier | Introducing Claude Opus 5.5Retrieved 2026-11-12 · Vendor-reported value. A point in the figure "AutomationBench / Accuracy vs Cost" on the Anthropic announcement page. values were taken from each point's aria-label (DOM); the page shows them only in the figure; measured by: Zapier (Opus 5.5 is from a Zapier early access evaluation; the others are from the Zapier public leaderboard); the figure's effort label "Low" was mapped to low in the OpenAI API reasoning effortSource excerptAutomationBench — Accuracy vs Cost: "GPT-6 Astra · Low: 30.3% $1.08" |
| 9 | Claude Opus 5.5 | Med | 28.6% | $0.64Cost per task | Zapier | Introducing Claude Opus 5.5Retrieved 2026-11-12 · Vendor-reported value. A point in the figure "AutomationBench / Accuracy vs Cost" on the Anthropic announcement page. values were taken from each point's aria-label (DOM); the page shows them only in the figure; measured by: Zapier (Opus 5.5 is from a Zapier early access evaluation; the others are from the Zapier public leaderboard)Source excerptAutomationBench — Accuracy vs Cost: "Opus 5.5 · Med: 28.6% $0.64" |
| 10 | Claude Opus 5.5 | Low | 23.3% | $0.50Cost per task | Zapier | Introducing Claude Opus 5.5Retrieved 2026-11-12 · Vendor-reported value. A point in the figure "AutomationBench / Accuracy vs Cost" on the Anthropic announcement page. values were taken from each point's aria-label (DOM); the page shows them only in the figure; measured by: Zapier (Opus 5.5 is from a Zapier early access evaluation; the others are from the Zapier public leaderboard)Source excerptAutomationBench — Accuracy vs Cost: "Opus 5.5 · Low: 23.3% $0.50" |
official results (Introducing Claude Fable 5.2 and Claude Mythos 5.2)
| Rank | Model | effort | Score | Source cost (metric) | measured/reported by | source |
|---|---|---|---|---|---|---|
| 1 | Claude Fable 5.2 | not stated | 51.4% | — | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 · Vendor-reported value. table note: Scores for Fable 5.2, Fable 5.1, and Opus 5.5 show results with safeguards on unless otherwise noted. All scores show each model’s highest-available effort unless otherwise noted.Source excerpt51.4%Location Table, AutomationBench row: 51.4 |
| 2 | Claude Opus 5.5 | not stated | 42.5% | — | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 · Vendor-reported value. table note: Scores for Fable 5.2, Fable 5.1, and Opus 5.5 show results with safeguards on unless otherwise noted. All scores show each model’s highest-available effort unless otherwise noted.Source excerpt42.5%Location Table, AutomationBench row: 42.5 |
| 3 | GPT-6 Astra | not stated | 41.4% | — | not stated | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 · Vendor-reported value. table note: Scores for Fable 5.2, Fable 5.1, and Opus 5.5 show results with safeguards on unless otherwise noted. All scores show each model’s highest-available effort unless otherwise noted. Values for other vendor models (as listed by Anthropic).Source excerpt41.4%Location Table, AutomationBench row: 41.4 |
All model rankings (records)
These rankings are recalculated by comparison group including legacy models and models unavailable in Claude Code / Codex. They differ from the rankings above because the set of models differs.
- View all model rankings (by metric)
- Cost balance (all models, records)
- Overall ranking (all models, records)
Show rankings including legacy and unavailable models (entries: 29)
official results (Introducing Claude Opus 5.5)
| Rank | Model | effort | Score | Source cost (metric) | measured/reported by | source |
|---|---|---|---|---|---|---|
| 1 | GPT-6 Astra | Max | 41.4% | $1.77Cost per task | Zapier | Introducing Claude Opus 5.5Retrieved 2026-11-12 · Vendor-reported value. A point in the figure "AutomationBench / Accuracy vs Cost" on the Anthropic announcement page. values were taken from each point's aria-label (DOM); the page shows them only in the figure; measured by: Zapier (Opus 5.5 is from a Zapier early access evaluation; the others are from the Zapier public leaderboard); the figure's effort label "Max" was mapped to max in the OpenAI API reasoning effortSource excerptAutomationBench — Accuracy vs Cost: "GPT-6 Astra · Max: 41.4% $1.77" |
| 2 | Claude Opus 5.5 | Max | 40.0% | $1.37Cost per task | Zapier | Introducing Claude Opus 5.5Retrieved 2026-11-12 · Vendor-reported value. A point in the figure "AutomationBench / Accuracy vs Cost" on the Anthropic announcement page. values were taken from each point's aria-label (DOM); the page shows them only in the figure; measured by: Zapier (Opus 5.5 is from a Zapier early access evaluation; the others are from the Zapier public leaderboard)Source excerptAutomationBench — Accuracy vs Cost: "Opus 5.5 · Max: 40.0% $1.37" |
| 3 | GPT-6 Astra | Xhigh | 39.0% | $1.53Cost per task | Zapier | Introducing Claude Opus 5.5Retrieved 2026-11-12 · Vendor-reported value. A point in the figure "AutomationBench / Accuracy vs Cost" on the Anthropic announcement page. values were taken from each point's aria-label (DOM); the page shows them only in the figure; measured by: Zapier (Opus 5.5 is from a Zapier early access evaluation; the others are from the Zapier public leaderboard); the figure's effort label "Xhigh" was mapped to xhigh in the OpenAI API reasoning effortSource excerptAutomationBench — Accuracy vs Cost: "GPT-6 Astra · Xhigh: 39.0% $1.53" |
| 4 | GPT-6 Astra | High | 37.1% | $1.45Cost per task | Zapier | Introducing Claude Opus 5.5Retrieved 2026-11-12 · Vendor-reported value. A point in the figure "AutomationBench / Accuracy vs Cost" on the Anthropic announcement page. values were taken from each point's aria-label (DOM); the page shows them only in the figure; measured by: Zapier (Opus 5.5 is from a Zapier early access evaluation; the others are from the Zapier public leaderboard); the figure's effort label "High" was mapped to high in the OpenAI API reasoning effortSource excerptAutomationBench — Accuracy vs Cost: "GPT-6 Astra · High: 37.1% $1.45" |
| 5 | Claude Opus 5.5 | Xhigh | 34.4% | $0.86Cost per task | Zapier | Introducing Claude Opus 5.5Retrieved 2026-11-12 · Vendor-reported value. A point in the figure "AutomationBench / Accuracy vs Cost" on the Anthropic announcement page. values were taken from each point's aria-label (DOM); the page shows them only in the figure; measured by: Zapier (Opus 5.5 is from a Zapier early access evaluation; the others are from the Zapier public leaderboard)Source excerptAutomationBench — Accuracy vs Cost: "Opus 5.5 · Xhigh: 34.4% $0.86" |
| 6 | GPT-6 Astra | Med | 34.1% | $1.28Cost per task | Zapier | Introducing Claude Opus 5.5Retrieved 2026-11-12 · Vendor-reported value. A point in the figure "AutomationBench / Accuracy vs Cost" on the Anthropic announcement page. values were taken from each point's aria-label (DOM); the page shows them only in the figure; measured by: Zapier (Opus 5.5 is from a Zapier early access evaluation; the others are from the Zapier public leaderboard); the figure's effort label "Med" was mapped to medium in the OpenAI API reasoning effortSource excerptAutomationBench — Accuracy vs Cost: "GPT-6 Astra · Med: 34.1% $1.28" |
| 7 | Claude Opus 5.5 | High | 32.0% | $0.70Cost per task | Zapier | Introducing Claude Opus 5.5Retrieved 2026-11-12 · Vendor-reported value. A point in the figure "AutomationBench / Accuracy vs Cost" on the Anthropic announcement page. values were taken from each point's aria-label (DOM); the page shows them only in the figure; measured by: Zapier (Opus 5.5 is from a Zapier early access evaluation; the others are from the Zapier public leaderboard)Source excerptAutomationBench — Accuracy vs Cost: "Opus 5.5 · High: 32.0% $0.70" |
| 8 | Claude Fable 5.1 | not stated | 31.4% | — | Zapier | Introducing Claude Opus 5.5Retrieved 2026-11-12 · Vendor-reported value. table note: Unless otherwise noted, all Claude Opus 5.5 results use adaptive thinking at max effort. production safeguards enabled. Run and reported by Zapier.Source excerpt31.4%Location Table, AutomationBench row: 31.4 |
| 9 | GPT-6 Astra | Low | 30.3% | $1.08Cost per task | Zapier | Introducing Claude Opus 5.5Retrieved 2026-11-12 · Vendor-reported value. A point in the figure "AutomationBench / Accuracy vs Cost" on the Anthropic announcement page. values were taken from each point's aria-label (DOM); the page shows them only in the figure; measured by: Zapier (Opus 5.5 is from a Zapier early access evaluation; the others are from the Zapier public leaderboard); the figure's effort label "Low" was mapped to low in the OpenAI API reasoning effortSource excerptAutomationBench — Accuracy vs Cost: "GPT-6 Astra · Low: 30.3% $1.08" |
| 10 | GPT-5.6 Sol | Max | 28.8% | $0.91Cost per task | Zapier | Introducing Claude Opus 5.5Retrieved 2026-11-12 · Vendor-reported value. A point in the figure "AutomationBench / Accuracy vs Cost" on the Anthropic announcement page. values were taken from each point's aria-label (DOM); the page shows them only in the figure; measured by: Zapier (Opus 5.5 is from a Zapier early access evaluation; the others are from the Zapier public leaderboard); the figure's effort label "Max" was mapped to max in the OpenAI API reasoning effortSource excerptAutomationBench — Accuracy vs Cost: "GPT-5.6 Sol · Max: 28.8% $0.91" |
| 11 | Claude Opus 5.5 | Med | 28.6% | $0.64Cost per task | Zapier | Introducing Claude Opus 5.5Retrieved 2026-11-12 · Vendor-reported value. A point in the figure "AutomationBench / Accuracy vs Cost" on the Anthropic announcement page. values were taken from each point's aria-label (DOM); the page shows them only in the figure; measured by: Zapier (Opus 5.5 is from a Zapier early access evaluation; the others are from the Zapier public leaderboard)Source excerptAutomationBench — Accuracy vs Cost: "Opus 5.5 · Med: 28.6% $0.64" |
| 12 | Claude Opus 5 | Max | 26.9% | $1.27Cost per task | Zapier | Introducing Claude Opus 5.5Retrieved 2026-11-12 · Vendor-reported value. A point in the figure "AutomationBench / Accuracy vs Cost" on the Anthropic announcement page. values were taken from each point's aria-label (DOM); the page shows them only in the figure; measured by: Zapier (Opus 5.5 is from a Zapier early access evaluation; the others are from the Zapier public leaderboard)Source excerptAutomationBench — Accuracy vs Cost: "Opus 5 · Max: 26.9% $1.27" |
| 13 | GPT-5.6 Sol | Xhigh | 26.3% | $0.74Cost per task | Zapier | Introducing Claude Opus 5.5Retrieved 2026-11-12 · Vendor-reported value. A point in the figure "AutomationBench / Accuracy vs Cost" on the Anthropic announcement page. values were taken from each point's aria-label (DOM); the page shows them only in the figure; measured by: Zapier (Opus 5.5 is from a Zapier early access evaluation; the others are from the Zapier public leaderboard); the figure's effort label "Xhigh" was mapped to xhigh in the OpenAI API reasoning effortSource excerptAutomationBench — Accuracy vs Cost: "GPT-5.6 Sol · Xhigh: 26.3% $0.74" |
| 14 | Claude Opus 5 | Xhigh | 25.3% | $1.15Cost per task | Zapier | Introducing Claude Opus 5.5Retrieved 2026-11-12 · Vendor-reported value. A point in the figure "AutomationBench / Accuracy vs Cost" on the Anthropic announcement page. values were taken from each point's aria-label (DOM); the page shows them only in the figure; measured by: Zapier (Opus 5.5 is from a Zapier early access evaluation; the others are from the Zapier public leaderboard)Source excerptAutomationBench — Accuracy vs Cost: "Opus 5 · Xhigh: 25.3% $1.15" |
| 15 | GPT-5.6 Sol | High | 24.8% | $0.64Cost per task | Zapier | Introducing Claude Opus 5.5Retrieved 2026-11-12 · Vendor-reported value. A point in the figure "AutomationBench / Accuracy vs Cost" on the Anthropic announcement page. values were taken from each point's aria-label (DOM); the page shows them only in the figure; measured by: Zapier (Opus 5.5 is from a Zapier early access evaluation; the others are from the Zapier public leaderboard); the figure's effort label "High" was mapped to high in the OpenAI API reasoning effortSource excerptAutomationBench — Accuracy vs Cost: "GPT-5.6 Sol · High: 24.8% $0.64" |
| 16 | Claude Opus 5 | Med | 23.9% | $0.89Cost per task | Zapier | Introducing Claude Opus 5.5Retrieved 2026-11-12 · Vendor-reported value. A point in the figure "AutomationBench / Accuracy vs Cost" on the Anthropic announcement page. values were taken from each point's aria-label (DOM); the page shows them only in the figure; measured by: Zapier (Opus 5.5 is from a Zapier early access evaluation; the others are from the Zapier public leaderboard)Source excerptAutomationBench — Accuracy vs Cost: "Opus 5 · Med: 23.9% $0.89" |
| 17 | Claude Opus 5.5 | Low | 23.3% | $0.50Cost per task | Zapier | Introducing Claude Opus 5.5Retrieved 2026-11-12 · Vendor-reported value. A point in the figure "AutomationBench / Accuracy vs Cost" on the Anthropic announcement page. values were taken from each point's aria-label (DOM); the page shows them only in the figure; measured by: Zapier (Opus 5.5 is from a Zapier early access evaluation; the others are from the Zapier public leaderboard)Source excerptAutomationBench — Accuracy vs Cost: "Opus 5.5 · Low: 23.3% $0.50" |
| 18 | Claude Opus 5 | High | 20.6% | $1.03Cost per task | Zapier | Introducing Claude Opus 5.5Retrieved 2026-11-12 · Vendor-reported value. A point in the figure "AutomationBench / Accuracy vs Cost" on the Anthropic announcement page. values were taken from each point's aria-label (DOM); the page shows them only in the figure; measured by: Zapier (Opus 5.5 is from a Zapier early access evaluation; the others are from the Zapier public leaderboard)Source excerptAutomationBench — Accuracy vs Cost: "Opus 5 · High: 20.6% $1.03" |
| 19 | Claude Opus 5 | Low | 20.4% | $0.75Cost per task | Zapier | Introducing Claude Opus 5.5Retrieved 2026-11-12 · Vendor-reported value. A point in the figure "AutomationBench / Accuracy vs Cost" on the Anthropic announcement page. values were taken from each point's aria-label (DOM); the page shows them only in the figure; measured by: Zapier (Opus 5.5 is from a Zapier early access evaluation; the others are from the Zapier public leaderboard)Source excerptAutomationBench — Accuracy vs Cost: "Opus 5 · Low: 20.4% $0.75" |
| 20 | GPT-5.6 Sol | Med | 19.6% | $0.57Cost per task | Zapier | Introducing Claude Opus 5.5Retrieved 2026-11-12 · Vendor-reported value. A point in the figure "AutomationBench / Accuracy vs Cost" on the Anthropic announcement page. values were taken from each point's aria-label (DOM); the page shows them only in the figure; measured by: Zapier (Opus 5.5 is from a Zapier early access evaluation; the others are from the Zapier public leaderboard); the figure's effort label "Med" was mapped to medium in the OpenAI API reasoning effortSource excerptAutomationBench — Accuracy vs Cost: "GPT-5.6 Sol · Med: 19.6% $0.57" |
| 21 | GPT-5.6 Sol | Low | 11.7% | $0.42Cost per task | Zapier | Introducing Claude Opus 5.5Retrieved 2026-11-12 · Vendor-reported value. A point in the figure "AutomationBench / Accuracy vs Cost" on the Anthropic announcement page. values were taken from each point's aria-label (DOM); the page shows them only in the figure; measured by: Zapier (Opus 5.5 is from a Zapier early access evaluation; the others are from the Zapier public leaderboard); the figure's effort label "Low" was mapped to low in the OpenAI API reasoning effortSource excerptAutomationBench — Accuracy vs Cost: "GPT-5.6 Sol · Low: 11.7% $0.42" |
official results (Introducing Claude Fable 5.2 and Claude Mythos 5.2)
| Rank | Model | effort | Score | Source cost (metric) | measured/reported by | source |
|---|---|---|---|---|---|---|
| 1 | Gemini 4 Carbon | not stated | 53.9% | — | not stated | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 · Vendor-reported value. table note: Scores for Fable 5.2, Fable 5.1, and Opus 5.5 show results with safeguards on unless otherwise noted. All scores show each model’s highest-available effort unless otherwise noted. Values for other vendor models (as listed by Anthropic).Source excerpt53.9%Location Table, AutomationBench row: 53.9 |
| 2 | Claude Fable 5.2 | not stated | 51.4% | — | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 · Vendor-reported value. table note: Scores for Fable 5.2, Fable 5.1, and Opus 5.5 show results with safeguards on unless otherwise noted. All scores show each model’s highest-available effort unless otherwise noted.Source excerpt51.4%Location Table, AutomationBench row: 51.4 |
| 3 | Claude Opus 5.5 | not stated | 42.5% | — | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 · Vendor-reported value. table note: Scores for Fable 5.2, Fable 5.1, and Opus 5.5 show results with safeguards on unless otherwise noted. All scores show each model’s highest-available effort unless otherwise noted.Source excerpt42.5%Location Table, AutomationBench row: 42.5 |
| 4 | GPT-6 Astra | not stated | 41.4% | — | not stated | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 · Vendor-reported value. table note: Scores for Fable 5.2, Fable 5.1, and Opus 5.5 show results with safeguards on unless otherwise noted. All scores show each model’s highest-available effort unless otherwise noted. Values for other vendor models (as listed by Anthropic).Source excerpt41.4%Location Table, AutomationBench row: 41.4 |
| 5 | Claude Fable 5.1 | not stated | 31.4% | — | Anthropic | Introducing Claude Fable 5.2 and Claude Mythos 5.2Retrieved 2026-11-12 · Vendor-reported value. table note: Scores for Fable 5.2, Fable 5.1, and Opus 5.5 show results with safeguards on unless otherwise noted. All scores show each model’s highest-available effort unless otherwise noted.Source excerpt31.4%Location Table, AutomationBench row: 31.4 |
official results (Gemini 4 Carbon (Google DeepMind models page, benchmark table))
| Rank | Model | effort | Score | Source cost (metric) | measured/reported by | source |
|---|---|---|---|---|---|---|
| 1 | Gemini 4 Carbon | High | 53.9% | — | Zapier | Gemini 4 Carbon (Google DeepMind models page, benchmark table)Retrieved 2026-11-12 · Vendor-reported value. Google published table. methodology PDF: Gemini 4 Carbon/Argon used the highest Gemini API thinking setting, pass@1, and a single trial. Row-specific source: Zapier official leaderboard (private set)Source excerptAutomationBench: Gemini 4 Carbon 53.9 |
| 2 | Gemini 4 Argon | High | 51.3% | — | Zapier | Gemini 4 Carbon (Google DeepMind models page, benchmark table)Retrieved 2026-11-12 · Vendor-reported value. Google published table. methodology PDF: Gemini 4 Carbon/Argon used the highest Gemini API thinking setting, pass@1, and a single trial. Row-specific source: Zapier official leaderboard (private set)Source excerptAutomationBench: Gemini 4 Argon 51.3 |
official results (Introducing Mistral Large 4)
| Rank | Model | effort | Score | Source cost (metric) | measured/reported by | source |
|---|---|---|---|---|---|---|
| 1 | Mistral Large 4 | not stated | 59.9% | — | Mistral AI | Introducing Mistral Large 4Retrieved 2026-11-12 · Vendor-reported value. Self-reported by Mistral (as of the 2026-10-06 preview version; a separate article says the later open weights version improved with additional RL)Source excerptAutomationBench: 59.9% |