ExploitGym
Success rate for exploiting vulnerabilities.
- Category
- Other
- Unit
- Percent (%)
- Higher is better
- Comparison groups
- 2total
- All model rankings (records) · Ranked results: 3
All model rankings (records)
These rankings are recalculated by comparison group including legacy models and models unavailable in Claude Code / Codex. They differ from the rankings above because the set of models differs.
Show rankings including legacy and unavailable models (entries: 3)
official results (Gemini 4 Carbon (Google DeepMind models page, benchmark table))
| Rank | Model | effort | Score | Source cost (metric) | measured/reported by | source |
|---|---|---|---|---|---|---|
| 1 | Gemini 4 Carbon | High | 54.8% | — | Gemini 4 Carbon (Google DeepMind models page, benchmark table)Retrieved 2026-11-12 · Vendor-reported value. Google published table. methodology PDF: Gemini 4 Carbon/Argon used the highest Gemini API thinking setting, pass@1, and a single trial. Row-specific source: self-measured. Antigravity harness, single trial, no internet, default 6-hour timeout (performance via Fairwind Program)Source excerptExploitGym: Gemini 4 Carbon 54.8 | |
| 2 | Gemini 4 Argon | High | 42.0% | — | Gemini 4 Carbon (Google DeepMind models page, benchmark table)Retrieved 2026-11-12 · Vendor-reported value. Google published table. methodology PDF: Gemini 4 Carbon/Argon used the highest Gemini API thinking setting, pass@1, and a single trial. Row-specific source: self-measured. Antigravity harness, single trial, no internet, default 6-hour timeout (performance via Fairwind Program)Source excerptExploitGym: Gemini 4 Argon 42.0 |
official results (Kimi K3.1 Pro (Hugging Face model card, moonshotai))
| Rank | Model | effort | Score | Source cost (metric) | measured/reported by | source |
|---|---|---|---|---|---|---|
| 1 | Kimi K3.1 Pro | Max | 20.7% | — | Moonshot AI | Kimi K3.1 Pro (Hugging Face model card, moonshotai)Retrieved 2026-11-13 · Vendor-reported value (self-reported by Moonshot AI). Table column heading Kimi K3.1 Pro (max), table note: all Kimi K3.1 Pro results use reasoning effort max, temperature 1.0, top-p 0.95. Kimi Code; 6 hours per 1 task.Source excerptExploitGym | 20.7Location HF model card "3. Evaluation Results" table, ExploitGym row, Kimi K3.1 Pro (max) column (leftmost numeric column). Supporting footnote: Cyber 2 |