SWE-Marathon v1.1
Long-running software engineering tasks (v1.1).
- Category
- Agentic coding
- Unit
- Percent (%)
- Higher is better
- Comparison groups
- 2total
- All model rankings (records) · Ranked results: 2
All model rankings (records)
These rankings are recalculated by comparison group including legacy models and models unavailable in Claude Code / Codex. They differ from the rankings above because the set of models differs.
Show rankings including legacy and unavailable models (entries: 2)
official results (Kimi K3.1 Pro (Hugging Face model card, moonshotai))
| Rank | Model | effort | Score | Source cost (metric) | measured/reported by | source |
|---|---|---|---|---|---|---|
| 1 | Kimi K3.1 Pro | Max | 51.7% | — | Moonshot AI | Kimi K3.1 Pro (Hugging Face model card, moonshotai)Retrieved 2026-11-13 · Vendor-reported value (self-reported by Moonshot AI). Table column heading Kimi K3.1 Pro (max), table note: all Kimi K3.1 Pro results use reasoning effort max, temperature 1.0, top-p 0.95. Evaluated with Claude Code.Source excerptSWE Marathon v1.1 | 51.7Location HF model card "3. Evaluation Results" table, SWE Marathon v1.1 row, Kimi K3.1 Pro (max) column (leftmost numeric column). Supporting footnote: Coding 3 |
official results (GLM-5.3 (Hugging Face model card, zai-org))
| Rank | Model | effort | Score | Source cost (metric) | measured/reported by | source |
|---|---|---|---|---|---|---|
| 1 | GLM-5.3 | Max | 42.5% | — | Z.ai | GLM-5.3 (Hugging Face model card, zai-org)Retrieved 2026-11-13 · Vendor-reported value (self-reported by Z.ai). Card Note: the default for reasoning_effort is max; use the default max to reproduce the benchmarks. Claude Code 2.1.207; maximum effort level. For some tasks, anti-cheat checks and build settings were changed.Source excerptSWE-Marathon (v1.1) | 42.5Location HF model card "Benchmark" table, SWE-Marathon (v1.1) row, GLM-5.3 column (leftmost numeric column). Basis: Footnotes, SWE-Marathon |