Real-SWE: Benchmarking AI models on private, real-world, enterprise codebases

Automated excerpt

Agents have to understand those conventions and make changes that work with what’s already there. Can a coding agent actually do the work of a software engineer in the real world? 1Fable 5. 1Claude CodeResolution rate: 38. 8%2GPT-6 AstraCodex CLIResolution rate: 33. 8%3Grok 4. 6Grok BuildResolution rate: 32. 5%4Gemini 3. 8 FlashGemini CLIResolution rate: 31. 2%5GLM 5. 3Claude CodeResolution rate: 28. 8%6Muse Spark 1. 3Muse CodeResolution rate: 23. 8%7Kimi K3Kimi CodeResolution rate: 18. 8%8GPT-5. 6 SolCodex CLIResolution rate: 16. 2%#ModelHarnessResolution rate1Fable 5. 1Claude Code38. 8%2GPT-6 AstraCodex CLI33. 8%3Grok 4. 6Grok Build32. 5%4Gemini 3. 8 FlashGemini CLI31. 2%5GLM 5. 3Claude Code28.

Selected automatically from source text; not independently written or fact-checked. Read the original for full context.