CWE-bench v2: More coverage, quality, and transparency

Automated excerpt

In our continued pursuit to push the horizon at Collinear AI, CWE-bench v2 opens up a public set of 100 tasks that anyone can run and evaluate their models on. To evaluate your model on the held-out set, contact cyber@collinear.ai. Of the 11 models evaluated, Gemini 4 Argon leads with 67.8% pass@1 and 81.0% pass@4, just ahead of Claude Opus 5.5 (67.3% pass@1, 78.0% pass@4). 95 tasks are solved by at least one model, an indication that the tasks are indeed solvable.

Selected automatically from source text; not independently written or fact-checked. Read the original for full context.