Bad benchmarks and evals: Senior SWE-Bench, napkin math, and winter tires

Automated excerpt

People frequently say that winter tires are superior to all-season tires in cold weather. What first jumped out to me, besides the memory latency number, was some of the other numbers. Some things that don't feel right about this are: The +1 makes every read unaligned. Since I have no idea what the numbers should be here, maybe we can look up some numbers. And if we look at other numbers, it seems like the broader point that having a few single numbers for specific read sizes isn't representative of disk performance in general.

Selected automatically from source text; not independently written or fact-checked. Read the original for full context.