AI article
I Set Every Seed and Trained Twice. The Models Diverged at Step 1
Community description: torch.manual_seed is not reproducibility. I measured how fast two identically-seeded runs drift apart on GPU, why it happens, and what
Dev.to | Oct 12, 2026 | b1ackjack12
Automated excerpt
Last week, while chasing a one-point accuracy difference I couldn't explain, I finally tested the belief directly: same seed, same data order, same model init, same machine, same everything — train ResNet18 for 200 steps, twice, and compare. The two loss curves matched at step 0 and diverged at step 1. Same numbers, different addition order, last-bit differences in the gradients.
Selected automatically from source text; not independently written or fact-checked. Read the original for full context.