AI article
LoRA & DoRA: The Math, Memory, and Trade-offs
Community description: LoRA and DoRA explained: matrix shapes, the geometry of weight decomposition, exact memory math for a 27B model, and what the DoRA benchmarks support.
Dev.to | Sep 26, 2026 | Aleksei Romanov
Automated excerpt
For example, (1, 0) points right with length 1. Adding (0, 1) gives (1, 1), which points diagonally and has length √2. In the 4,096 × 4,096 example, DoRA adds 4,096 scales, bringing the total to 135,168 trainable parameters.
Selected automatically from source text; not independently written or fact-checked. Read the original for full context.