Will it fit? A free planner for fine-tuning models that do not fit on your GPU

Automated excerpt

Measured on a 6 GB laptop card: a 32B model at 1024 tokens/step, 36.7 tok/s, 4.61 GB peak. On Windows, PEFT does not OOM for 7B/14B — it silently spills into shared system memory (10 GB and 16 GB reserved on a 6 GB card) and pays 5.3x and 2.8x the step time. Fixed, both start at the same number (1.42087). 2k context does not fit on 6 GB (~0.6 GB short, bf16 lm_head); --head nf4 is broken in the bitsandbytes version I tested. 3B/7B fine-tuning is faster with PEFT.

Selected automatically from source text; not independently written or fact-checked. Read the original for full context.