Show HN: Which Mac runs which LLM, and how fast

Automated excerpt

Llama 3. 3 70B at Q4_K_M runs on M4 Pro with 64 GB. Expect roughly 5. 7 tokens/sec of generation. Each chip people commonly use for inference also has its own page with the full model by model breakdown at each memory configuration. ChipBandwidthMemory optionsFound inM168 GB/s8, 16 GBMacBook Air, 13-inch MacBook Pro, Mac mini, iMac (2020-2021)16 GB ceiling makes 14B the hard limit, and only at Q4. M1 Pro200 GB/s16, 32 GBMacBook Pro 14/16-inch (2021)M1 Max400 GB/s32, 64 GBMacBook Pro 14/16-inch (2021), Mac Studio (2022)Still one of the best value-per-GB/s machines on the used market.

Selected automatically from source text; not independently written or fact-checked. Read the original for full context.