Why Your Phone Runs LLMs 80x Slower Than It Should (And What I Found)

Automated excerpt

Tags: on-device inference / llama. cpp / Android / performance I ran Qwen2. 5-1. 5B-Instruct Q4_K_M fully offline on a Google Pixel 4 (Snapdragon 855, 2019). Suspect 2: missing ARM optimization? (HIT) llama. cpp does its heavy lifting in ggml. Suspect 3: mmap pages getting evicted? (ruled out) Android reclaims mmap'ed pages under memory pressure.

Selected automatically from source text; not independently written or fact-checked. Read the original for full context.