The KV Cache From Scratch

Automated excerpt

In every part of this series, one thing kept showing up: the notebook. In Part 1 it grew by 72 KB with every word. The notebook is just those two cards, K and V, saved for every earlier word, on every floor, for every head.

Selected automatically from source text; not independently written or fact-checked. Read the original for full context.