In 2018 I hand-wrote a C++ deep learning framework so I'd never pad a batch. In 2023 LLM serving landed on the same structure.

Automated excerpt

Same skeleton as InsNet: build the graph lazily, compute a signature per node, batch same-signature ready nodes. Only identically-shaped operations ever share a batch. Same enemy (padding), same weapon (flat memory + offset bookkeeping), different battlefield (serving scheduler vs. training executor).

Selected automatically from source text; not independently written or fact-checked. Read the original for full context.