NanoGPT Slowrun: Language Modeling with Limited Data, Infinite Compute

Automated excerpt

The solution is to build new learning algorithms that work in limited data, practically infinite compute settings. NanoGPT Slowrun baseline: 2. 4x data efficiency Last week we released NanoGPT Slowrun , an open repo for data-efficient learning algorithms. The baseline sits at ~2. 4x data efficiency against modded-nanogpt.

Selected automatically from source text; not independently written or fact-checked. Read the original for full context.