Infinite-Parameter LLMs: Generating and Adapting Weights from Live Data

Automated excerpt

That success is built on static pretraining data. A conventional model cannot learn from this data, because its weights are frozen after training. A compact hypernetwork turns the data given at run time into a low-rank modulation of a shared base network, so the feed-forward weights are generated from live data rather than stored in a fixed bank.

Selected automatically from source text; not independently written or fact-checked. Read the original for full context.