LLMs
- Home /
- Categories /
- LLMs
vLLM Internals: PagedAttention, Continuous Batching, and Where the 2-4x Comes From
A decode step moves every weight in the model from HBM into the SMs to produce exactly one token per sequence. At batch size 1 that is a …
Read More
Scaling Laws: Kaplan, Chinchilla, and Why Nobody Trains Chinchilla-Optimal
Say you’ve got loss numbers from a 40M-parameter run and a 400M-parameter run, same data, same architecture family, same optimizer. …
Read More