Scaling-Laws

Scaling Laws: Kaplan, Chinchilla, and Why Nobody Trains Chinchilla-Optimal

• Lakshmanan LN
• Machine learning, Deep learning, Llms

Say you’ve got loss numbers from a 40M-parameter run and a 400M-parameter run, same data, same architecture family, same optimizer. …