Scaling laws

Prescriptive Scaling Reveals the Evolution of Language Model Capabilities
Prescriptive scaling laws: estimate stable capability boundaries vs. pretraining compute, track temporal shifts, and release the Proteus 2k evaluation dataset.
Hanlin Zhang, Jikai Jin, Vasilis Syrgkanis, Sham Kakade
PDF DOI ArXiv Blog Dataset