How to Scale Mixture-of-Experts: From muP to the Maximally Scale-Stable Parameterization
Paper • 2605.14200 • Published • 1
Leena Chennuru Vankadara
leenachennuru
AI & ML interests
None yet
Recent Activity
authored a paper 15 days ago
How to Scale Mixture-of-Experts: From muP to the Maximally Scale-Stable Parameterization authored a paper 15 days ago
On the Surprising Effectiveness of Large Learning Rates under Standard Width ScalingOrganizations
None yet
upvoted a paper 11 days ago