- Author
- Yacine Mahdid
- Published
- 19 August 2025
- Kind
- Short Substack post in terse note form
- Original
- www.yacinemahdid.com/p/muon-optimizer-explained-to-a-toddler
- Subject
- Language models and training
- Themes
- Getting good
What it is
Yacine Mahdid writes a newsletter called learning with yacine. This post explains the Muon optimizer for training neural networks. It uses a boat-on-a-cost-surface image and plain one-line steps.
What it says
- The post says AdamW is the most used optimizer for deep networks as of 2025.
- It says momentum optimizers favor a few update directions, so updates look low-rank.
- Muon aims to even out the update by pushing all singular values to roughly one.
- A Newton-Schulz iteration approximates this cheaply, and the post says 5 steps are enough.
- The post gives the quintic polynomial's coefficients as 3.4445, -4.7750 and 2.0315.
- It says Muon suits only 2D hidden layers. AdamW should handle the first and last layers.
What evidence it gives
The post offers argument and analogy only. It shows no benchmark, code or loss curve. It links a 32 min breakdown and four reading links for detail.
Interests and limits
The post is short and says it simplifies. Many claims are marked empirical, with no source in the post. The credit for the idea goes to an unnamed contributor. The linked breakdown and reading links were not opened.