【学习】分析momentum在梯度下降中的作用的好文章

Here’s a popular s tory about momentum [1, 2, 3] : gradient descent is a man walking down a hill. He follows the steepest path downwards; his progress is slow, but steady. Momentum is a heavy ball rolling down the same hill. The added inertia acts both as a smoother and an accelerator, dampening oscillations and causing us to barrel through narrow valleys, small humps and local minima.