Plot

Plot

Legend

  • Green: basic gradient descent
  • Red: gradient descent with momentum
  • Blue: gradient descent with momentum using Nesterov & Sutskever method
  • Magenta: RProp+
  • Cyan: iRProp- (tweaked)

Optimization parameters

Basic gradient descent
Gradient descent with momentum

Image parameters