Published in ICLR'2018 Links : openreview, Reddit

TDLR; We investigate the convergence of popular optimization algorithms like Adam , RMSProp and propose new variants of these methods which provably converge to optimal solution in convex settings.