First-order methods such as gradient descent (GD) are foundational in optimization. In unconstrained problems with exact gradients, momentum-based methods—most notably Nesterov’s accelerated gradient descent (AGD) and Polyak’s heavy-ball (HB) method—achieve faster convergence by improving dependence on the condition number. However, this acceleration comes at a cost: momentum amplifies gradient noise, making these methods less robust than GD under standard parameter choices and requiring more accurate gradient estimates to attain comparable accuracy. Similar challenges arise in convex and nonconvex min–max optimization.
Motivated by applications in machine learning, this talk studies unconstrained and min–max optimization under deterministic, unbiased stochastic, and biased stochastic gradient noise. I will present new algorithms that achieve optimal robustness against different noise types, using control-theoretic tools such as the H_2​ norm, the H_∞​ norm, and the risk-sensitivity index, together with coherent risk measures. I will also discuss worst-case noise constructions and high-probability convergence guarantees. This perspective builds a bridge between optimization and robust control theory and enables the design of noise-robust and risk-sensitive accelerated methods.
Representative Publications:
M. Gürbüzbalaban, Y. Syed, N. S. Aybat, Accelerated gradient methods with biased gradient estimates: Risk sensitivity, high-probability guarantees, and large deviation bounds, Journal of Nonlinear and Variational Analysis, 2026 (Special Issue). https://jnva.biemdas.com/archives/2927
M. Gürbüzbalaban, Robustly Stable Accelerated Momentum Methods with a Near-Optimal L_2​ Gain and H_∞​ Performance, Mathematics of Operations Research, 2025.
https://pubsonline.informs.org/doi/abs/10.1287/moor.2023.0321
B. Can and M. Gürbüzbalaban, Entropic risk-averse generalized momentum methods, Optimization Methods and Software, 2025. https://www.tandfonline.com/doi/abs/10.1080/10556788.2025.2549356