Abstract:
General first order methods (GFOMs), including various gradient descent variants and approximate message passing algorithms, constitute a broad class of iterative algorithms widely used in modern statistical learning problems. Some GFOMs also serve as constructive proof devices, iteratively characterizing the empirical distributions of statistical estimators in the asymptotic regime of large system limits for a fixed number of iterations.
This talk develops a non-asymptotic mean-field characterization of the dynamics of a general class of GFOMs. Our characterizations capture the precise stochastic behavior of each coordinate of the GFOM iterates and, more importantly, hold universally across a broad class of heterogeneous random matrix models.
We demonstrate the utility of these general results through two applications. In the first application, we characterize the mean-field behavior of gradient descent algorithms in a broad class of empirical risk minimization problems. Our theory also facilitates a generic iterative algorithm that consistently estimates key state evolution parameters, which can be used for statistical inference via gradient descent.
In the second application, we develop an algorithmic method for proving the universality of regularized regression estimators. Specifically, we systematically improve universality results for regularized regression estimators in the linear model and resolve the universality conjecture for (regularized) maximum likelihood estimators in the logistic regression model.
About the Speaker:
Qiyang Han is an Associate Professor of Statistics at Rutgers University. He received a Ph.D. in Statistics in 2018 from University of Washington under the supervision of Professor Jon A. Wellner. His research expands broadly in mathematical statistics and high dimensional probability, including empirical process theory, constrained nonparametric inference, mean-field high dimensional statistics, and more recently, large random iterative algorithms. He is a recipient of the NSF CAREER award, the Bernoulli Society New Researcher Award, and the David G. Kendall’s Award in Mathematical Statistics.

Your participation is warmly welcomed!

欢迎扫码关注北大统计科学中心公众号,了解更多讲座信息!