讲座题目：An algorithmic view of L_2 regularization and some path-following Algorithms

　　报告人： 刘仁雄 博士候选人

　　报告时间：2019年5月15日(周三)下午：15点30分

　　报告地点：经管院B127

　　主办单位：数理经济与数理金融系

　　主持人:李汛

　　摘要: We establish an equivalence between L_2-regularized solution path and the solution of an ordinary differentiable equation (ODE).Importantly, this equivalence reveals that the solution path can be viewed as the flow of a hybrid of gradient descent and Newton method applying to the empirical loss, which is similar to a widely used optimization technique called trust region method. This provides an interesting algorithmic view of L_2 regularization, and is in contrast to the conventional view that the L_2 regularization solution path is similar to the gradient flow of the empirical loss. Interestingly, the limit of the solution path is shown to coincide with the minimum L_2 norm minimizer of the empirical loss provided that it is finite.Path-following algorithms based on homotopy method and numerical ODE solvers are considered to numerically approximate the solution path. In particular, we consider Newton update and gradient descent update as the basis algorithm for the homotopy method, and establish their the global approximation error rates. In terms of computational cost, these path-following algorithms are shown to require roughly thesame computational cost needed for computing a single solution on the path. Finally, we use L_2-regularized logistic regression as an illustrating example to demonstrate the effectiveness of the proposed path-following algorithms.

　　报告人简介：刘仁雄，999策略白菜手机论坛数理金融实验班本科毕业，现就读于美国俄亥俄州立大学统计系，研究方向为机器学习，计算统计和最优化。