News & Events

Subscribe to email list

Please select the email list(s) to which you wish to subscribe.
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.

Enter the characters shown in the image.

User menu

You are here

Statistical and computational phenomena in deep learning

Thursday, February 9, 2023 - 10:30 to 11:30
Spencer Frei, Postdoctoral Fellow, Simons Institute for the Theory of Computing, University of California, Berkeley
Statistics Seminar
ESB 4192 / Zoom

To join via Zoom: To join this seminar virtually, please request Zoom connection details from headsec [at]

Title: Statistical and computational phenomena in deep learning

Abstract: Deep learning's success has revealed a number of phenomena that appear to conflict with classical intuitions in the fields of optimization and statistics.  First, the objective functions formulated in deep learning are highly nonconvex but are typically amenable to minimization with first-order optimization methods like gradient descent.  And second, neural networks trained by gradient descent are capable of 'benign overfitting': they can achieve zero training error on noisy training data and simultaneously generalize well to unseen data.  In this talk we go over our recent work towards understanding these phenomena.  We show how the framework of proxy convexity allows for tractable optimization analysis despite nonconvexity, while the implicit regularization of gradient descent plays a key role in benign overfitting.   In closing, we discuss some of the questions that motivate our current work on understanding deep learning, and how we may use our insights to make deep learning more trustworthy, efficient, and powerful.