Revisiting neural network approximation theory in the age of generative AI

Date Range
Tue October 24th 2023, 4:30pm
Sloan 380C
Song Mei, UC Berkeley

Textbooks on deep learning theory primarily perceive neural networks as universal function approximators. While this classical viewpoint is fundamental, it inadequately explains the impressive capabilities of modern generative AI models such as language models and diffusion models. This talk puts forth a refined perspective: neural networks often serve as algorithm approximators, going beyond mere function approximation. I will explain how this refined perspective offers a deeper insight into the success of modern generative AI models.