Seminar: Mathematics of Learning - From Hilbert's 13th Problem to Fluctuating Hydrodynamics
Description
Neural networks (NN) constitute a class of functions that nonlinearly depend on a finite number of parameters which in turn are typically used for (optimal) approximation of an unknown function from a given set of function values.
In this seminar, we plan first to investigate the richness and approximation power of neural networks by considering topics like Hilbert's 13th problem, approximation error estimates, and discretization errror estimates for NN-discretizations of partial differential equations.
Furthermore we will adress the algebraic minimization problems for (optimal) NN-parameters. Popular numerical solution methods like stochastic gradient-type descent will finally lead us to novel reinterpretations of NN in terms of fluctuating hydrodynamics that are currently investigated in a MATH+ project of the organizers.
Participants are expected to have basic knowledge in numerical analysis as communicated, e.g. in the lectures Numerik I - III at FU Berlin.
Schedule
The introduction of the topics takes place on 11.04.2019.
Date | Title | Speaker | Literature |
23.05.2019 | Hilbert's 13th problem | Issagali | [1][2] |
06.06.2019 | The Kolmogorov-Arnold representation theorem and extensions | Merten | [3][4][5] |
13.06.2019 | Towards neural networks: Constructive proofs of Kolmogorov-Arnold | Boisserée | [6][7][8][9] |
20.06.2019 | Towards neural networks: Approximation instead of representation | Srinivasan | [10] |
27.06.2019 | Iterative parameter computation by gradient-type methods | Kostré | [11][12][13] |
04.07.2019 | Fluctuating hydrodynamics: The Dean-Kawasaki model | Kornhuber | |
11.07.2019 | Neural networks as interacting particle systems | Gräser |
Organizational Matters
- The seminar takes place in Arnimallee 6, SR 009, on Thursday, 14-16.
- Each participant prepares a talk.
- In addition, every participant prepares a handout, preferably using LaTeX.
Literature
- A.G. Vitushkin: On Hilbert's thirteenth problem and related questions. Russian Mathematical Surveys, 59(1), 11-24 (2004)
- V. Brattka: From Hilbert’s 13th Problem to the theory of neural networks: constructive aspects of Kolmogorov’s Superposition Theorem. In Kolmogorov’s heritage in mathematics. Springer, Berlin, Heidelberg, 253-280 (2007)
- A. N. Kolmogorov: On the representation of continuous functions of many variables by superposition of continuous functions of one variable and addition. Dokl. Akad. Nauk SSSR, 114:5,953–956 (1957)
- V. I. Arnol'd: On the representability of a function of two variables in the form \(\chi \phi(x) + \psi(y)\). Uspekhi Matematicheskikh Nauk 12.2, 119-121 (1957)
- D.A. Sprecher: On the structure of continuous functions of several variables. Transactions of the American Mathematical Society, 115, 340-355 (1965)
- J. Braun, M. Griebel: On a constructive proof of Kolmogorov’s superposition theorem. Constructive approximation, Springer (2009)
- D.A. Sprecher: A Numerical Implementation of Kolmogorov’s Superpositions. Neural Networks, 9(5), 765-772 (1996)
- D.A. Sprecher: A Numerical Implementation of Kolmogorov’s Superpositions II. Neural Networks, 10(3), 447–457 (1997)
- V. Kurkova: Kolmogorov's theorem is relevant. Neural Computation 3(4), 617-622, (1991)
- A. Pinkus: Approximation theory of the MLP model in neural networks. Acta Numerica, 143-195 (1999)
- A. Botev, H. Ritter, D. Barber: Practical Gauss-Newton optimisation for deep learning. arxiv:1706.03662, 2017
- Y. A. LeCun, L. and Bottou, G. B. Orr, K.-R. Müller: Efficient {BackProp}. In Neural networks: Tricks of the trade. Springer, 6-50 (1998) doi
- D. P. Kingma, J. Ba: Adam: A method for stochastic optimization. arxiv:1412.6980
Kontakt
Prof. Dr. Carsten Gräser | graeser@mi.fu-berlin.de | Arnimallee 6, Raum 121 Sekretariat Frau Engel: Arnimallee 6, Room 131 |
Prof. Dr. Ralf Kornhuber | ralf.kornhuber@fu-berlin.de | Arnimallee 6, Raum 130 Sekretariat Frau Engel: Arnimallee 6, Room 131 |