How deep neural networks learn — a dynamical-systems perspective

B. Mehlig

Document data

How deep neural networks learn — a dynamical-systems perspective

B. Mehlig

Abstract

After giving an introduction to deep learning, I will discuss how deep networks learn. This can be analysed and understood, in part, using concepts from dynamical-systems and random-matrix theory [1]. For deep neural networks, the maximal finite-time Lyapunov exponent forms geometrical structures in input space, akin to coherent structures in dynamical systems such as turbulent flow. Ridges of large positive exponents divide input space into different regions that the network associates with different classes in a classification task. The ridges visualise the geometry that deep networks construct in input space, and help to quantify how the learning depends on the network depth and width [2].

[1] Bernhard Mehlig, Machine Learning with neural networks, Cambridge University Press (2021).
[2] Storm, Linander, Bec, Gustavsson & Mehlig, Finite-time Lyapunov exponents of deep neural networks, Phys. Rev. Lett. 132 (2024) 057301.

How to cite

B. Mehlig, How deep neural networks learn — a dynamical-systems perspective, in: Data-Driven Mechanics and Physics of Materials 2025. URL https://www.scipedia.com/public/Mehlig_2025a

Read document

Discussion

History

Revision as of 12:07, 21 July 2025 (view source)

Scipediacontent (talk | contribs)

(Created page with "==1 Title, abstract and keywords<!-- Your document should start with a concise and informative title. Titles are often used in information-retrieval systems. Avoid abbreviatio...")

Newer edit →

(No difference)