Official description

Neural networks are the backbone of artificial intelligence and machine learning systems. Despite the immense success of neural networks at a variety of real-world problems, the theory of deep (multi-layer) neural networks is still in its infancy. There are many tantalizing analogies between neural networks and situations we encounter in all branches of physics: the interactions of a large number of entities which give rise to simple collective behaviour is strongly reminiscent of statistical mechanics and condensed matter physics, and the data structures encountered in physics may provide tractable models for how neural networks learn from complex real-world data.

These courses will explore the perspective that physics may bring towards understanding neural network architectures and algorithms. PHY2108S will review some basic statistical concepts and introduce neural networks and deep learning. The bulk of the course will focus on the infinite-width limit of fully-connected neural networks and the connection to Gaussian processes and criticality, as well as empirically-observed scaling laws and connections to statistical learning theory and kernel regression. PHY2109S will explore the phenomenon of feature learning through Feynman-diagram perturbation theory and mean-field theory, and will investigate modern architectures from the perspective of physics, including diffusion models and large language models. Note that these courses will not focus on applications of neural networks to physics problems, but rather will try to build a theoretical foundation for studying neural networks using the toolbox of theoretical physics.

The perspective of physics as an experimental and phenomenological science will be a key component of these courses. Rigour will be ruthlessly sacrificed in favour of flexibility and empirical investigations. Coding assignments will be crucial components of the learning process; use of AI coding assistants is permitted (and encouraged!), but students will be expected to understand each component of the code and run extensive numerical experiments themselves.

Additional information

PHY2108S lectures will run for the first 6 weeks of the S term, and PHY2109S lectures will run for the last 6 weeks