Probabilistic graphical models play a central role in modern statistics, machine learning, and artificial intelligence. Within these contexts, researchers and practitioners alike are often interested in modeling the (conditional) independencies among variables within some complex system. Graphical models address this problem combinatorially: A probabilistic graphical model associates jointly distributed random variables in the system with the nodes of a graph and then encodes conditional independence relations entailed by the joint distribution in the edge structure of the graph. Consequently, the combinatorial properties of the graph play a role in our understanding of the model and the associated inference problems. We will see in this course how the graph structure provides the scientist with (1) a transparent and tractable representation of relational information encoded by the model, (2) effective approaches to exact and approximate probabilistic inference, and (3) data-driven techniques for learning graphical representatives of relational information, such as conditional independence structure or cause-effect relationships.
Course type:
- AS track: elective
- AI track: mandatory
- Joint curriculum: advanced
Time: Given odd years, Autumn
Teachers: Svante Linusson, Liam Solus (KTH), Fredrik Lindsten, Johan Alenlöv (LiU), Vera Koponen (UU)
Examiner: Liam Solus (KTH)
The participants are assumed to have a background in mathematics corresponding to the contents of the WASP-course “Introduction to Mathematics for Machine Learning, 4hp”.
The course requires basic understanding of probability theory and graph theory. In particular, the participants need to be comfortable with reasoning about (in-)dependence of random variables and factorizing joint probability distributions/density functions using the chain rule of probability, as well as reasoning about basic properties of graphs.
Module 2 requires familiarity with Bayesian statistics (as well as graphical models and basic message-passing algorithms, covered in module 1).
After completing the course, the students will have functional knowledge of the theory of graphical models, graphical model structure learning methods as well as methods for Bayesian inference with graphical models. Specifically, students should be able to
- Distinguish between the use-cases of different classes of graphical models (undirected and directed acyclic graphical models).
- Solve typical graphical model structure learning problems.
- Formulate the Bayesian inference problem for graphical models and distinguish between scenarios in which this problem can be solved exactly and when approximate inference algorithms are needed.
- Solve typical Bayesian inference problems for graphical models.
- Compare and contrast between different graphical model learning algorithms.
In module 1 we will introduce undirected graphical and directed acyclic graphical models and explore how these models encode independencies between variables in complex systems via the graph structure. Methods for learning graphical representations of the independences in a data-generating distribution will be discussed, along with exact algorithms for the inference of posterior probabilities.
In module 2 we consider Bayesian inference in graphical models where the involved distributions and/or the structure of the graph prevents exact solutions. We introduce several approximate inference algorithms to tackle this issue, such as expectation propagation, variational inference, and Markov chain Monte Carlo. We highlight some key differences, but also similarities, between these types of algorithms.
- Diestel, Reinhard. Bakground on graph theory. Volume 173 of Graduate texts in mathematics (2012):7.
- Koller, Daphne, and Nir Friedman. Probabilistic graphical models: principles and techniques. MIT press, 2009.
- Lauritzen, Steffen. Graphical models. Vol. 17. Clarendon Press, 1996.
Participants in the course will be assessed using hand-in assignments. To pass each individual module the student must receive a passing grade for their work on the module’s hand-in assignments and have actively participated in the in-person meetings for both modules. To pass the course the student must pass both modules.
A re-examination is prepared about 6 months after the course covering completion of missing parts.
