Modern information processing systems deal with massive amounts of data in the presence of large statistical uncertainty. The performance and behaviors of such systems often exhibit phase transitions (e.g., threshold SNR, cutoff rates, etc.), due to the interactions of a large number of random variables with complex correlation structures. Methods developed in statistical physics have been tremendously successful in understanding the macroscopic properties of many-body interactive systems. They have also been gradually recognized as an indispensable tool in the study of information processing systems and algorithms.

The goal of this course is to introduce students to several fundamental notions and methods in statistical physics that have been successfully applied to the analysis of information processing systems. Discussions will be focused on studying such systems in the infinite-size limit, on analyzing the emergence of phase transitions, and on understanding the behaviors of efficient algorithms. This course seeks to start from basics, assuming just undergraduate probability and analysis, and in particular assuming no knowledge of statistical physics. It is essentially self- contained, developing the necessary concepts, tools, algorithms, and rigorous theory step-by- step, in as much detail as possible. Students will take an active role by exploring and applying what they learn from the course to their preferred applications, including but not limited to signal processing, communications, machine learning, imaging, control, power networks, and biological data analysis.