Tutorials

Tutorial 1

Why Deep Neural Networks Extract Informative Feature:
An Information Geometric View

Tutorial Speakers:
Shao-Lun Huang (Tsinghua University Shenzhen Institute) and
Lizhong Zheng (MIT)

Monday September 29, morning

Abstract

When applying information theoretic analysis to machine learning problems, we often face the difficulty of describing the relation between a number of different distributions: the ground truth distribution, the empirical distribution of the training and the testing sets, the parameterized family of distributions we can use for approximations, the learned models and the updates in each iteration, etc. We argue in this tutorial that a good way to describe this complex situation is often a geometric approach. We introduce the machinery of a simplified information geometry analysis, with the basic techniques of local approximations and the key concepts including the Fisher information metrics, i-projection, mismatched statistics. We show some learning theory applications of these tools in the analysis of the strong data processing inequality, generalization error, model selection, as well as more applied problems like understanding deep neural networks, transfer learning and multi-modal learning problems.

Tutorial 2

Information Theory in Molecular Communication:
Channel Modeling and Analysis

Tutorial Speakers:
Christian Deppe (TU Braunschweig)
Johannes Rosenberger (TU München)
Marcel Mross (TU Braunschweig)​

Monday September 29, morning

Abstract

Molecular Communication (MC) is a fascinating communication paradigm where information is carried by molecules — a method naturally employed by biological cells. As a truly interdisciplinary field, MC has attracted attention from biologists, communication engineers, and increasingly, information theorists. This tutorial is designed to introduce MC specifically to the information theory community. We start by highlighting the unique mathematical challenges that arise when analyzing the capacity of molecular channels. Tackling these challenges often demands adapting existing mathematical tools — or even creating entirely new ones. Interestingly, it turns out that classical Shannon-based information theory doesn’t always provide the right metrics for molecular communication. Therefore, we also explore MC through the lens of Post-Shannon theory. In particular, identification theory, as developed by Ahlswede and Dueck, emerges as a promising framework for MC. Finally, to bridge theory and practice, we present an overview of potential experimental testbeds that could help bring molecular communication concepts into reality.

Tutorial 3

Advances of Lattice Coding for Post-Quantum Cryptography

Tutorial Speakers:
Shanxiang Lyu (Jinan University) and
Ling Liu (Guangzhou Institute, Xidian University)

Monday September 29, afternoon

Abstract
Lattice-based cryptography has emerged as a cornerstone of post-quantum
cryptography (PQC), offering robust security guarantees against quantum attacks. This
tutorial explores the synergy between lattice coding and its cryptographic applications,
focusing on cutting-edge techniques such as polar lattices, and error correction in
lattice-based cryptosystems. Attendees will gain insights into the mathematical
underpinnings of lattice coding, practical challenges in ciphertext compression and error resilience, and novel applications in secure communication. The tutorial aims to equip researchers with the tools to transit from coding theory to post-quantum cryptography.

Tutorial 4

Nanopore DNA sequencing: Coding and Information Theory

Tutorial Speakers:
Adrian Vidal (Monash University, University of the Philippines Diliman),
Brendon McBain (Monash University), and
Emanuele Viterbo (Monash University)

Monday September 29, afternoon

Abstract

DNA storage is a cutting-edge technology that has the potential to store all of the promised information of the future, which approximately doubles each year, without erasing the information of today. An important part of DNA storage systems is the reading process, i.e., the sequencing of DNA. This tutorial is an overview of current day research on the nanopore DNA sequencer, which is rapidly growing in interest within the coding and information theory community due to the rich problems that it poses. The tutorial begins with a soft introduction to DNA storage, then it narrows down to the nanopore sequencer and covers important topics including channel modelling, achievable rates, model evaluation, decoding algorithms, and coding schemes.