Dipartimento di Informatica

Date: Monday, 6 March 2023
Time: 11:00-13:00
Venue: Aula Seminari - Via Salaria 113, 3rd floor
Speaker: Luigi Gresele, Max Planck Institute for Intelligent Systems, Tübingen
Title: Learning Identifiable Representations: A Gentle Introduction
 

Abstract

Representation learning is motivated by the fact that learning systems, both biological and artificial, receive unstructured information from the external world: artificial networks trained for object recognition take collections of pixels as inputs; visual information processing in biological systems starts in photoreceptors, where incoming light is converted into biological signals. To make certain aspects of this information explicit and easily accessible (e.g., the position, dimension and colour of objects in an image), complex processing is required. Central questions are what information should be made explicit in a representation, and how to do so. In this talk, we will approach these problems based on two metaphors. The first one is the cocktail-party problem, where a number of conversations happen in parallel in a room, and the task is to recover (or separate) the voices of the individual speakers from recorded mixtures—also termed blind source separation. The second one is what we call the independent-listeners problem: given two listeners in front of some loudspeakers, the question is whether, when processing what they hear, they will make the same information explicit, identifying similar constitutive elements. Rather than the reconstruction of a ground truth, in the second problem we are interested in comparing the representations extracted by the two listeners. These questions can be studied with the approach of independent component analysis (ICA). This entails establishing whether, under some technical assumptions, representations can be uniquely specified—up to some ambiguities deemed tolerable, and except for a small number of corner cases. In technical terms, this corresponds to characterizing identifiability of the model. We will explore the motivations and objectives of research on identifiability in representation learning, highlighting the challenges and the proposed solutions. Although the talk will be mostly based on literature on (nonlinear) ICA, we will also discuss the implications for unsupervised learning and probabilistic modelling in general.

Short Bio: Luigi Gresele is a last-year PhD student in the Empirical Inference department, supervised by Prof. Bernhard Schölkopf. His work focuses on the development of unsupervised learning and causal inference methods. In particular, he works on the theory of novel linear and nonlinear models for independent component analysis, as well as on novel estimation techniques. He is broadly interested in causality, and how causal modeling and reasoning help interpreting and understanding real world phenomena. Additionally, he is interested in exploring connections between machine learning and causality. He is also interested in statistical physics. During his master, Luigi took part in an international program on the Physics of Complex Systems involving institutes located in Trieste (SISSA and ICTP), Turin (Politecnico di Torino) and Paris (Universities Pierre & Marie Curie, Paris Diderot, Paris-Sud and the École Normale Supérieure at Cachan).