Transfer learning
Transfer Learning. Theory, methods & applications
Part of the ENBIS-24 Leuven conference.
This half-day course is a joint initiative from ENBIS and ECAS (http://ecas.fenstats.eu/) which has provided courses since 1987 to achieve training in special areas of statistics for both researchers and teachers for universities and professionals in industry fields.
Instructor
Mathilde Mougeot, ENS Paris Saclay
Overview
It is well known the greater the volume of data the better the performance of a model. However, for given applications, access to large databases to train a model is not necessary easy. The aim of transfer learning is to take advantage of basic model sources or pre-existing data in order to adapt a high-performance model to a given target application for which sufficient data is not directly available.
This course will present various transfer learning methods (instance-based, feature-based, parameter-based) that have been introduced in the literature depending on the information available for the source and the target. Different approaches will be presented depending on the class of models used: models based on neural networks, models based on decision trees, models based on Gaussian processes, etc.
Concrete application cases will also be presented in which transfer learning has been used to obtain very high-performance machine learning models, even when dedicated data are scarce or even non-existent.
All the concepts presented in this lesson will be illustrated from a practical point of view using the ADAPT library, which offers free access to a large majority of transfer learning algorithms presented in the literature. These algorithms will be used both on simulated and real databases.
By taking into account all the former elements (presentation of the methods, illustration on concrete cases and operational practice on simple examples), this course offers all the ingredients necessary to tackle problems of transfer learning on your own using new data.
Outline
Below is given an outline of the course:
- Machine learning and transfer learning
- The different methods: Feature-based, instance-based, parameter-based.
- Benefits for real-life applications
- Presentation of practical experiments using the ADAPT library.
References
- A survey on transfer learning. Pan, S. J., & Yang, Q. (2009). IEEE Transactions on knowledge and data engineering, 22(10), 1345-1359.
- From theoretical to practical transfer learning: The ADAPT library. A de Mathelin, F Deheeger, M Mougeot, N Vayatis - Federated and Transfer Learning, 2022. hal-04529706.
- ADAPT library: https://adapt-python.github.io/adapt/
Short bio
Mathilde Mougeot is Professor of Data Science at Ecole Nationale Supérieure d'Informatique pour l'Industrie et l'Entreprise (ENSIIE) and adjunct Professor at ENS Paris Saclay where she holds the Industrial Research Chair "Industrial Data Analytics & Machine Learning". Since the beginning of her career, Mathilde Mougeot has been interested in machine learning for artificial intelligence applications. Her research activity is motivated by questions related to concrete applications stemming from collaborative projects with the socio-economic world. Her research focuses mainly on scientific issues related to predictive models in various contexts, such as those of high dimensionality, model aggregation, domain adaptation, data frugality by model transfer or by hybrid models.
From 1999 to 2005, she has been contributed to the creation and the development of the start-up Miriad Technologies, specialized in the development of mathematical solutions for the industry based on machine learning, statistics and signal processing techniques.
She offers a strong experience in leading projects at the interface of academics and industry.