Reinforcement Learning of Ordinal User Preferences on Wearable Devices: An Industrial Perspective

Thursday 20 Feb 2025, 12:30 → 13:30 Europe/Amsterdam

Davide Cacciarelli (Imperial College London)

Reinforcement Learning of Ordinal User Preferences on Wearable Devices: An Industrial Perspective

Speaker: Simon Weinberger (ERIC laboratory, Univ. Lyon 2, France & Essilor International)

Chair: Davide Cacciarelli (Imperial College London, UK)

Date: 20th February 2025, at 12:30-13:30 CET

Abstract:

From personalized treatment plans for patients to robotics and automation and even smart grid optimization, Reinforcement Learning (RL) has proven effective in numerous problems thanks to its capacity to address complicated sequential decision-making problems.  

Indeed, unlike supervised learning, RL allows addressing dynamic environments where one must take actions depending on some state value in such a way that rewards are maximized over the long run. In the RL paradigm, taken actions influence future states and future rewards, and this is accounted for, allowing to approach complex real-world problems where data is unlabeled. 

For example, wearable devices allow collecting data at an individual level, which can be used to propose an unseen degree of personalization for a broad domain of applications, such as the adaptive automatic control of a setting. This can be done using RL: depending on sensor data (state) some setting of the wearable device must be automatically piloted (action) in such a way that the user does not interact often with its device (reward).  

 

In this ENBIS webinar, we provide a brief introduction to Reinforcement Learning and some of its applications and illustrate its use for the adaptive control of the tint of electrochromic eyewear. With this objective, we use an ordinal regression model to control the tint level using environmental information from sensors in the eyewear. The time-dependent nature of the predictor is accounted for by considering the state space as a functional data space and adapting the Trust Region Policy Optimization (TRPO) update accordingly. 

Bio:

Simon Weinberger is a PhD student at ERIC laboratory, Univ. Lyon 2, France, currently working at the Connected Devices department of EssilorLuxottica. His current role involves addressing learning strategies for complex wearables devices, which has brought him working on Reinforcement Learning and functional data. Since he completed a master's degree in applied mathematics, Simon has been interested in the use of statistics to solve a wide range of industrial problems. 

 

1 Reinforcement Learning of ordinal user preferences on wearable devices: an industrial perspective

Speaker: Simon Weinberger (EssilorLuxottica)