In the dynamic landscape of online businesses, recommender systems are pivotal in enhancing user experiences. While traditional approaches have relied on static supervised learning, the quest for adaptive, user-centric recommendations has led to the emergence of the formulation of contextual bandits. This tutorial investigates the contextual bandits as a powerful framework for personalized recommendations. We delve into the challenges, advanced algorithms and theories, collaborative strategies, and open challenges and future prospects within this field. Different from existing related tutorials, (1) we focus on the exploration perspective of contextual bandits to alleviate the ``Matthew Effect'' in the recommender systems, i.e., the rich get richer and the poor get poorer, concerning the popularity of items; (2) in addition to the conventional linear contextual bandits, we will also dedicated to neural contextual bandits which have emerged as an important branch in recent years, to investigate how neural networks benefit contextual bandits for personalized recommendation both empirically and theoretically; (3) we will cover the latest topic, collaborative neural contextual bandits, to incorporate both user heterogeneity and user correlations customized for recommender system; (4) we will provide and discuss the new emerging challenges and open questions for neural contextual bandits with applications in the personalized recommendation, especially for large neural models.

Compared with other greedy personalized recommendation approaches, Contextual Bandits techniques provide distinct ways of modeling user preferences. We believe this tutorial can benefit researchers and practitioners by appreciating the power of exploration and the performance guarantee brought by neural contextual bandits, as well as rethinking the challenges caused by the increasing complexity of neural models and the magnitude of data.

The topics of this tutorial include (but are not limited to) the following:

Dr. Yikun Ban obtained his Ph.D. degree in the Department of Computer Science at the University of Illinois at Urbana–Champaign. He is a member of DAIS (Data and Information Systems) Research Lab. He received his M.CS. degree from Peking University in 2019 and B.Eng. degree from Wuhan University in 2016. His research interests lie in multi-armed bandits/Reinforcement Learning to design and develop principled exploration strategies in sequential decision-making. He has published more than plenty of papers, including 10 first-author papers at top conferences in Machine Learning and Data Mining (WWW, KDD, NeurIPS, ICLR). He has been a reviewer or program committee member of mainstream machine learning journals and conferences. He was an applied scientist intern at Amazon Web Service, and his research works have been powering primary applications in Amazon and Instacart.

Mr. Yunzhe Qi is a Ph.D. candidate in the School of Information Sciences at the University of Illinois at Urbana–Champaign (UIUC). He received his Master's degree from UIUC in 2021, and his Bachelor's degree in Beijing University of Posts and Telecommunications in 2019 respectively. His research interests mainly focus on utilizing Contextual Bandit methods to solve the exploitation-exploration dilemma for machine learning tasks, such as online recommendation. He has published several papers at top machine learning / data mining conferences (e.g., KDD, NeurIPS), and has been serving as the reviewer as well as PC member for multiple machine learning / data mining conferences and journals. He was a Machine Learning Engineer Intern at Instacart, who designed and implemented Contextual Bandit frameworks for personalized recommendation that have been generating actual business growth.

Dr. Jingrui He (Corresponding Tutor) is a professor in the School of Information Sciences at the University of Illinois Urbana-Champaign. She received her Ph.D. from Carnegie Mellon University in 2010. Her research focuses on heterogeneous machine learning, rare category analysis, active learning and semi-supervised learning, with applications in security, social network analysis, healthcare, and manufacturing processes. Dr. He is the recipient of the 2016 NSF CAREER Award, the 2020 OAT Award, three-time recipient of the IBM Faculty Award in 2018, 2015, and 2014, and was selected as IJCAI 2017 Early Career Spotlight. Dr. He has more than 100 publications at major conferences (e.g., WWW, IJCAI, AAAI, KDD, ICML, NeurIPS) and journals (e.g., TKDE, TKDD, DMKD), and is the author of two books. Her papers have received the Distinguished Paper Award at FAccT 2022, as well as Bests of the Conference at ICDM 2016, ICDM 2010, and SDM 2010. She has several years of course teaching experience as an instructor and has offered several tutorials at major conferences, e.g., KDD, AAAI, IJCAI, SDM, IEEE BigData, etc. in the past few years. For more information, please refer to her homepage at https://www.hejingrui.org/.