| Title | : | Sequential Learning under Model Ambiguity: Theoretical Foundations and Algorithm Design |
| Speaker | : | Dr. Kishan Panaganti Badrinath (Postdoctoral Scholar Research Associate at Caltech) |
| Details | : | Tue, 10 Dec, 2024 11:00 AM @ SSB 334 |
| Abstract: | : | Sequential decision-making algorithms such as Reinforcement learning (RL) has achieved remarkable successes, such as outperforming humans in games like Atari and Go, fast chip design, robotics, and fine-tuning large language models. However, these successes are largely confined to structured or simulated environments. My research enables robustness to the standard RL algorithms, allowing them to perform reliably in unstructured and complex environments. Training RL algorithms directly on real-world systems is expensive and potentially dangerous, so they are typically trained on simulators. This leads to a significant issue: the simulation-to-reality gap. Due to modeling approximation errors, changes in real-world parameters over time, and possible adversarial disturbances, there are inevitable mismatches between the training and real-world environments. This gap degrades the real-world performance of RL algorithms. My work addresses this fundamental challenge by developing novel robust reinforcement learning (RL) theories and algorithms. Unlike standard RL, which considers a single model, robust RL considers a set of models (an ambiguity set covering training and testing models) to find an optimal robust policy that performs best under the worst possible model within this set. Additionally, to address the large-state space issue in real-world applications, I design general function architecture-based robust RL algorithms to utilize deep learning tools in practical implementations. Through my work, I aim to provide the statistical theory and practical foundations necessary to bridge the simulation-to-reality gap and enable reliable deployment of RL in real-world applications.
Research Bio for Kishan Panaganti: Kishan is a PIMCO Postdoctoral Fellow at Caltech. He received his PhD in August 2023 from the Department of Electrical and Computer Engineering at Texas A&M University. His research interests in machine learning are motivated via addressing challenges for autonomous solutions in real-world problems. His work on theoretical foundations of reinforcement learning algorithms tackling the simulation to real-world performance gaps through providing robust solutions has been published in multiple machine learning conferences. His research interests also extends beyond reinforcement learning to address environmental ambiguities in more general frameworks--like multi-agent learning, imitation learning, and learning from human feedback--aiming to address challenges in more safety-critical real-world systems. Prior to Texas A&M University, he obtained a Master's degree in Communication & Networks at the Indian Institute of Science, Bengaluru. |
