Dhruv Shah: A General-Purpose Robotic Navigation Model

Abstract: The advent of large-scale machine learning models ("foundation models") has been paradigmatic in the fields of computer vision and natural language processing. What would a similar consolidation look like in robot learning, where learned models typically train on data from a single research group and on a specific robot embodiment? In this talk, I will share some recent progress we have made such a consolidation for the task of visual navigation in challenging real-world environments. I will discuss how sharing data across robots and tasks can enable remarkable generalization and adapt to new skills by a mechanism similar to soft prompting. Lastly, I will also discuss how such pre-trained models can enable exciting applications such as kilometer-scale navigation, open-vocabulary instruction following, and autonomous online improvement with reinforcement learning.

Bio: Dhruv Shah is a graduate student in EECS at UC Berkeley, where he is advised by Sergey Levine. His research spans the fields of machine learning and robotics, with the general goal of enabling real-world robotic systems perceiving and acting "in-the-wild". His research is supported by the Berkeley Fellowship for Graduate Study, and was nominated for the Best Systems Paper Award at RSS 2022. Earlier, he graduated with honors from IIT Bombay, where he received the Undergraduate Research Award and the Institute Academic Prize.