TCOptRob Seminar: Video World Models That Know When They Don’t Know by Ani Majumdar of Princeton

TCOptRob Seminar: Video World Models That Know When They Don’t Know by Ani Majumdar of Princeton University

Abstract:
World models in the form of action-conditioned video generation models have the potential to serve as general-purpose simulators for robotics. Their ability to generate photorealistic observations, simulate complex physical interactions, and be improved with data make them an attractive alternative to traditional physics-based simulation for policy evaluation, data generation, and inference-time planning. In this talk, I will highlight our recent work from Google DeepMind on using video models as "simulators" for evaluating generalist robot policies for performance, generalization, and safety. I will then describe how the data-driven nature of video world models also leads to outputs that are not always trustworthy. Indeed, current models exhibit a variety of hallucinations: objects can appear or disappear, deform in unrealistic ways, or move in a manner that defies physics. I will argue that knowing when and where to trust the generations of world models is critical in robotics applications. I will describe recent work from my group at Princeton on training world models that output dense confidence estimates at the subpatch (channel) level, precisely localizing the uncertainty in each generated video frame. Through extensive experiments on large-scale robotics datasets (DROID and Bridge), we demonstrate how our method allows us to identify uncertainty in a calibrated manner, while also enabling out-of-distribution detection in unseen domains. To our knowledge, this is the first work on calibrated uncertainty quantification for action-conditioned video models.

Speaker Bio:
Anirudha Majumdar is an Associate Professor at Princeton University in the Mechanical and Aerospace Engineering (MAE) department, and founding co-Director of the Princeton Robotics Initiative. He also holds a 20% research scientist position at Google DeepMind in the Robotics Safety & Alignment team. Majumdar received a Ph.D. in Electrical Engineering and Computer Science from the Massachusetts Institute of Technology in 2016, and a B.S.E. in Mechanical Engineering and Mathematics from the University of Pennsylvania in 2011. Subsequently, he was a postdoctoral scholar at Stanford University from 2016 to 2017 at the Autonomous Systems Lab in the Aeronautics and Astronautics department. Majumdar is a recipient of the Sloan Fellowship, ONR Young Investigator Program (YIP) award, NSF CAREER award, Google Faculty Research Award (twice), Amazon Research Award (twice), Young Faculty Researcher Award from the Toyota Research Institute, Best Student Paper Award (as advisor) at the Conference on Robot Learning (CoRL), Paper of the Year Award from the International Journal of Robotics Research (IJRR), Best Conference Paper Award at the International Conference on Robotics and Automation (ICRA), Alfred Rheinstein Faculty Award (Princeton), and the Excellence in Teaching Award (Princeton SEAS).

The IEEE RAS Technical Committee on Model Based Optimization for Robotics is focused on building and supporting a community of researchers and practioners focused on the development and application of model-based optimization techniques for the generation and control of dynamic behaviors in robotics and their practical implementation. You can find our more about the TC at: https://www.ieee-ras.org/model-based-....

Copyright (c) 2026 TCOptRob. All rights reserved.
License: CC BY-NC-SA 4.0