1X and NVIDIA to Work Together to Develop AI for Robots, Safety, Dexterity, Visual Understanding

Developing AI for humanoid robots involves tackling many open research challenges, including safety, dexterity, visual understanding, and more. It helps to compare notes with other labs tackling similar challenges to accelerate progress toward a future where NEOs autonomously perform all the tasks required to keep your home tidy.

To that end, 1X AI and NVIDIA are pleased to announce our research collaboration effort. As a first step, the teams worked together to produce an autonomy demo for Jensen Huang’s GTC 2025 Keynote, where NEOs autonomously performed a dish-loading task.

Below, you can find where, how, and when we taught NEOs how to wash dishes together with the NVIDIA team.

To enable this collaboration, the 1X AI Team created a dataset API that would allow NVIDIA to access data collected from 1X offices and employee homes, and an inference SDK to deliver model predictions at a continuous 5Hz image-action cycle using either the onboard NVIDIA GPU in NEO’s head or an external GPU.

The most important step when integrating a new learning codebase into NEO is to verify accuracy—that is, overfitting the base model to a small amount of demonstration data and ensuring that the time synchronization between images and actions is consistent from data collection to training to runtime inference.

We demonstrate this by working with the NVIDIA GEAR team to train a single end-to-end neural network based on the NVIDIA GR00T N1 model, showing how NEO can fit compactly into a kitchen space by autonomously grasping a glass, grabbing another, and placing it in the dishwasher, while still having kinematic access to move the glass from the sink to the dishwasher.

This is a good “first task” to learn because it checks basic compatibility with the logging and inference architecture of an external research codebase. After verifying correctness, the obvious next step is to feed thousands of hours of NEO data collected internally into the model.

Over the course of a week, our teams trained this model in a 1X employee’s home, exchanging notes about action fields, control frequencies, and other imitation learning tricks needed to get good performance on NEO Gamma. Moments like these—a NEO washing dishes in the background while your friends hang out at home—will soon become a daily occurrence.

NEO Gamma’s safety is especially evident when working in homes. NEO’s mechanically compatible and secure design allowed engineers to get into extremely close proximity with the robot while testing various experimental architectures.

Source: https://www.1x.tech/discover/1X-NVIDI...