AGILE: A Generic Isaac-Lab Based Engine for Humanoid Loco-Manipulation | Robotics Office Hour

In this Robotics Office Hours livestream, NVIDIA engineers introduce AGILE, a generic Isaac Lab–based engine for humanoid locomotion and manipulation learning.

Learn how AGILE tackles sim-to-real transfer, reinforcement learning workflows, MDP design, training at scale, and deployment to MuJoCo and real robots.

Key Takeaways
AGILE provides a practical RL workflow for humanoid locomotion and manipulation using Isaac Lab.
Model verification and deterministic evaluation are critical for sim-to-real success.
Reward curves alone are insufficient; joint limits and tracking metrics matter.
Deployment is streamlined via TorchScript and IO descriptors for sim-to-sim and real robots.

Chapters
0:20 – Welcome to Robotics Office Hours 2026
1:04 – Meet the Hosts and AGILE Team (Isaac Sim & Isaac Lab)
2:22 – What Is AGILE? A Generic Isaac Lab Engine for Humanoid Loco-Manipulation
3:06 – From Flashy Humanoid Demos to Real-World Deployment
4:21 – AGILE Mission and System Overview (RL, MDPs, Tasks, Robots)
6:08 – End-to-End Training Workflow: 5 Core Components
7:16 – Pre-Training Model Verification GUI (Joints, PD Gains, Contacts)
10:24 – Scaling Training with OSMO + Weights & Biases (Sweeps & Reproducibility)
14:12 – Deterministic Evaluation and What Predicts Sim-to-Real Transfer
24:32 – Deployment Pipeline: TorchScript, IO Exporter, and MuJoCo Transfer
26:50 – Live Q&A: Robustness, Failure Recovery, and “Child-like” Behavior
28:20 – VLA Pipeline Question + Task-Agnostic Policies (Sonic)
29:16 – Modeling Contact + Friction for Sim-to-Real (USD + Randomization)
31:05 – Adding a New Embodiment (H1) + Cross-Embodiment Training Strategy
34:46 – Is the Verification UI Part of Isaac Sim? (AGILE Tooling Clarification)
35:33 – Teleoperation Options Beyond Keyboard (Isaac Lab Teleop + VR First-Person View)
36:21 – Joint Control Models: Delayed Motor vs Implicit Actuators
38:35 – Modular Architecture: Lower-Body RL + Upper-Body Control + Distillation
41:09 – IK + Locomotion: Improving Manipulation Accuracy
43:31 – Learning From Human Video: Retargeting and Physics Grounding
46:40 – AGILE Wrap-Up: What the Framework Delivers
47:55 – Q&A: Preventing Policy Exploitation Between Upper/Lower Controllers
48:41 – Q&A: Heatmaps and “Pilot View” Visualization (What’s Missing + VR Option)
49:35 – Q&A: 5-Finger vs 3-Finger Retargeting and Grounding Requirements
50:51 – Q&A: Physics Solver Transfer (PhysX vs Newton)
52:07 – Call to Action: Download AGILE from GitHub + Community Support Channels
52:53 – Live Demo: AGILE GitHub Repo Walkthrough (Robot Configs + Tasks)
57:31 – Q&A: Training Multi-Arm Coordination (Two Franka Arms, etc.)
58:16 – Q&A: Best Domain Randomization Strategy

---------------------------------------

AGILE Q&A FAQ:   / discord  
AGILE GitHub: https://github.com/nvidia-isaac/WBC-A...
Omniverse Discord:   / discord  
Isaac Sim & Isaac Lab Study Group: Wednesday at 12:30 PM in the NVIDIA Omniverse Discord Community Room
AddEvent Livestream Calendar: https://www.addevent.com/calendar/ae4...

#NVIDIA #Robotics #IsaacSim #IsaacLab #HumanoidRobotics
#ReinforcementLearning #Sim2Real #PhysicalAI #MuJoCo