NVIDIA AI Senior Research Scientist Jim Fan, Thursday Nights in AI

Outset Capital's Ali Rohde and Kanjun Qiu interview Dr. Jim Fan, a Senior Research Scientist at NVIDIA AI and the co-creator of NeurIPS 2022 best paper winner, MineDojo. Special thanks to Notion for welcoming us at their beautiful HQ!

This fireside chat is part of our 'Thursday Nights in AI" series, co-hosted by Outset Capital and Imbue and sponsored by WndrCo.

Join our upcoming events! Full list here: https://www.outsetcapital.com/thursda...

About Outset Capital: Outset Capital is led by Ali Rohde, Kanjun Qiu, and Josh Albrecht — AI practitioners investing in AI, deeptech, and the future of work. We back companies at the outset and love to be the first check in.

About Imbue: Imbue is an AI research company aiming to rekindle the dream of the personal computer—by creating practical AI agents that can accomplish larger goals and safely work for us in the real world. They are hiring! https://imbue.com/careers

About WndrCo: Founded in 2016 by Sujay Jaswa, Jeffrey Katzenberg, and ChenLi Wang, WndrCo builds & invests in companies whose products are revolutionizing how people live and work. WndrCo's investments include 1Password, Aura, Databricks, Figma, Material Security, and Pilot.

About MineDojo: MineDojo is a new framework built on the popular Minecraft game for embodied agent research. MineDojo features a simulation suite with 1000s of open-ended and language-prompted tasks, where the AI agents can freely explore a procedurally generated 3D world with diverse terrains to roam, materials to mine, tools to craft, structures to build, and wonders to discover.

Learn more about MineDojo: https://minedojo.org/
Learn more about Dr. Jim Fan: https://jimfan.me/

00:00 Intro
00:30 Jim's journey
03:06 2015: Jim at Baidu
05:11 2016: Jim at OpenAI
07:59 Jim at Stanford PhD
11:10 2020: NVIDIA AI
12:20 Jim & embodied agents
15:33 What is embodiment?
16:03 LLM's and exploratory behaviors
20:17 Use cases of embodied agents
20:55 Uses of AI Agents: Software
21:41 Existing problems that needs to be solved
24:40 Skeptical applications of AI agents
27:02 Jim's Twitter/X following
28:44 Next compelling thing in Game AI
33:38 Q&A
33:38 Future tools for AI Engineers
34:50 What do you think about Reinforcement Learning?
37:09 Using pretraining to improve Reinforcement learning
41:06 Things that haven't gotten enough attention
44:30 Scaling embodied AI agents
46:38 How do you decide what is important in ML research