D-Flow: Differentiating through Flows for Controlled Generation | Heli Ben-Hamu
Автор: Valence Labs
Загружено: 2024-05-06
Просмотров: 1948
Portal is the home of the AI for drug discovery community. Join for more details on this talk and to connect with the speakers: https://portal.valencelabs.com/logg
Abstract: Taming the generation outcome of state of the art Diffusion and Flow-Matching (FM) models without having to re-train a task-specific model unlocks a powerful tool for solving inverse problems, conditional generation, and controlled generation in general. In this work we introduce D-Flow, a simple framework for controlling the generation process by differentiating through the flow, optimizing for the source (noise) point. We motivate this framework by our key observation stating that for Diffusion/FM models trained with Gaussian probability paths, differentiating through the generation process projects gradient on the data manifold, implicitly injecting the prior into the optimization process. We validate our framework on linear and non-linear controlled generation problems including: image and audio inverse problems and conditional molecule generation reaching state of the art performance across all.
Speakers: Heli Ben-Hamu
Twitter Hannes: / hannesstaerk
Twitter Dominique: / dom_beaini
~
Chapters
00:00 - Intro + Background
8:19 - Controlled Generation
19:38 - D-Flow
39:13 - Theoretical Intuition
45:05 - Experiments
52:23 - Conclusions
54:50 - Q+A
Доступные форматы для скачивания:
Скачать видео mp4
-
Информация по загрузке:
