Anticipation and the Anticipatory Music Transformer

Speaker: John William Thickstun (https://johnthickstun.com/) from Stanford
Time: Nov 10, 2023, 12:30 PM – 1:30 PM
Location: Engineering Centers Building (1550 Engineering Drive) M1059 (M floor)
Abstract: Imagine you are given a melody, and asked to compose a harmonizing accompaniment; this melody is an example of a temporal point process: a sequence of (musical) events that arrive stochastically at points in time. An accompaniment is also a temporal point process, one that is tightly correlated with—but asynchronous from—the melody. Motivated by this example, we are interested in constructing generative models of a temporal point process (the event process) that can be conditioned on realizations of a second, correlated point process (the control process). We achieve this using anticipation: a new method of controllable generation that interleaves sequences of events and controls, such that controls appear following stopping times in the event sequence. We apply anticipation to construct anticipatory infilling models of music; these models unlock new control capabilities for music generation, without sacrificing the performance of unconditional generation.
Bio: John Thickstun is a Postdoc at Stanford University, working with Percy Liang and members of the Center for Research on Foundation Models. John’s work has been recognized by an NSF Graduate Fellowship, a Qualcomm Innovation Fellowship, and outstanding paper awards at the Neurips and ACL conferences. His current research focus is on advancing the capabilities, controllability, and governance of generative models.