MotionFix: Text-Driven 3DHuman Motion Editing (SIGGRAPH Asia 2024)

The focus of this paper is on 3D motion editing. Given a 3D human motion and a textual description of the desired modification, our goal is to generate an edited motion as described by the text. The key challenges include the scarcity of training data and the need to design a model that accurately edits the source motion. In this paper, we address both challenges. We propose a methodology to semi-automatically collect a dataset of triplets comprising (i) a source motion, (ii) a target motion, and (iii) an edit text, introducing the new MotionFix dataset. Access to this data allows us to train a conditional diffusion model, TMED, that takes both the source motion and the edit text as input. We develop several baselines to evaluate our model, comparing it against models trained solely on text-motion pair datasets, and demonstrate the superior performance of our model trained on triplets. We also introduce new retrieval-based metrics for motion editing, establishing a benchmark on the evaluation set of MotionFix. Our results are promising, paving the way for further research in fine-grained motion generation.

Authors: Athanasiou, Nikos, Ceske, Alpar and Diomataris, Markos and Black, Michael J. and Varol, Gül

project: https://motionfix.is.tue.mpg.de/index...
arXiv: https://arxiv.org/abs/2408.00712
code & data: https://github.com/athn-nik/motionfix
live demo: https://motionfix.is.tue.mpg.de/explo...
bibtex:
@inproceedings{athanasiou2024motionfix,
title = {{MotionFix}: Text-Driven 3D Human Motion Editing},
author = {Athanasiou, Nikos and Ceske, Alp{\'a}r and Diomataris, Markos and Black, Michael J. and Varol, G{\"u}l},
booktitle = {SIGGRAPH Asia 2024 Conference Papers},
year = {2024}
}