This work introduces MotionLCM, extending controllable motion generation to a real-time level. Existing methods for spatial control in text-conditioned motion generation suffer from significant runtime inefficiency. To address this issue, we first propose the motion latent consistency model (MotionLCM) for motion generation, building upon the latent diffusion model (MLD). By employing one-step (or few-step) inference, we further improve the runtime efficiency of the motion latent diffusion model for motion generation. To ensure effective controllability, we incorporate a motion ControlNet within the latent space of MotionLCM and enable explicit control signals (e.g., pelvis trajectory) in the vanilla motion space to control the generation process directly, similar to controlling other latent-free diffusion models for motion generation. By employing these techniques, our approach can generate human motions with text and control signals in real-time. Experimental results demonstrate the remarkable generation and controlling capabilities of MotionLCM while maintaining real-time runtime efficiency.
“a person slightly bent over with right hand pressing against the air walks forward slowly”
“a person runs forward and stops short.”
“with arms out to the sides a person walks forward”
“a person walks in a counter counterclockwise circle.”
“a person does a jump”
“a person waves both arms in the air.”
“a person is doing jumping jacks”
“the man is throwing his right hand”
“this person bends forward as if to bow.”
“a person holds their arms near their face and searches left and right.”
“a man paces back and forth along the same line.”
“a person walks clockwise in a large curve while swinging their arms.”
“the person is jogging around.”
“a man walks forward in a snake like pattern.”
“the person is doing a dance move.”
“a hunched individual slowly wobbles forward in a drunken manner.”
“the person was pushed but did not fall”
“a person jumps to his left.”
“a person walks using a handrail with his right hand.”
“a man walks around in a clockwise circle.”
“a person walks forward, turns around and sits on a chair.”
“a person taking a huge diagonal step.”
“a person jauntily skips forward”
“a person walks quickly and intentionally in zig-zag pattern forward.”
“a man crawls forward on his stomach”
“person is walking with his arms out like he is balancing.”
@article{motionlcm,
title={MotionLCM: Real-time Controllable Motion Generation via Latent Consistency Model},
author={Dai, Wenxun and Chen, Ling-Hao and Wang, Jingbo and Liu, Jinpeng and Dai, Bo and Tang, Yansong},
journal={arXiv preprint arXiv:2404.19759},
year={2024}
}