MotionLCM: Real-time Controllable Motion Generation via Latent Consistency Model

1Tsinghua University, 2Shanghai AI Laboratory

Online Demo

MotionLCM Teaser


This work introduces MotionLCM, extending controllable motion generation to a real-time level. Existing methods for spatial control in text-conditioned motion generation suffer from significant runtime inefficiency. To address this issue, we first propose the motion latent consistency model (MotionLCM) for motion generation, building upon the latent diffusion model (MLD). By employing one-step (or few-step) inference, we further improve the runtime efficiency of the motion latent diffusion model for motion generation. To ensure effective controllability, we incorporate a motion ControlNet within the latent space of MotionLCM and enable explicit control signals (e.g., pelvis trajectory) in the vanilla motion space to control the generation process directly, similar to controlling other latent-free diffusion models for motion generation. By employing these techniques, our approach can generate human motions with text and control signals in real-time. Experimental results demonstrate the remarkable generation and controlling capabilities of MotionLCM while maintaining real-time runtime efficiency.


Text-to-Motion (1-step, ~30ms/sample)

Motion Control (1-step, ~34ms/sample, Dense signals on pelvis)

Motion Control (1-step, ~34ms/sample, Sparse signals on pelvis)






      title={MotionLCM: Real-time Controllable Motion Generation via Latent Consistency Model},
      author={Dai, Wenxun and Chen, Ling-Hao and Wang, Jingbo and Liu, Jinpeng and Dai, Bo and Tang, Yansong},
      journal={arXiv preprint arXiv:2404.19759},