Stochastic Scene-Aware Motion Prediction

Mohamed Hassan, Duygu Ceylan, Ruben Villegas, Jun Saito, Jimei Yang, Yi Zhou, and Michael Black

Abstract

A long-standing goal in computer vision is to capture, model, and realistically synthesize human behavior. Specifically, by learning from data, our goal is to enable virtual humans to navigate within cluttered indoor scenes and naturally interact with objects. Such embodied behavior has applications in virtual reality, computer games, and robotics, while synthesized behavior can be used as a source of training data. This is challenging because real human motion is diverse and adapts to the scene. For example, a person can sit or lie on a sofa in many places and with varying styles. It is necessary to model this diversity when synthesizing virtual humans that realistically perform human-scene interactions. We present a novel data-driven, stochastic motion synthesis method that models different styles of performing a given action with a target object. Our method, called SAMP, for Scene-Aware Motion Prediction, generalizes to target objects of various geometries while enabling the character to navigate in cluttered scenes. To train our method, we collected MoCap data covering various sitting, lying down, walking, and running styles. We demonstrate our method on complex indoor scenes and achieve superior performance compared to existing solutions. Our code and data are available for research at https://samp.is.tue.mpg.de.

Video

Paper

pdf

Code

Github

Data

Please register and accept the License agreement in this website in order to get access to the downloads.

Poster

pdf

Referencing SAMP

@inproceedings{hassan_samp_2021,
  title = {Stochastic Scene-Aware Motion Prediction},
  author = {Hassan, Mohamed and Ceylan, Duygu and Villegas, Ruben and Saito, Jun and Yang, Jimei and Zhou, Yi and Black, Michael},
  booktitle = {Proceedings of the International Conference on Computer Vision 2021},
  month = oct,
  year = {2021},
  event_name = {International Conference on Computer Vision 2021},
  event_place = {virtual (originally Montreal, Canada)},
  month_numeric = {10}
}

Contact

For questions, please contact samp@tue.mpg.de.

For commercial licensing, please contact ps-licensing@tue.mpg.de.