FlyCap: Markerless Motion Capture Using Multiple Autonomous Flying Cameras

 Lan Xu, Yebin Liu, Wei Cheng, Kaiwen Guo, Guyue Zhou, Qionghai Dai, Lu FANG


Fig. 1 Reconstruction results of FlyCap in the local view. For each result, we show the correspondent color and depth input captured by the UAV in the front of the model, and the reconstruction result.

Abstract

Aiming at automatic, convenient and non-instrusive motion capture, this paper presents a new generation markerless motion capture technique, the FlyCap system, to capture surface motions of moving characters using multiple autonomous flying cameras (autonomous unmanned aerial vehicles(UAV) each integrated with an RGBD video camera). During data capture, three cooperative flying cameras automatically track and follow the moving target who performs large scale motions in a wide space. We propose a novel non-rigid surface registration method to track and fuse the depth of the three flying cameras for surface motion tracking of the moving target, and simultaneously calculate the pose of each flying camera. We leverage the using of visual-odometry information provided by the UAV platform, and formulate the surface tracking problem in a non-linear objective function that can be linearized and effectively minimized through a Gaussian-Newton method. Quantitative and qualitative experimental results demonstrate the competent and plausible surface and motion reconstruction results.

Capture illustration of FlyCap system. (a) The Capture scene; (b) The RGB images of the three UAVs; (c) The depth maps of the three UAVs; (d) The reconstructed dynamic model.

The pipeline of the proposed joint optimization scheme.


Content on this page requires a newer version of Adobe Flash Player.

Get Adobe Flash player

Content on this page requires a newer version of Adobe Flash Player.

Get Adobe Flash player

Download for : Supplemental Video Supplemental Video2

Citation:

FlyCap: Markerless Motion Capture Using Multiple Autonomous Flying Cameras

@article{xu2017flycap,   title={FlyCap: Markerless motion capture using multiple autonomous flying cameras},   
author={Xu, Lan and Liu, Yebin and Cheng, Wei and Guo, Kaiwen and Zhou, Guyue and Dai, Qionghai and Fang, Lu},   
journal={IEEE transactions on visualization and computer graphics},   
year={2017},   
publisher={IEEE} }