|
It is a difficult problem to reconstruct human shape and motion from three dimensional data which is continuous in chronological order. In recent years we use a statistical shape model constructed on the basis of an accurate whole body scan of a human being to compensate for parts that can not be measured by occlusion. Such a statistical model can be used in a three-dimensional scan of a human who wears tight clothing, but it can not express deformation such as wrinkles of human clothing worn in relaxed clothes. Therefore, we aim to reproduce the surface shape in arbitrary posture by measuring the surface shape of the human body with RGB-D sensor etc. and learning the difference between this and the basic 3D model by template.
|
|
 input |
 |
 vertex fitted model |
 pose fitted model |
|
In this way, two models are created from the three-dimensional point group measured with the RGB-D sensor, and the difference between them is obtained.
|
|  3 bases |
 5 bases |
 10 bases |
 Cumulative contribution ratio |
|
Calculation of difference is obtained by vector difference between three dimensional points. Calculate each coordinate of body parts so that it does not depend on the position of the 3D model.
Principal component analysis is performed with the difference obtained and dimension compression is performed.It expresses RMSE of the original data and the model restored by principal component analysis in color. RMSE is larger for red, and smaller for blue.
It can represent over 90% of the original data with 10 principal components.Regress the matrix compressed by principal component analysis.
|
|
The above movie is the result of restoration by our method. The right video uses 0 to 6 frames in sequence 544 frame for learning and 7 to 9 frames interpolated.
The left video interpolated 100 frames in the sequence. Both express RMSE with ground truth in color.
Blue indicates ground truth and its difference 0, green 5 cm, red 15 cm.
|
Publications
|
