hide
Free keywords:
-
Abstract:
Real-time simultaneous tracking of hands manipulating and interacting with
external objects has many potential applications in augmented reality, tangible
computing, and wearable computing. However, due to dicult occlusions,
fast motions, and uniform hand appearance, jointly tracking hand and object
pose is more challenging than tracking either of the two separately. Many
previous approaches resort to complex multi-camera setups to remedy the occlusion
problem and often employ expensive segmentation and optimization
steps which makes real-time tracking impossible. In this paper, we propose
a real-time solution that uses a single commodity RGB-D camera. The core
of our approach is a 3D articulated Gaussian mixture alignment strategy tailored
to hand-object tracking that allows fast pose optimization. The alignment
energy uses novel regularizers to address occlusions and hand-object
contacts. For added robustness, we guide the optimization with discriminative
part classication of the hand and segmentation of the object. We
conducted extensive experiments on several existing datasets and introduce
a new annotated hand-object dataset. Quantitative and qualitative results
show the key advantages of our method: speed, accuracy, and robustness.
