Hybrid method for hand articulations tracking using kinect

In order to track position and orientation of hand and its articulations, we implement a hybrid solution for markerless hand articulation tracking with electromagnetic tracker and visual observations from Kinect. The position tracking of hand uses an electromagnetic tracker, and tracking of 23 DOFs of articulations is treated as an optimization problem which is solved using Particle Swarm Optimization (PSO), and it is essentially the progress of seeking for the hand model parameters that minimize the discrepancy between the hypothesized hand model and actual hand observations. The system input is RGB and depth images from Kinect sensor. This system does not require special markers and is capable to track hand articulations frame by frame. Experiments show that robust 3D tracking of hand articulations can be achieved in 12FPS, and average error is less than 10mm.