Computable Minds -

Kinect: How works its 3D body tracking

Posted on: Nov, 23rd 2010
Microsoft wants that we can play with the Xbox 360 moving us in front of the screen. For that have spend hundreds of millions of dollars in develop a device to realize the recognition and the tracking, not only of a complete person but several ones, including also 3D information, its name is Kinect. In this post you will discover how works the hardware and the artificial intelligence algorithms that uses.

To get the 3D image have a Infrared light projector and a camera that only captures this kind of light, so the intensity of the infrared light that arrives bounced to the camera, giving to the video console the depth of every pixel, creating a height map in black and white of 320x240 pixels, where the color more white is more near and the color more dark is more distant. This method isn't the only to give a machine stereoscopic vision, also can reconstruct a 3D image of the reality using two or more cameras, but the chosen method works also with bad illumination conditions.

schema of the Kinect

Also have a color camera of 640x480 pixels with CMOS sensor and autofocus to record video of the players, make facial recognition and separate the persons from the background, because the color and textures information is fundamental to accomplish this task. Separate something from the background, technically is called segmentation and before of do it also have to do an algorithm to align the color image with the height map, knowing the space between the cameras and, I suppose, that would be calibrated, because if not its lenses would provoke different deformation making difficult the task of do the pixel correspondence, while with the calibration the deformations can be deleted before do the correspondence.

When the person is segmented, Kinect must recognize every part of the person and assign it to a virtual skeleton, where its position will determine the action that will go to realize in the game.

In the base have an engine to calibrate its position with the help of an internal accelerometer, that's mean that the algorithms that use to identify the parts of a person produce errors if the cameras are not seeing just in front of the person.

Ended the body recognition, in the following frames will use some algorithm of tracking to know where we are going to move, this will avoid some calculus in the segmentation and recognition of our parts of the body in the next frames.

As all artificial intelligence system, it needs a training to learn to recognize us. This consist in leaving the hand in front of the device and making some movements in front of the screen. In this way the system extract the needed patterns to identify the characteristics our hands (skin color, size, etc.) and the characteristics of our body (width, height, etc.)

Certainly is an interesting device and don't surprise me if someone would use its technology to build a robot. I would like to probe it to see if surpass the level of precision of the Wii remote plus or the Playstation Move.

In the next post I will explain another interesting functionality of Kinect, the multiarray microphone.

Comments (1): See comments Comment
Categories: , ,


Copy and paste in your page:

How about you!? Don't give your opinion?

Replying to the next comments:

To check if you are human answer the question correctly:

I don't like this question, change it!

None of these data will be stored.

(Write the e-mail)

Required field.

(Write the e-mail or several e-mails separated by coma)

Required field.

To check if you are human answer the question correctly:

I don't like this question, change it!

Date: Apr, 29th 2011 Time: 12:52:24
i always wonderd how a kinect works
Daiatron on Google+