Copied from the README.txt file

KinectDTW - Kinect SDK Dynamic Time Warping (DTW) Gesture Recognition
By Rhemyst
Implementation by Rymix
Copyright (c) 2011, Rymix

This source code is distributed freely as open source. We'd rather you kept
this message in tact and included our copyright headers in your project.

Microsoft's SDK is not for commercial use, so by extension neither is this
library. Always adhere to Microsoft's terms of Kinect SDK use:

No warranty or support given. No guarantees this will work or meet your needs.
Some elements of this project have been tailored to the authors' needs and
therefore don't necessarily follow best practice. Subsequent releases of this
project will (probably) not be compatible with different versions, so whatever
you do, don'toverwrite your implementation with any new releases of this

Release information
Release version 0.1
Full of bugs, bad practice and not flexible enough yet, but a good starting
place for the community to build upon.

The Kinect SDK is rocking a whole bag of awesome, of that there's little doubt,
but until the open source community make some of its most tasty stuff
compatible with the SDK, Microsoft's offering will remain a little
under-featured and under-developed.

Amongst the missing features is a gesture recognition system. Lots of people
are building their own, bespoke implementations but no package (that I've
seen, at any rate) providesfast, accurate and programmable gesture libraries.

That's where this project could help. Using a relatively novel approach,
KinectDTW uses a vector-based nearest neighbour algorithm to track and classify
your gestures. The Dynamic Time Warping element means that this can recognise
gestures performed at different speeds.

1. Open up the Solution and check that you have all the prerequisite
software/dlls. Put Visual Studio into Debug mode and build and run the project.
You should see the MainWindow XAML window appear. If anything goes wrong at
this point, check your references, any paths that might be wrong or any other
dependencies that I might have forgotten.

2. Step into view of the Kinect sensor. Your skeleton should be tracked almost
immediately. It's probably best is you do this with only you in Kinect's sight
as this release only deals with one player.

3. Load the sample gestures by clicking Load gesture file and navigating to
the supplied RecordedGestures<date>.txt file

4. Start performing some gestures. You can see the names of the gestures from
the select box, and hopefully most of them are obvious to perform. You will
see matches appear in the results text panel at the top of the screen.

5. Nuke the app and start again.

6. Try recording your own gestures. Make sure your skeleton is being tracked,
select the gesture name you want to record, then click the Capture button.
You have three seconds to get into place and start recording your gesture. The
gesture is currently hard-coded to look at 32 frames (which is actually every
other frame over 64 expended frames). You may want to tweak this setting.

When recording gestures it is important that you start your gesture as soon
as the recording starts and that you finish on the 32nd frame. This might
mean that you have to perform your gestures for the recording slower (or
perhaps more quickly) than you would do in real life. Stick with it. The
DTW algorithm doesn't care about how quickly the gesture is performed.

7. When recording of each gesture is finished it automatically switches back
into Read mode, so test your new gesture a few times to see if you're happy
with it. If not, re-record it and try again.

8. When you're happy with your results, save your gestures to file. I'd
really love it if you shared your gestures with the community so that we can
build up a library of reliable gestures.

9. Make your own gestures - simply amend or add to the selectbox items with a
unique name and record your gesture. Note that a gesture name must start with
an @

That's it for this demo. Of course, it's not production specification yet, nor
would you want to release the gesture recorder in your project (only the
recogniser), but hopefully this gives you a good start in producing your own
gestures. DTW is probably not the perfect solution for all gestures, either, so
you'll need to experiment to find out what works and what doesn't. However, it
is a powerful tool for general gesture recognition.

Contribution and community
(Almost) anyone is free to contribute to this project. Just get in contact via

This framework's strength could well be in a community-built libray of tried
and tested gestures. Please contribute your gestures!

  • 2D gesture recognition using all the joints of the upper torso (Skeleton
  • Fast and customisible gesture recogniser
  • Gesture recorder, so you can create your own gestures
  • Save gestures to file for future use (and load from file)
  • Sample WPF project with skeletal viewer and optional depth and RGB viewers

Future developments
  • 3D gesture recording and recognition
  • More customisible parameters for finer control
  • Dynamic gesture length
  • Voice control for starting, stopping and switching gestures in the recorder
(so that you don't have to keep returning to the keyboard/mouse and messing up
your gesture recording)
  • Customisible 'active joints' for gesture recognition (i.e. track just the
right arm and hand rather than the whole upper body)
  • Ability to deal with more than one player
  • Smooth out the recognition of similar gestures so that only one is selected

Windows 7
Kinect SDK
Visual Studio 2010
Coding4Fun toolkit
Probably the latest .NET and maybe some other stuff too. You'll soon find out.
Look at the project References if you're stuck.

Tested on
Windows 7 Professional 32-bit SP1
Visual Studio 2010 Ultimate
Intel Pentium Dual CPU E2180 @ 2.00 GHz
4.00 GB RAM (3.25 GB usable)
ATI Radeon X1950 Pro 512MB

The SDK runs pretty slowly on my PC (which is actually beneath Microsoft's
recommended specification for the SDK). It was unusably slow when I turned on
the depth and RGB streams. Hopefully if you have a faster PC you'll have a
better experience. However, the gestures are recognised very quickly.

KinectDTW project on Codeplex:

Microsot's Kinect SDK terms of use:

Wikipedia's ovbiously 100% reliable explanation of dynamic time warping:

You'd probably better take a look at a real scientific paper :

Kinect SDK fora on MSDN:

Microsoft Coding4Fun Toolkit

Stock disclaimer statement, but I mean it: I'm releasing this to you as a
learning tool, nothing more. Don't expect anything from me or expect to hold
me liable for anything whatsoever, because I herein expunge myself of all
responsibility for this code. This software is provided by the Rymix and its
licensors “as is” and any express or implied warranties, including, but not
limited to, the implied warranties of merchantability and fitness for a
particular purpose are disclaimed. In no event will Rymix or its licensors
be liable for any direct, indirect, incidental, special, exemplary, or
consequential damages (including, but not limited to, procurement of
substitute goods or services; loss of use, data, or profits; or business
interruption) however caused and on any theory of liability, whether in
contract, strict liability, or tort (including negligence or otherwise) arising
in any way out of the use of this software, even if advised of the possibility
of such damage.

Last edited Aug 30, 2011 at 1:31 PM by Simboubou, version 4


No comments yet.