3D Support (Not Sure But Think I Did It)

Oct 24, 2011 at 7:34 AM

Hi everyone,

I think I've added 3D Gesture Recognition support. i'm not too sure as I haven't had a good opportunity to test it but please do check it out and correct me (with some patience ;) ) if I'm wrong. Thanks a ton.

LINK: http://dl.dropbox.com/u/11030595/KinectDTW3D/kinectdtw3D-1.0-modified.zip

Best regards,

Mikhail.

Oct 29, 2011 at 10:52 AM

Hi,

I downloaded your version and can't find much difference to the 2D version? Am I right that you only extended one dimension in "Skeleton3DDataExtract" and "Skeleton3DdataCoordEventArgs" class?

I can't test your code because I'm working on porting this technique to OpenNI and don't run the official SDK on my machine... But a simple test to check if your code works is that you train a gesture that only move in the Z-Axis, something like a push without vertical or horizontal movement! To get 100% sure you can adjust the values in the saved file for X and Y Axis (think have to be the same value).

Thomas

Oct 29, 2011 at 8:49 PM
Ya that's exactly what I did. And by the logic of the code it should
work. I tried a gesture sideways that was detected equally well when
it was moved along the Z axis as well preserving the X-Y coordinate
change. However now it isn't being detected (meaning that the Z Axis
is being checked as well). But i could be wrong. Thanks.

Best regards,
Mikhail.
Oct 31, 2011 at 8:56 AM

That sounds great. I will try the changes in my own code the next days and will give you feedback.

If your idea with only changing these two classes works, this would be absolutely fabulous....

Thomas

Oct 31, 2011 at 11:06 AM
Edited Oct 31, 2011 at 11:33 AM
Hi Thomas,
There's a tiny bit more. You need to also change the dimensions of the arrays used to hold the data. It will grow to 18 doubles from 12 originally used, in each frame. My newer code let's you isolate the joints to check for for each gesture rather than checking all joint coordinates at all times for all gestures. Also it checks the legs also. I will upload it soon, though in truth the changes are not too large.

Best regards,
Mikhail Jacob
Oct 31, 2011 at 11:34 AM

New Code up for testing. Adds support for selecting which joints to monitor for which gestures.

LINK: http://kinectdtw.codeplex.com/discussions/277748

Nov 2, 2011 at 8:09 AM

Hi Mikhail,

thanks for the fresh uploaded Code. I will have a look at it right now!

The Array size is no problem to me because I use a complete other struct to save everything (more dynamic). For the leg support I've got no need because my application is only for the upper body, but I will definitly check that out.

Thomas

Nov 2, 2011 at 9:10 AM
Actually, the more important upgrade there is the option of selecting joints that the gesture monitors. A right handed gesture should not rely on the left hand being at the same position it was in when the gesture was recorded during the recognition phase. So once I record a right handed punch relying on the position changes of wrist, elbow and hand joints of only the right hand, the right handed punch should be detected regardless of the position of any of the other joints.

Best regards,
Mikhail Jacob

On 02-Nov-2011, at 4:09 AM, "Kipaboy" <notifications@codeplex.com> wrote:

From: Kipaboy

Hi Mikhail,

thanks for the fresh uploaded Code. I will have a look at it right now!

The Array size is no problem to me because I use a complete other struct to save everything (more dynamic). For the leg support I've got no need because my application is only for the upper body, but I will definitly check that out.

Thomas

Nov 2, 2011 at 11:20 AM
Edited Nov 2, 2011 at 12:09 PM

Yeah that's a great feature! Tested it on my laptop, where Windows SDK is now running. I have to say, the recognition is much more accurate!!

Trying to implement that on my own project, but I'm not sure if I understood your code right... You idea behind is that u create a pattern which says that if a joint has to be compared or not (true and false). If the pattern says false you assign the value "0.0" to both sequences (the saved one in the dtw class and the actual framebuffer). Am I right?

 

Thomas

 

edit:

I've got another short question to your code: DtwGestureRecognizer::+Recognize(Arraylist seq) - Line 188-198

Would be "seqReplaced = seq;" the same?

Nov 2, 2011 at 12:10 PM
Yup... That's it. The gesture signature frames are permanently masked with zeros for joints that are not to be checked. However the example sequence to compare against is only temporarily masked so that other gesture signatures with different joint selections can be checked later. One defect I've got to fix is that since the gestures are checking selective joints, if two gesture signatures with movements such that one gesture's set of movements is a superset of the other's is recorded to memory such that the signature with less joints checked for is first in the list of joints, it is detected even if the second gesture is done. Eg. If i do a one handed punch with joints hand, wrist and elbow and a two handed punch with both hands, wrists and elbows. If i record the single handed one first it is detected even if i do the double handed punch since according to the joints selected it is correct. To fix this you need to sort gesture signatures according to decresing number of joints checked for. Simple fix. :)

Best regards,
Mikhail Jacob

On 02-Nov-2011, at 7:20 AM, "Kipaboy" <notifications@codeplex.com> wrote:

From: Kipaboy

Yeah that's a great feature! Tested it on my laptop, where Windows SDK is now running. I have to say, the recognition is much more accurate!!

Trying to implement that on my own project, but I'm not sure if I understood your code right... You idea behind is that u create a pattern which says that if a joint has to be compared or not (true and false). If the pattern says false you assign the value "0.0" to both sequences (the saved one in the dtw class and the actual framebuffer). Am I right?

Thomas

Nov 2, 2011 at 12:21 PM

Yeah already modified my struct to identify the joints that have to be compared for the specific gesture.

Thanks for all the stuff :) Hope this will work as good with OpenNI as with the official SDK