Windows Kinect Driver/SDK - CL NUI Platform Release!!! *** Updated ***
Posted: 24 November 2010 03:54 AM   [ Ignore ]   [ # 16 ]
New Member
Rank
Total Posts:  1
Joined  2010-11-15

Hello,

I’m having trouble understanding the depth image that is displayed in the example app. I was expecting a greyscale z-depth image, but I can’t see how the depth is encoded into the RGB channels the example. Is there a way to just get a grayscale image? Thank you.

Profile
 
 
Posted: 24 November 2010 02:12 PM   [ Ignore ]   [ # 17 ]
New Member
Rank
Total Posts:  6
Joined  2010-11-21
KrisM - 21 November 2010 10:19 PM

@Liam that is beautiful work man! I hope you don’t mind if we highlight this as example video for SDK usage… really amazing work!

@Pixel I’ll try and grab the latest processing and try out your wrapper, is it using JNI?


No problem whatsoever

also… using ben x tan’s source (he made midi controller using your SDK) i made a touch wall what controls my computers mouse

VERY basic, very MESSY, just for demo purposes really…

In action:

source: http://essexconsolerepairs.co.uk/programs/openkinect/KinectTouchSource.rar

Edit: my tweaked sample source: http://essexconsolerepairs.co.uk/programs/openkinect/KinectSampleSource.rar

Enjoy smile

Profile
 
 
Posted: 24 November 2010 03:09 PM   [ Ignore ]   [ # 18 ]
New Member
Rank
Total Posts:  2
Joined  2010-11-23

I managed to get the depth RGB data into openCV (2.1) and output using the C API.

Here’s the relevant code.

 

cvNamedWindow"Kinect");
    
CLNUICamera KinectCamera;
 
    
HMODULE kinectDll LoadLibrary(TEXT("CLNUIDevice.dll"));

    if (
kinectDll==NULL){
        printf
("Error loading DLL file:\n");
        return -
1;
    
}

    KinectCamera 
CreateNUICamera();

    if(!
StartNUICamera(KinectCamera)){
        printf
("Could Not Start Kinect Camera");
        return -
1;
     
}
    
    PDWORD imageName 
= (PDWORDmalloc(640*480*4);

    while((
charkey != 27 ){
         frame_count 
++;  
    
         if(
GetNUICameraDepthFrameRGB32(KinectCameraimageName)){
        
             kinectImage 
cvCreateImageHeader(cvSize(640,480),8,4);
             
cvSetData(kinectImage,imageName,kinectImage->widthStep);
        
             
cvCvtColor(kinectImage,img,CV_RGB2BGR);
      
             
cvShowImage"Kinect"kinectImage);
        
             
cvReleaseImageHeader(&kinectImage;); 
          
}

         
//...Do a whole bunch of other stuff here
    
Profile
 
 
Posted: 28 November 2010 12:18 PM   [ Ignore ]   [ # 19 ]
New Member
Rank
Total Posts:  6
Joined  2010-11-21

Hey, i recommend that in a new version of your SDK that you can add the option for a greyscale depth feed as im sure it would be useful to many, i for one really need it as a greyscale conversion is taking 84ms per frame what is a no go!

It would be great if you could add it..


Thanks

Liam

Profile
 
 
Posted: 05 December 2010 11:22 AM   [ Ignore ]   [ # 20 ]
New Member
Rank
Total Posts:  3
Joined  2010-11-24
sam.muscroft - 24 November 2010 03:09 PM

I managed to get the depth RGB data into openCV (2.1) and output using the C API.

Here’s the relevant code.

 

cvNamedWindow"Kinect");
    
CLNUICamera KinectCamera;
 
    
HMODULE kinectDll LoadLibrary(TEXT("CLNUIDevice.dll"));

    if (
kinectDll==NULL){
        printf
("Error loading DLL file:\n");
        return -
1;
    
}

    KinectCamera 
CreateNUICamera();

    if(!
StartNUICamera(KinectCamera)){
        printf
("Could Not Start Kinect Camera");
        return -
1;
     
}
    
    PDWORD imageName 
= (PDWORDmalloc(640*480*4);

    while((
charkey != 27 ){
         frame_count 
++;  
    
         if(
GetNUICameraDepthFrameRGB32(KinectCameraimageName)){
        
             kinectImage 
cvCreateImageHeader(cvSize(640,480),8,4);
             
cvSetData(kinectImage,imageName,kinectImage->widthStep);
        
             
cvCvtColor(kinectImage,img,CV_RGB2BGR);
      
             
cvShowImage"Kinect"kinectImage);
        
             
cvReleaseImageHeader(&kinectImage;); 
          
}

         
//...Do a whole bunch of other stuff here
    

Great job L14M333
i really like ur app but i could not success to do the left click by taping as u show in the video
regards

Profile
 
 
Posted: 05 December 2010 10:15 PM   [ Ignore ]   [ # 21 ]
New Member
Rank
Total Posts:  1
Joined  2010-12-05

Thanks Vitorio for starting the ball rolling.

The python code below works great except for the depth data.

I picked up a Kinect this weekend and was frustrated that on Windows I couldn’t use the Python interface from OpenKinect, then I found CLNUI, but with no Python interface had to resort to ctypes (thankfully Vitorio got the ball rolling with it).

Here are some updates that I found helpful (I’ll post more later if folks find it useful - not sure if this is the best place for it).


If you only want to save the RGB depth image add the following to Vitorio’s code :

#Get 32 bit depth image
kinect.GetNUICameraDepthFrameRGB32(camera, nuiimg32)


#Save the 32bit depth map image
outimg32 = Image.fromstring(‘RGBA’, [640, 480], nuiimg32, ‘raw’,‘RGBA’, 0, 1)
outimg32.save(“depth_img.jpg”)

Now the raw data is done differently, this seems to be (mostly) working for me:

import numpy
import struct


#get raw depth data
kinect.GetNUICameraDepthFrameRAW(camera, nuiimg16)

#unpack and save the depth data
s=struct.Struct(‘H’)
depth_image=numpy.empty((640, 480), dtype=numpy.uint16)

for i in range(640):
  for j in range(480):
      x=s.unpack_from(nuiimg16,(j*640+i)*2)[0]
      depth_image[i, j]=x

depth_image*=255.0/depth_image.max()

i = Image.fromarray(numpy.uint8(depth_image))
i.save(“depth_data.jpg”)


Rob

Vitorio - 21 November 2010 03:27 PM

Here is an example using the CL NUI Platform DLL with Python to pull data from the Kinect.  It saves out a PNG of the RGB camera, and what I believe is a valid 16-bit greyscale TIFF of the depth camera.

# Requires at least Python 2.5 and PIL
import ctypes
import time
import Image

# Kinect image buffers
nuiimg24 ctypes.create_string_buffer(640*480*(24/8))
nuiimg16 ctypes.create_string_buffer(640*480*(16/8))

# Init the Kinect
kinect ctypes.cdll.CLNUIDevice

camera 
kinect.CreateNUICamera()
kinect.StartNUICamera(camera)

# Let the RGB camera warm up
time.sleep(2)

# Get the Kinect data
kinect.GetNUICameraColorFrameRGB24(cameranuiimg24)
kinect.GetNUICameraDepthFrameRAW(cameranuiimg16)

# Shut down the Kinect
kinect.StopNUICamera(camera)
kinect.DestroyNUICamera(camera)

# Output with PIL
outimg24 Image.fromstring('RGB'[640480]nuiimg24'raw''BGR'01)
outimg24.save("outimg24.png")

outimg16 Image.fromstring('I;16'[640480]nuiimg16)
outimg16.save('outimg16.tiff'

http://python.pastebin.com/ZQw6ttzZ

Profile
 
 
Posted: 09 December 2010 10:20 PM   [ Ignore ]   [ # 22 ]
New Member
Rank
Total Posts:  2
Joined  2010-12-09

Hey all, I just downloaded the SDK and driver today, but I’m having issues. I included CLNUIDevice.dll in the project, but it’s saying “The program can’t start because CLNUIDevice.dll is missing from your computer. Try reinstalling the program to fix this problem.” I also tried the C# version, and while the sample compiled and ran (and I was able to play with the Kinect by modifying the sample), I could not for the life of me get an entirely new project to run with C#, as when I tried to add a reference for CLNUDevice.dll it said it wasn’t a valid .dll file. Any suggestions?

Profile
 
 
Posted: 10 December 2010 01:28 AM   [ Ignore ]   [ # 23 ]
New Member
Rank
Total Posts:  6
Joined  2010-11-21
phoenixnyt - 09 December 2010 10:20 PM

Hey all, I just downloaded the SDK and driver today, but I’m having issues. I included CLNUIDevice.dll in the project, but it’s saying “The program can’t start because CLNUIDevice.dll is missing from your computer. Try reinstalling the program to fix this problem.” I also tried the C# version, and while the sample compiled and ran (and I was able to play with the Kinect by modifying the sample), I could not for the life of me get an entirely new project to run with C#, as when I tried to add a reference for CLNUDevice.dll it said it wasn’t a valid .dll file. Any suggestions?

make sure its in the directory of wherever the exe you are running it (Bun/Debug?)

Profile
 
 
Posted: 10 December 2010 04:07 AM   [ Ignore ]   [ # 24 ]
New Member
Rank
Total Posts:  4
Joined  2010-11-23

Finger tracking to tuio

http://www.youtube.com/watch?v=lCuItHQEgEQ

vs2010 project. opencv + (modified) TUIO reference implementation + CL NUI driver ver. 1.0.0.1121

source code:

http://dl.dropbox.com/u/5505209/FingertipTuio3d.zip

Profile
 
 
Posted: 10 December 2010 04:16 AM   [ Ignore ]   [ # 25 ]
New Member
Rank
Total Posts:  6
Joined  2010-11-21
spotgratis - 10 December 2010 04:07 AM

Finger tracking to tuio

http://www.youtube.com/watch?v=lCuItHQEgEQ

vs2010 project. opencv + (modified) TUIO reference implementation + CL NUI driver ver. 1.0.0.1121

source code:

http://dl.dropbox.com/u/5505209/FingertipTuio3d.zip


NICE WORK!

Profile
 
 
Posted: 10 December 2010 11:19 AM   [ Ignore ]   [ # 26 ]
New Member
Rank
Total Posts:  2
Joined  2010-12-09
L14M333 - 10 December 2010 01:28 AM
phoenixnyt - 09 December 2010 10:20 PM

Hey all, I just downloaded the SDK and driver today, but I’m having issues. I included CLNUIDevice.dll in the project, but it’s saying “The program can’t start because CLNUIDevice.dll is missing from your computer. Try reinstalling the program to fix this problem.” I also tried the C# version, and while the sample compiled and ran (and I was able to play with the Kinect by modifying the sample), I could not for the life of me get an entirely new project to run with C#, as when I tried to add a reference for CLNUDevice.dll it said it wasn’t a valid .dll file. Any suggestions?

make sure its in the directory of wherever the exe you are running it (Bun/Debug?)

L14M333, you are officially my favorite person.

Profile
 
 
Posted: 11 December 2010 06:11 AM   [ Ignore ]   [ # 27 ]
New Member
Rank
Total Posts:  4
Joined  2010-12-11

Hi,
I’ve just produced a Delphi Wrapper for the CL NUI SDK, and am right now rewriting the sample applications in native Delphi/Pascal code…
The problem I have is that I don’t currently possess a Kinect device with which to test my efforts.

Wondering if anyone would like to volunteer?

The wrapper and demos will be released on my blog and SVN shortly as open source, and will include compiled executable demos.

Anyone happy to test it for me with a Kinect and let me know if it works as expected?

Profile
 
 
Posted: 12 December 2010 12:57 PM   [ Ignore ]   [ # 28 ]
New Member
Rank
Total Posts:  1
Joined  2010-12-12
spotgratis - 10 December 2010 04:07 AM

Finger tracking to tuio

http://www.youtube.com/watch?v=lCuItHQEgEQ

vs2010 project. opencv + (modified) TUIO reference implementation + CL NUI driver ver. 1.0.0.1121

source code:

http://dl.dropbox.com/u/5505209/FingertipTuio3d.zip

Okay, needless to say, I am not very familiar with C++ in Visual Studio but I have gotten the application to compile, however I seem to have issues with my installation.

1)
imshow(“debugFrame”, debugFrame);
crashes as soon as I start the application and I get this error:

Unhandled exception at 0x0075fca1 (msvcr90d.dll) in FingertipTuio3d.exe: 0xC0000005: Access violation reading location 0x61724667.

Call Stack:
>  msvcr90d.dll!strcat(unsigned char * dst=0x0080d12c, unsigned char * src=0x002eece4)  Line 167   Asm

2)
If I take that line of code out, it compiles and starts running 0’s down the screen, however, once I put my hand in front of the Kinect it fails again. This time with the following error.

Unhandled exception at 0x00984a79 in FingertipTuio3d.exe: 0xC0000005: Access violation reading location 0x00000008.

Call Stack:
>  FingertipTuio3d.exe!std::vector

,std::allocator

> >::size()  Line 879 + 0x6 bytes   C++

Can anyone help me with this problem?

Profile
 
 
Posted: 14 December 2010 03:28 AM   [ Ignore ]   [ # 29 ]
New Member
Rank
Total Posts:  1
Joined  2010-12-14

Hi everyone, a few things I can’t understand.
I made some little changes to the CLKinectData included as example, I added the other DepthFrame in the dropdown so that I could see the different output formats. However, something ain’t quite right:

The RGB32 depth image works as well, but corrected12 and raw appear duplicated, while corrected8 has 4 images.

http://img714.imageshack.us/img714/8551/depth1.png

http://img638.imageshack.us/img638/1122/48581763.png
http://img217.imageshack.us/img217/8101/c12j.png
http://img8.imageshack.us/img8/9669/rawkr.png

I guess it’s something related to the pixel format and probably stride, but I can’t understand much more.

My second question is: do someone know how you can get a value proportional to the depth from either GetNUICameraDepthFrameRAW, GetNUICameraDepthFrameCorrected12, GetNUICameraDepthFrameCorrected8 or GetNUICameraDepthFrameRGB32?

Or: how many byte per pixel should I consider for those images, and what is the order of those pixel?

Thank you very much - I hope I can give back some help once I’m inducted into the mysteries of the API and the basics of image manipulation smile

Profile
 
 
Posted: 01 January 2011 04:49 PM   [ Ignore ]   [ # 30 ]
New Member
Rank
Total Posts:  1
Joined  2011-01-01

Are the functions thread-safe? In particular, can I retrieve colour images and depth images in separates threads without explicit safe guarding against race conditions? Moreover, according to my test, it seems heap corruptions are sometimes triggered by calling StopNUICamera. I am wondering if anyone could confirm my observation…

P.S. I’m using C++ in Visual Studio 2005 (SP1), Windows 7 (64 bit, but I was using the 32 bit compiler).

Many thanks!

Profile
 
 
2 of 3
2
 


RSS 2.0     Atom Feed