Home ForumsCL Studio LiveAnnouncementsThread
Windows Kinect Driver/SDK - CL NUI Platform Release!!! *** Updated ***
 
AlexP
Posted: 17 November 2010 02:48 AM   [ Ignore ]
Administrator
Avatar
RankRankRankRank
Total Posts:  585
Joined  2009-09-17

Updates: The CL NUI Platform Official 1.0 Release; now supports multiple Kinect cameras, signed driver and much more…

You can review the updates here to get started using the SDK Samples and more.

~~~ Nov 16th ~~~

We are proud to release the first preview release of the new CL NUI Platform which consists of an SDK, API and a stable driver for Microsoft’s new Kinect Hardware (Xbox NUI Audio, NUI Camera, NUI Motor and Accelerometer) devices on Windows machines.

To start we have a WPF/C# (.NET 3.5) Visual Studio 2010 Sample Applications as well as C API (CLNUIDevice.h, DLL, LIB) and plan on extending the SDK similar to our CL Eye SDK which has Samples for C/C++/C#, Java and DirectShow.

Learn more about the platform or download the latest installer to get started…

Video/Screenshot of the Sample WPF Application - 2 Cameras 640x480@30FPS, NUI Motor/Accelerometer and LED Control:

CL_NUI_PLATFORM_SAMPLE.png

An overview of the platform:

CL_NUI_PLATFORM_OVERVIEW.png

The CL NUI API:

//////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
//
// This library is part of CL NUI SDK
// It allows the use of Microsoft Kinect cameras in your own applications
//
// For updates and file downloads go to: http://codelaboratories.com/get/kinect
//
// Copyright 2010 (c) Code Laboratories, Inc.  All rights reserved.
//
//////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
#pragma once
#include <windows.h>

#define IMPORT(type) extern "C" __declspec(dllimport)## type __cdecl

//////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
// NUIDevice  API
//////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
// Device enumeration
IMPORT(int) GetNUIDeviceCount();
IMPORT(PCHAR) GetNUIDeviceSerial(int index);

//////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
// CLNUIMotor  API
//////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
// Motor instance type
typedef void *CLNUIMotor;

// Library initialization
IMPORT(CLNUIMotor) CreateNUIMotor(PCHAR serial);
IMPORT(bool) DestroyNUIMotor(CLNUIMotor mot);

// Motor control
IMPORT(bool) SetNUIMotorPosition(CLNUIMotor mot, SHORT position);

// Get accelerometer data
IMPORT(bool) GetNUIMotorAccelerometer(CLNUIMotor mot, SHORT &x, SHORT &y, SHORT &z);

IMPORT(bool) SetNUIMotorLED(CLNUIMotor mot, BYTE value);
//////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////

//////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
// CLNUICamera API
//////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
// Camera instance type
typedef void *CLNUICamera;

// Library initialization
IMPORT(CLNUICamera) CreateNUICamera(PCHAR serial);
IMPORT(bool) DestroyNUICamera(CLNUICamera cam);

// Camera capture control
IMPORT(bool) StartNUICamera(CLNUICamera cam);
IMPORT(bool) StopNUICamera(CLNUICamera cam);

// Camera video frame image data retrieval
IMPORT(bool) GetNUICameraColorFrameRAW(CLNUICamera cam, PBYTE pData, int waitTimeout = 2000);
IMPORT(bool) GetNUICameraColorFrameRGB24(CLNUICamera cam, PBYTE pData, int waitTimeout = 2000);
IMPORT(bool) GetNUICameraColorFrameRGB32(CLNUICamera cam, PDWORD pData, int waitTimeout = 2000);

// Camera depth frame image data retrieval
IMPORT(bool) GetNUICameraDepthFrameRAW(CLNUICamera cam, PUSHORT pData, int waitTimeout = 2000);
IMPORT(bool) GetNUICameraDepthFrameCorrected12(CLNUICamera cam, PUSHORT pData, int waitTimeout = 2000);
IMPORT(bool) GetNUICameraDepthFrameCorrected8(CLNUICamera cam, PBYTE pData, int waitTimeout = 2000);
IMPORT(bool) GetNUICameraDepthFrameRGB32(CLNUICamera cam, PDWORD pData, int waitTimeout = 2000);
///////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////// 

Thanks and please share any videos of it in use or feedback while getting started with the CL NUI Platform.

AlexP

Profile
 
 
mitch
Posted: 17 November 2010 10:23 AM   [ Ignore ]   [ # 1 ]
New Member
Rank
Total Posts:  6
Joined  2010-11-11

As told in nuigroup, very easy and working solution, I do appreaciate this piece of software and the work behind it.

Just wondering something : are there any other methods that can be exposed by DLL to achieve more control on hardware and onboard computation ?

Profile
 
 
Robert Appleton
Posted: 17 November 2010 02:27 PM   [ Ignore ]   [ # 2 ]
New Member
Rank
Total Posts:  1
Joined  2010-11-17

Great Work Guys, Installed and Working not glitches on my 7 Box.

smile Okey, so when can i start using my PC like they do on minority report?
Whats the development pathway for what you doing? OR simply providing a community driver?

You guys entering Googles competition for “Cool App”?

Profile
 
 
mitch
Posted: 17 November 2010 04:50 PM   [ Ignore ]   [ # 3 ]
New Member
Rank
Total Posts:  6
Joined  2010-11-11

i did this 4 years ago , it’s a matter of ideas not of tech tools :D

http://vimeo.com/853894

Profile
 
 
jorgemagic
Posted: 18 November 2010 12:27 AM   [ Ignore ]   [ # 4 ]
New Member
Rank
Total Posts:  1
Joined  2010-11-18

Great job, I think it was one of the devices that have been hacked sooner. I have a question.

When you set Kinect in the console menu you can see the recognition of the skeleton that is capable of. Is this the image processing is done with software from the device or from the console?

Profile
 
 
mikex
Posted: 21 November 2010 11:54 AM   [ Ignore ]   [ # 5 ]
New Member
Rank
Total Posts:  1
Joined  2010-11-21

Hi,
How do you get this to work on XP? Do I need any other redistributables? I have Visual Studio 2010 and .NET 3.5 SP1 and i get the following error:

Cannot create instance of ‘MainWindow’ defined in assembly ‘CLNUIDeviceTest, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null’. Exception has been thrown by the target of an invocation.  Error in markup file ‘MainWindow.xaml’.

Profile
 
 
Vitorio
Posted: 21 November 2010 03:27 PM   [ Ignore ]   [ # 6 ]
New Member
Rank
Total Posts:  1
Joined  2010-11-21

Here is an example using the CL NUI Platform DLL with Python to pull data from the Kinect.  It saves out a PNG of the RGB camera, and what I believe is a valid 16-bit greyscale TIFF of the depth camera.

# Requires at least Python 2.5 and PIL
import ctypes
import time
import Image

# Kinect image buffers
nuiimg24 = ctypes.create_string_buffer(640*480*(24/8))
nuiimg16 = ctypes.create_string_buffer(640*480*(16/8))

# Init the Kinect
kinect = ctypes.cdll.CLNUIDevice

camera = kinect.CreateNUICamera()
kinect.StartNUICamera(camera)

# Let the RGB camera warm up
time.sleep(2)

# Get the Kinect data
kinect.GetNUICameraColorFrameRGB24(camera, nuiimg24)
kinect.GetNUICameraDepthFrameRAW(camera, nuiimg16)

# Shut down the Kinect
kinect.StopNUICamera(camera)
kinect.DestroyNUICamera(camera)

# Output with PIL
outimg24 = Image.fromstring('RGB', [640, 480], nuiimg24, 'raw', 'BGR', 0, 1)
outimg24.save("outimg24.png")

outimg16 = Image.fromstring('I;16', [640, 480], nuiimg16)
outimg16.save('outimg16.tiff') 

http://python.pastebin.com/ZQw6ttzZ

Profile
 
 
L14M333
Posted: 21 November 2010 03:51 PM   [ Ignore ]   [ # 7 ]
New Member
Rank
Total Posts:  6
Joined  2010-11-21

Just thought i would share a little bit of tweaking i did to the sample…


All i really did was improve the tilt, add accelerometer calibration (set to zero) then plotted 2 of the 3 axis into a graph live


Hope you like

Liam

Profile
 
 
pixelnerve
Posted: 21 November 2010 05:30 PM   [ Ignore ]   [ # 8 ]
New Member
Rank
Total Posts:  23
Joined  2010-03-20

I made a wrapper for java: http://victamin.googlecode.com/files/CLNUI4J_01.zip

It works on eclipse but when running with processing it seems to have the same problem i had with recent ps3eye drivers.
It keeps saying it can’t find the DLL file.

This was tested on win7 64bit. If there is anyone out there using java/processing, i would appreciate some feedback.

ps: In a next few days i will release a new version

Profile
 
 
CL
Posted: 21 November 2010 10:19 PM   [ Ignore ]   [ # 9 ]
Sr. Member
Avatar
RankRankRankRank
Total Posts:  162
Joined  2009-09-17

@Liam that is beautiful work man! I hope you don’t mind if we highlight this as example video for SDK usage… really amazing work!

@Pixel I’ll try and grab the latest processing and try out your wrapper, is it using JNI?

Profile
 
 
gianluca_fuse
Posted: 22 November 2010 03:24 AM   [ Ignore ]   [ # 10 ]
New Member
Rank
Total Posts:  12
Joined  2010-02-16

Can’t find DLL here too..

is there any way to specify the path?

Profile
 
 
pixelnerve
Posted: 22 November 2010 10:32 AM   [ Ignore ]   [ # 11 ]
New Member
Rank
Total Posts:  23
Joined  2010-03-20

Krism: Uses JNA.

—-
It seems we’re having problems with processing as PS3Eye driver did.
Even if setting up a custom path it simply can’t find the dll or load it. Nonetheless it runs just fine with java.
I will try to get see how to get this to run under processing, but i have to say i had no luck with ps3eye java lib, so probably won’t with this one :(

I would like people to try the lib anyway, as other OS versions might give different results. So far only tested on Windows7 64bit.

Thanks

Profile
 
 
mitch
Posted: 22 November 2010 02:34 PM   [ Ignore ]   [ # 12 ]
New Member
Rank
Total Posts:  6
Joined  2010-11-11

what’s new in release 1121 ?

Profile
 
 
spotgratis
Posted: 23 November 2010 06:43 AM   [ Ignore ]   [ # 13 ]
New Member
Rank
Total Posts:  4
Joined  2010-11-23

a software like WIITUIO but with kinect is beautiful. i think you can develop this software in 2 days.

http://code.google.com/p/wiituio/

Profile
 
 
sam.muscroft
Posted: 23 November 2010 10:26 AM   [ Ignore ]   [ # 14 ]
New Member
Rank
Total Posts:  2
Joined  2010-11-23

Fantastic job getting a Windows driver and API out there so quickly!

I’m trying to use the C/C++ API and have successfully built and run a project in Visual Studio and can connect and control the Kinect ok…start / stop camera etc… I’m having trouble displaying the images from the camera API functions I am trying to do this with openCV like so, this produces a blank image…


 

PBYTE imageName = (PBYTE) malloc(640*480*3);
    GetNUICameraColorFrameRAW(KinectCamera, imageName);

    kinectImage = cvCreateImageHeader(cvSize(640,480), IPL_DEPTH_8U, 3); 
    cvSetData(kinectImage,imageName,640*3);
    cvShowImage( "Kinect", kinectImage ) ; 

I previously tried to save out the image to a static bitmap but did not have any luck…I notice in the C# implementation that there is a NUIImage class which does the conversion but have not been able to port this to C++ as yet.

Has anyone had any luck with the C/C++ API or does anyone have any advice? I have been able to compile and run the C# program ok, but would really like to use C++ for best performance and compatibility reasons.

Best regards,

Sam.

Profile
 
 
bryanlivingston
Posted: 23 November 2010 10:48 AM   [ Ignore ]   [ # 15 ]
New Member
Rank
Total Posts:  1
Joined  2010-11-23

I’m trying to get a depth map out of the driver to work with inside of C#.

I’ve been able to get them out as integers from the 32bit image by doing the following:

const int width = 640;
        const int height = 480;
        int[] calibrationPixels = new int[width * height];

        static void ImageToDepthField(InteropBitmap source, int[] destination)
        {
            int stride = width * 4;
            source.CopyPixels(destination, stride, 0);
        }

        // call this right after depthImage.Invalidate() on the main worker thead.
        ImageToDepthField(depthImage.BitmapSource, pixels); 

But that is not ideal at all.  I’d like to be able to use GetCameraDepthFrameRAW but can’t figure it out.  The closest I’ve come is the following but it doesn’t appear to be correct.  What is the format of the data returned from GetCameraDepthFrameRAW?

short[] depths = new short[width * height];

        void GetDepths(IntPtr source, short[] destination)
        {
            if (source != IntPtr.Zero)
            {
                for (int count = 0; count < destination.Length; count++)
                {
                    destination[count] = Marshal.ReadInt16(source, count * 2);
                }
            }
        } 

What is the resolution of the depth camera?  I’ve read that it’s 320x240 @15fps and 11 bit-depth but that doesn’t appear to be happening.

Ideally there should be a method added to the library that would wrap all this up:

bool GetCameraDepthFrameRAW(IntPtr camera, short[] destination, int timeout); 

Thanks for your help and thanks for the library.

Bryan

Profile
 
 
1 of 3
1
2
3
Next
 
RSS 2.0     Atom Feed
Home ForumsCL Studio LiveAnnouncementsThread