Code: Kinect with OpenCV 2.2 in C++
Posted: 03 February 2011 09:45 PM   [ Ignore ]
Jr. Member
RankRank
Total Posts:  47
Joined  2010-02-15

A lot of people on this forum and around these here interwebs have been asking how to get the kinect working with OpenCV in C++. Here is the basic code to display the RGB image and the depth image as both a colormap and as greyscale. This is for OpenCV 2.2 which is the latest version. If you want to use an earlier version your includes will be different.

NOTE: The calls to StopNUICamera(), DestroyNUIMotor(motor), and DestroyNUICamera() are commented out because they still throw an error when called. It doesn’t seem to be a problem though.

// Jim Vaughn
// Displays the color and depth information using OpenCV

#include "stdafx.h"

#include <opencv/cv.h>
#include <opencv/cxcore.h>
#include <opencv/highgui.h>

#include <CLNUIDevice.h>


using namespace std;

void displayKinectImage();

int _tmain(int argc_TCHARargv[])
{
    displayKinectImage
();
}

void displayKinectImage
() {
    PDWORD rgb32_data 
= (PDWORDmalloc(640*480*4);
    
PDWORD depth32_data = (PDWORDmalloc(640*480*4);

    
CLNUICamera cam CreateNUICamera(GetNUIDeviceSerial(0));
    
CLNUIMotor motor CreateNUIMotor(GetNUIDeviceSerial(0));

    
StartNUICamera(cam);

    
cvNamedWindow("Image"CV_WINDOW_AUTOSIZE);
    
cvNamedWindow("Depth"CV_WINDOW_AUTOSIZE);
    
cvNamedWindow("Grey"CV_WINDOW_AUTOSIZE);

    
IplImage *rgb32 cvCreateImageHeader(cvSize(640480), 84);
    
IplImage *grey;
    
IplImage *depth32 cvCreateImageHeader(cvSize(640480), 84);;

    do 
{
        GetNUICameraColorFrameRGB32
(camrgb32_data);
        
GetNUICameraDepthFrameRGB32(camdepth32_data);

        
rgb32 cvCreateImageHeader(cvSize(640,480), 84);
        
grey cvCreateImage(cvSize(640480), 81);
        
depth32 cvCreateImageHeader(cvSize(640480), 84);

        
cvSetData(rgb32rgb32_datargb32->widthStep);
        
cvSetData(depth32depth32_datadepth32->widthStep);

        
// Convert RGB32 to greyscale
        
cvCvtColor(depth32greyCV_RGB2GRAY);

        
cvShowImage("Image"rgb32);
        
cvShowImage("Grey"grey);
        
cvShowImage("Depth"depth32);

        
cvReleaseImageHeader(&rgb32;);
        
cvReleaseImage(&grey;);
        
cvReleaseImageHeader(&depth32;);

        
cvWaitKey(1);

    
while (!GetAsyncKeyState(0x50));


    
free(rgb32_data);
    
free(depth32_data);

    
//StopNUICamera(cam);

    //DestroyNUIMotor(motor);

    //DestroyNUICamera(cam);

Profile
 
 
Posted: 04 February 2011 12:26 PM   [ Ignore ]   [ # 1 ]
New Member
Rank
Total Posts:  2
Joined  2011-02-04

Nice work! I was trying to do the same and having no success until now.


Tks thesmileman!

Profile
 
 
Posted: 04 February 2011 12:39 PM   [ Ignore ]   [ # 2 ]
Jr. Member
RankRank
Total Posts:  47
Joined  2010-02-15
ruinalmeida - 04 February 2011 12:26 PM

Nice work! I was trying to do the same and having no success until now.


Tks thesmileman!

No problem. I saw a lot of people were having trouble with this and I had it done a while back so I thought I would help.

Profile
 
 
Posted: 10 February 2011 03:13 AM   [ Ignore ]   [ # 3 ]
New Member
Rank
Total Posts:  2
Joined  2011-02-10

Hey im currently using VS2010 and openCV 2.1

I tried just adding alle the libearies and running a “hello world” without problems.

However, as soon as I add your code I firstly get som Syntax errors as you have some strange semicolons wink like cvReleaseImageHeader(&rgb32;);

however, fixing those and it runs, EXCEPT it doesnt raspberry

It simply says all the sudden that highgui210d.dll doesnt exists, or it simply choses another at random it seems, sometimes its cxcore sometimes cv sometimes highgui, and I have absolutely NO idea whats wrong and what to do about it. Any suggestions would be very appriciated :D

EDIT: NEVER MIND! :D for ppl who run into same problem, I just had to copy the .dll files i needed to the projects Debug folder

Profile
 
 
Posted: 10 February 2011 08:39 AM   [ Ignore ]   [ # 4 ]
Jr. Member
RankRank
Total Posts:  47
Joined  2010-02-15
Olle - 10 February 2011 03:13 AM

Hey im currently using VS2010 and openCV 2.1

I tried just adding alle the libearies and running a “hello world” without problems.

However, as soon as I add your code I firstly get som Syntax errors as you have some strange semicolons wink like cvReleaseImageHeader(&rgb32;);

however, fixing those and it runs, EXCEPT it doesnt raspberry

It simply says all the sudden that highgui210d.dll doesnt exists, or it simply choses another at random it seems, sometimes its cxcore sometimes cv sometimes highgui, and I have absolutely NO idea whats wrong and what to do about it. Any suggestions would be very appriciated :D

EDIT: NEVER MIND! :D for ppl who run into same problem, I just had to copy the .dll files i needed to the projects Debug folder

Apparently the site is adding those funny “;” automatically because I just tried to remove them. I was wondering how they got there. I guess it is mistaking the “&” for an escape character.


Also you includes are drastically different between OpenCV 2.1 and OpenCV 2.2

Profile
 
 
Posted: 16 February 2011 06:28 AM   [ Ignore ]   [ # 5 ]
New Member
Rank
Total Posts:  2
Joined  2011-02-04

I ‘ve managed to integrate OpenCV and obtained both RGB camera and depth sensor calibration parameters and matrices, using the procedures at http://nicolas.burrus.name/index.php/Research/KinectRgbDemoV4?from=Research.KinectRgbDemo

Now I’m trying to consolidate the information of both and make a 3D reconstruction using the functions available at
http://opencv.willowgarage.com/documentation/python/calib3d_camera_calibration_and_3d_reconstruction.html # stereorectify

I do not know if anyone has managed to do this and wants to share experience.

Tks,

Rui

Profile
 
 
Posted: 17 February 2011 09:21 PM   [ Ignore ]   [ # 6 ]
Jr. Member
RankRank
Total Posts:  47
Joined  2010-02-15
ruinalmeida - 16 February 2011 06:28 AM

I ‘ve managed to integrate OpenCV and obtained both RGB camera and depth sensor calibration parameters and matrices, using the procedures at http://nicolas.burrus.name/index.php/Research/KinectRgbDemoV4?from=Research.KinectRgbDemo

Now I’m trying to consolidate the information of both and make a 3D reconstruction using the functions available at
http://opencv.willowgarage.com/documentation/python/calib3d_camera_calibration_and_3d_reconstruction.html # stereorectify

I do not know if anyone has managed to do this and wants to share experience.

Tks,

Rui

These guys use OpenCV and some other tools to do what you are talking about. I have been working on a new solution what I believe is a far superior. My first solution was very similar to these guys and it also looks similar to theirs which is to say not that great. But I am starting to get results that are much much better even in the early stages of research. Here is the solution I mentioned for the ROS guys:

http://www.ros.org/wiki/openni/Contests/ROS 3D/RGBD-6D-SLAM

Profile
 
 
Posted: 12 April 2011 12:49 AM   [ Ignore ]   [ # 7 ]
New Member
Rank
Total Posts:  2
Joined  2011-04-11

How can I do an equivalent thing in Emgu CV?
I used the CLNUDevice.cs class form the sample programs.
but the CLNUIDevice. GetCameraColorFrameRGB32() always returns a IntPtr that shows a blank when i cast it to Emgu.CV.Image.

What am I doing wrong?

Profile
 
 
Posted: 03 May 2011 06:41 AM   [ Ignore ]   [ # 8 ]
New Member
Rank
Total Posts:  1
Joined  2011-05-03

AAAh thanks for this!!! I really nedded a clean explanation but now I ran into a problem while trying to compile with OPen CV2.1 and VS2008

.\Main.cpp(14) : error C2061: syntax error : identifier ‘_TCHAR’


any ideas?? thanks!!!

also I have no idea where to find “stdafx.h”, my system cannot find it. Is it essential?


EDIT: Nevermind solved it! I deleted the _tmain function and made void displayKinectImage() the new main, and now he does everything! thank you so much for this!


Only question I have now is how do I let the Kinect capture rawframe data like we can in OpenCV with a normal webcam ..?

Profile
 
 
Posted: 20 August 2011 10:14 AM   [ Ignore ]   [ # 9 ]
New Member
Rank
Total Posts:  1
Joined  2011-08-20

I have this error in my VS2010
error LNK2019: unresolved external symbol __imp__GetNUICameraDepthFrameRGB32 referenced in function “void __cdecl displayKinectImage(void)” (?displayKinectImage@@YAXXZ)
Can you tell me which file i need to add in my linker or my compiler options???

Profile
 
 
 
 


RSS 2.0     Atom Feed