Although my entry is not eligible for winning (with me being a part of The CodeLabs) here’s my idea:

I would add 4 cameras on my car; two would be forward-looking, two rear-looking. I would constantly run those cameras with a small car-pc that would be on while I was driving. I would then have software that (a) constantly records video and (b) runs analysis algorithms on the video to extract license plate numbers.

Why? The video part I’d use to make time-lapse records of trips. The license-plate number bit, I do not have exact use for, but it’s a nice way to leverage the technology.

Another cool idea would be to have a camera looking at the driver, and run a facial-recognition algorithm. If the software detects that that the driver closed his eyes for more than 2 seconds, it would sound an alarm to wake them up.

Now it’s your turn. We know you are all creative people, so let’s hear what you would do with 4 eyes.

Object:Touchless Multipoint Sytem
I would like to use 4 cameras to build a Touchless Multipoint System.
In General, Multipoint Touch system use a lot of stuffs like projecter, cameras, LED and acrylic board ...etc.
But in Touchless Multipoint System, we only need to use four cameras to simulate the whole function of Multipoint Touch Sytem.

My Idea:
a 360° Pano-camera, like this one: http://www.vrhotwires.com/InexpensivePanoramicVideo.html
or this: http://photocreations.ca/panhead/index.html .
It s only a rough idea but i will make some more thoughts about the camera rig and on the software.
There should be some stitching tool and some undistortion posibility to get a clean panorama with different objectives. The best would be to develop this in openFrameworks.
Perhaps we could develop an opensource streetview camera system like this one (but only a little bit cheaper) :ephel  

“A Multi-surface , Multi-touch gaming cockpit.”
tw0 Camera for main-forward display
one camera for touch control / gesture recognition of right hand
one camera for touch control / gesture recognition of left hand

Object:Cheap Motion Capture System
I would like to build two kinds of cheap motion capture system.

Type 1:Using the blob detection and camera calibration to detect blobs to calculate the 3D axes.
Type 2:Using gesture recognition to keep track the whole body to detect head, torso and limbs etc .

Have a medium to large display projected onto glass (can either be MT enabled or not). This table is surrounded by four lcd MT screens (one on each side). These screens are tilted so that when you’re standing at any one you cannot see what is on the other screens.

This design could be used as a multiplayer games machine. Each screen being a different players view and the screen in the middle showing a map or infill of the games middle section - similar to the Nintendo DS dual screen.

It could also be used as an information directory in a shopping centre. Each of the four screens allowing four different people to find information at the same time. And showing a logo or map in the middle screen.

Although having the middle screen MT enabled will require a total of five cameras, it will increase the functionality.

I have also included a couple badly drawn CAD concept drawings.

I’ll probably try to remove the IR filter on one and the other 3 to try it again after I broke the first.  :D

or…

Make me a 360 degrees nightvision helmet that keeps tracks of my steps and help me walk within a dark room without hitting a wall.

I’d definitely use the four cameras to build an all optical multi-touch system. I’d position the cameras in such a way to maximize their view of the touch surface. Then I’d stitch the multiple images together and create a map of the touch surface that contains only the position of the fingers. This system, being optical, would be in a unique place where it could recognize fiduciary tracker markings as well as gestures above the touch surface.

I would consider coming up with the idea of integrating 4 cameras into one compact “Smart Camera” solution capable of producing uncompressed 1280*960 (more then 720p HD) @60fps and competing with Sony’s XCI smart camera series (http://pro.sony.com/bbsc/ssr/cat-industrialcameras/resource.solutions.bbsccms-assets-mkt-indauto-Solutions-smartcameras.shtml).
For that I would consider some small XP based system, having 4USBs, 4PS3Eyes properly alligned through mechanical and/or software means eitheer to produce 2*2 video input matrix (and 1280*960 output resolution) and/or 4 sequential output streams of 640*480 (VGA resolution, if properly alligned at the same axes).
Each PS3Eye will be “sitting” on it’s own USB transport, thus each proivding 60FPS parallel processing, so parallel result would be the same 60FPS at 1280*960, however sequentially or at VGA resolution result would be 60*4=240FPS (if USB clocks would be properly shifted).
Output port could be either directly monitors/digital output and/or analog output.
For digital output processing: uncompressed GigE, compressed => parallel/COM
Input: parallel/COM
Possibly Support for Sony’s VISCA interface

In 1280*960 mode it should allow windowing into one of 4 available subwindows, as well as downshift to 320*240 in each of 4 subwindows with correspondent increase in FPS.

In addition to HD and/or high FPS video, system might provide ability to preprocess 16 channels of Audio (8*2rows for example).
Most important point is the following:
assuming price of computer is less then $1000
price of 4PS3Eyes <= $200
price of OS and software <= $500
========================
full system cost => $1700

Forget about technical aspects (as obviously our system is much more powerfull) and compare to XCISX100/XP
http://pro.sony.com/bbsc/ssr/cat-industrialcameras/cat-smart/product-XCISX100/XP/ sold for 2.5times more ($4,508.00)

I would like to use multiple camera’s to make interaction with VR environments more immersive.  Most VR users are distracted from the immersive aspects of VR because of trackers stuck on their heads and hands. 

An interface that combines a multiouch surface and and gesture recognition through a volume is what I would like to try.  Cameras are needed for both parts of the interface.  Optical gesture recognition suffers from occlusion.  The use of multiple cameras will help to overcome this shortcoming.  An affordable multi-touch surface model has been developed using commodity cameras and near infra-red lighting. 

Then to go one step further, a camera can be used for head tracking.  This will remove another hardware distraction and should help users to be more immersed in the designer’s virtual environment.

Your give away would jump start this exciting project.

cnso

Hi all!

I’m starting my multitouch project now. I’m really not a programming guy, so no code to share as I had this idea when I saw the contest.

I could use the 4 cameras in a multitouch table, where I’d use 2 cameras to “see” the screen in infra-red and 2 to “see” in visible wavelength.

The IR cameras would capture the touches, fiducials or some IR marks. The image processing would detect the gestures, motion and the IR marks.

The visible wavelength cameras would make optical recognition of objects and users. These cameras could detect the face of the users and login to the desktop. The cameras could detect the user expressions and adapt to it (think of games or learning software). Also could detect objects put over the table and use them, like detect a cellphone and try a bluetooth connection. Also make OCR of documents.

Not sure if this is pratical as I have not tested. Problems will occur because of environment light.

Anyway, will give it a shot. This could be very interesting for games/entertainment and education (specially for the disabled).

Cheers,
Nuno

Get rid of IR reflections / hotspots in MT Rear DI setups.

When using one powerful IR source centered
at the surface, there will be the issue with an annoying reflection
where blob sensing will fail when using background suppression.

I thought of overlaping camera views where the hotspot
areas are masked and replaced by views of other cameras from other view angles.

the composed camera view would be reflection / hotspot free.

i attached some sketches.

I’m planning on getting the new PlayStation Move (aka Arc, Gem, Sphere, Motion Controller) controller working on the PC (like I I did with the Wii Remote). PlayStation Move is designed to work with the PlayStation Eye for fast 6DOF tracking. But I don’t want to be tied to my computer and forced to face forwards the whole time, since I have a VR920 Head Mounted Display. So I reckon I need four PlayStation Eye cameras set up around the room to give myself maximum tracking area.

On the other hand, I might also like to try augmented reality. But the VR920 is a non-see-through stereoscopic HMD, so I would need two identical cameras, one for each eye. I would then use the other two cameras for tracking my hands and body.

I’ve also been working on designing my own Virtual Reality library for the PC, but I want to support full body tracking over a large area, and it would be really cool to try out some configurations using four PlayStation Eyes around the environment.

Of course if I want to track multiple body parts over a really wide area, I wonder if it would be cool to put fiducial markers (white and black squares with patterns) around the environment and just mount a camera to each arm and a camera to each leg. Normally people just use one camera for their head, but where’s the fun in only having head tracking? I want people to be able to kick and punch, or even dance, in a VR game.

hi @all

ive learned a lot about MT in the past 4 months..thanks to the NUI-community!

and after helping a friend to build his own (ftir) MT screen for his master-thesis, i started my own MT project. even though im not a programmer myself, i was able to to take part in the programming part and this helped a lot to understand the technology as a multidirectional interaction instrument whose theoretical limits are the hardware specs but still the showns solutions (software and hardware) are in a early stage. except two setups all MT system / configurations i saw (at least 20 tables live) - either tracking accuracy or latency were suboptimal…especially object/fidicual tracking. same for changing light conditions - most of the tables that performed “ok”, were in enviroments with low light but as soon there is e.g. ceiling light (like the ones i saw @ CEBIT) the overall performance falls rapidly and touch errors occur more often.

therefore i would use the quadpack + 4 ps3 cams for a new approach: explained here

good luck @all!!

Our proposal is as follows:

We have been looking at a way to create DSI MT devices in the 40+ inch range which are thin enough to hang on the wall.

Essentially, we are looking at dividing the screen into 4 areas which will each be monitored by a single camera with a fisheye lens:

Since the aspect ratio of the cameras is 4:3, we would get the desired overlap for tracking input across the camera boundries making seamless dragging possible.

The surface would be front projected using a short throw projector from an arm placed overhead (similar to how an overhead projector lens appears).  This gives us nearly shadowless tracking for a front projected table.

The only limits on this kind of setup are the focal range of the cameras and the distortion caused by adding a fish eye lens.

We estimate we can get a front projected, 4 camera surface down to about 25cm thick which is suitable to be able to hang the system from a wall.

This type of setup could also be adapted for use with a large LCD with the same results.

Layering is as follows:

With proper software correction of fisheye distortion, we could probablly make the system even thinner.

We are also looking at providing interconnecting cabling to allow us to chain these displays together into groups.  So we could essentially use the cl-lab hexa-packs to enable larger displays to be created out of multiple smaller displays:

We are also looking at creating a single long (~10m) display using the same multiple camera method mentioned above.