Real Head-Tracking 3D Desktop

[klange]

So you've seen those headtracking videos with the box and the targets and what not, but they never have a real interface. They're all just proof-of-concepts and whatnot - they aren't real usable products.

Until now! (or actually a few months ago, technically, but let's not get worked up on the details)

askKFSe3aLk

The first truly useful application of head tracking outside of 3d-modeling, and tailored for regular users. Yes, I made this with the help of a "colleague". Yes, I'm promoting my own video. Shut up.

After about a year total working on this from the point where it tracked IR lights with a Wiimote (it still supports that, but I sure as heck don't, and the interface is broken anyway), we've finally reached a stable point in webcam tracking.

Again, as I even show in the video, there is absolutely no extra hardware used in this: Just a webcam on a netbook with a Linux desktop.

Basically, it works the same way as those other videos, but instead of just applying a glFrustrum to a generic OpenGL environment, we're doing it to a full-featured 3d desktop engine. Windows are then stacked on the Z-axis so that you can look around them (assuming they aren't full screen). Windows can be moved back and forth with a set of keybindings (including moving them off the screen towards your face, but that doesn't work well without some stereoscopy, which we don't have right now).

And so as not to make this all about me: Let's discuss the future of such technologies.

[thehoodedsmack]

Let's discuss the future of such technologies.

Let's. What exactly is this technology good for? I can't think of a reason why I'd want a 3D desktop, but is there anything else practical you can do with this?

[SMASH]

Can I has program?

[klange]

Let's. What exactly is this technology good for? I can't think of a reason why I'd want a 3D desktop, but is there anything else practical you can do with this?
The usual example is something quite common here - 3D modeling. It makes seeing a model a lot more realistic. (Of note: You can view 3d models in Compiz with the "Cube Model" plugin, which works fine with this)

@SMASH: It's a plugin for the Linux compositing system, Compiz. So, yes, you can, if you really want it.

[SMASH]

Ah, linux, ew... nothing like this for the PC yet?

[Maniac]

http://www.youtube.com/watch?v=lcbiX2UnKyI&feature=PlayList&p=DE900D1F4F7E00A2&playnext=1&index=1

[itszutak]

http://www.youtube.com/watch?v=lcbiX2UnKyI&feature=PlayList&p=DE900D1F4F7E00A2&playnext=1&index=1
The point of the OP is that his does NOT require a wii or any other IR headset/receiver. This is purely through tracking the eyes.

[Maniac]

Neat.

[Llama Juice]

that made me dizzy...

[klange]

http://www.youtube.com/watch?v=lcbiX2UnKyI&feature=PlayList&p=DE900D1F4F7E00A2&playnext=1&index=1
That's where our original Wiimote tracking system came from - his sample code.

@Llama: Do you often experience motion sickness from things like this? It could be a medical condition. Unfortunate, you may not be able to enjoy the advantages of this sort of thing.

[Limited]

On a laptop? Useless.

I dont really see the need of it, its definitely cool and alot of great work Admiral, I just dont see it being useful.

[Maniac]

Nice one man, the technology and games that comes from this will be something to look forward to. Never thought about it like being able to look round a 3d model before, with some type of adjustments im sure that could result in fully rotating models just by moving your head. :)
edit: just read that and i guess you would not actually be rotating the model , you would be doing something else, ill figure it out after i have this stroke from trying to think about that.

[ExAm]

Nice one man, the technology and games that comes from this will be something to look forward to. Never thought about it like being able to look round a 3d model before, with some type of adjustments im sure that could result in fully rotating models just by moving your head. :)
edit: just read that and i guess you would not actually be rotating the model , you would be doing something else, ill figure it out after i have this stroke from trying to think about that.
TrackIR (another combination of software and infrared gear) can already be used with several games that are already on the market :/
This is nothing new.

[klange]

TrackIR (another combination of software and infrared gear) can already be used with several games that are already on the market :/
This is nothing new.
TrackIR is great and all, but it is tracking infrared sources. FaceAPI (a professional face tracking system, much more powerful than the OpenCV sample code I'm using) would be far superior (I've seen some amazing videos, wish it were totaly free and open-source...)


On a laptop? Useless.

I dont really see the need of it, its definitely cool and alot of great work Admiral, I just dont see it being useful.I don't quite see why it would be so useless on a laptop (this is probably the only form that would be useful on a laptop, as most people don't do as much gaming). Laptops also move more, so the effect would serve more of a purpose.

But thanks anyway ;)


Ah, linux, ew... nothing like this for the PC yet?To my knowledge there is no actual effort to do this for Windows yet. There was a partial attempt a while back that got some movement out of windows (on XP) but that was about it. One of the many Comp-clones may start working on it, keep looking.

[Limited]

I don't quite see why it would be so useless on a laptop (this is probably the only form that would be useful on a laptop, as most people don't do as much gaming). Laptops also move more, so the effect would serve more of a purpose.
Most of the time laptops are used as the name describes, on someones lap, therefore you cant really move your head enough to really use the technology to the fullest.

Also you need to be looking at the screen dead on with most laptops, otherwise screen gets dark and becomes invisible.

[Heathen]

Most of the time laptops are used as the name describes, on someones lap, therefore you cant really move your head enough to really use the technology to the fullest.

Also you need to be looking at the screen dead on with most laptops, otherwise screen gets dark and becomes invisible.
LOLOL, none that I have ever used...

[Huero]

I'd like to see this combined with something like BumpTop. Can't see your programs? Just turn your head, they're on the wall! :D