First, take a look at the demo. The cool part of the demo starts at about 50 seconds in and goes a little past 3 minutes. Watch at least that and you’ll be up to speed well enough to understand the following. It does help if you’re also familiar with Qik and Photosynth

Reading some of the comments on TechChrunch’s post, I found several people calling this “a solution without a problem” so I figured I’d give some potential “problems” this could solve. You just have to think about how this technology can be combined with other technologies we already have available, as well as how this kind of stitching technology can be evolved. Two game changing applications spring to mind immediately:

Sports and Live Event Broadcasting

Just think outside the box a little bit. Sure, they’re using camera phones in the demo, but that doesn’t mean you can’t scale up to using 1080p high definition video sources. Or 2K, 4K and even higher resolution sources for that matter. At that point you’d need extremely heavy computing power to do the processing, but it isn’t like we aren’t seeing the silicon showing up to provide the number crunching horsepower. Newspapers think they have it bad because of the internet, just think about the implications of every football stadium installing arrays of high definition cameras around the field. Traditional broadcast single-feed TV won’t be able to compete…

Subscription satellite TV services already allow viewers to jump between different angles of a play. How much would the sports fanatics pay to access a real-time stream that would allow them to “fly” the camera to any vantage point around the field that they wish to view? A custom camera viewpoint for each viewer of the stream, always getting to see what they want to see instead of what the guy in the production studio wants them to see.

And eventually the three dimension mapping abilities of Photosynth will be applicable in real-time as well. This would not only allow you to pan the camera around the field, but also through it. Want to see what a play looks like standing 20 feet from the sideline, thirty feet in the air, on the 50 yard line while zoomed in on the wide receiver? Not a problem. How about watching the play while tracking over the head of the quarterback? Not a problem.

Emergency Response

How would the outcome of disasters like September 11th and Katrina have differed if we had technology such as this in place? Well, if we had enough people on site with high resolution video phones, these citizens could have been a vital part of information gathering for first responders. Granted, we don’t need bystanders getting in the way and hindering rescue efforts. But, if executed properly, these cameras could provide a logistics team with a real-time three dimension view of a disaster site. This kind of information would allow for an unprecedented level of organization among emergency response crews.

And again, this does not have to rely on people with camera phones. A team in a van filled with monitoring equipment could harness the streams coming in from any available networked camera equipment. Any civilians in need of assistance could be identified so their position and any other vital details can be radioed to the closest responder able to provide assistance. Extra sets of eyes on the situation could also identify threats that may not be immediately visible to personnel “on the ground” such as overhead debris that is likely to fall.

Then there is also a whole new level of information that could be made available to emergency personnel when you combine this technology with augmented reality heads up displays, HUDs:

• Overlay data about the locations of all civilians in the area who need assistance and provide information about their injuries, vitals and triage priority.
• Pass various instructions along from off-site support staff such as doctors. This would allow personnel in the field to perform treatments they may not have prior training to perform in situations where the civilian won’t survive without immediate treatment.
• Off-site civil engineers would be able to provide live structural consults to help avoid unnecessary structure collapses and identify problem areas.

However, I am getting a bit off topic at this point, these were just a couple of related ideas that popped up that I wanted to suggest.

So hopefully this gives the naysayers something to chew on. I’ve learned to never question a new technology’s worth. Solutions with out problems don’t exist. We just haven’t found the problem yet.

For those who aren’t familiar with Qik, definitely check it out. Think live-action YouTube—you never know what you’re going to find! And be sure to read TechCrunch’s post about the demonstration.

To pull off such a feat, augmented reality comes to the rescue! Augmented reality, or AR, is a blending of “true” reality and virtual reality. Generally AR systems use computers to track the motion of objects and use that data to overlay useful information into our field of vision–such as adding virtual objects to the real world. Most applications to date have involved camera setups which track 2D “barcodes” to achieve motion tracking.

A great example of this can be seen in the YouTube video I have embedded below. It shows a demo of such barcode tracking being used to turn everyday objects into fantastical imagery from Harry Potter. A baseball cap turns into the Sorting Hat, a stick turns into a sword, a piece of paper becomes The Marauder’s Map and a cardboard box holds a Golden Snitch.

One hurdle to putting augmented reality into general use is getting such functionality off a computer monitor and placed directly in front of our eyes. Head mounted displays with optics to overlay a screen onto the real world are the common solution. At this point in time, however, commercially available HMDs are bulky and generally in need of wired power sources. But that is changing quickly as more companies put R&D into augmented reality technologies.

So how does this relate to interior design and painting your living room walls? Take a look at the diagram below:

So here we have a gentleman who is ecstatic (if you could see his face, he’s got a huge grin) that he doesn’t have to get covered in paint in order to change the look of his wall. The funny looking symbols on the four corners of the wall are what an AR system would use to track the location of the wall. This would then be used to overlay whatever color, pattern or texture our happy ex-painter wishes. The next diagram shows that he settled on a Victorian wallpaper. Note that the tracking symbols would not be seen by the user of the system, I have simply left them visible as a reference point.

As I mentioned earlier, the easiest way of pulling this off with today’s technology is with a head mounted display. So a major limitation would be that only people who are wearing a HMD would be able to see the texture on the wall. In addition to the HMD, the motion tracking system would also need to be head mounted so the wall doesn’t require a camera to track the user’s head. But as technology proceeds forward, the limitations will disappear. People like Johnny Chung Lee are showing us how to use easily accessed devices like the Nintendo Wii remote to do high quality head tracking. Nanotechnology is driving the size of all technology smaller and smaller so that HMDs will turn into contact lens displays and allow batteries to supply the necessary power to make the systems mobile.

This demo hits home for me, as I am a proponent of simulated reality technologies. The graphics hardware of tomorrow combined with advanced rendering software like CryENGINE 2 get us one step closer to simulated reality becoming…reality. The demo shows hundreds of thousands of (apparently rubber) teapots bouncing down a modeled street among many cars as bystanders watch wondering what the hell is going on. Take a peek and drool as you realize that video on par with Pixar movies is being rendered in real time.