Update: This article now also in Dutch.

This year Augmented Reality (AR) is out of the box. It has escaped from the universities and is spreading like wild fire. It’s being written about more and more, especially by marketing people, who go for hypes first. This last month I’ve personally heard about at least 4 advertising agencies working on it. On Twitter its mentioned at least 5 times an hour. What types of Augmented Reality are there and where do we stand?

To frame this and differentiate the levels of AR I made the Augmented Reality Hype Cycle. It is inspired by Robert Rice’s article “Is it too early for Augmented Reality?” and expands on the AR Levels which he describes in “Augmented Vision and the Decade of Ubiquity “. The Gartner Hype Cycle is a great method to plot new technologies and applications.

In the Hype Cycle I focus on technology that is already out there and which application is aimed at the individual. Not Industrial. Also I focus on mobile when ever possible.

There are 4 Levels of AR (the larger grey titles):

• Level 0 - Physical World Hyper Linking
• Level 1 - Marker Based AR
• Level 2 - Markerless AR
• Level 3 - Augmented Vision

LEVEL 0 - Physical World Hyper Linking

This is the oldest form and an addition to Rices’s 3 initial Levels of AR. It is a way to link the physical world to the virtual world. It starts with the 1D (UPC) bar code that’s on every product you have. It’s the identifier as registered in the database. The same can be said for 2D codes of which QR-codes are best known. Last part of this group is 2D image recognition. Most used mobile application is the recognition of a company logo or film poster which will then redirect you to a site for more information.

Being the oldest, this form of AR is the most developed. Its also the simplest and doesn’t involve real time rendering and display of graphics. Some may not even call it AR. General adoption of this technology will happen within a year. Most Nokia phones are already outfitted with the software for 1D and 2D bar code scanning. 2D scanning is stuck on the slope of enlightenment and will be surpassed by 2D image recognition in the coming year.

Examples of 1D & 2D Barcodes:

• Compare everywhere
• Snapptell (also image recognition)
• ShopSavvy
• Al in one review video of these three applications
  
• Densowave
• Kaywa
• Shotcode

Examples 2D Image Recognition

• Nokia Point and Find, video
• SKUair / Deam Interactive, video
• Kooaba, video
• Fugu, video

LEVEL 1 - Marker Based AR

This is where the hype is now. Augmented reality based on markers. Its the first step for real AR because it enables real time processing of ‘reality’ through the recognition of markers and subsequently real time rendering and display of graphics on top of this reality.

The first type is the 2D Marker AR which is PC and web cam based. Almost all applications work use an Internet connection to retrieve more information. The marker is the black and white square image you print out and hold in front of your web cam to see a 3D animation.

This is what most people are doing. There are also free toolkits to develop applications and now also Flash based which explains a lot of the popularity.

Next step up is 2D Marker AR with a mobile device. This is tricky-er because it involves a lot or real time processing and a very capable phone. For instance it is not possible on the iPhone because the OS does not allow for real time video processing. You also can’t record video with it as yet. It is possible with a hacked iPhone though.

2D Marker AR is most seen on Windows Mobile devices surprisingly enough. Coolest is 3D recognition, where the mobile recognizes a chair or another physical object and goes on to augmented it.

Level 1 - Marker Based AR is moving from the technology trigger to just before the actual peak of inflated expectations. For now it will not go away and in the coming year PC and Mobile 2D marker based AR will be the IN thing. Where the mobile version will be lagging in comparison to the PC based version due to the lack of computing power and other mobile issues. When the novelty is worn off the actual application value will prove to be thin. Its no real money maker except for marketing and PR applications. On the mobile it will last a bit longer when good games come out. The most promising, 3D recognition, is in the tail of Level 1 - Marker Based AR and is hardly a reality yet.

Examples of 2D Marker AR - PC & Web cam based:

• ARtoolkit
• ARtag
• EasyON
• Total Immersion, video of trading card application
• Metaio, Vuzix demo GDC 2009
• Beyond Reality - Augmented Reality Games, Video PIT Strategy Board Game
• Geisha Tokyo AR-figure, video
• Augmented Reality Toys , video
• PS3 game - The Eye of Judgement

Examples of 2D Marker AR - Mobile:

• Metaio Mobile, video
• Int13, video
• Imagination
• Studierstube Tracker

Examples 3D Object Recognition - Mobile:

• iVisit , video

LEVEL 2 - Markerless AR

Augmented Reality without markers is powerful. Pull out your mobile phone and experience an augmented reality. It can be that simple. Only G1 (the Android - Google phone from T-mobile) owners had the opportunity to experience this with Wikitude from Mobilizy and later ING Wegwijzer. Its based on GPS data and the compass. Because the phone knows where it is (the GPS) and in what direction you are looking (the compass) it can augment reality on the screen correctly.

Level 2 - Markerless AR is halfway the technology trigger and will be nearing the peak of inflated expectations in a year when more phones have the combination of GPS and compass. The Nokia N97 which will be out in the summer will have a compass and will trigger the first non Android Markerless AR applications. It is unknown if the new iPhone will have this capability. It would be HUGH if it did…

Examples of GPS - Compass based AR:

• Mobilizy / Wikitude , video
• SPRXmobile / ING Wegwijzer
• Sixth Sense
• NRU /Last Minute
• Enkin (not heard from since the Android challenge, bought by Google according to rumours)

LEVEL 3 - Augmented Vision

This does not exist yet. As Rice says:

“We must break away from the monitor and display to lightweight transparent wearable displays (in an eyeglasses form factor). Once AR becomes AV, it is immersive. The whole experience immediately changes into something more relevant, contextual, and personal. This is radical and changes everything. As I have said before, this will be the next evolution in media. Print, Radio, Television, Internet, Augmented Reality (well, Vision). L3 must also be mobile massively multi-user, persistent, shared, dynamic, and ubiquitous.”

It will be more then 3 years before anything remotely capable will be available. Until then we will have to make do with research news like that of the University of Washington where they are working on displaying pixels in contact lenses .

In Conclusion

To me it’s daunting to see how far there is to go. Yet knowing where we are now, what is coming and how to frame it enables realistic planning and choices. I hope it helps you. I’ll be enjoying the ride and developing some cool AR initiatives with my SPRX partners.

Thanks to all AR bloggers and twitterers who directly and indirectly helped me put this together. Looking forward to see your feedback and more examples in the comments.

- April 2009

 Retweet This Post

PEW has a new report out and it tells us that the mobile will be the primary device for online access. Also voice-recognition improves and artificial and virtual reality become more embedded in everyday life. THats nice to hear.

It based on a survey of internet leaders, activists and analyst.

Here are the key findings on the survey of experts by the Pew Internet & American Life Project that asked respondents to assess predictions about technology and its roles in the year 2020:

• The mobile device will be the primary connection tool to the internet for most people in the world in 2020.
• The transparency of people and organizations will increase, but that will not necessarily yield more personal integrity, social tolerance, or forgiveness.
• Voice recognition and touch user-interfaces with the internet will be more prevalent and accepted by 2020.
• Those working to enforce intellectual property law and copyright protection will remain in a continuing arms race, with the crackers who will find ways to copy and share content without payment.
• The divisions between personal time and work time and between physical and virtual reality will be further erased for everyone who is connected, and the results will be mixed in their impact on basic social relations.
• Next-generation engineering of the network to improve the current internet architecture is more likely than an effort to rebuild the architecture from scratch.

View PDF of Report

 Retweet This Post

I have the G1 now for a week. It is a good phone which is still very new. This means that it doesn’t have the 10.000 apps like the iphone. Yet it already does some amazing things apart from seamless Gmail, Google Calander & Talk integration. The application that I keep going back to is Wikitude.

Wikitude is projects wikipedia location data on your screen on the camera input. You hold the camera level in front of you and see the world through its lens. Wikitude projects wiki data over it. As you turn the phone around you see the different points of interested projected in the screen. It a simple idea but very powerful. Here is the developer demoing their product:

My next tourist guide is in my phone. When I arrive I just hold my phone up and see what’s up, superimposed on the camera image.

It a good start. Its in the same category as Point and Find from Nokia. Here is Scobeizer checking it out. Point and Find works with image recognition instead of looking up data based on a GPS position like Wikitude. Point and Find is a QR-code like application. It scans an image and the get an action from it. Yet it can ’scan’ any image, even your house, and then go to your private website.

Both are examples of combining the real world with the virtual world. It will make the webbrowser based internet look very old.

 Retweet This Post

Just read a post from Helen Keegan (who joined the MoMo London team..yeahh) about the relevant simple approach of the current wave of Location Based Services. I totally agree with her that we are still not very creative nor intelligent with the context information we currently have to our disposal. As I commented on her blog I believe that having the complete context in place is necessary to offer relevant contextual services. The first step after having the context information in place is being great in pattern recognition.  Spotting the Nodal Points(as Jyri calls them).

Well what is the complete context? Here is my list:

• Taskrelated context: What are you doing?
• Social context: With whom are you?
• Spatio-Temporal context: Where are you? What time is it?
• Physiological context: Heartrate, movement, temparature(see post about the pill below)
• Environmental context: In what environment are you?
• Mental context: How are you feeling?

These context domains are all about the NOW! So not ” what are you plans tomorrow?” or “what is your favourite music?”.  These are all legitimate profiling questions and offer great insight about a person BUT I question if in a mobile services context(hehe) they are relevant..

Some questions I am still puzzling with is what to do with informaton about your history or the complete context of the people which are important to you. Would you need to have this information to become more relavent to people in specific situations?

Curious about your thoughts..

P.S. Image is from Helen’s blog.. Just had to post it. LOVE IT!!

 Retweet This Post