Handheld Augmented Reality

Augmented Reality on mobile phones : Exact location based information anytime and anyplace

The Christian Doppler Laboratory for Handheld Augmented Reality at Graz University of Technology was founded in 2008. Its mission is research and development of Augmented Reality (AR) technology for mobile phones. AR superimposes computer graphics on top of a live video image, for the display of location and situation dependent information. The display operates in real time, and every piece of information related to an exactly measured point in the world - the desired accuracy is about 1cm and 1 degree. This quality of location based service significantly differs from previous mobile services, which only use coarse GPS measurements.

AR is primarily a method for human-computer interaction. Using mobile devices, such as cell phones, users can interact with digital information anywhere and mostly without tedious input, just by using point & shoot on the mobile phone viewfinder. For example, rather than typing a web address, a user can point to the object of interest to retrieve related information. Rather than bringing the interaction to the computer, it can happen at the task location. Every location and every object can potentially bear information.

This allows a new way of handling information retrieval. The AR device becomes a personal assistant, which can be used in a variety of situations. For example, pedestrian navigation required routes to be displayed in 3D, for example indoors in a multi-floor building. Tourist can find historical artefacts more easily and do no longer need to consult a guidebook for explanations. Shoppers can see descriptions of goods and price comparisons directly in the shopping window. Other electronic devices can be explained or even remote operated using the AR cell phone. Finally, one can place virtual sticky notes to remind oneself or to provide hints for others.

A similarly potent market are computer games. The first high quality AR games are already on the market, for example Eye of Judgement (Playstation 3). The Christian Doppler Lab provided consulting for another successful game, Invizimals (Playstation Portable). In marketing, we are going to see games and other entertainment financed by advertising. For example, soft drink maker Red Bull has recently launched a magazine Red Bulletin, which has pictures that come to live in "Harry Potter" style using AR. The team at Graz has also started working on a magic mirror, which allows a customer in a fashion store to virtually try out items of clothing. Other areas of application are industry, medicine and architecture.

The most important pieces of an AR system are:

(1) Tracking of location, in 3D and in real time

(2) Display of video blended with computer graphics

(3) Visualization techniques determining what should be displayed, and in which form

(4) A database of virtual and real objects to interact with

The accuracy requirements of tracking are currently the most significant challenge. GPS and compass are not precise enough. Image analysis of the live video is the only feasible approach, however the computational requirements often exceed what can be done in real time on a phone. A good approach is the sensor fusion of camera, GPS and inertial sensors. Another issue is the creation of large image databases, which allow the identification of a place. However, it is sometimes feasible to let the phone learn the environment online.

The display of AR must address the question which information to display, and in which form. Too much information can easily lead to image clutter, in particular on a small phone display. For realistic scenes in AR games, real and virtual objects should be seamless and cast shadows on each other. This requires photometric reconstruction of the lighting and high dynamic range image techniques.

And where do we get the information that is shown? Databases with geo-referenced information are very popular, but do not store the information in a form suitable for AR. A promising direction are geo-mashups, which are often used for online map services. However, AR required more detailed and precise data than conventionally available. The raw data will be drawn from commercial providers, but also crowd-sourced to the phone users. It is very likely that mobile AR is the next revolution after Web 2.0 social networking.

Augmented Reality auf dem Mobiltelefon Exakte ortbezogene Informationen überall und jederzeit

Das Christian Doppler Labor für Handheld Augmented Reality an der TU Graz wurde 2008 gegründet und befasst sich mit der Erforschung und Entwicklung von Augmented Reality-Technologie für Mobiltelefone. Unter Augmented Reality (AR) versteht man die Überlagerung von Computergrafik in einem Live-Videobild, um erklärende, auf Ort und Situation bezogene Inhalte anzuzeigen. Die Darstellung erfolgt in Echtzeit, und jede Information bezieht sich auf einen genau spezifizierten Punkt in der Welt - die angestrebte Genauigkeit lässt sich grob mit 1cm und 1 Grad abschatzen. Diese Art eines ortbezogenen Informationsdienstes unterscheidet sich daher deutlich von bisherigen mobilen Lösungen, die zumeist nur auf groben GPS-Angaben basieren.

AR ist in erster Linie eine neue Form der Mensch-Maschine-Interaktion, die mittels mobiler Geräte wie Handys

Augmented Reality Anywhere and Anytime

Handheld Augmented Reality

Augmented Reality on mobile phones : Exact location based information anytime and anyplace

Augmented Reality auf dem Mobiltelefon Exakte ortbezogene Informationen überall und jederzeit