handheld augmented reality

Augmented Reality Anywhere and Anytime   

Projects

   Social Augmented Reality

   Information Presentation

   Real-Time Self Localization

   Structural Modelling

   AR Graphics

   AR Navigation

   Augmented Reality Games

   Past Projects

 

Technology

   Hybrid SLAM

   Panorama SLAM

   Planar SLAM

   Model-Based Tracking

   Marker-Based Tracking

 

Software Libraries

   Studierstube ES

   Studierstube Tracker

   Muddleware

   ARToolkitPlus

 

More

   Videos

   Media/Press

   Team

   Publications

   Collaborations

   Student projects

   FAQ

   Site map

 

The Handheld Augmented Reality
Project is supported by the
following institutions:

Qualcomm

 

Christian Doppler Forschungsgesellschaft

 

Graz University of Technology

 

 


 

Come to ISMAR 2011

ISMAR2011

 

Follow us on twitter

SignPost (2003)

Introduction

To test our PDA-based augmented reality system with an actual application within the Studierstube framework, we choose to implement a client for the Signpost project. The application guides a user through an unknown building by showing a variety of navigation hints, including a wireframe visualization of the building structure superimposed on the video image. An arrow points in the direction to go (see Figure 1), and a 2D overview map is available too (see Figure 2).


Figure 1: SignPost showing the way.
Figure 2: SignPost Overview map. The green spot marks the current position.

How does it work?

General Setup

Figure 3 shows a SignPost setup. The camera takes a picture of the surrounding environment. SignPost uses ARToolKit to analyze the image and calculates the PDA's position relative to the marker. Since this is a CPU intensive task it can be outsourced if a server and a WLAN connection is available. SignPost does not exclusively rely on ARToolKit. The architecture is open to use any kind of mobile tracking technology. Additionally to the ARToolKit support we implemented an ART module on the SignPost server which allows benefiting from the high quality of ART tracking data when available.

 

Figure 3: SignPost setup.

Server/Client Communication

Figure 4 shows how the server/client communication works in the case of using ARToolKit. After the camera image is read it is converted to 32-bits RGBX for easier handling later on (and since this is the format ARToolKit expects the image anyway). If there is no workstation available the image is passed to the local ARToolKit module on the PDA, which calculates the camera position. Since this is a very CPU intensive task the PDA tries to out-source it if a server is available. The raw image as retrieved from the camera takes 225 Kbytes which is far too large to effectively transfer it over a WLAN. Considering ARToolKit does a thresholding anyway we can do this on the PDA to reduce the amount of data from 24 bits to 1 bit per pixel. Since this is still to large we apply a run-length encoding to the image and finally reduce the data size to around 3 Kbytes. The compressed image now fits nicely into a single UDP packet and can now effectively be transferred to the server which decompresses the image and queries ARToolKit to analyze it. The resulting transformation matrix is then sent back to the client.
After that the PDA starts drawing the video background plus the arrow that points the way to go and (if turned on) the wireframe of the building.

 

Figure 4: server/client communication.

BauML

SignPost uses BAUML (Building AUgmented Modelling Language) as its native data file format. BAUML is an XML dialect developed for the original SignPost project. A BAUML database contains information about a building such as the building’s geometry and all the markers' positions and orientations. Each floor is split into rooms which are connected through portals, yielding the aforementioned adjacency graph. Our PC-based SignPost client uses style sheets to translate BAUML files into OpenInventor file structures that can be directly used. On the PDA there is no Open Inventor. Therefore we use Expat to directly read the BAUML XML file and built proprietary run-time structures.
Rendering on a PDA

A PDA normally runs in "GDI-mode". Using the direct video memory access mode is quite complicated because of the different memory layouts on different PDAs. To ease the task we used the 2D library PocketFrog which abstracts the video memory to allow uniform access on all devices.

There is no built in 3D functionality on current PDA although we expect such devices to be available at the end of 2003. There are quite a lot free 3D libraries for the PocketPC platform but all of them are targeted to the game area and did not suite our needs. Porting mesa was no option since it aims at fully implementing the OpenGL API, which results in slow speed.
Instead we started implementing our own subset of OpenGL called Klimt (formerly known as SoftGL), which is targeted for portability and performance. On the PocketPC it runs on top of PocketFrog, but the only thing that is required is access to a memory buffer.

 

Video

A high quality version of this video can be found in the Media/Press section.

 

 

 

 

 

copyright (c) 2014 Graz University of Technology