Augmented Reality Anywhere and Anytime
The Handheld Augmented Reality
Come to ISMAR 2011
To test our PDA-based augmented reality system with an actual application within the Studierstube framework, we choose to implement a client for the Signpost project. The application guides a user through an unknown building by showing a variety of navigation hints, including a wireframe visualization of the building structure superimposed on the video image. An arrow points in the direction to go (see Figure 1), and a 2D overview map is available too (see Figure 2).
How does it work?
Figure 3 shows a SignPost setup. The camera takes a picture of the surrounding environment. SignPost uses ARToolKit to analyze the image and calculates the PDA's position relative to the marker. Since this is a CPU intensive task it can be outsourced if a server and a WLAN connection is available. SignPost does not exclusively rely on ARToolKit. The architecture is open to use any kind of mobile tracking technology. Additionally to the ARToolKit support we implemented an ART module on the SignPost server which allows benefiting from the high quality of ART tracking data when available.
Figure 4 shows how the server/client communication works in the case of using ARToolKit. After the camera image is read it is converted to 32-bits RGBX for easier handling later on (and since this is the format ARToolKit expects the image anyway). If there is no workstation available the image is passed to the local ARToolKit module on the PDA, which calculates the camera position. Since this is a very CPU intensive task the PDA tries to out-source it if a server is available. The raw image as retrieved from the camera takes 225 Kbytes which is far too large to effectively transfer it over a WLAN. Considering ARToolKit does a thresholding anyway we can do this on the PDA to reduce the amount of data from 24 bits to 1 bit per pixel. Since this is still to large we apply a run-length encoding to the image and finally reduce the data size to around 3 Kbytes. The compressed image now fits nicely into a single UDP packet and can now effectively be transferred to the server which decompresses the image and queries ARToolKit to analyze it. The resulting transformation matrix is then sent back to the client.
SignPost uses BAUML (Building AUgmented Modelling Language) as its native data file format. BAUML is an XML dialect developed for the original SignPost project. A BAUML database contains information about a building such as the building’s geometry and all the markers' positions and orientations. Each floor is split into rooms which are connected through portals, yielding the aforementioned adjacency graph. Our PC-based SignPost client uses style sheets to translate BAUML files into OpenInventor file structures that can be directly used. On the PDA there is no Open Inventor. Therefore we use Expat to directly read the BAUML XML file and built proprietary run-time structures.
A PDA normally runs in "GDI-mode". Using the direct video memory access mode is quite complicated because of the different memory layouts on different PDAs. To ease the task we used the 2D library PocketFrog which abstracts the video memory to allow uniform access on all devices.
There is no built in 3D functionality on current PDA although we expect such devices to be available at the end of 2003. There are quite a lot free 3D libraries for the PocketPC platform but all of them are targeted to the game area and did not suite our needs. Porting mesa was no option since it aims at fully implementing the OpenGL API, which results in slow speed.
A high quality version of this video can be found in the Media/Press section.
copyright (c) 2014 Graz University of Technology