Augmented Reality Anywhere and Anytime
The Handheld Augmented Reality
Participate to ISMAR 2012
Model-Based Natural Feature Tracking
In recent years, model-based tracking has become state-of-the-art on mobile devices. In 2008 the team members of the Christian Doppler Laboratory for Handheld Augmented Reality presented the world's first 6DOF real-time natural feature tracking system running on a mobile phone. Since then we have made many advances towards our goal of wide area markerless tracking. We show that robust and high frame-rate tracking of a-priori known 3D objects is not prohibitive on mobile phones anymore.
This page summarizes the various steps we made and the current status of our work onto our goal of tracking anywhere and anytime. On our way to this goal, we always only consider methods that are suitable for the mobile phone platform in terms of processing power and memory requirements.
Summary: The techniques developed for our work on wide area localization are now also used for our latest high-speed natural feature tracker that can detect and track real 3D objects rather than planar objects only. We perform real-time scale space keypoint detection (20Hz) on a mobile phone. The applied tracker is an extended version of the tracker used for our work on "Multiple Target Detection and Tracking".
Summary:The detection and tracking method described above can only detect and track one object at a time, because the processing power of mobile phones does not permit to run both detection and tracking simultanously without seriously degrading overall speed. We therefore invented new methods to run both techniques simultanously at high frame rate. The generall idea is to only detect in those areas of the camera image that are not used for tracking yet as well as splitting the slower detection task over multiple frames. The whole process is designed to not exceed a predefined time budget in order to guarantee a certain frame rate. The resulting detector/tracker combinaiton can detect and track up to 6 independent objects at a frame rate of 23Hz on a mobile phone.
Publication: Multiple Target Detection and Tracking with Guaranteed Framerates on Mobile Phones, Daniel Wagner, Dieter Schmalstieg, Horst Bischof, ISMAR 2009
Summary: In our next step to improve robustness and speed we replaced the integrated tracker of our previous approach with a dedicated tracker that takes over once an object is detected. Dedicated tracking uses active search and can be implemented extremely fast, requiring as little as ~1ms on a PC and ~6-10ms on a mobile phone per tracked object. We use exhaustive normalize cross correlation (NCC) template matching over a search window rather than the popular KLT tracker since the latter is too slow for our purposes. Additionally, NCC is highly robust to affine brightess changes. Due to the active search approach the tracker's speed is largely independent to the camera's resolution.
Summary: Our first attempt towards markerless tracking on mobile phones (also known as PhonySIFT) uses a heavily modified version of the well known SIFT descriptor. We replaced the slow DoG approach of the original SIFT with a FAST corner detector. The approach runs in 3-5ms on a PC and 20-40ms on a mobile phone allowing AR applications at interative frame rates.
Publication: Pose Tracking from Natural Features on Mobile Phones, Daniel Wagner, Gerhard Reitmayr, Alessandro Mulloni, Tom Drummond, Dieter Schmalstieg, ISMAR 2008
Summary: Marker tracking has revolutionized Augmented Reality about a decade ago. However, this revolution came at the expense of visual clutter. We developed methods that allow tracking beyond the visibility of markers to improving robustness. These techniques are highly efficient in their memory and CPU usage and run at interactive frame rates on mobile phones.
Publication: Robust and Unobtrusive Marker Tracking on Mobile Phones, Daniel Wagner, Tobias Langlotz, Dieter Schmalstieg, ISMAR 2008
website maintained by
copyright (c) 2011 Graz University of Technology