handheld augmented reality

Augmented Reality Anywhere and Anytime   

Projects

   Social Augmented Reality

   Information Presentation

   Real-Time Self Localization

   Structural Modelling

   AR Graphics

   AR Navigation

   Augmented Reality Games

   Past Projects

 

Technology

   Hybrid SLAM

   Panorama SLAM

   Planar SLAM

   Model-Based Tracking

   Marker-Based Tracking

 

Software Libraries

   Studierstube ES

   Studierstube Tracker

   Muddleware

   ARToolkitPlus

 

More

   Videos

   Media/Press

   Team

   Publications

   Collaborations

   Student projects

   FAQ

   Site map

 

The Handheld Augmented Reality
Project is supported by the
following institutions:

Qualcomm

 

Christian Doppler Forschungsgesellschaft

 

Graz University of Technology

 

 


 

Come to ISMAR 2011

ISMAR2011

 

Follow us on twitter

High Speed Natural Feature Tracking

High Speed Natural Feature Tracking

 

Mobile phones have highly restricted resources compared to PCs. If everything is done right, a high-speed mobile phone is about 10 times slower than an average PC. Making natural feature tracking practical on mobile phone therefore requires development of new methods that take the specific weaknesses of modern mobile phones into account.

We developed a new high speed natural feature tracking mechanism that works in real time on any PC as well as any smart phone. The specific strengths of the new approach are:

  • Very fast (1-2ms on PC, 10-20ms on mobile phone)
  • Robust to blur
  • Robust to reflections
  • Robust to partial occlusion
  • (Re)intialization from single image
  • Large scale changes (working volume)
  • Can track extreme tilts

Speed

The new natural feature tracker was developed specifically with mobile phones in mind. It minimizes data access (it does not have to process the whole image) and is therefore largely independent of the camera resolution. The tracking can run without floating point support. Although a floating point unit is helpful (gives a minimal speedup) it is not required due to optimized fixed-point code. Tests done on a 2Ghz single-core PC showed tracking times of 1-2 milliseconds per frame (from a 320x240 camera stream). Tests on mobile phones show tracking times between about 10ms (600MHz) and 20ms (300MHz).

 

Robust to Blur

Due to a new approach the tracker is highly robust to blur.

Robust to Blur
Robust tracking despite of strong blur in the camera image.

 

Robust to Reflections

The tracking uses an NCC-like approach for feature matching that makes it largely independent of brightness changes.

Robust to Reflections
Robust tracking despite of strong reflection on the tracking target.

 

Robust to partial occlusion

Tracking continues as long as there is "something" still visible of the tracking target.

Occlusion
Tracking despite major occlusions.

 

Large Scale Changes

Tracking works over large changes of scale. In the right image below the tracking target is about 20 times smaller than in the left image.

Large Scale Changes
Tracking over large scale changes.

 

Extreme Tilts

Strong Tilt
Tracking under strong tilt.

 

 

Video

All videos can also be found in our media section.

 

High-speed AR at 30Hz on a mobile phone

(Tracking and rendering are done in software only)


YouTube has a limited quality and frame rate. The original video was recorded at 50Hz.
Click here to download a higher quality version that shows the full performance of the demo.

 

 

 

Natural Feature Tracking of multiple Targets on a Mobile Phone




Tracking and rendering are done in software only.
Click here to download a high quality version of this video.

 

 

 

Natural Feature Tracking in 2ms per frame on a PC




The natural feature tracking running on a 2GHz notebook.
Click here to download a high quality version of the video.

 

 

 

 

copyright (c) 2014 Graz University of Technology