Mobile phones have highly restricted resources compared to PCs. If everything is done right, a high-speed mobile phone is about 10 times slower than an average PC. Making natural feature tracking practical on mobile phone therefore requires development of new methods that take the specific weaknesses of modern mobile phones into account.
We developed a new high speed natural feature tracking mechanism that works in real time on any PC as well as any smart phone. The specific strengths of the new approach are:
Very fast (1-2ms on PC, 10-20ms on mobile phone)
Robust to blur
Robust to reflections
Robust to partial occlusion
(Re)intialization from single image
Large scale changes (working volume)
Can track extreme tilts
Speed
The new natural feature tracker was developed specifically with mobile phones in mind. It minimizes data access (it does not have to process the whole image) and is therefore largely independent of the camera resolution. The tracking can run without floating point support. Although a floating point unit is helpful (gives a minimal speedup) it is not required due to optimized fixed-point code. Tests done on a 2Ghz single-core PC showed tracking times of 1-2 milliseconds per frame (from a 320x240 camera stream). Tests on mobile phones show tracking times between about 10ms (600MHz) and 20ms (300MHz).
Robust to Blur
Due to a new approach the tracker is highly robust to blur.
Robust tracking despite of strong blur in the camera image.
Robust to Reflections
The tracking uses an NCC-like approach for feature matching that makes it largely independent of brightness changes.
Robust tracking despite of strong reflection on the tracking target.
Robust to partial occlusion
Tracking continues as long as there is "something" still visible of the tracking target.
Tracking despite major occlusions.
Large Scale Changes
Tracking works over large changes of scale. In the right image below the tracking target is about 20 times smaller than in the left image.
(Tracking and rendering are done in software only)
YouTube has a limited quality and frame rate. The original video was recorded at 50Hz. Click here to download a higher quality version that shows the full performance of the demo.
Natural Feature Tracking of multiple Targets on a Mobile Phone
Tracking and rendering are done in software only. Click here to download a high quality version of this video.
Natural Feature Tracking in 2ms per frame on a PC
The natural feature tracking running on a 2GHz notebook. Click here to download a high quality version of the video.
website maintained by
Tobias Langlotz
last updated on 2011-08-02
