By
Marios Savvides
Kiran Bhat
 
 

Initial Outlook

Initially we thought that feature tracking could be performed using active contours (snakes) where a user would initially manually specify facial features (lips, eye brows, nose etc) for the snakes to slap on and track temporally. However we found that it was not easy to implement snakes, and we could not find any existing snake code available.  Therefore we started to look at other techniques to track features (MAD combined with edge-map tracking) until we found  the KLT feature tracking method. We also switched our platform implementation from Java to Visual C++.

KLT Feature Tracking Method

Our current system tracks features in the image based on the method proposed by Tomasi.et.al (KLT system). The KLT (Kanade Lucas Tomasi) feature tracking system identifies and tracks features by monitoring a measure of feature dissimilarity (that quantifies the change of appearance of a feature between the first and current frame).
 Features with good texture are selected and are tracked using an affine tracking model, which can account/compensate for translation and linear warping. We tested the tracker on a few face sequences and observed that features which were identified were tracked satisfactory.
 
 

Current System

We have implemented an automatic feature tracker using the Kanade/Lucas/Tomasi  (KLT) feature Tracking method using routines written at Stanford.
We have incorporated and modified these routines to produce the a visual testing framework. An appropriate GUI was designed and produced using Visual C++ and provided the following features:
 

GUI

• Choice of loading from 5 video streams.
• VCR type control panel.
• Automatic frame-by-frame feature tracking (500ms timed update interval).
• Manual step-through feature tracking process.
• Ability to specify the number of features to be found and tracked through a sequence.
• Ability to specify the number of frames to elapse before replacing lost features.
• Image display of base frame(image when feature tracker replaces lost features)
• Image display of current frame ( for comparison between the two).
• Number of features succesfully tracked in the current frame (relative to features in base frame).
• Implemented user-friendly/program safe- interface (ie. depending on user action, GUI disables un-available options controls to avoid user from performing an illegal action like pressing Load Stream 2 button while in auto mode viewing video stream 1. Using this method, the GUI directs the user to what options are valid at each moment during the execution of the program.

(Snap-shot of our system)

Sequence Capture

Video Sequences were obtained by modifying available framegrabber  'C' code to grab and store a sequence of color Windows BMP files. These files were then batch converted to grayscale BMP using Graphic Workshop, and converted to raw binary PGM format (the input to our system) using Paint Shop Pro 5.
Overrall 5 video sequences were recorded:

• 2 Sequences of a face moving with a uniform color  background (500 frames)
• 2 Sequences of a face moving with a feature background (300 frames)
• 1 Sequence of CMU campus (200 frames)

Current Accomplishments

Our current system using the KLT tracking method, is actually a much more better framework than what we originally goaled for :

• Good features are automatically identified and temporally tracked using the KLT method. Therefore no user is required to select features as originally thought. Hence the whole process can be fully automated without any user intervention.
• Feature Tracking is performed in Real-Time! We tested the system using a 300Mhz PII  PC, and found that we could track 100 features on a 300x200 grayscale image on average around 3-4Hz!
• User-Friendly GUI for visual testing and optimal parameter selection.

            We plan to incorporate morphing capability (by calculating the sparse to dense correspondence matrix between chosen key frames).Unix code is available, but needs to be ported to windows and tested for incorporation. We also would like to experiment with simpler triangulation methods for morphing, and depending on performance of these methods, we will try to implement a real-time feature based morphing system using input from a camera.