Morphing Based Interpolation Techniques for Motion Compensation in a MPEG Videostream

By
Marios Savvides
Kiran Bhat
 
 

Initial Outlook

Initially we thought that feature tracking could be performed using active contours (snakes) where a user would initially manually specify facial features (lips, eye brows, nose etc) for the snakes to slap on and track temporally. However we found that it was not easy to implement snakes, and we could not find any existing snake code available.  Therefore we started to look at other techniques to track features (MAD combined with edge-map tracking) until we found  the KLT feature tracking method. We also switched our platform implementation from Java to Visual C++.

KLT Feature Tracking Method

Our current system tracks features in the image based on the method proposed by Tomasi.et.al (KLT system). The KLT (Kanade Lucas Tomasi) feature tracking system identifies and tracks features by monitoring a measure of feature dissimilarity (that quantifies the change of appearance of a feature between the first and current frame).
 Features with good texture are selected and are tracked using an affine tracking model, which can account/compensate for translation and linear warping. We tested the tracker on a few face sequences and observed that features which were identified were tracked satisfactory.
 
 

Current System

We have implemented an automatic feature tracker using the Kanade/Lucas/Tomasi  (KLT) feature Tracking method using routines written at Stanford.
We have incorporated and modified these routines to produce the a visual testing framework. An appropriate GUI was designed and produced using Visual C++ and provided the following features:
 

GUI

 
 
 

(Snap-shot of our system)


 
 

Sequence Capture

Video Sequences were obtained by modifying available framegrabber  'C' code to grab and store a sequence of color Windows BMP files. These files were then batch converted to grayscale BMP using Graphic Workshop, and converted to raw binary PGM format (the input to our system) using Paint Shop Pro 5.
Overrall 5 video sequences were recorded:


Current Accomplishments

Our current system using the KLT tracking method, is actually a much more better framework than what we originally goaled for :
Future Plans

            We plan to incorporate morphing capability (by calculating the sparse to dense correspondence matrix between chosen key frames).Unix code is available, but needs to be ported to windows and tested for incorporation. We also would like to experiment with simpler triangulation methods for morphing, and depending on performance of these methods, we will try to implement a real-time feature based morphing system using input from a camera.