Author: zvw

Over the past two weeks our project took a rather large change. The entire way we structured our project needed to change for us to use Mediapipe. Overall my part of the project was implementing  the erase feature which I got done this past Thursday. Along with that I added multiple colors and cleaned up some of the code to make it more reliable and cleaner. Cleaner in terms of how it would react when the gesture turns on and off in the middle of it detecting stuff. At first the straight edge mode would not work unless the gesture was being detected already when the mode is entered.

This week I worked on presentation slides, the presentation video also made a couple tweaks and changes to the code base.  Along with that I met with my partners to talk over how we were planning on doing the video and to portion out who does what. Along with that my team and I worked out on what the evaluation process should be.  I finished my part of the video along with that I finished my part of the poster. All that is left for this week is finishing the final Paper.

This week and the past weeks I caught up on a lot of the stuff I needed to get done. Last week I ported the code to C++ and got everything done on the schedule. This week the team has been trying to pivot to using media pipe to get out of using a glove. For the first half of next week likely I will try to get media pipe working on my system as well. That way we should have a better time creating build packages that work in general for Linux systems.

This week The team got together and got the entire pipeline put together. We can now read camera input modify it in software and allow other applications to use that software camera as an input. We have it working on Arch Linux and Linux Mint (Which as a Fork of Ubuntu). Jenny is using Ubuntu 20.04 and just got it up and running this week and will be setting her stuff up this week as well. On my own front I began to find a more sophisticated way of doing background subtraction for the purpose of hand detection. So far in my research I found a good paper which also pointed to other papers which give a survey of many other methods that are used. Along with that I started looking at media pipe which seems to be even more ideal for what we want to do. Unfortunately this week I have not been able to get these new methods tested at yet due to assignments being due.

Paper that gives a survey of background subtraction methods. https://ipsjcva.springeropen.com/articles/10.1186/s41074-017-0036-1

This week I worked to change the method for removing the background from the camera feed and making the hand detection much cleaner I accomplished that by using a blue glove and doing contour detection of that color. I am going to use the recommendations from Course staff to change the method I am using so that I won’t have to use a blue glove anymore. Along with that this week I added code to respond to Jenny’s model to make integration of the whole thing easier.  In doing this I also added code so that I can now draw on the screen and do writing. I added controls to also allow the user to erase the screen as well. Right now everything is bound to the keyboard but all that needs to change is replacing that with signals from Jenny’s Model. For next week we should be able to fully integrate the pipeline after I port all my code to C++.

The first part of this week I spent the majority of my time working on my teams design presentation, and presented on Monday. The second part of this week I spent my time getting my portion of the project off the ground. My goal was to get all of the finger tracking code in the method that I found online ported into C++. So far I have not been able to test any of the code written but I have been able to get OpenCV and have been able to do some basic things with it. This portion has been going a bit slower for me as the majority of my background is not in software.

This week I made progress on my portion of the project. I needed to figure out how to do real time finger tracking. I found a method that was used and this will be my starting point. I worked this week with the  team on the design presentation and began to think about how I will be presenting on Monday.

This week I worked with my group to fixup our presentation. Along with that I met with my group to work on and expand on our Gantt Chart. Following this I also did research into how finger tracking works. Along with this I also did some research into Hand Pose Estimation and found several research papers on this along with the code and the data sets for hand pose estimation that the researchers used. This will help me set up for next week.

This week was spent refining our ideas for our project. We spent the week making the presentation and fleshing out design decisions for our project.  We made plans for how the entire execution will go down and made a schedule that we are going to follow and update accordingly with how things go. Further we also assigned tasks to everyone so I can now start reading up on object tracking. We fleshed out our ideas and are preparing for the presentation we have next week.