Author: jifengy

For this week, I looked over the feedback given by peers from the design review presentation and discussed some plan changes with my group. I have decided to train my gesture detection ML model in Python and then export it as a json or txt file, since training is done separately and doesn’t affect application speed. Then, in our actual application, we will be using c++ and using that to run a classifier with our model.

In terms of coding, I have been able to do 3 different types of maskings, one which does threshing by detecting pixels within a skintone range of colors, one that contours areas of skintone, and one that does dynamic background subtraction, essentially detecting changes in motion. I have taken a screenshot below for demo purposes:

For next week, I will be working on figuring out a way to combine information from the three to 1) filter out noise and 2) only focus on the hand and ignore the face.

For this week, our team changed from using Python on dataflow and our ML implementations to using C++ in favor of faster performance, even though the code itself may be a bit more difficult. We also completed the slides for the upcoming Design Review presentation. Due to the change in language and some other minor shifts in our application, we changed around the Gantt chart a bit. Each of the team members made progress in their own portions of the project as outlined in the individual status reports.

For this week, I discussed with my team and we made the decision to switch to C++ as our language of choice instead of Python due to efficiency of performance. I also helped with creating the design review presentation slides due on Sunday and created a basic logo for us. I also decided upon the two models that I will be testing using (CNN and SVMs) to see which better fits our purpose for gesture detection. I also began trying to implement background subtraction in our C++ repo. Finally, I also wrote our team status report.

For this week, our team presented our project proposal on Wednesday. Our team also adjusted our deadlines and pre-existing tasks on our Gantt chart, as well as adding some more new tasks and subtasks. We created a project repo for Whiteboard Pal on github, and we now have some scaffold code for our dataflow, aka how we will collect hand gesture and tracking data from webcam frames using opencv. Our members also did more research into the two individual areas of our requirements, the hand tracking and gesture detection, and we also looked at pre-existing literature and code surrounding those tasks.

For this week, I worked with my group to adjust and expand our Gantt chart. I changed some projected deadlines around, added some more tasks and subtasks. Additionally, I did research into how opencv and hand detection work, while playing around with some sample code to test different features of opencv. I also worked on our team’s status report for the week. For next week, I intend to work on implementing our Whiteboard Pal version of detecting hands vs. no hands in a frame and then hands vs. pinching gesture.

For this week, our team managed to finalize our project idea and expand upon the plans. We had started the week with several iterations of different ideas, but decided upon the hand tracking whiteboard idea. We met up and spoke with our project TA, Uzair, and instructors, Professor Kim and Professor Mukherjee, who gave us advice for the project itself and the proposal.

For our project idea, we were advised to really narrow down our problem statement and idea to do one thing really well. After brainstorming, we begun and finished the project proposal slides. We were given feedback that we should cut down on the words and include more images and diagrams, which we went back to and finished.

For this week, I worked on finalizing our project idea and the proposal slides with my team. We oscillated between several iterations of the idea before setting on a final one. I created the sketches/diagrams for the slides, as well as participated in work sessions together with my teammates to write down the content on each of the slides. In addition, I also set plans and a timeline this week for what the team should seek to accomplish regarding the proposal and blog posts. I also worked on the team status report for WordPress.