Author: karenson

• What did you personally accomplish this week on the project? Give files or
  photos that demonstrate your progress.

This week I worked on switching from a hand tracking model (tracking using mediapipe) to a color tracking model because mediapipe is relatively slow since it needs to keep track of many points on the hand, while we only need to know where the hand is. Instead, we used colorful tapes and finger cots to keep track of the user’s middle finger position and identify where it is. I worked mainly on adjusting the RGB value identified for each color (so far the system should support Red, green, yellow, blue, and purple). In the end, we probably won’t need that many colors, but this is just for testing purposes (and in the extended case where the user has specific colored clothes that will require a different color finger to be tracked)

• Is your progress on schedule or behind? If you are behind, what actions will be
  taken to catch up to the project schedule?

I think my progress is still a little behind because changing from hand tracking to color tracking is not part of our intended schedule. However this does kind of solve the problem of time delay, so it is sort of progress speed up in a different place. Next week because of carnival there will  be fewer classes so more time for work

• What deliverables do you hope to complete in the next week?

Next week I will try to get the color recording fully integrated with the synthesizer (with note area classification) and continue to work on the gesture recognition section

• Now that you are entering into the verification and validation phase of your project, provide a comprehensive update on what tests you have you run or are planning to run. In particular, how will you analyze the anticipated measured results to verify your contribution to the project meets the engineering design requirements or the use case requirements?

Currently tests that I am currently running/have finished include color detection accuracy(using different color objects under different lighting settings to see if the system can still find the assigned color), color detection speed (time elapse between color input (middle finger locating) and sound output based on quadrant. Tests will be conducted in the following week include battery testing and gesture recognition accuracy, which are based on unfinished components.

• What did you personally accomplish this week on the project? Give files or
  photos that demonstrate your progress.

This week I worked with Oscar to get the hand tracking module integrated with the synthesizer. So far we were able to get the two components working together, although it is slightly slower than expected. We are still working on trying to improve the speed.

• Is your progress on schedule or behind? If you are behind, what actions will be
  taken to catch up to the project schedule.

I think my progress is slightly behind because some integration work are not working as intended, and I did not have much time last week due to exams. I am working on this class most of the week next week.

• What deliverables do you hope to complete in the next week?

Next week I will try to get the gesture recognition section of the software to work properly and try to get it working together with the gesture mapping section. In addition I will work with Oscar to get the synthesizer-gesture recognition system to work faster.

• What did you personally accomplish this week on the project? Give files or
  photos that demonstrate your progress.

This week I am mainly focusing on understanding the recognition models and setting up the system on my computer. I already have the yoha demo set up on my computer, and working on how to integrate this to what is already existing.

• Is your progress on schedule or behind? If you are behind, what actions will be
  taken to catch up to the project schedule.

I am still a little behind process because of the ethics assignment and some technical difficulty that came out of nowhere ;-; So far I decide to switch to a different device to hopefully avoid the issue.

• What deliverables do you hope to complete in the next week?

Next week I hope to try to get a gesture recognition model binded with the ui and starting to get hand tracking synced with the synthesizer.

• What did you personally accomplish this week on the project? Give files or
  photos that demonstrate your progress.

This week I worked on finishing the mapping function, and saving data using python pickle (was considering joblib as a backup idea). So far I am using some dummy gestures to test the buttons and data loading/saving function.

• Is your progress on schedule or behind? If you are behind, what actions will be
  taken to catch up to the project schedule.

I am still a little behind process because of a pop up event during spring break and I suddenly have to travel to somewhere with bad internet connection. I am planning to allocate next Tuesday entirely to sit down and go over the project so I can start planning integration with the ges rec mode.

• What deliverables do you hope to complete in the next week?

Next week I hope to this piece of software to interact with the real model instead of dummy placeholders.

 

• What did you personally accomplish this week on the project? Give files or
  photos that demonstrate your progress.

This week I created a python file that creates a pop up window showing the possible commands and music effects that gestures can control. So far no actual mapping is done because we did not come up with actual gestures we are using yet. I end up using pygame because it supports a more fancy ui compared to pysimpleGUI and should be better for the users to use.

• Is your progress on schedule or behind? If you are behind, what actions will be
  taken to catch up to the project schedule.

I am a little behind process because I did not get the data saving part to work. every time the mapping will just reset, which is not good for usability. I am planning read some python documentation and tutorials to see if this function is possible and switch to a website if this is not possible.

• What deliverables do you hope to complete in the next week?

Next week I hope to get the scroll bar and data saving to work in the interactive part, and get the gestures that we are going to use confirmed after looking at the models and their datasets.

 

• This week I spent most of the time researching existing hand gesture recognition models (and part of its corresponding database) to observe the kind of items it consist and reevaluate the practicality of collecting our own data. In case we are not able to collect and train our own data, I want to find a list of practical existing models that we can pick from.
• This week I think I am pretty much on schedule in terms of researching for project implementation and design for the first half of the week, then I got covid so there is not as much progress as I would like in the later half of the week
• Next week (Feb 19) I am planning on finishing our project design proposal presentations slides with my group. I will be preparing and delivering the design proposal presentation (presentation 2) for my group.