What did you personally accomplish this week on the project? Give files or
photos that demonstrate your progress. Prove to the reader that you put sufficient effort into the project over the course of the week (12+ hours).
This past week I worked on the CV code to segment out both black and white keys. I made several changes to my code and I will go through them here:
- Resize the image to make things consistent (used some factor so that every image would have the same number of pixels while keeping the same aspect ratio).
- Add a gray border to the image to help with thresholding. Can’t be black nor white because then the border would be thresholded with either the white or black keys which we don’t want
- Do the grayscale, blurring, and edge detection. Then do binary thresholding on the edge detection. This gives us areas dividing all keys to the borders seen here:
- Then we do segmentation on the image so that each of the white spaces are assigned enumerated labels. I used the cv.connectedComponentsWithStats function for this task. These labels were converted by me to greyscale values between 0 and 255 normalized with an even distribution. The output image has layers of shades of gray each value essentially representing a segmented region for each piano key or as I define them as “sectors”:
- I then used a mouse cursor window function called cv.setMouseCallback that allows me to create a window for the image where my mouse can read information on the 2D matrix. By getting its value I can find it’s respective sector information for that segmented region from a list I created to store that information. When the user hovers over a key, the enumerated number of that sector will show up. Here is some of my code below and the terminal output showing this:
Per last week, last Monday I ordered the phone mount that i decided to go with. It has arrived to the ECE corridor so I can try it out this coming Monday.
Lastly, I worked on finishing the ethics assignment this week too. This includes having the task 3 discussion with my team in addition to going to the ethics lecture for task 4.
“ Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?
My progress is on schedule. I need to do some small adjusting for the image sizes to make that more test images are more consistent. I will test this with the new phone holder stand to start testing with real images with the ring light to make further adjustments there.
“ What deliverables do you hope to complete in the next week?
By next week according to the schedule, I plan on incorporating the hands in there. How I will do that is essentially have a picture shot of the piano with no fingers on it to have it as a background. Then the colored tips of the gloves will hover over a piano key, the CV will easily be able to tell that the colored region overlapping with a segmented region. So it will be able to determine which piano key the finger is over then.