This week, I worked on identifying our selected ingredients from our video dataset. I successfully used blob detection to identify fish balls and bounded contours to identify tofu. Originally, I wanted to do both with blob detection but after experimenting, contouring worked much better to identify the tofu. In the coming week I will see if tweaking the thresholding method allows us to detect tofu with blobs, but for now the contouring is working well for that ingredient. Applying both detectors at the same time, I also confirmed that the fish ball blob detector does not have false positives on any other ingredients (does not detect anything except for fish balls). The tofu detector is more prone to false positives, but I have fixed most of them by checking the aspect ratio of the contour’s bounding box. Many of the false positives were due to chopsticks, which could be filtered out with their very extreme width to height ratio. There were also occasional false positives detected outside of the pot, but this will be fixed once we crop the video to the hot pot using the known location of our camera on the control tower in our final setup. I have also started doing the HSV color segmentation for identifying mushrooms, bok choy, and sliced beef. This puts me about a day behind schedule, but I have slack time until next weekend to finish with all the testing and tweaking of the algorithm developed from this initial dataset. I hope to have the initial identification of our remaining ingredients based on HSV done this week after finishing helping my team complete our design report.
Below are some screenshots of certain frames with tofu (blue) and fish balls (red) identified by the algorithm, as well as an example of bok choy masked out by its green color:
Note the blue square around the fish balls in the tray in the background… the non-hot-pot areas of the frame will be cropped out in future versions of our algorithm.
