This week, our team filmed thorough hot pot footage to use as a basis for our computer vision algorithm. The data included shots of all five of our chosen ingredients being placed into the hot pot from several angles and in different poses. We also confirmed the validity of our use case – Hot Pot Bot would solve several issues we ran into during our regular hot pot, including searching through the broth for food bits, overcooking vegetables, and getting food out of the pot quickly.
A significant risk with the computer vision algorithm is whether or not we will be able to distinguish ingredients being added to the pot from the background of our video feed, which is very non-uniform since it will be against the boiling hot pot broth with ingredients floating in it. Also, the appearance of the shiitake mushrooms varies greatly with different rotations, so it may be difficult to consistently identify it. In the case that the identification of the shiitake mushrooms fails, we may need to substitute this ingredient for something else. We also obtained footage of sliced lotus root and enoki mushrooms as backup ingredients. In the worst-case scenario where our computer vision is completely unviable, we can incorporate ingredient selection into our UI instead at a cost to the user experience, since the process would not be as seamless without automatic detection.
