Progress Update:
I completed the Object Re-Identification logic and finish implementing it. This feature can detect the ingredients grid in the UI then see which cells are occupied. Once it identifies the occupied cell locations, it parses the json file that contains the list of ingredients that should be present in the grid. Each item has a property called “cell_pos” that has the expected row and column of the ingredient along with properties such as “name”, “filename”, and “filepath”. This allows the program to retrieve the ingredient name and label the image of the ingredient captured from the grid.
As a result of this feature, I created modular functions that can easily add and remove ingredients from the json file and scan singular cells for an ingredients vs scanning the entire grid at once.
I used the following template of the UI (made by Caroline) to do my testing. I added images of ingredients to reID. This is what the camera will see projected on to the table:
Here is a video demonstrating the reID process:
https://drive.google.com/file/d/1fGxNk6h5AN5JJqAoFRP-cgjNKqWtU1hy/view?usp=sharing
Schedule: On track!
Next Week Plans:
- Implement button actions using gesture recognition
- Implement Swipe Gesture only in a region of interest
- currently this has only been implemented for functionality and not integrated into TC
- Consider adding a cursor feature that follows user’s finger tip (beyond MVP)