This week I mostly helped Alex to build the platform for the camera setup, and then attempted some basic tests on the setup. I also am currently training a model based on the data collected from previous weeks (and some from this week after getting the platform up)
Figure 1: Example of Fully-Constructed Platform
Figure 2: Example of self-collected data (here, of yellow squash) taken on new platform
More notably, I have been trying to improve the robustness of the change detector since I predict issues with the CV algorithm falsely detecting a change due to illumination changes from the opening of the door or a person walking around etc.
To this end, I tried using division normalization, where we divide the image by a very blurred version of the same image as per https://stackoverflow.com/a/64357685/3141253
While this allowed the algorithm to be very robust against illumination it also reduced the sensitivity significantly. One particularly problematic aspect occured because my skin color was similar to the birch wood color and so my hand was “merged” into the background. When displaying with a red apple, however, there was a better response, but not always enough to trigger changes. With this in mind, we have plans to color the platform white and hope that this might cause a larger differential between natural colors like that of skin and fruit, vs those of the artificially white background.
Another alternative is to play around with the saturation value of the HSV dimension. Or, since the main issue is going to be falsely detecting a change that moves the FSM into a background state falsely, we can potentially check for a true extra object using a Harris corner detector.
Next week, in addition to trying the aforementioned items, I will also be helping to integrate the CV algorithm on the Jetson. We are currently on schedule but might need to speed up the integration component so we can spend more time on robustness testing.