This week we ran into the roadblock of being unable to capture camera data from our Jetson. Our team decided to have some workable data by temporarily pivoting to using a phone camera to capture the bird’s eye data.
We are currently still waiting on the UWB hardware to arrive, so we all shifted focus to capturing a usable map from the bird’s eye view camera, as that is something that all the other components would rely on, and we were behind schedule on both capturing the image and generating a CV model of the space. While my teammates captured data from their houses, I was also able to simultaneously capture data from the HH lab by placing the phone up on a ceiling light.
We were also trying to find an appropriate CV model for this task. With a segmentation model which only produced a border outline of the map, I tried to fill in the obstacles so that the inside of the border was outlined as well:
This was unsuccessful as I struggled to correctly identify the floor compared to actual obstacles. Charles was able to use the Meta SAM model which accomplished this and we will likely proceed with that model.
I also played around with using CV to remove the fisheye effect. This will require more tuning.
I think we are behind schedule, but it is manageable. I would like to be able overlay the path planning on top of our occupancy matrix next week. I also hope that if the UWB sensors arrive next week, that we can set that positioning system up.