Accomplishments this week:
Finished debugging and integrated my fine-tuned YOLOv8 model that now classifies stairs along with objects (image included below — note that our Jetson has to be plugged in now, so we couldn’t bring it to actual stairs but I emulated stairs with the cart in the image). Helped Kaya with fixing pyrealsense a little, then once it worked I used his code that gets distance from grid points to make the code that runs the model and creates bounding boxes now get the distance to the center points of the objects too. I also worked with Maya to get the haptics working from the Jetson. Lastly, I wrote the code to integrate/trigger the haptics and decide what action to suggest with the object detection model, based on objects detected, their location, and their distance — this currently works but the suggested actions are not all correct yet.
Reflection on schedule:
I believe I did a lot this week and caught up to what we planned.
Plans for next week:
Work with Kaya and get the wall detection working. Fix the recommended action decision making code.