Posted on

Kobe Individual Status Report 4/27

What did you personally accomplish this week on the project? 

This week I focused on debugging the depth image sensing and image pipeline. In the beginning of the week our depth image and color image was no longer triggering our main callback, after a long debug process we found that the main issue was that our images were being synced together correctly with the TimeSynchronizer object. To get around this, we decided to use an ApproximateTimeSynchronizer with an adjustable slop value. Additionally I continued to work on making our custom yolov8 model. This week I added more images of hexapods and refined our dataset with more images of people of different ethnicities so that our model would not be biased.

Is your progress on schedule or behind? If you are behind, what actions will betaken to catch up to the project schedule.

I think we are on schedule if we count the addition of slack time. We’re doing some final search behavior testing right now and we’ll be testing our multiple hexapod system soon.

What deliverables in the next week?

In the next week we just hope to complete our posters and refine our hexapod behaviors.

Posted on

Team Status Report 27th April

What are the most significant risks that could jeopardize the success of the project? How are these risks being managed? What contingency plans a re ready?

As discussed in Casper’s individual report, the most significant risk currently is the “Out of frame resources” error that is occurring when trying to run our program. This is causing indeterminate crashes when we try to run the program sometimes. If we are unable to find out when exactly this error occurs, and how to solve it, we can revert back to when we are only running YoloV8 without VSLAM.

Were any changes made to the existing design of the system (requirements, block diagram, system spec, etc)? Why was this change necessary, what costs does the change incur, and how will these costs be mitigated going forward?

As discussed above, it is possible that we might have to discard VSLAM due to resource limitations. This means that we will not be able to visualize the map and employ the compass coordinates from the IMU.

 

Provide an updated schedule if changes have occurred

 

List all unit tests and overall system test carried out for experimentation of the system. List any findings and design changes made from your analysis of test results and other data obtained from the experimentation.

Scalability speedup test:

We found that after 6 hexapods our speedup decreased so we shouldn’t ever scale above the number 6.

Battery test:

 

Object classification test

Through the object classification tests we decided to change our design to use a Jetson Orin Nano instead of a Jetson Nano due to the need for better computation speed and our desire to run YOLOv8 which is hardware accelerated and requires a Jetpack version that the Jetson Nano could not run. The mean accuracy readings led us to no longer need to extensively train a new model since the accuracy of our COCO dataset pretrained model was already very strong.

 

 

Posted on

Casper Individual Status Report 4/27

What did you personally accomplish this week on the project? Give files or photos that demonstrate your progress. Prove to the reader that you put sufficient effort into the project over the course of the week (12+ hours).
In the past week, I created the majority of slides and wrote the script for our final presentation – this included conducting some validations such as battery life calculations. I also spent a few hours creating cue cards and practicing.
In terms of advancing the project, I spent a lot of time deploying, testing, and tuning the search algorithm on the actual Hexapods. Testing on physical hardware has taken longer than anticipated. Furthermore, I suspect that the performance requirements from running both YoloV8 and VSLAM is causing too much resource depletion and overheating, where the Jetson encounters a “Out of Frame Resources” error after running the program too many times. We will need to find the root cause of this error and fix it, since it might be a bigger issue in our demo.
Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?
We are still quite behind schedule unfortunately, as we need to finish integrating the robot (though we have performed most of the component-wise validations already). We only need to finish fine tuning the search algorithm and add communications, however there might be many roadbumps (ie. the “Out of Frame Resources” error). Our team is prepared to dedicate the next several days on completing the project before our demo.
What deliverables do you hope to complete in the next week?
We have an early demo on Wednesday, so we are planning to finish our integrated robot by Monday ideally. This will involve solving the error caused by running VSLAM and Yolov8 together. If we are still unable to resolve this by early Monday, we may have to pivot off VSLAM and simply just hardcode turns and movements.
Posted on

Akash’s Individual Status Report 27th April

• What did you personally accomplish this week on the project? Give files orphotos that demonstrate your progress. Prove to the reader that you put sufficient effort into the project over the week (12+ hours).

I got the communications working last week but this week I had to integrate it with ROS and have it done as a Node so we can use it for communications. As I was traveling for an award ceremony from Thursday to Saturday, I worked really hard before Thursday to get this completely integrated. Challenges were to make it completely fault-tolerant and detect disconnections and new connections on both ends. Taking Distributed Systems helped with this. Moreover, I had previously just sent strings but since we need to send a grid between the hexapods, I had to modify it to be able to send 2d arrays. I also implemented a heartbeat system to detect faults in communication.
|• Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?

In terms of communication, I am on schedule. Overall we are behind schedule as we are still integrating right now.

• What deliverables do you hope to complete in the next week?

At this point, it’s just going to be the 3 of us working very hard to get a demo up on running before the demo day so the deliverable is just to get the final product done.

Didn’t have any flashy photos for communication so here is some debugging output while I was testing the sytem
Posted on

Kobe Individual Status Report 4/20

What did you personally accomplish this week on the project? Give files or photos that demonstrate your progress. Prove to the reader that you put sufficient effort into the project over the course of the week (12+ hours).
This week I integrated yolov8 and VSLAM together with hardware acceleration enabled. I initially tried to launch the launch files of both packages together but I pivoted to just combining the VSLAM container with the yolov8 nitros container in order to have hardware acceleration and have both packages work with a single realsense camera node. This integration is pretty complex and took the vast majority of my time. Earlier during the week I was also focused on testing out VSLAM and seeing how we can use pose data to keep track of our orientation.
Slam and Object Detection working
Is your progress on schedule or behind? 
We are admittedly behind schedule, mainly due to the various intricacies in ISAAC ros hardware acceleration that we’ve run into. The issues we’ve seen are not very well documented because of how new Isaac ROS but we’ve been able to overcome them with a bit more time. To make up for falling behind in schedule, we are all working around the clock for our project.
What deliverables do you hope to complete in the next week?In this next week I want to have our entire system running together with our state machine. I want to integrate the search algorithm with our pose data from VSLAM and then ensure that the hexapod has the proper behavior for our search and rescue tasks.

 

As you’ve designed, implemented and debugged your project, what new tools or new knowledge did you find it necessary to learn to be able to accomplish these tasks? What learning strategies did you use to acquire this new knowledge?

Understanding ROS and specifically Isaac ROS has been vital to our project and making the various parts of our hexapod software work together. Another skill that we learned and needed was being able to delve deep into documentation in order to figure out what we needed to do in order to resolve various software issues. We also found the use of docker containers to be very important for running Isaac ROS, this took a lot of trial and error to acquire new knowledge.

 

Posted on

Casper’s Individual Status Report Apr 20 2024

What did you personally accomplish this week on the project? Give files or photos that demonstrate your progress. Prove to the reader that you put sufficient effort into the project over the course of the week (12+ hours).
Over this past week (and during Carnival), I was able to write the search algorithm for our Hexapod, which can be seen here. I had to make some modifications in behaviour compared to our simulated search algorithm due to some physical constraints that we did not have in the simulation. Furthermore, I was able to assemble and 3D print all the required parts for a second Hexapod, as well as setup the Raspberry Pi for actuation. 
What deliverables do you hope to complete in the next week?

In this next week, I am planning to test and polish up the search algorithm on the Hexapod. This will involve interfacing and calibrating the compass bearings provided from the VSLAM library.

I am also the speaker for our final presentation this week, so I will need to prepare the slide deck and script.

Finally, I will work with the rest of the team in integrating all the Hexapod systems together (ie. image detection, search algorithm communication) then begin validating the whole system.

As you’ve designed, implemented and debugged your project, what new tools or new knowledge did you find it necessary to learn to be able to accomplish these tasks? What learning strategies did you use to acquire this new knowledge?

Being completely new to robotics before this capstone, I learned a lot about ROS throughout this project! I did so through auditing the 18448 class, as well as reading documentation as well as video tutorials.

Although I already knew the fundamentals, I also learned a lot more about 3D printing, as well as all the issues that come with it and how to debug them (ie. bed leveling, nozzle temperature, filament type). A majority of this learning came from simple trial and error, as well as asking others more knowledgeable about the issues that I encountered.

Posted on

Team Status Report 20th April

•What are the most significant risks that could jeopardize the success of theproject? How are these risks being managed? What contingency plans are ready?

Our most significant risk at this point is not having enough time for us to finish integration. This risk has been managed by having us work on different parts of the project parallel leading us to just need to just integrate and test the search algorithm over the next week. Contingency plans are various simplifications we can make to the search algorithm to complete our final demo.

• Were any changes made to the existing design of the system (requirements,block diagram, system spec, etc)? Why was this change necessary, what costsdoes the change incur, and how will these costs be mitigated going forward?

We switched back to using SLAM cause we got it working and we figured out how to get SLAM and our object detection working together, hence we didn’t need to pivot to a magnetometer which was a great finding. SLAM also gives us accurate heading data which helps our search algorithm. Additionally, we are now no longer running an image publisher node and instead launching a hardware accelerated realsense camera node within our node container alongside yolov8 and VSLAM

• Provide an updated schedule if changes have occurred.
Next week’s sole focus right now is integration and final testings, we have most of the components we need for our project individually but we need to combine them together.

• This is also the place to put some photos of your progress or to brag about athe component you got working.

Slam and Object Detection working

Two hexapods assembled and beginning to test communication and collaborative behavior.

https://docs.google.com/presentation/d/146rbdmwq4HQAA4EFEV7s1tSQf6OJx3z_CFxVmjaCUdc/edit#slide=id.p

Posted on

Akash’s Individual Status Report 20th of April

• What did you personally accomplish this week on the project? Give files orphotos that demonstrate your progress. Prove to the reader that you put sufficient effort into the project over the week (12+ hours).


Successfully got inter-hexapod communication working over TCP and worked with Kobe extensively to get the Realsense camera Node in ros working rather than using pyrealsense2 like before. We also got slam and yolov8 finally working simultaneously. Too many hours were spent debugging the above.

Slam and Object Detection working

• Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?

Communications are on schedule, but I’ve just today uncovered a weird bug in our realsense camera node that could derail us, we are remedying it by working around the clock.

• What deliverables do you hope to complete in the next week?

Since we have the robot physically constructed, hardware setup, and search algorithm implemented we are still mainly bottlenecked by the bug in the camera node. So after getting it resolved, it is mainly completing the integration and demoing it in an actual grid.

• As you’ve designed, implemented and debugged your project, what new tools or new knowledge did you find it necessary to learn to be able to accomplish these tasks? What learning strategies did you use to acquire this new knowledge?

We recognize that there are quite a few different methods (i.e. learning strategies) for gaining new knowledge — one doesn’t always need to take a class, or read a textbook to learn something new. Informal methods, such as watching an online video or reading a forum post are quite appropriate learning strategies for the acquisition of new knowledge.

I feel like the most important learning strategy I acquired was being able to parse useful information from online forums. Moreover to figure out needed information from poorly written documentation. For example in our project we use Nvidia’s issac ros to facilitate Vslam and object detection, however we had to figure out how to do everything almost from scratch because their examples only work for a particular use case and they don’t explain how to extend it. I also reached out to professors, friends, and others who have worked on similar platforms and their guidance was also essential to our learning.

Posted on

Kobe Individual Status Report 4/6

This week our main goal was to finish up a hexapod with enough of the desired behavior to be a demo. We were able to get our image to hexapod command pipeline working. I specifically worked on the software for the hexapod, focusing on the depth and the location of object bounding boxes from the Yolov8 detection outputs. I used the location of the bounding box centers as a way for our hexapod to turn toward people and follow them to a certain distance. The rest of the week I focused on restructuring our code to have our main callback function be in charge of running our state machine. I implemented more of our desired behavior so that the hexapod actually goes through various states from Search to Investigate to Found. I debugged this on the actual hexapod and ensured that it worked well with the rest of the package. I also added various checks into the code to make sure that the data we’re basing our hexapod behaviors off is accurate. Here is a snippet from one of our states:

We adjusted our gannt charts to be more realistic and we are currently on schedule. Next week I hope to implement a more complex search algorithm.

Posted on

Akash’s Individual Status Report Apr 6th

What did you personally accomplish this week on the project? Give files orphotos that demonstrate your progress. Prove to the reader that you put sufficient effort into the project over the course of the week (12+ hours).

This week was one with a lot of success for our team, Casper, Kobe and I were successfully able to integrate our code to consistently work from Sunday to Wednesday to ultimately be able to be able to be demo something we were proud of. We were able to get one complete robot working detecting a human and maneuvering towards it. After the demo I worked with my teammates to come up with an effective ros architecture and implementing a better state machine. My individual task right now has switched to focusing on communications between the hexapods and getting the other hexapods set up. I’m currently working on the code on of them since we are waiting on more SSD’s to arrive to effectively set up our other 2 Jetsons
• Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?

This week we are on track with our modified schedule, we do need to keep the pace up if we need to deliver three robots that are working with each other.
• What deliverables do you hope to complete in the next week?

Deliverables are get the communication done and then setting up our other Jetsons and have them effectively communicate with each other such as heartbeats and messages which can cause state transitions.