Category: Team Status Reports

What are the most significant risks that could jeopardize the success of the project? How are these risks being managed? What contingency plans a re ready?

As discussed in Casper’s individual report, the most significant risk currently is the “Out of frame resources” error that is occurring when trying to run our program. This is causing indeterminate crashes when we try to run the program sometimes. If we are unable to find out when exactly this error occurs, and how to solve it, we can revert back to when we are only running YoloV8 without VSLAM.

Were any changes made to the existing design of the system (requirements, block diagram, system spec, etc)? Why was this change necessary, what costs does the change incur, and how will these costs be mitigated going forward?

As discussed above, it is possible that we might have to discard VSLAM due to resource limitations. This means that we will not be able to visualize the map and employ the compass coordinates from the IMU.

Provide an updated schedule if changes have occurred

List all unit tests and overall system test carried out for experimentation of the system. List any findings and design changes made from your analysis of test results and other data obtained from the experimentation.

Scalability speedup test:

We found that after 6 hexapods our speedup decreased so we shouldn’t ever scale above the number 6.

Battery test:

Object classification test

Through the object classification tests we decided to change our design to use a Jetson Orin Nano instead of a Jetson Nano due to the need for better computation speed and our desire to run YOLOv8 which is hardware accelerated and requires a Jetpack version that the Jetson Nano could not run. The mean accuracy readings led us to no longer need to extensively train a new model since the accuracy of our COCO dataset pretrained model was already very strong.

What are the most significant risks that could jeopardize the success of the project? How are these risks being managed? What contingency plans are ready?

We have made good progress with the Hexapod for the interim demonstration, where it can track and follow people. However, we are still yet to implement a proper search/path planning algorithm to find people in the first place, which is an important component of our project.  To mitigate this risk, we have many ideas for search algorithms of varying levels of difficulty, so that we can always fall back on a simple implementation if needed. Our ideas range from using SLAM map data to track all explored and unexplored areas, to simply walking straight when possible and turning in a random direction when blocked.

For localization and mapping, we are almost finished with setting up VSLAM using live data from the RealSense cameras. However, there is still worry that VSLAM may be too compute heavy, both in terms of compute resources and also memory requirements (since offline data can be gigabytes to terabytes large), so this might not be feasible. As a contingency plan, we might be able to simply use the movement commands we send to create a map, as it seems that the drifts on the Hexapod robots are tolerable.

Were any changes made to the existing design of the system (requirements, block diagram, system spec, etc)? Why was this change necessary, what costs does the change incur, and how will these costs be mitigated going forward?

As discussed above, we might need to replace SLAM from our system requirements if we deem it is computationally infeasible. This means the robot will not be as accurate in localization due to drift, but will have more computing power for object detection and other algorithms.

We also made some additions and changes to our hardware, such as powering the RPi using an additional battery pack, so that all the 18650s are dedicated to powering the servo motors for longer life.

Provide an updated schedule if changes have occurred.

Our schedule is actively updated in our Gantt chart, which can be seen in this spreadsheet.

Now that you have some portions of your project built, and entering into the verification and validation phase of your project, provide a comprehensive update on what tests you have run or are planning to run. In particular, how will you analyze the anticipated measured results to verify your contribution to the project meets the engineering design requirements or the use case requirements?

Controls test (already done): After setting up the hardware and software for the Hexapods, we tested that the movement and controls algorithm works as intended.

Search algorithm test: We will test that the hexapods are able to navigate around a room with obstacles without colliding. We can do so by cordoning off a section of a room and addingobstacles in front of the robot’s paths.

People detection test: We will test that our object classifier is able to detect people in a variety of poses, lighting conditions, and other environment variables. We will use test data that we took ourselves or from online.

Mock real-world test: We will create a 5m x 5m space / room, with walls and other obstacles as a simulation of a real-world scenario that the hexapods might find themselves in. We will test that the hexapods are able to navigate through the room and find all humans (ie. human figurines or images).

What are the most significant risks that could jeopardize the success of the project? How are these risks being managed? What contingency plans are ready?
The main risk we are facing right now is implementing some form of SLAM to allow our hexapods to localize and determine where other hexapods are in the area. Our contingency plan for this risk is to utilize a prebuilt Isaac-ROS april tags library to use april tags for pose estimation of our hexapods instead of having a full SLAM implementation. We think that this would be a lot easier to implement though we have to constrain our scope a bit more.

Were any changes made to the existing design of the system (requirements, block diagram, system spec, etc)? Why was this change necessary, what costs does the change incur, and how will these costs be mitigated going forward?
One of the changes made to the existing design of the system is that we’re now using Isaac-ROS on our Jetson Orin Nano. This is because Isaac-ROS supports our Yolov8 object detection algorithm and is already hardware accelerated with the NITROS nodes that it provides. This hardware acceleration is very important for us because we want to make our object detection fast.

Part A: … with consideration of global factors. Global factors are world-wide contexts and factors, rather than only local ones. They do not necessarily represent geographic concerns. Global factors do not need to concern every single person in the entire world. Rather, these factors affect people outside of Pittsburgh, or those who are not in an academic environment, or those who are not technologically savvy, etc.
With consideration of global factors, the main “job to be done” for our design is to provide accessible search and rescue options for countries around the world to deploy. With the rise of natural disasters and various conflicts around the globe, it’s important that we are armed with the appropriate, cost effective, and scalable solutions. Our hexapod swarm will also be trained with a diverse dataset ensuring that we can account for all kinds of people from around the world. Our hexapod’s versatility and simplicity will allow it to be deployed around the world by people with limited technology ability. (Written by Kobe)

Part B: … with consideration of cultural factors. Cultural factors encompass the set of beliefs, moral values, traditions, language, and laws (or rules of behavior) held in common by a nation, a community, or other defined group of people.
With consideration of cultural factors, it is part of our moral belief and instinct to help those in times of need, such that we are also able to receive help when we are vulnerable ourselves. Our product solution is designed to address this common moral, across all different cultures and backgrounds. We will train the object detection models such that it can recognize people from all cultures and backgrounds, and design the product generally so that it can be deployed in many versatile places. (Written by Casper)

Part C: … with consideration of environmental factors. Environmental factors are concerned with the environment as it relates to living organisms and natural resources.
With respect to environmental factors the reason we chose a hexapod robot is so that the robot can traverse a lot of different terrains. We have to do physical testing to ascertain this. We also will account for domestic pets like dogs and cats and have that in our training dataset so we can identify and rescue them as well. We also will test our model with dolls and humanoid lookalikes to check if it doesn’t get confused by them. We have a fault tolerant requirement to account for changing environments that could compromise robots such as water damage or collapsing infrastructure. (Written by Akash)

Were any changes made to the existing design of the system (requirements, block diagram, system spec, etc)? Why was this change necessary, what costs does the change incur, and how will these costs be mitigated going forward?

One significant change we made was switching from the Jetson Nano to the Jetson Orin Nano. This was necessary as we became cognizant that the Nano’s compute might not have been sufficient to run ROS, Object Detection, and SLAM simultaneously. We also saw that it took the Nano 40 seconds to run YOLOv7. In terms of costs we incur we were lucky that the Orins were available in inventory so we didn’t have the shell money out of them from our budget. An additional cost is that the Orin needs to be supplied with a higher voltage so we have to upgrade a hexapod’s power supplies to meet its demands.

What are the most significant risks that could jeopardize the success of the project? How are these risks being managed? What contingency plans are ready?

Currently, our greatest troubles has been in bringing up the Jetson and Raspberry Pi boards to run all the relevant software that we need (ie. ROS, YOLO, Jetpack, & Python versions meet requirements). This has taken a bit more time than previously planned, though we have learnt many things.

Since we did just swap from Jetson Nano to Jetson Orin Nano, we will also need to make sure that the Jetson Orin Nano does not have any hidden requirements.

Provide an updated schedule if changes have occurred.
We will have to extend our scheduled time for setting up Jetsons (since we changed to the Orin). It’s a good thing we have slack!

What are the most significant risks that could jeopardize the success of the project? How are these risks being managed? What contingency plans are ready?

Currently, we are running into some issues with testing out YOLOv5 on our Jetson due to some of the packages and dependencies being out of date. We’re putting a lot of effort into figuring out which parts of the dependencies are outdated and trying to manually install alternatives. A contingency plan is to try out a newer version of YOLO in hopes that the dependencies would be more up to date and better documented. 

Were any changes made to the existing design of the system (requirements, block diagram, system spec, etc)? Why was this change necessary, what costs does the change incur, and how will these costs be mitigated going forward?

If possible, we are considering removing the Raspberry Pi from the Hexapod and using the Jetson Nano to perform all the actuation of servos. We will just need to confirm that the Raspberry Pi shield from the Hexapod is compatible with the Jetson Nano.

Provide an updated schedule if changes have occurred.

No major changes. We have completed the assembly and bring-up of one robot, as well as setting up a Jetson; now we are trying to train a YOLO network on the Jetson. One thing is we have decided to only order and start with one hexapod rather than all 3, so we are a little behind in that aspect. However, we believe that we will be able to do the bring up for later hexapods much faster.

This is also the place to put some photos of your progress or to brag about a component you got working

Please write a paragraph or two describing how the product solution you are designing will meet a specified need…

• with respect to considerations of public health, safety or welfare. 

The Crisis Critters are intended to help find victims of natural disasters (ie. Earthquakes, building collapse) fast and efficiently. We believe that by finding victims earlier, we can prevent deaths and further deterioration of the victim’s health. Furthermore, these robots also reduce the workload and risk that human rescue teams face in these dangerous environments. 

• With consideration of social factors. 

Computer vision algorithms have historically had problems with biases in the dataset that its being trained with. For example if we train our product with images of people from only one race it might not be able to identify other races. To remedy this we need to have people from different races to make our solution race invariant. 

• with consideration of economic factors. 

With the increase in natural disasters in addition to conflicts around the world, the need for cost effective methods of search and rescue are rising dramatically. The integration of sophisticated autonomous robots would be integral to expediting the SAR process in areas of need. The production of these hexapods will be cheap and would be widely available due to the relatively low cost of the hardware and the public nature of the required software.

The most significant risk right now is whether or not the hardware for the hexapod works well and whether our current search algorithm is effective for the use case. As contingencies we have other hexapod models that we can order if our current one does not function well and we also have looked into alternative search algorithms for our swarm.

Upon further design discussions, we realized that we would need to implement some sort of SLAM algorithm to ensure that each hexapod can know where they are within the search area. This is important because internal tracking of distance can be erroneous due to any slippage on the hexapod. We also need SLAM in order to guarantee that our hexapods can find each other if there are cases where a hexapod carrying first aid needs to reach a survivor that a separate hexapod has located. This adds a bit of time cost to our project and it also may require us to purchase a better camera module or a lidar. We believe that we can still get these parts within our budget and we will do more research into finding the most cost effective solution.

This change means that we need to add an extra week of research and testing on SLAM into our schedule. We will all be working on this during an upcoming week within February.