Posted on

Ronit’s Status Report for 3/16/2024

Tasks accomplished this week

This week, I primarily worked on getting the PTZ Camera working with Nina. I spent most of my time working on installing libraries, upgrading the Raspberry Pi OS from Buster to Bullseye, and debugging installation dependencies (6+ hours). We were able to make significant progress and we would be able to get the camera up and running by the weekend.

Furthermore, I wrote code to integrate the YOLOv5 algorithm in the CV server nodes as planned (2+ hours). This essentially means that everything in the distributed CV server is completed, with the remaining task of relaying information to different components in the system. I also wrote a lot of unit tests to ensure the integrity of the system (6+ hours). With this done, I can fully devote my time to the PTZ Camera and communication between components.

Progress

In general, I still believe I am a little behind, as I hoped to have the camera up and running; however, we are extremely close to getting this done. I am going to work extensively this week to make sure that the PTZ Camera is working correctly, and that we are able to relay video data to the CV server from the Raspberry Pi.

Deliverables for next week

As mentioned above, I still need to work with Nina to get the PTZ Camera completely up and running. This should be straightforward now, as a significant amount of work has been done; only some basic libraries need to be installed. I also plan on working on the communication and integration of the Camera and the Distributed server next week, allowing us to have functional live computer vision in time before the interim demo.

Posted on

Nina’s Status Report For 3/16/2024

Accomplished Tasks

This week, I received the Arducam with PTZ Gimbal from inventory along with a Raspberry Pi 4 in order to control it. I was able to attach the Arducam to the PTZ Gimbal using an acrylic sheet and screws in order to have 180 degree panning. In addition, I used wires to connect the Arducam and gimbal to the RPI and set up the RPi as well. David helped me with the setup process on the RPi as it was our first time using one and we were missing extra tools such as a monitor, keyboard, and wired mouse. To test the camera on the RPi desktop, I found some sample code online and the corresponding libcamera packages used for the Arducam.

Progress

I spent a majority of my time setting up the Raspberry Pi as there were issues connecting it to CMU-DEVICE. In addition, since the RPi  wasn’t the latest model or updated to the latest OS, we were unable to download essential packages and dependencies. In order to fix this, I had to manually convert the OS version from Buster to Bullseye since the picamera2 library is not supported. There was many issues regarding installations (couldn’t use pip3 to install since it was deprecated, deprecated config file name in RPi sources list, lack of swap to install dependencies, and deprecated commands in general from the outdated set up guide of this camera). Because of these issues, I faced a lot of errors just running the code. After finally getting a preview window when running ‘libcamera-hello’ to show the camera’s perspective, it stopped working after updating the OS. However, the code now runs without import errors, so I believe it’s fixable after rebuilding and compiling the libcamera library from source.

Next Week’s Deliverables

Next week, I plan on getting a working stream of the camera visible from the RPI desktop and have it attached to the camera mount that will be on top of the rover and continue refining the code that will move the PTZ gimbal. In addition, I will help with the laser modules feature that will be connected to the camera in order to have our spotlight implemented.

Posted on

Team Status Report for 3/9/2024

Significant Risks and Contingency Plans

The most significant risks to our project right now is getting everything to work. The last two weeks, we worked on quickly readjusting our project to accommodate for the fact that we are now using a rover instead of a drone. This meant developing new plans for our implementation and our overall system so that our project could maintain the same functionality as before. Now, the key risks are making sure that our new implementation strategies will work. To ensure this is the case, we made sure to try and not change what we had from before, and then analyze what differences occurred now that we had a rover. This was easier to handle since in the past, we had focused on modularity, meaning the CV servers and the website were not affected as heavily. To add to this modularity approach to add an information hub to help with information flow in our project, and also providing a barrier to note where one module starts and ends.

Regarding the rover, our main concern is having controlled communication with the rover. We are working on using a pre-made web app to learn existing methods of communication, then interface/hack off these concepts. Should this method not work, we also plan on having a Raspberry Pi aboard the rover (for the camera video data), and we can also send JSON commands to this Raspberry Pi instead. The JSON commands can then be transferred to the rover using UART communication.

System Changes

We have conducted a significant amount of research to figure out how communication and functionality of the rover would work. We decided that the easiest and practical way for communication would be through the ESP32 module on the rover. Additionally, using a Raspberry Pi module on the rover would allow us to control the laser pointer and gather video data from the camera. Apart from these inclusions, there have been no major system changes, and it seems like integration of all the components would be seamless through WiFi communication. There might be some changes in the way the distributed CV server and the Django Web server would receive and send out information, and we would know of any changes once we work on figuring out the communication protocol of the WAVESHARE rover via the Raspberry Pi.

Other Updates

There have been no schedule changes nor other updates. Potential schedule changes may occur after we work on the rover next week after its delivery, notably as we attempt to accomplish our goal (as requested by Prof. Kim) of having an end-to-end solution by our meeting Wednesday. If this is not accomplished, then the plan is to have something close to this by the end of the week.

Specified Needs Fulfilled Through Product Solution

We also make some further considerations that are important to think about while we develop our product. We identify these considerations by breaking them down into three more groups. The first group (A) deals with global factors. The second group (B) deals with cultural factors. The third group (C) involves environmental considerations. A was written by Ronit, B was written by Nina, and C was written by David.

Consideration of global factors (A)

Our product of a search and rescue rover heavily deals with global factors. The need for efficient and cost-effective search and rescue operations is inherently a global concern, as disaster and war could affect any country in the world. In general, our rover would be able to leave a positive economic impact on the global economy, as risks to human life are mitigated by removing the need for manual intervention in search and rescue operations. Furthermore, our rover caters to different terrains, making it usable in various geographical locations and scenarios. 

The technologies our rover uses also considers global factors. The use of a web application that could be accessed by users anywhere in the world could allow for real-time collaboration between various international rescue agencies. Using WiFi to communicate with various components in our system could allow us to expand our system to incorporate satellite WiFi, allowing for the distributed server to communicate with any rover anywhere in the world. Using a distributed system also improves scalability of the product, as multiple rovers in multiple rescue missions could send their video data for analysis, improving efficiency of rescue operations that might be happening concurrently in different locations.

Cultural considerations (B)

During the development of our search and point rover, it’s important to be mindful of its use case across different communities and cultures. Some features to consider would be adapting the rover’s interface to support different languages, communication styles, and symbols. In addition, abilities to customize and personalize the rover would allow users to tailor the rover’s interface to their cultural preferences. Ensuring that the rover does not impose on sacred cultural areas, being mindful of gender sensitivities, and privacy considerations are also vital. Additionally,  receiving feedback from these different cultural communities in the rover’s design facilitates optimized technology that respects cultural and ethical values of diverse populations. By integrating these cultural considerations, we can create a search and point rover that is not only technologically advanced but also respectful across various cultural backgrounds.

Environmental considerations (C)

with consideration of environmental factors. Environmental factors are concerned with the environment as it relates to living organisms and natural resources.

Search and Point would have no environmental side effects whatsoever. As the rover performs its tasks of searching and pointing the laser pointer, it does not leave behind any sort of residual presence. The rover is battery-powered, allowing it to not have any harmful byproducts (unlike gasoline-powered motors). The batteries are also rechargeable, allowing for a renewable and safe way of maintaining the rover’s functionality. The decentralized system revolving around a centralized information hub means that the actual part of the project that is deployed into the environment is actually quite small; all the main computing power is offloaded to servers elsewhere. The only part of the environment our project actually leaves an imprint on is the rover running over the landscape, which is an arguably negligible part of environmental damage. The laser we chose is also a relatively low power laser, meaning that there are not any harmful side effects from using the laser.

Posted on

Ronit’s Status Report for 3/9/2024

Tasks accomplished this week

This week, I primarily focused on writing the design report. I was responsible for completing the sections for our system architecture (2+ hours), design requirements (4+ hours), portions of trade studies (2+ hours), and testing and validation (3+ hours). I also took up responsibilities for making the diagrams in the report and general formatting (2+ hours).

Furthermore, I was able to implement the sequential YOLOv5 algorithm (3+ hours). This will serve as a baseline for comparisons of speedup and also gave me a significant insight into how I can implement it with the Distributed server. This shouldn’t take long to implement now, and then I can focus on writing test cases to verify the integrity of the distributed system.

Progress

I am still a little bit behind on my progress, as I hoped to have all testing and verification for the distributed server. However, this is okay, as I still have some slack time left. Furthermore, finishing unit testing will reduce time taken in the future to verify everything. I will work next week to complete all testing and implementations so that I can focus on the rover.

Deliverables for next week

As mentioned above, I still need to implement the YOLOv5 algorithm into the children server nodes. This should be straightforward now that the sequential algorithm is implemented. I also plan on finishing writing more comprehensive unit tests to ensure that all parts of the distributed server work as expected. This would involve somehow gathering a validation dataset, which I have to look into thoroughly.

Posted on

David’s Status Report for 3/9/2024

Accomplished Tasks

These weeks were the weeks for the Design Review and Spring Break. A large chunk of my first week was spent on writing portions of the design review. I was primarily in charge of writing the Design Implementation portion, along with Trade Studies. This took a rather long time, as due to the recent change in our plans (eg. drone -> rover), I had to rework how exactly our rover project would work. The details of the plans are inside the report, but essentially speaking, our project now consists of a rover with a mounted camera/laser that will search and point as previously planned. The overall structure of our system now has an information hub, which will help with information control/parsing/propagation to the three other parts (the rover, the website, and the CV servers).

During Spring Break, I was mostly travelling. However, I also devoted what time I could into looking into the rover communication methods. Since this section of communication is the most critical to be able to have rover guidance, I looked into how the pre-made web application was able to send the JSON commands to the rover. I am currently working on interfacing and/or hacking this web application to be able to send the commands that we desire instead.

Progress

The original goal was to work out all the rover controls by this Wednesday. Prof. Kim had an even larger request of managing to have a full end-to-end product working by our meeting on Wednesday. Our rover, which we had ordered prior to Spring Break was set to arrive over Spring Break, but we have not had any notification of its arrival yet, which is concerning. Since I am in the process of working out the rover controlling right now, I am somewhat on track, though depending on how much progress I am able to accomplish over the coming few days, I may end up slightly behind schedule. Following Prof. Kim’s request would put us ahead of schedule.

Next Week’s Deliverables

Next week, I plan to have the rover be fully controllable using programming. The stretch goal is to have an end-to-end version of the final rover running by our meeting with Prof. Kim on Wednesday. For me, these goals should be essentially the same, as due to the modularity of our system design, the interactions between the three components should be relatively small. Rover movement may need to be packaged and encoded when received from the CV servers, but this should not be a small end portion; getting the rover to move in a general controlled manner is the main plan and goal.

Posted on

Nina’s Status Report For 3/9/2024

Accomplished Tasks

For these two weeks, I was busy working on ordering parts (rover, batteries, camera, etc). In addition, I worked mostly on the design review report and was tasked with the introduction, use-case requirements, part of the design requirements, revising the system implementation, project management, related work, and the summary. Much of the time was spent on researching related works that would help augment our proposal as well as how we should be formatting our system.

During spring break, I was away on a trip. I did pick up some inventory requests and made additional orders of parts that we needed . I also began looking into the camera mount 3D printing that Ronit and I would need to work on and how to build a frame of the mount through AutoCAD.

Progress

Since our rover has yet to arrive, I will just be working primarily on designing the camera mount that would be hoisting the camera 2-3+ ft on top of the rover. Regardless, of whether or not it will be done through 3D print or not, I will try to have this done by the end of the week. This is doable without the rover as we have the dimensions of the rover as well as the camera itself. I’m also working on getting a live feed of the video from the ArduCam using its API that works with OpenCV.

Next Week’s Deliverables

Next week, I plan on working on finishing the camera mount and have it set up properly on the rover with the laser pointer incorporated as well. I will be checking the camera’s perspective and have the camera feed be shown on the website prior to it being mounted. Then, I will work on adding additional features to the site listed in the design review such as an emergency overdrive button to enable manual control of the rover.