Posted on

David’s Status Report for 4/6/2024

Accomplished Tasks

This week was the week of the Interim Demo. In regards to this, I managed to present the biggest success I had this project: getting the rover to move under my command through my own code! It involved a lot of hours of work, running into trouble after trouble. The inconsistency of documentation certainly did not help, since the JSON commands were unclear. However, even worse was the number of Raspberry Pi errors. My original Raspberry Pi had a broken TX pin that caused the JSON commands to be unable to be sent over UART communication; this was only debugged with TA Aden’s help (THANK YOU!). Another RPi I tried struggled to connect to the Internet despite visibly being connected. Fortunately, the last RPi I tried (the FIFTH one) managed to work, and I was able to control the rover through the RPi. As a reminder, this means that I am now able to control the rover through a computer connected to CMU wifi, effectively allowing for the system communication to all come together.

In order to verify the rover movement system works correctly, I have implemented benchmark testing to get a better understanding as to how much power needs to be given for the rover to move as directed (eg. how much power is needed to turn left). I also test whether or not the rover can drive straight. This involves testing the rover with predetermined simple paths (like a rectangle), and seeing if it can return to the original position, and facing in the right way.

See the rover move!

Progress

My progress is quite on track, though the end-to-end demonstration has not been put together yet. We had ideally wanted this to work a week or so ago, so all my focus will be on how to put everything together, including system communication and the infrastructure on the rover.

Next Week’s Deliverables

Next week, I plan to have the communication working out between the CV servers and the rover controlling system. This involves investigating the threading abilities of the RPi, along with figuring out how to translate the CV-detected person to directional commands. Working these out would finalize communication amongst the whole system, enabling everything to be put together.

Posted on

Nina’s Status Report For 4/6/2024

Accomplished Tasks

This week, I was able to get a TCP stream running on my personal computer rather than just having the camera stream show on the RaspberryPi desktop by running a libcamera-vid script. I did so through the use of picamera2 documentation that allowed me to capture the video stream and point it to a web browser. In addition, I embedded this stream link using iframe into my website application to introduce formal monitoring of where our rover will be searching. Furthermore, I was able attach and implement a separate imx219 camera to our rover using the ribbon cable but still keep the i2c connections for the PTZ gimbal. This way, we can still have an ongoing camera feed but also have the ability to move our camera around for 180 degree panning.

Progress

Currently, I am working on introducing more features to my web application such as incorporating keypresses that allow for manual movement of the camera PTZ gimbal, however, I am struggling with communication across devices since the keypresses must register on the RaspberryPi desktop. In addition, I am trying to implement concurrent ability to move and stream from the camera. Right now, I am only able to have mutually exclusive use of the two features since both requires the camera. However, I am planning on using threading to possibly have 2 threads spawning in order for the camera to work or add an additional RaspberryPi for one of the features and a power bank to power it separately.

Next Week’s Deliverables

Next week, I will be working on the camera mount design since we finally have the camera working. After inquiring about the 3D print and seeing how expensive it may be, I will likely be working with laser cutting a mount for the camera, laser, and gimbal combo in order to attach to the rover. Additionally, I will work on the frontend portion of the web application and get it hosted on an AWS server where I will further work on the security of the website.

Verification

Since I will be working on the verification of the camera stream latency as well as the security of the monitoring site to make it resistant to hacker attacks, I will be checking for the real-time processing of stream data to my website application and ensure it is under 50ms to minimize communication delay of where a person is to the object detection server. To do so, I will use Ping, a simple command-line tool that sends a data packet to a destination and measures the round-trip time (RTT). I will ensure the live camera feed from the RPi to the stream on my website will be under 50ms through this method. Furthermore, I will be checking the security of the website by using vulnerability scanning tools that will check for site insecurities such as cross-site scripting, SQL injection, command injection, path traversal and insecure server configuration. I will use Acunetix, a website scanning tool, that performs these tests in order to formally check the site. In addition, to prevent unwanted users from accessing the site, I will use Google OAuth to authenticate proper users (rescue workers) to be the only ones to access the site. I will get my friends to test or “break” the site by asking them to perform a series of GET and POST requests to see if they can access any website data as an authorized user. To prevent these from happen, I will introduce cross-site request forgery tokens to ensure data will not be improperly accessed.

Posted on

Team Status Report for 3/30/2024

Significant Risks and Contingency Plans

The most significant risk to our project right now is getting the entire rover to work together. We managed to overcome our most significant concern last week by getting the PTZ camera feed to display. That said, it turned out to be a hardware concern, so we are debating if we want to purchase the same camera again, or to use a different camera which also functions. Using a new camera would involve trying to learn the library again (and it is unable to Zoom), but the camera can be displayed. As a result, we simply need to decide between the two cameras, with the contingency plan being the other camera. We also need to make sure everything can come altogether into one system, which can definitely have some unforeseen issues. Right now, we work on two different RPi’s, which would be the contingency plan if we cannot get everything to work on one RPI.

System Changes

As of right now, no major system changes from the last Team Status Report exist. Other than that, we still plan on continuing to implement the system with the original PTZ Camera motor. If we choose to switch to a different camera, that may be a hardware change. If we decide to order the same camera again, then there will be no difference.

Other Updates

Our schedule has been updated, and is reflected in our new Gantt Chart. Our progress has been a bit delayed, since the camera had been very troublesome, but now that that has been mostly resolved, the team can go back to focusing on their individual portions. We plan on having a preliminary end-to-end system working by the interim demo.

Posted on

David’s Status Report for 3/30/2024

Accomplished Tasks

This week was a week of good progress… towards the latter half. I immediately started the week with burning my Raspberry Pi that I was working on. This was because while attempting to hook the RPi up with the rover, I did not realize that the rover shell was metallic. As a result, it shorted the RPi’s bottom, and burned the SD card (and myself). Fortunately, this was good insight for the future, as when we fully assemble the rover, I will remember to be more cautious about parts on the rover. I had to obtain a new RPi, and re-set it all up to be used again. I also obtained the new batteries, and they successfully fit inside the rover, allowing for the rover bottom to be closed appropriately. Slowly the rover is coming together. The JSON commands are still to be sorted out, as it took a while to find the correct port for the RPi to be read from.

On a more important point, working together with my team, we managed to get the camera to work! The camera can now be controlled with the PTZ motor, and the camera feed is displayed. It turned out to be a hardware issue all along, and we are investigating if we should switch to a different camera, but maintain the PTZ portion.

Progress

My progress is still somewhat on track, though I absolutely need to work on the rover JSON commands, which I fortunately can focus on now that the camera data is set up properly. I also need to discuss with how the CV servers will be sending the appropriate movement controls.

Next Week’s Deliverables

Next week, for the interim demo, I plan to have the rover be successfully controlled using programming; in other words, sort out the exact JSON commands needed to move the rover. I also plan on making the interfacing with instruction controls to be as straightforward as possible, so that when the CV part is linked up, it is as painless as possible. While this is still similar to last week’s goals, it mostly stems from being diverted to helping with the camera and being stalled by the RPI burning. The end-to-end version of the rover is a strong goal to work for towards the interim demo.

Posted on

Ronit’s Status Report for 3/30/2024

Tasks accomplished this week

This week, I worked on getting the laser diode working by constructing a small circuit on the breadboard (3+ hours). This involved using the Raspberry Pi to turn on or turn off the diode, so I also had to write a small script to control the pins (3+ hours). This task now allowed us to achieve the laser pointing functionality of the rover.

In addition, as mentioned in the status report last week, I worked on communication from the distributed CV server to the rover (5+ hours). This involved researching on the best way to send instructions, and I implemented a communication protocol over WiFi. I also started working a little bit on the control logic for how much the rover has to turn and where the rover should stop (2+ hours); however, this isn’t fully implemented yet.

Progress

I believe I am on track now, as many of the important components of our system are now implemented; it’s just a matter of testing it out and refining certain components. I am going to work extensively next week with my partners to test out our overall system in time for the interim demo.

Deliverables for next week

For next week, I hope to implement and test out the control logic for the rover. This involves trigonometric calculations and measuring the latency of communication between the CV server and the rover.

Posted on

Nina’s Status Report For 3/30/2024

Accomplished Tasks

This week, I was finally able to get the camera up and running with a stable and high definition video stream on the RaspberryPi desktop. It involved a lot of firmware updates as well as trying out different pieces of hardware such as the ribbon cable used to send camera data to the RPi. I also managed to deal with the stale external trigger issue that would cause the stream to become stale by increasing the timeout of the camera within the config file. Finally, I asked another team to borrow their PTZ camera since they ordered the same one from inventory as us. I realized the camera itself was the issue and ultimately decided to request a new one from inventory albeit without the Pan Tilt Zoom feature. I hope to attach the camera module to the top of the old PTZ Camera in order to continue using the panning and tilting feature without needing to order another PTZ camera and use our budget.

Progress

Currently, I am working on using OpenCV to get the CSI camera to open and begin streaming on my own personal computer. This way, I can create a central hub of communication for the video stream to be extracted and used in our monitoring web application and also to perform real time object detection on. This has proven to be somewhat difficult as the library our camera uses doesn’t currently work with openCV, so I may need to try another method of streaming the camera footage from my computer or else I may have to stream the video from the RPi and send it to my computer. Either way, I will work on reducing latency between our independent elements.

Next Week’s Deliverables

Next week, I will work on the video communication between Ronit’s object detection server and my web application since the video will be used in both of our subsystems. Then, I will work on adding more features to my web application which was previously delayed to the new camera issue caused by our conversion from drone to rover.

Posted on

Ronit’s Status Report for 3/23/2024

Tasks accomplished this week

This week, I primarily worked on getting video feed from the PTZ Camera on the Raspberry Pi. This task was crucial as it handled communication between the rover and the distributed CV server. I spent most of my time researching and implementing logic for receiving, decoding, and interpreting the byte stream received from the Raspberry Pi (8+ hours). I also worked on using the video frames and feeding it into YOLOv5 object detection (2+ hours)

Furthermore, I worked with Nina on the PTZ functionality (4+ hours). This involved debugging the stock code received with the camera to ensure the movability of the PTZ motors. This brings us significantly closer to our goals with the rover, as we would now be able to capture different angles.

Progress

I believe I am a little behind, as communication from the distributed CV server to the rover must be worked out. However, this is not much of a concern as I was able to figure out how exactly communication over WiFi can be implemented between the two components. I am going to work extensively this week to make sure the rover is able to receive instructions and execute them in a timely manner before the interim demo.

Deliverables for next week

As mentioned above, I am going to work extensively on communication from the CV server to the rover. Additionally, I am hoping to make headway on the trigonometric calculations (control logic) that must be performed on the server to ensure that the rover orients itself correctly if a human is detected.

Posted on

Team Status Report for 3/23/2024

Significant Risks and Contingency Plans

The most significant risk to our project right now continues to be getting the PTZ Camera to work. There had been extensive debugging performed, with little to no success. Upon obtaining a new Raspberry Pi and setting it up with a new, proper Bullseye OS, we achieved a huge goal: the PTZ camera movements were now controlled. However, the camera video feed data proves to not be working, showing errors of unable to find camera. This could potentially be a hardware issues, in which case there are plans to immediately order a new camera, or it could still continue to be a software issue. Contingency plans still involve getting a new camera, or possibly switching the camera out entirely.

We also made progress on the rover, and have managed to set up wireless communication with the RPi over wifi successfully. This means that the communication link between the rover and the main “Local Base” information computer has been established and works. This was a critical component needed for controlling the rover in our final implementation. The main goals now are ensuring that the hardware functions properly.

System Changes

As of right now, no major system changes from the last Team Status Report exist. Other than that, we still plan on continuing to implement the system with the original PTZ Camera. If the camera video feed data can still not be properly functioning soon, we may need to resort to finding new hardware.

Other Updates

There have been no schedule changes nor any other updates. Our progress has been a bit delayed, since the camera is proving to be very troublesome. As stated last week, we plan on having a full end-to-end system working by the end of this week, though that is mostly a stretch goal by now. The camera video data must be obtained soon, or we will end up behind schedule.

Posted on

David’s Status Report for 3/23/2024

Accomplished Tasks

This week was the week of the Ethics Lecture. As such, I spent out first class discussing ethics with other teams regarding our and their projects. In regards to the project, the camera has been running into issues, causing concurrent work to be difficult. I set up an entirely new Raspberry Pi, going through the process of connecting it to wifi, and installing a (correct!) Bullseye OS. In terms of actual progress, I installed VNC Viewer onto the RPi, allowing for remote work to the RPi, making it much less of a hassle of needing to always have a spare monitor and keyboard with me. I set up SSH and successfully got scp commands to work from my own computer (the non-VNC Viewer desktop), meaning that there is a successful established connection with the RPi from our “Local Base” information computer in our final design.

It turns out that setting up the new RPi also made progress with the camera, allowing us to control the camera, but unfortunately the camera video data still has issues. That work is still to be done by my teammates. As of writing this post, I am still working on working out the correct JSON commands to send to the rover. Also, the new smaller batteries for the rover came in!

Progress

My progress is still somewhat on track, though dragged notably behind by the lack of being able to work concurrently on the project as my teammates. By setting up a whole new RPi, I have managed to set my work progress back on target. The camera video data is still a concern.

Next Week’s Deliverables

Next week, before our meeting with Prof. Kim, I plan to have the rover be successfully controlled using programming; in other words, sort out the exact JSON commands needed to move the rover. While this is similar to last week’s goals, it mostly stems from being diverted to helping with the camera and being stalled on my own side. The end-to-end version of the rover is still a stretch goal, due to the troubles with a camera, but still a very important deliverable to try and achieve.

Posted on

Nina’s Status Report For 3/23/24

Accomplished Tasks

This week, I was working on the ArduCam and PTZ gimbal and trying to get it to display a camera stream on the RaspberryPi Desktop. Due to issues with the old RPi4 OS installation, I wiped the SD card and did a clean installation of the Bullseye OS on the SD card and installed the OpenCV and Libcamera libraries. Thankfully, now the PTZ gimbal interface showed up and I am able to use keypresses to move and tilt the camera around.

Progress

Although the camera is able to pan 180 degrees and tilt, the camera stream is not showing up and only a black preview window is showing.  I’ve been dealing with possibly a firmware error that prevents the camera from showing due to the deprecated libcamera -apps library that has now become rpicam-apps. Even if I wanted to download the rpicam-apps library, it is not compatible with and RPi4 and would need me to order an RPi5. On the other hand, it could be a hardware issue due to a damaged ribbon cable which I will request a new one from Quinn and reattach it. I’ve been struggling a lot with staying on track due to all these unforeseen problems with the camera and the lack of documentation and tech support online. If this camera doesn’t work, I may potentially request a different one or buy a new one altogether that doesn’t require RaspberryPi.

Next Week’s Deliverables

Next week, I plan on getting a video stream whether it comes from the ArduCam or a new camera altogether to integrate with Ronit’s object detection software. In addition, I will make sure that it’s integrated within my web application for monitoring.