Posted on

Team Status Report for 3/30/2024

Significant Risks and Contingency Plans

The most significant risk to our project right now is getting the entire rover to work together. We managed to overcome our most significant concern last week by getting the PTZ camera feed to display. That said, it turned out to be a hardware concern, so we are debating if we want to purchase the same camera again, or to use a different camera which also functions. Using a new camera would involve trying to learn the library again (and it is unable to Zoom), but the camera can be displayed. As a result, we simply need to decide between the two cameras, with the contingency plan being the other camera. We also need to make sure everything can come altogether into one system, which can definitely have some unforeseen issues. Right now, we work on two different RPi’s, which would be the contingency plan if we cannot get everything to work on one RPI.

System Changes

As of right now, no major system changes from the last Team Status Report exist. Other than that, we still plan on continuing to implement the system with the original PTZ Camera motor. If we choose to switch to a different camera, that may be a hardware change. If we decide to order the same camera again, then there will be no difference.

Other Updates

Our schedule has been updated, and is reflected in our new Gantt Chart. Our progress has been a bit delayed, since the camera had been very troublesome, but now that that has been mostly resolved, the team can go back to focusing on their individual portions. We plan on having a preliminary end-to-end system working by the interim demo.

Posted on

David’s Status Report for 3/30/2024

Accomplished Tasks

This week was a week of good progress… towards the latter half. I immediately started the week with burning my Raspberry Pi that I was working on. This was because while attempting to hook the RPi up with the rover, I did not realize that the rover shell was metallic. As a result, it shorted the RPi’s bottom, and burned the SD card (and myself). Fortunately, this was good insight for the future, as when we fully assemble the rover, I will remember to be more cautious about parts on the rover. I had to obtain a new RPi, and re-set it all up to be used again. I also obtained the new batteries, and they successfully fit inside the rover, allowing for the rover bottom to be closed appropriately. Slowly the rover is coming together. The JSON commands are still to be sorted out, as it took a while to find the correct port for the RPi to be read from.

On a more important point, working together with my team, we managed to get the camera to work! The camera can now be controlled with the PTZ motor, and the camera feed is displayed. It turned out to be a hardware issue all along, and we are investigating if we should switch to a different camera, but maintain the PTZ portion.

Progress

My progress is still somewhat on track, though I absolutely need to work on the rover JSON commands, which I fortunately can focus on now that the camera data is set up properly. I also need to discuss with how the CV servers will be sending the appropriate movement controls.

Next Week’s Deliverables

Next week, for the interim demo, I plan to have the rover be successfully controlled using programming; in other words, sort out the exact JSON commands needed to move the rover. I also plan on making the interfacing with instruction controls to be as straightforward as possible, so that when the CV part is linked up, it is as painless as possible. While this is still similar to last week’s goals, it mostly stems from being diverted to helping with the camera and being stalled by the RPI burning. The end-to-end version of the rover is a strong goal to work for towards the interim demo.

Posted on

Ronit’s Status Report for 3/30/2024

Tasks accomplished this week

This week, I worked on getting the laser diode working by constructing a small circuit on the breadboard (3+ hours). This involved using the Raspberry Pi to turn on or turn off the diode, so I also had to write a small script to control the pins (3+ hours). This task now allowed us to achieve the laser pointing functionality of the rover.

In addition, as mentioned in the status report last week, I worked on communication from the distributed CV server to the rover (5+ hours). This involved researching on the best way to send instructions, and I implemented a communication protocol over WiFi. I also started working a little bit on the control logic for how much the rover has to turn and where the rover should stop (2+ hours); however, this isn’t fully implemented yet.

Progress

I believe I am on track now, as many of the important components of our system are now implemented; it’s just a matter of testing it out and refining certain components. I am going to work extensively next week with my partners to test out our overall system in time for the interim demo.

Deliverables for next week

For next week, I hope to implement and test out the control logic for the rover. This involves trigonometric calculations and measuring the latency of communication between the CV server and the rover.

Posted on

Nina’s Status Report For 3/30/2024

Accomplished Tasks

This week, I was finally able to get the camera up and running with a stable and high definition video stream on the RaspberryPi desktop. It involved a lot of firmware updates as well as trying out different pieces of hardware such as the ribbon cable used to send camera data to the RPi. I also managed to deal with the stale external trigger issue that would cause the stream to become stale by increasing the timeout of the camera within the config file. Finally, I asked another team to borrow their PTZ camera since they ordered the same one from inventory as us. I realized the camera itself was the issue and ultimately decided to request a new one from inventory albeit without the Pan Tilt Zoom feature. I hope to attach the camera module to the top of the old PTZ Camera in order to continue using the panning and tilting feature without needing to order another PTZ camera and use our budget.

Progress

Currently, I am working on using OpenCV to get the CSI camera to open and begin streaming on my own personal computer. This way, I can create a central hub of communication for the video stream to be extracted and used in our monitoring web application and also to perform real time object detection on. This has proven to be somewhat difficult as the library our camera uses doesn’t currently work with openCV, so I may need to try another method of streaming the camera footage from my computer or else I may have to stream the video from the RPi and send it to my computer. Either way, I will work on reducing latency between our independent elements.

Next Week’s Deliverables

Next week, I will work on the video communication between Ronit’s object detection server and my web application since the video will be used in both of our subsystems. Then, I will work on adding more features to my web application which was previously delayed to the new camera issue caused by our conversion from drone to rover.

Posted on

Ronit’s Status Report for 3/23/2024

Tasks accomplished this week

This week, I primarily worked on getting video feed from the PTZ Camera on the Raspberry Pi. This task was crucial as it handled communication between the rover and the distributed CV server. I spent most of my time researching and implementing logic for receiving, decoding, and interpreting the byte stream received from the Raspberry Pi (8+ hours). I also worked on using the video frames and feeding it into YOLOv5 object detection (2+ hours)

Furthermore, I worked with Nina on the PTZ functionality (4+ hours). This involved debugging the stock code received with the camera to ensure the movability of the PTZ motors. This brings us significantly closer to our goals with the rover, as we would now be able to capture different angles.

Progress

I believe I am a little behind, as communication from the distributed CV server to the rover must be worked out. However, this is not much of a concern as I was able to figure out how exactly communication over WiFi can be implemented between the two components. I am going to work extensively this week to make sure the rover is able to receive instructions and execute them in a timely manner before the interim demo.

Deliverables for next week

As mentioned above, I am going to work extensively on communication from the CV server to the rover. Additionally, I am hoping to make headway on the trigonometric calculations (control logic) that must be performed on the server to ensure that the rover orients itself correctly if a human is detected.

Posted on

Team Status Report for 3/23/2024

Significant Risks and Contingency Plans

The most significant risk to our project right now continues to be getting the PTZ Camera to work. There had been extensive debugging performed, with little to no success. Upon obtaining a new Raspberry Pi and setting it up with a new, proper Bullseye OS, we achieved a huge goal: the PTZ camera movements were now controlled. However, the camera video feed data proves to not be working, showing errors of unable to find camera. This could potentially be a hardware issues, in which case there are plans to immediately order a new camera, or it could still continue to be a software issue. Contingency plans still involve getting a new camera, or possibly switching the camera out entirely.

We also made progress on the rover, and have managed to set up wireless communication with the RPi over wifi successfully. This means that the communication link between the rover and the main “Local Base” information computer has been established and works. This was a critical component needed for controlling the rover in our final implementation. The main goals now are ensuring that the hardware functions properly.

System Changes

As of right now, no major system changes from the last Team Status Report exist. Other than that, we still plan on continuing to implement the system with the original PTZ Camera. If the camera video feed data can still not be properly functioning soon, we may need to resort to finding new hardware.

Other Updates

There have been no schedule changes nor any other updates. Our progress has been a bit delayed, since the camera is proving to be very troublesome. As stated last week, we plan on having a full end-to-end system working by the end of this week, though that is mostly a stretch goal by now. The camera video data must be obtained soon, or we will end up behind schedule.

Posted on

David’s Status Report for 3/23/2024

Accomplished Tasks

This week was the week of the Ethics Lecture. As such, I spent out first class discussing ethics with other teams regarding our and their projects. In regards to the project, the camera has been running into issues, causing concurrent work to be difficult. I set up an entirely new Raspberry Pi, going through the process of connecting it to wifi, and installing a (correct!) Bullseye OS. In terms of actual progress, I installed VNC Viewer onto the RPi, allowing for remote work to the RPi, making it much less of a hassle of needing to always have a spare monitor and keyboard with me. I set up SSH and successfully got scp commands to work from my own computer (the non-VNC Viewer desktop), meaning that there is a successful established connection with the RPi from our “Local Base” information computer in our final design.

It turns out that setting up the new RPi also made progress with the camera, allowing us to control the camera, but unfortunately the camera video data still has issues. That work is still to be done by my teammates. As of writing this post, I am still working on working out the correct JSON commands to send to the rover. Also, the new smaller batteries for the rover came in!

Progress

My progress is still somewhat on track, though dragged notably behind by the lack of being able to work concurrently on the project as my teammates. By setting up a whole new RPi, I have managed to set my work progress back on target. The camera video data is still a concern.

Next Week’s Deliverables

Next week, before our meeting with Prof. Kim, I plan to have the rover be successfully controlled using programming; in other words, sort out the exact JSON commands needed to move the rover. While this is similar to last week’s goals, it mostly stems from being diverted to helping with the camera and being stalled on my own side. The end-to-end version of the rover is still a stretch goal, due to the troubles with a camera, but still a very important deliverable to try and achieve.

Posted on

Nina’s Status Report For 3/23/24

Accomplished Tasks

This week, I was working on the ArduCam and PTZ gimbal and trying to get it to display a camera stream on the RaspberryPi Desktop. Due to issues with the old RPi4 OS installation, I wiped the SD card and did a clean installation of the Bullseye OS on the SD card and installed the OpenCV and Libcamera libraries. Thankfully, now the PTZ gimbal interface showed up and I am able to use keypresses to move and tilt the camera around.

Progress

Although the camera is able to pan 180 degrees and tilt, the camera stream is not showing up and only a black preview window is showing.  I’ve been dealing with possibly a firmware error that prevents the camera from showing due to the deprecated libcamera -apps library that has now become rpicam-apps. Even if I wanted to download the rpicam-apps library, it is not compatible with and RPi4 and would need me to order an RPi5. On the other hand, it could be a hardware issue due to a damaged ribbon cable which I will request a new one from Quinn and reattach it. I’ve been struggling a lot with staying on track due to all these unforeseen problems with the camera and the lack of documentation and tech support online. If this camera doesn’t work, I may potentially request a different one or buy a new one altogether that doesn’t require RaspberryPi.

Next Week’s Deliverables

Next week, I plan on getting a video stream whether it comes from the ArduCam or a new camera altogether to integrate with Ronit’s object detection software. In addition, I will make sure that it’s integrated within my web application for monitoring.

Posted on

David’s Status Report for 3/16/2024

Accomplished Tasks

This week was a week of intense progress. Early into the week, I realized that hacking the web application was not a suitable approach, due to the fact that the web app was hosted on the rover’s own wifi. As such, it would not be possible, or at least, rather inconvenient to try and mesh this rover wifi together with school wifi. Thus, I went with the backup plan, which was to communicate with the rover through the onboard Raspberry Pi, and then send the JSON commands through UART communication. This required reading Python documentation on how UART communication worked, and how to send the appropriate JSON commands. The code for this has been flushed out nearly entirely; the only main confusing part is the exact JSON command that should be serialized. The documentation is somewhat confusing and inconsistent, and trying to sort out exactly what is required is where I am currently at. I am also helping with my teammates in working out the camera, which has hit some difficulties in setting up. There appears to be issues involving the OS version of the RPi causing inconsistencies in downloaded packages.

As a side note, the rover works! It can be controlled with the web app, albeit the batteries we purchased were too large. I ordered smaller replacement batteries to resolve this issue.

Progress

My progress is now much more on track. Working out this control code, and having functioning hardware was a huge success. I am concerned about the camera’s status, which is falling behind, so I will devote efforts to making that work as well.

Next Week’s Deliverables

Next week, I plan to have the rover be successfully controlled using programming; in other words, sort out the exact JSON commands needed to move the rover. I also want to have an end-to-end version of the rover running, which, at this point in time, means getting the camera to work properly. This is highest priority, and I plan to help to make sure the camera can function. This could mean working out the bugs, installing a new OS, getting a new camera, etc.

Posted on

Team Status Report for 3/16/2024

Significant Risks and Contingency Plans

The most significant risk to our project right now is getting the PTZ Camera to work. The past week, we worked extensively on downloading libraries and debugging dependencies to get the software API of the PTZ Camera to work to display video feed. We still have to get this functioning correctly, and are working extremely hard to get it done. We are optimistic that we can get the camera to function correctly, as 3/4 of the libraries have been installed correctly, and we are planning on getting the last installed latest by tomorrow to get video feed. In the scenario where the camera doesn’t work correctly, we might consider changing the MCU to the NVIDIA Jetson Nano, which has a lot of easy to use cameras available for cheap. However, this is a worst case scenario and is unlikely to occur.

We also made progress on the rover, and the code to control the search pattern of the rover is complete. We will now focus on tuning the search pattern. As all of our individual components are nearing the end of their implementation, a significant risk is the communication between components and how to use the WiFi module to relay information. A lot of online documentation exists on this, so we plan on conducting a lot of research to identify the best way to handle communication between components. We also have some recommendations from our TA Aden to use scp to transfer information to/from the computer (communication hub) to the rover as well.  If WiFi doesn’t work, we have a contingency plan of using bluetooth communication on the built-in RaspberryPi bluetooth module to relay video data to a central hub (computer) that handles communication. Again, however, this is an unlikely scenario as preliminary research shows that WiFi communication should be feasible to implement.

System Changes

As of right now, no major system changes from the last Team Status Report exist. There was a decision to switch from hacking the web app to using the RPi and UART communication to send JSON commands to the rover. Other than that, we still plan on continuing to implement the system with the PTZ Camera and WiFi communication, and would only consider a pivot from these to alternate technologies mentioned above if absolutely necessary. We would know of these changes as we work more on the communication protocol of the rover.

Other Updates

There have been no schedule changes nor any other updates. Potential schedule changes may occur after we work on the communication protocol of the rover next week. We plan on having a working interim demo with communication between the rover and the server where the rover would stop and turn towards a human if detected in its search pattern by the end of next week.