12/7/2024
I first started this week by making the finishing touches on the lane detection system in order to work properly with our demo environment. Thus mainly involved figuring out how to do color distinction between the lanes and the logic between the colors detected, their positions, and how that determined the lane that the car was located in. Next, I worked on various tasks involved with setting up our testing environment, such as assembling our lanes, testing the video feed when placed on top of the RC car and setting up the lights. I spent the rest of the week figuring out how to SSH so that we could connect one of our local computers to the Jetson in order to control it and see output without a need for a monitor. We intended to use this method in order to see the video feed outputs, both for unit testing and for our final demo. However, this method proved to be too slow, as the communication time made it such that only a few frames showed up every second. Thus, in order to solve this speed issue, I set up a socket script on both my personal computer and the Jetson in order to connect the two, and thus communicate the video feed significantly quicker.
I think my progress is a little bit behind schedule, because I wanted to run full verification tests for the lane detection system and develop metrics of its success rate this week. However, everything is now set in place for these metrics to be obtained, which is how I will start the coming week. Additionally, we will finalize everything for our demo and prepare our final report, poster, and video.
11/30/2024
This week, I helped finalize the purchases for the RC car with Eunice and the traffic light demo equipment with Emily. Additionally, I spent the majority of my time trying to redesign the Lane Detection algorithm approach in order to hopefully make it more efficient at detecting the lanes. This involved going back through many of the opencv tutorials to try and find different methods and parameters when it came to steps such as noise reduction, canny edge detection, and performing houghlines transforms. Currently, I split the focus on the canny edge detection into two separate edge types in order to hopefully capture more of the edges within a frame.
I am still looking to experiment more with the HSV filtering in an attempt to try and differentiate the detected lines between yellow and white lane markers. This should overall help me fully flesh out the new lane detection. Additionally, next week, I plan to integrate the LD with the TLD for the full system and test it out on our demo environment. As of now, my progress is just slightly behind and I hope to remedy this by finalizing the lane detection early into next week.
11/16/2024
This week, I helped verify many different systems. In the beginning, I worked with Eunice on modifying the TLD. Our previous system involved using the YOLO model to detect both the actual traffic light and the color, however, while the traffic light position was often incredibly accurate, the color was often wrong during testing. Thus, I helped her adapt the system to instead split each detected traffic light into thirds, and used alpha intensity to determine which light was actually on. I additionally collected more image data to run tests on. Afterwards, I shifted focus onto the radar. Here, we were having troubles with the system having interference from close range signals, specifically those that kept the detection at 0.7m. Emily and I consulted with the sales representative from the company and used some of the API commands to be able to tell the radar to ignore signals from within a certain distance. Afterwards, we tested the initial FCD code that Emily wrote, tweaking as we say necessary. Finally, Emily and I went out to grab real video data footage to run with both the LD and TLD systems. Additionally, I also debugged much of the LD system. I focused on mainly shifting the region of interest for the Hough Transform to detect lines within, having a better idea of where the road will actually show up on video, as well as write code that filtered out detected lines based on slope.
At the end of this week, the TLD is getting better at detecting the correct colors, but still needs better integration tests with the LD and with real video footage we gathered. The LD system needs better calibration on the slope detection in order to ensure we only keep road lines. Afterwards, we will need to verify that the lane numbering subsystem of the LD works. Finally, for the FCD, we want to run tests in more open fields, with less obstacles that could cause interference, and to better understand the full range of our radar, as well as test it through a real car window as opposed to the clear acrylic we used throughout the week.
Overall, the verification work done throughout this week has helped us hone in on the specific areas of each subsystem where our project needs more attention, and we will be sure to look at them throughout the upcoming week.
11/9/2024
Personally this week, I worked on much of the data synchronization and working on connecting the components together. I first managed to adapt the original radar script to work on the Jetson. Specifically, our original script worked on Windows ports, but now it has been adapted to work on the serial ports of the Jetson Orin. Additionally, I wrote a script to sample the camera video and the radar at the time, able to produce the frame and corresponding radar read at the same time. Furthermore, I attempted to connect the GPS to our current interface. However, I realized that while I could read from the GPS into our Jetson, our GPS is unable to work correctly because it is missing a receiver which allows it to communicate with a satellite to obtain information such as latitude, longitude, and speed. Towards the end of the week, Emily and I worked with testing her code for the forward car departure code as well as figuring out the limitations of the radar sensor, such as being able to detect through glass, figuring out the detection cone of the sensor, and testing the FMCW relative speed sensor. Finally, I helped train the YOLOv8n model for traffic light detection.
Due to a lot of the parallel work our team did throughout this week, I feel we managed to catch up a lot on our progress with our project, especially given our initial setbacks. Throughout next week, we hope to fully catch up by doing more testing and refinement of our traffic light detection and forward car departure systems. Additionally, we will try to begin integration between the traffic light and lane detection systems.
11/2/2024
This week, I was primarily responsible for trying to fix our old Jetson Orin as well as order a new one under the circumstances I was unsuccessful. Additionally, I helped research for a new camera model and GPS to replace our incompatible components. Additionally, I procured datasets for the traffic light detection process and started development on the lane detection algorithm.
Currently, I feel that we are very behind. Many components needed to be ordered this week, we still do not have a working computer board, and I think we are behind on the development of our main project. Overall, for the upcoming week, I plan to give fixing the Jetson one more try before moving onto the back up Nano, and fully commit to developing the lane detection and start work on the traffic light detection.
10/26/2024
This week, I worked on setting up some of the data synchronization programs for our project. Specifically, I installed the Tera Term program that allowed us to access the OPS243C radar API and wrote a python script to run the program and pipe its output to the python script. Furthermore, I helped troubleshoot our Jetson Orin after it stopped working. Finally, I also started working on the data synchronization from the dashcam, where I quickly realized we needed a different product, as it did not come with a reliable real time streaming protocol (RTSP) that we could use to stream the video directly to the computer.
My progress is certainly behind schedule. Between the Jetson not working and the dashcam not having certain capabilities, I am adjusting my personal schedule to try and accommodate more tasks. Over the next week, I plan to order a new camera model with RTSP and finish off the data synchronization. Additionally, I plan to also write starter code for the lane detection model which I can unit test on my personal computer.
10/20/2024
This week, I spent time writing the System Specification and Design Trade Studies sections of our team’s design report. Additionally, I spent time researching and studying the specific machine learning techniques and detection algorithms we might potentially need to use throughout our development process. These included concepts such as the Gaussian Filters, Canny Edge Detector, Hough Line Transform, and Convolutional Neural Networks. Additionally, I set up the Jetson Orin this week and managed to power it on and load the OS.
My progress is currently slightly behind, so I will combine certain tasks this week and hopefully work with my team to get back on schedule.
In particular, I need to obtain a USB keyboard, as the one issue I ran into with the Jetson Orin is that we are still unable to input keystrokes to it. Furthermore, I will also work with Emily and Eunice in doing the data synchronization and testing how both the radar and dashcam will communicate with the Jetson. Finally, I will start working on the lane detection process and doing unit testing for it.
10/5/2024
This week, I helped with preparing the design presentation slides, specifically the sections on the solution approach, implementation plan, and system specification. I further detailed how each data element will interact with our 4 major detection processes and how their outputs will be defined and utilized. Furthermore, I have also started working on the Design Report and have made more progress into getting the Jetson working. I acquired the microSD card and adapter necessary to flash the OS to the Jetson, and now am simply waiting for a displayport to HDMI cable to come in to be able to connect to a monitor to finish the initial boot for the Jetson.
My progress is on schedule.
Next week, I plan to finish working on my sections of the Design Report. Furthermore, I hope to get the Jetson up and running and to see if we can connect and start processing data from the dash cam and radar to the Jetson, depending on when they come in.
9/28/2024
Throughout this week, I looked up some of the material components we will need to construct our project including the dashcam we plan to move forward with. Additionally, I did individual research into radar technology, specifically what are FMCW radars, how they work, and why they are necessary for our project, as well as, the importance of 24 GHz vs 77 GHz and how they affect radar performance, and which would be suitable for our project. Some of my resources include: Detecting Targets by 24GHz FMCW Radar Technique, Moving from legacy 24 GHz to state-of-the-art 77 GHz radar, and Radar Solutions for Vehicle Detection, Collision Avoidance, & Positioning Feedback. Finally, I tested and did some of the setup for the Nvidia Jetson Orin Nano Developer Kit.
We are currently on schedule.
For next week, I plan to finish my slides for the design presentation and start writing the design report. Additionally, I would like to do further research and testing of the Nvidia Jetson Orin Nano and start testing out the Dash Cam and Radar Sensor per their arrival.
9/21/2024
Throughout this week, I assisted my teammates in helping prepare our proposal presentation slides and producing images for the presentation. Additionally, I developed a rough sketch for the inter connections of the system that was later refined and expanded upon by Eunice. As I was in charge of presenting this week, thus, I spent a large portion of time this week rehearsing with my team and by myself for our presentation. Finally, I began reading through the design review presentation requirements and further expanded some of the details of the implementation of our project ideas.
We are currently on schedule. For next week, we hope to specify all necessary components for our project such as the dashcam/camera model and power requirements, and put in the purchasing forms for all equipment. Additionally, we will fully flesh out our solution approach to our project and build a block diagram detailing all requirements of the system. Finally, we will produce design review slides and get prepared to give our design presentation about our project.