Posted on

Zachary’s status report for 9/24

This week, I was mainly focused on editing the slides and preparing for the presentation, which I presented on Monday. I appreciated the feedback and questions that we received from the Instructors and classmates, and particularly the pushback on the false positive rate, which I feel is a valid concern. As an aside, I felt that some of the feedback I received saying that I did not know the material well, or that I was underprepared, was unwarranted, as I had spent a substantial amount of time on the presentation. However, perhaps due to being too softspoken and having technical difficulties during the presentation, I was not able to reflect that.

Additionally, I have also spent a bit of time doing research on object detection algorithms for the implementation of the walk sign detection.

I am currently on schedule, as our team has put aside time in our schedule for the first four weeks, to specifically do research and flesh out our design (as well as prepare for presentations) before we start implementing.

Since I have limited experience with ML, I really want to get a head start on the material and implementation. In this upcoming week, I will be doing more research, as well as working with Eshita to find/create a dataset for walk signs fo the ML model that I will implement. Additionally, I also hope to set up a github repo and start writing down some code, if possible. Lastly, I will talk with my teammates to see if AWS may be needed for model training, and talk with the TAs and professor if we do to set that up.

Posted on

Eshita’s Status Report for 9/24

Personal reflection

I looked at all the feedback this week for our team and wrote down some ideas that Colin, Zach, and I were brainstorming already. Generally, I feel like I should not have interrupted Zach in the presentation, he knew his material and I felt embarrassed to see the feedback that peers/faculty thought otherwise. It is a teammate attitude change that I will uphold moving forward. I do also think I got a little carried away with predictions on the statistics of our application. I feel like I’m used to classes and assignments where the stakes are quite different, and even compared to the other CV projects within the class, ours stands out in its real user base. I am excited to work for what we’re doing, but worried about how much we can achieve. It is not only a change of number (say from 1% to 0.05%), but we have to follow through on actually doing it.

Some design ideas

We do plan to keep the camera mounted on the shoulder using some sort of strap-on that comes for GoPros. I have a GoPro and there seems to be some vague documentation on connecting and streaming video to a Jetson. (An example is here.)

Another idea would be to keep the Jetson and the camera in a self-contained unit (maybe 3D-printed), but I do not know how much power/energy it would give off, and whether that would be harmful to the user. I wouldn’t want to wear something hot to my chest or near my body.

Algorithm work

I also research the background of crosswalk detection itself. I wrote down some potential steps based on my research of edge detection algorithms I think I would use. More research is needed on how to actually track objects in video, which is not something I’ve delved too much into before. Here are the steps I came up with:

  1. I would like to start with processing a frame and filtering out all the white I notice in that frame.
  2. Erode frames by structuring the frame. This is necessary to remove the noise and identify some finer points in the image. (A link on why this is necessary)
  3. Identify contours using cv.findContours, and draw tracking lines on each of the contours
  4. Find the median points in the contour. I have some ideas for the actual linear regression, which could be a simpler model, but also found this specific application for image alignment called RANSAC https://learnopencv.com/tag/ransac/
  5. Basically, after finding the median points, we want to use those along with the contours found to create a middle intersection point. This will be used to calculate the direction vector from the camera compared to that points.
  6. This alignment vector will help decide the walking direction for the user.

Next week?

I am a little behind on project work due to attending career fairs and interviews this past week. I feel good about my algorithm idea and want to start a github repo in the coming week and sketch out some outline code for it. I also want to, if not complete the pseudocode, find images and videos of crosswalks to test my algorithm on, as well as do research on how video tracking would work. I have a direction vector and a camera’s center point, but how do these two connect? I also hope to help Zach with collecting pictures of walk signs and I’m trying to take pictures of as many as I can while walking to and from campus! I also let the professors know I would need AWS, but I’ve always worked locally to train my ML algorithms even for classes, so I would like to play around with how that works!

Posted on

Team Status Report for 9/24

At the moment the most significant risk that could jeopardize the success of the project is the accuracy of our object detection algorithms. We do not want to tell a blind person to cross the road when they are not supposed to. We are currently looking into options to mitigate this risk, one option may be to reduce the scope of the project to just doing crosswalk detection or cross sign detection to allow us to focus more time to one the algorithms and to hopefully make it better. We should also focus on the rate being less than 1%, a metric we were thinking of would probably be 0.5% for the detection of whichever application we pick, and if we pusue both as well. The design of the system has been unchanged, however we are looking into how to get the false positive rate as close to 0% as possible.

Posted on

Colin’s Status Report for 9/24

This week I did research about all aspects of the hardware for the project. I wanted to tie in all of the components at a high level and see how they would all interact together in the project. In particular, I have decided to go with a BMA180 accelerometer to feed in data into the Jetson to determine if the user is stationary or walking. I can use a python library for this particular accelerometer to get the data, or I can write a small c program to gather the data and run some algorithms to determine the state of the user. I figured it would be nice to be able to easily gather the data using python given that we will be using python for the object detection portion of the project, and that the data from the accelerometer must be communicated to the object detection portion. I believe that doing both of these in the same python code would significantly increase both robustness and speed of development. I have also been looking into cameras that can stream data to the Jetson, and I believe that the SainSmart IMX219 would work well with a Jetson Nano, which is what we plan on using. Currently, I am on track according to the schedule given that for now all of us are working towards the design proposal, and the work that I have done this week all has to do with the design of the project on the hardware side. My primary goal next week is to look into the options involving the audio communication to the user of whether or not they should cross the street and what direction to go. I would also like to receive a Jetson Nano within the next week and start to install python/OpenCV on it. When installing python, I would also like to look into the option of building a multi-threaded python program to be able to get the accelerometer data at regular intervals and to communicate that data to the thread that decides whether to look at the walk sign detection or the crosswalk detection.

Posted on

Introduction to Project

Our project aims to aid the visually impaired to travel safely over crosswalks. Specifically, we limit our scope to solving two problems: The first is the identification and recognition of traffic walk signs using image detection. While some walk signs have audio cues indicating when it is turned on, most lack this feature. As a result, visually impaired people are inconvenienced and must rely on more subtle audio signals, and/or others around them, to gauge when it’s time to cross. Having a device that can recognize walk signs, and detect when they are on,  gives people who are visually impaired safety and independence when crossing the road. The device will also tell the user if they are staying on the white lines of the crosswalk, and will notify the user what direction to turn in so that they can stay within the white lines of the crosswalk.

Project by Colin Hackwelder, Eshita Shrawan, Zachary Zhao