Category: Evann’s Status Reports

What did you personally accomplish this week on the project? Give files or photos that demonstrate your progress. Prove to the reader that you put sufficient effort into the project over the course of the week (12+ hours).

This week I worked mainly on creating our final demo video. I worked on ensuring that all the hardware would work inside of the car as well as mounting all of the components. I also worked on editing all of the film that we recorded as well as voicing over explanations of the system. I also collected more test data and performed more testing and validation to update our results.

Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?

Our progress on the board is on schedule. We’ve completed all of our major goals and are just finishing up the final report.

What deliverables do you hope to complete in the next week?

We are planning on doing more testing as well as finishing up the final report.

What did you personally accomplish this week on the project? Give files or photos that demonstrate your progress. Prove to the reader that you put sufficient effort into the project over the course of the week (12+ hours).

This week I completed the accelerometer integration. I was able to make a physical connection between the accelerometer and board using some spare Arduino wires. After ensuring that the connection was established, I then checked that the board was receiving data from the accelerometer by checking that the memory was mapped to the device. I then used the smbus python library to read the data coming in from the memory associated with the accelerometer. I also completed the sound output. This was accomplished by using the playsound python library. I also worked on implementing pose detection using the facial landmarks we already use for eye detection.

Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?

Our progress on the board is slightly ahead of schedule. We’ve completed all of our major goals and have begun work on stretch goals.

What deliverables do you hope to complete in the next week?

We are planning to focus work next week to get our system working in a vehicle for a demo. I want to finish the pose detection next week as well.

What did you personally accomplish this week on the project? Give files or photos that demonstrate your progress. Prove to the reader that you put sufficient effort into the project over the course of the week (12+ hours).

This week I spent the majority of my time integrating the updated application onto the board. There was an issue with how some of the calibration code was interacting with the images that were captured using the onboard camera. I also spent time collecting data on performance and working to optimize the landmarking. There are some dependency issues with converting the landmarking model to a tensorrt optimized graph and I am currently working through these.

Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?

Our progress on the board is slightly ahead of schedule. We finished integration this week, as well as testing and validation. We are planning to start work on some of our reach goals.

What deliverables do you hope to complete in the next week?

I hope to finish accelerometer input and sound output by next week.

What did you personally accomplish this week on the project? Give files or photos that demonstrate your progress. Prove to the reader that you put sufficient effort into the project over the course of the week (12+ hours).

This week I spent the majority of my time integrating the primary application onto the board. There was an issue with an incompatible version of Jetson’s SDK with the version of TensorFlow that we were using which required reflashing our SD card with a new OS image. Using Nvidia’s JetPack SDK required us to change some of our dependencies such as switching to Python 3.6 and TensorFlow2.4 which required us to refactor our code to handle some of the deprecated functions. Working through these issues, I was able to get the preliminary code running on the board. A demo is shown below.

Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?

Our progress on the board is slightly behind schedule. We intended to finish integration earlier this week, however issues with dependencies and platform differences caused some unforeseen delays. I will use some of the dedicated slack time to adjust the schedule.

What deliverables do you hope to complete in the next week?

I hope to implement accelerometer input and sound output by next week. I also intend to begin work on pose estimation if time allows.

This week, I worked on testing the camera following Nvidia’s guide linked here https://developer.nvidia.com/embedded/learn/tutorials/first-picture-csi-usb-camera. We are now able to capture image data. I also worked on completing the integration of our primary computer vision application onto the Xavier board. There was an issue with compatibility with one of the libraries we are using so I am rewriting some of the code that handles loading the pre trained keras model. In addition I read through a research article describing how pose estimation can be done using landmarks linked here: https://www.cs.cmu.edu/~deva/papers/face/face-cvpr12.pdf. Progress is on schedule and my plan is to create a test suite and get baseline measurements for our MVP by next week.

This week I spent the majority of my time working on the Design Review presentation/paper. I reviewed the presentations of my peers as well as worked on writing the design paper. I was able to set up the hardware and test the core I/O and ensured that everything was functional. My progress is on schedule and I hope to have a preliminary version of our program running on the board by next week.

This week, I worked on setting up a development environment for developing for the Jetson Xavier NX board. Our team has decided to move forward with Ubuntu as our OS as the Jetson Xavier has Ubuntu based software packages optimized specifically for the board that we want to use. Additionally, I worked with the rest of my team to create a Design Review slide deck. I used google doc’s drawing feature to create simple, straightforward block diagrams of the hardware system and the calibration process. Finally, I received some of the hardware that we requested and have been exploring how to set everything up.

This week I worked with my team on creating our project proposal presentation as well as reviewed the presentation of the other teams in my section. In addition, I worked to finalize the list of hardware that our project requires and submitted the order request forms to have the parts purchased. I started researching further into the computer vision algorithms that we will use by reading through documentation of similar projects.

This week, I focused on determining the components of hardware system for the project. I first worked with the team to narrow the scope of our project to focus more on drowsy driving and potentially distracted driving (i.e. using cellphone, looking away). Then after solidifying a plan for the project, first working on eye detection the facial pose analysis, we concluded which components we would need to use for the project. We decided to use the Nvidia Jetson Xavier NX as it was the most powerful board with a GPU at our price range. After settling on this decision, I researched the other components we would need for the system. I found a few cameras that would be suitable for our applications but settled on a screen module/camera kit partly due to the price savings. Next I found a suitable sound card, accelerometer, and wifi card. Then I researched what OS to use for the boards as well as potentially SDKs such as Jetpack. Finally, I spent time working on the proposal presentation with my teammates working primarily on the Requirements, Hardware Solution Approach, and Scheduling.