Posted on

Adriana’s Status Report for 3/27

This week the majority of my time went to working on our classification algorithm. Now that we have a working mouth and eye detection implementation we need to know when the user should be notified that they appear distracted. For distraction concerning the eyes, we are looking into when their eyes  are closed. Given the eye calibration values that Jananni has worked on, I use that number as a basis to detect when their eyes appear to be about half of that value. Similarly, for the mouth detection, I check to see if their mouth appears to be yawning by using the open mouth calibration value and check if the user is currently opening their mouth at a larger distance. When either of these conditions are met then an alert will sound to warn the user.

This week I also managed to help both Evann and Jananni with putting our eye detection algorithm on the board. We are currently in the process of improving how the code runs on the board and Evann is spearheading that progress. Another portion of the project that I have worked on is integration of our eye tracker and mouth tracker. Before they used to be two seperate algorithm but I have combine them in order for them to be able to run simultaneously. This involved refactoring a lot of the code that we had but it is something that will be very useful for us. My current progress is on schedule and I will be looking into how to incorporate an actual sound for an alert instead of text on the screen. I also hope on starting to work on improving our algorithm in both accuracy and speed since putting this on the board will degrade the performance of our algorithms.

Posted on

Team Status Report for 3/27

  1. This week we continued integration and the biggest potential issue we ran into was the compute power of the board.  Once the camera was connected, we quickly realized that the board had a delay of about 2 seconds.  With a decreased frame rate of 5 frames per second we had a delay of about half a second.  We are worried that with our heavy computation the delay will get larger.  We expected this and we are continuing forward with our plan to optimize for faster speeds and worst case put some computation on AWS to lighten the computation load.
  2. No changes were made to our requirements, specs or block diagrams.
  3. This week we have caught up to our original schedule and we plan to continue front loading our work so that integration will have more time.
Posted on

Jananni’s Status Report for 3/27

  1. Accomplished this week:  This week I have been focused on building the calibration for our project.  This process asks the user to take pictures of their front and side profiles so that our algorithms can be more accurate for the specific user.  This week’s goal is to get the user’s eye height (in order to check for blinking) and the user’s mouth height (in order to check for yawning).

I first did research on various calibration code I could potential use.  One potential idea was using this project (https://github.com/rajendra7406-zz/FaceShape) where certain face dimensions are outputted.

Unfortunately these dimensions were not enough for our use case.  So I focused on using the DNN algorithm for a single frame and calculating the dimensions I required from the landmarks.

Then I started working on building the backend for the calibration process.  I spent a lot of time understanding the different pieces of code including the eye tracking and mouth detection.  Once I understood this, I worked on integrating the two and retrieving the specific values we need.  For the eye height, I calculated the average of the left and right eye height and for the mouth I return the max distance between the outer mouth landmarks.

2. Progress according to schedule:  My goal for this week was to build the basics of the calibration process and get the user’s facial dimensions which was accomplished on schedule.

3. Deliverables for next week:  Next week I hope to start working on the frontend of the calibration process and the user interface.

Posted on

Evann’s Status Report for 03/27

This week, I worked on testing the camera following Nvidia’s guide linked here https://developer.nvidia.com/embedded/learn/tutorials/first-picture-csi-usb-camera. We are now able to capture image data. I also worked on completing the integration of our primary computer vision application onto the Xavier board. There was an issue with compatibility with one of the libraries we are using so I am rewriting some of the code that handles loading the pre trained keras model. In addition I read through a research article describing how pose estimation can be done using landmarks linked here: https://www.cs.cmu.edu/~deva/papers/face/face-cvpr12.pdf. Progress is on schedule and my plan is to create a test suite and get baseline measurements for our MVP by next week.

Posted on

Adriana’s Status Report for 03/13

This week I worked on the design review presentation along with the design review paper. That is where the majority of my time went towards along with  working on testing out the face detection algorithm locally in which we will be migrating to our board at the start of next week. Now that we have the majority of our parts we can begin assembling it together. This will be our first big task in seeing how to different components of our project get to work together. So far, I am on schedule and I’m making good progress.

Posted on

Jananni’s Status Report for 3/13

This week I focused on finalizing the block diagrams and research Adriana needed in order to present the Design Review Presentation.  I also started working on the Design Report due next week.  As a group we worked on the report and began putting the device together to start preliminary testing.  We are a little behind on our schedule but next week we plan to ramp up because we have more time.  I plan on completing a rough draft for the Design Report by Monday so that next week we can turn in the Final report and begin serious testing.  I will also start working on the calibration procedure.

Posted on

Team Status Report 03/13

The most significant risks that could jeopardize the success of the project right now is whether of not the program will run at a desirable frame rate on our board. This risk is mitigated by our contingency plan to use AWS for faster compute. There have been no major changes to our design this week. Our hardware components have been tested to ensure functionality and we are excited to see how our board will handle our preliminary implementation of the CV application.

Posted on

Evann’s Status Report for 03/13

This week I spent the majority of my time working on the Design Review presentation/paper. I reviewed the presentations of my peers as well as worked on writing the design paper. I was able to set up the hardware and test the core I/O and ensured that everything was functional. My progress is on schedule and I hope to have a preliminary version of our program running on the board by next week.

Posted on

Team’s Status Report for 03/06

As a team we all worked on our design review presentation. We had the chance to flush out all our block diagrams for the hardware and software components of our project. We decided on utilizing AWS compute power in case the board blows up or is not enough for our project. This also allowed us to obtain a better breakdown of our project and how all of our components are going to interact with one another. We are currently following our original schedule that we have planned out. Now that we have our Xavier board we can begin making some exciting progress by writing our preliminary code onto it.

Posted on

Jananni’s Status Report for 3/6

This week we focused on ordering our most essential parts and finalizing our design.  First, we put in orders for the Jetson Xavier and the Kit since those are essential.  When we placed the order, we realized our TA wanted an official Budget so I created that.

Once the budget was finalized and our orders were approved, I started working on the PowerPoint for the design review presentation.  I primarily worked on the metrics slides for the driver and the device.  I used the comments and feedback I received from my Proposal Presentation and tried to incorporate that.  After, I helped work on the block diagrams.  Specifically, I worked on the System Specification: Hardware and System Specification: Software diagrams.

After, our parts arrived and we started assembling the device.

We are currently on track with our schedule.  Next week we plan to start testing the software we have on the device and hopefully begin some optimization.