Category: Team’s Status Reports

We finished our final presentation in which Evann presented on Monday. We also worked on our final demo video and poster which are almost completed. Lastly, we have completed our car set up which includes a small box placed in the car dashboard with the screen being held up by a phone holder.

1. Significant Risks:  At this stage there is not many significant risks that can jeopardize our project.  There are small things such as improving testing and our dataset.
2. Changes to the Existing Design: No changes for this week except we did add
3. Schedule: There are no changes to the schedule

 

1. Significant Risks:  At this stage there is not many significant risks that can jeopardize our project.  There are small things such as not being able to classify the eyes accurately enough. We are still working on best improving this algorithm depending on the distance the user is from the camera / the lighting in the car.
2. Changes to the Existing Design: No changes for this week
3. Schedule: There are no changes to the schedule
4. Short demo Video:

https://drive.google.com/file/d/1gHNVyxSBz6iphaPCnnBXD8wDqWc6w8aR/view?usp=sharing

This shows the program working on our touchscreen. Essentially, after calibration, there is a sound played (played on the headphones in this video) when the user has their eyes closed / mouth open for a longer period of time.

1. One risk that could potential impact our project at this point is the buzzer integration.  If there is an issue with this aspect of the project, we may need to figure out a new way to alert the user.  This could potentially be addressed by alerting the user with an alert on the screen.
2. No, this week there were no changes to the existing design of the system.  For testing and verification, we more precisely defined what we meant by 75% accuracy against our test suite.
3. This week we hope to complete the integration of the buzzer and the UI interface of our project.  Next week we hope to create a video for the head turn thresholding to showcase the accuracy and how it changes as you turn your head.  We also need to potentially work on pose detection and more test videos. Finally, we are going to be testing our application in a car to obtain real world data.

1. The most significant risk that could happen to our project is not meeting one of our main requirements of at least 5 frames per second.  We are working on improving our current rate of 4 frames per second by having some openCV algorithms run GPU, in particular the dnn module which detects where a face in the image is.  If this optimization does not noticeably increase our performance then  we plan on using our back-up plan of moving some of the computation to AWS so that the board is faster.
2. This week we have not made any major changes to our designs or block diagrams.
3. In terms of our goals and milestones, we are a little ahead of our original plan.  Because of this, we are deciding which stretch goals we should start working on and which of our stretch goals are feasible within the remaining time we have. We are currently deciding between pose detection or phone detection.

1. This week we focused on integrating with the board.  We finalized our calibration and main software processes.  We did more testing on how the lighting conditions affect our software and optimized our eye tracker by using the aspect ratio.  One big risk that we are continuously looking out for is the speed of our program.  On our laptops the computation speed is very quick but if it takes longer on the board our contingency plan is to use AWS to speed it up.
2. For the board we are now using TensorFlow 2.4 and python 3.6 which changes a few built-in functions.  This is necessary for the compatibility with the Nvidia Jetpack SDK we are using on the board.  There are no costs for this only minor changes in the code are necessary.
3. So far we are keeping to the original schedule.  We will be focusing on optimization and next start working on our stretch goals.
4. Eyes closed detection:

1. This week we continued integration and the biggest potential issue we ran into was the compute power of the board.  Once the camera was connected, we quickly realized that the board had a delay of about 2 seconds.  With a decreased frame rate of 5 frames per second we had a delay of about half a second.  We are worried that with our heavy computation the delay will get larger.  We expected this and we are continuing forward with our plan to optimize for faster speeds and worst case put some computation on AWS to lighten the computation load.
2. No changes were made to our requirements, specs or block diagrams.
3. This week we have caught up to our original schedule and we plan to continue front loading our work so that integration will have more time.

As a team we all worked on our design review presentation. We had the chance to flush out all our block diagrams for the hardware and software components of our project. We decided on utilizing AWS compute power in case the board blows up or is not enough for our project. This also allowed us to obtain a better breakdown of our project and how all of our components are going to interact with one another. We are currently following our original schedule that we have planned out. Now that we have our Xavier board we can begin making some exciting progress by writing our preliminary code onto it.

This week, we presented our CarMa project proposal. We obtain informative feedback about issues we might face in the future. As a team, we also reviewed the presentations of all the other teams in our section. We met as a group this week and ordered the essential hardware components for our project. We also began further research into our chosen computer vision algorithms.

Our group worked on two main things this week: narrowing the scope of our project and spending some time research various technology.

Narrowing our Scope

We first worked on narrowing down the scope of our project.  For our MVP, we are focusing on preventing drivers from falling asleep or from getting distracted.

Research

In terms of hardware components, we compared the tradeoffs of various boards and we have decide on the Jetson Xavier board.  In terms of software components, we have done preliminary research on various face-tracking and eye-tracking algorithms that work with the hardware.  We also began some basic testing on some open source codebases to test for accuracy.  We plan to spend some more time looking into the specific OS to put on our Xavier board and to narrow done the extra hardware such as the camera and screen.