The biggest change to our system at the moment is converting all of our features over to DLib. We are trying to make this change in time because doing so would increase our accuracies for testing in nonideal conditions. The costs incurred are that it will take time to make the switch as well as retest with the new model, and we don’t have a definitive answer as to how this change will affect the current accuracies we have. However, we believe it is worth trying for the increase to the frame rate and more accurate head pose estimation. We will also continue testing what we currently have in the case that the change can’t be made in time so we still have a good final product.
Another risk we are currently having is related to integrating the full CV code onto the Jetson Nano. We were having issues installing tensorflow onto the Jetson, but the current problem is that the microSD card in the Jetson Nano is too small for our code and needed libraries. We have ordered a 128gb microSD card that will arrive on 04/30, at which time we will set up the Jetson again with the new SD card and restart the process of installing the necessary libraries and be able to run our full code on the Jetson directly.
We have pushed entire system testing on the Jetson again (to this upcoming week) because of issues we are having with integrating the CV subsystem code onto the Jetson Nano. We have also added tasks for converting head pose and mouth detection from CNN facial landmarks to DLib.
| Unit Test | Notes |
| Accelerometer (in Hamerschlag) | Acceleration data fluctuates significantly when the accelerometer is stationary – inaccurate acceleration measures also leads to inaccurate speed estimations. Need to still test in a moving car to determine feasibility of this unit. |
| Head pose estimation (CNN facial landmarks) | When tested pre-integration, head pose worked very accurately. However, the lower fps caused by integration makes the head pose ineffective. This led to the design change to convert Head pose and mouth detection from CNN to DLib because the use of the same algorithm for all of our components would decrease runtime and increase fps. Will need to retest after converting to DLib. |
| Mouth detection (CNN facial landmarks) | Worked accurately for both pre and post integration tests. Will need to retest after converting to DLib. |
| Eye-tracking | Worked accurately pre and post integration |
| Blink detection | Worked accurately pre-integration, but fps was too low for accurate blink detection after integration. We are expecting performance to improve after |
| Connecting to WiFi hotspot in car and ssh-ing in | Set up the Jetson to auto-connect to Elinora’s hotspot and checked that it was connected in the car. Did this 3x successfully in the car. There is a long connection time ~ 5 min and not waiting previously led us to think that the connection wasn’t working. |
| Retrieving data from Firestore (in React) and displaying on the web app | Created a test collection on Firestore and confirmed that the data from that test collection was retrieved and shown on the web app in table format. |
| Hosting the web app | Able to connect remotely from all 3 of our computers |
| Firebase authentication | Able to create 3 accounts and log in. |
| Audio feedback | Tested the speaker separately and then connected it to a laptop to play audio feedback integrated into the classification code. Audio feedback was played virtually instantaneously (passing our latency requirement) |