Author: sbrahman

I spent the beginning of the week working on the final presentation slides and practicing the presentation since it was my turn to present.  I also worked on debugging the issue with setting up the Jetson in the car due to the personal hotspot not wanting to connect, and Elinora and I were able to fix this issue.  I also helped Yasser with getting the code to properly run on the Jetson, but it was taking much longer than predicted due to the libraries we needed to download and the lack of storage.  The most recent update with this is that we will need to get an SD card with more storage if we want to be able to run the code on the Jetson.  I also worked on the website and updated it for the demo.  Since all of the final assignment are due soon, I started to work on the final poster as well.

We are a little behind at the moment because we had another setback due to the discovery that we didn’t have enough storage for all of our code.  It was unfortunate that we had to find this out so late, especially because we started to remedy this by deleting unneccesary files and folders from the Jetson and it would tell us there was enough storage, but once we installed the libraries one at a time, we would see that there still wasn’t enough storage.

We have an SD card for the storage now and are going to work on this during the week.  We also want to make the switch from cnn-facial-landmarks to DLib to improve the accuracy for testing in non ideal conditions.  We will first make this change for the features that currently work so we can see if any changes in their accuracies is worth making the change for head pose estimation, or if we should eliminate head pose estimation.  We are also going to continue working on the poster and start the final report, video, and setup for the demo.

This past week, I spent time with integrating the software and hardware, finishing up the final parts of the software features, and started testing. Some of the last things that needed to be done with the software included tuning the accelerometer code and figuring out how to get audio feedback to play from the speaker.  This part took a little longer than expected because since we switched from the Jetson Xavier to the Jetson Nano, the ports we originally assumed we’d have were not on the Nano, so we had to wait for some new adapters to arrive so we could work with the ports we have.  There were also some issues with the most common ways to play audio files that we could find, so it took a little longer to find a method that worked for us, but we were able to find one and it was able to be played from the speaker connected to the Jetson during classification.  We started doing testing with the laptop to get some baseline accuracies.  We weren’t able to start testing with the webcam on the Jetson until later because when we ran the code on the Jetson, we had to download a lot of the libraries, but they were causing some issues with the Jetson and it required us to do a lot of more setup.  This issue is practically fixed now.  I also spent a lot of time working on the slides for the final presentation and practicing the presentation since it is my turn for this week.

As of right now, we are on track to finish.  All that is left is testing in various settings in the most accurate way we can find, aside from being in a car, for purposes of the final demo.  The code for the facial detection and eye tracking is all done, code for feedback is done, and accelerometer is almost done, the web application is basically done, and everything is integrating together well.

For next week, we hope to have testing finished and have a test environment that we can use for the demo since we can’t use a car.

This week, I worked a lot on setting up the Jetson and preparing the accelerometer so it can be integrated with the Jetson.  It took a lot of time to get to a point where the sensor was able to be detected and the program to run the accelerometer code worked because versions of certain dependencies needed to updated in very specific ways.  We were able to get it working to a point where we could classify the sensor moving backwards and the sensor slowing down motion.  I also did more work with the computer vision code and helped with increasing the accuracy and classification of the head pose estimation code.

We are on track according to the updated Gantt chart.

By next week, we hope to have the hardware and software integration completely done and start testing in the car with the new camera.

The tests we plan to run include user testing in the car with the fully integrated project.  We will analyze the anticipated measured results by timing how long it takes for the computation and feedback, as well as how useful the feedback is for the user and correcting their driving.

This week, I spent most of the time working on the web application.  I worked on getting the initial setup of all of the pages done, making the HTML and CSS as close as possible to what we designed a few weeks earlier, making the JavaScript allow us to go between pages and interact with them properly, and figuring out how to incorporate ways to display graphs and metrics on the web application.  I also worked on integrating the web application with Firebase, but there have been a few issues in terms of configurations that I am still in the process of figuring out.  At this point, the web application is as finished as it can be before we integrate the data and aside from a few finishing touches.

Progress is a bit behind at the moment, and this is due to some unexpected changes popping up with the computer vision code.  The teammate who is working on the eye tracking and facial detection has mentioned that in order to implement a certain part, we will need a different hardware component, so that also has to be ordered now and we need to wait for it.  In the meantime, we have still been able to test with our computer’s webcam.  Aside from the coding pushing back our integration with hardware and the need for new hardware components, everything else is on track.  Hopefully once, these new parts are ordered, we should be able to make up any lost time.  We also allocated a significant amount of buffer time, so everything will still be able to be completed.

For next week, we hope to have all of the hardware components finalized and delivered as well as the computer vision code done so we can start the integration process.

This past week, I worked mainly on the web application and testing and starting to set up the hardware.  I continued to implement the designs of the pages that were worked on earlier.  At the moment, almost all of the pages have been set up, and we are working on them making look like our designs.  We are looking into Firebase as well as an authentication system to incorporate into the web application.  I also started doing some testing of the hardware to make sure each component is working individually and, in the event we find something that requires us to use a different component and purchase something new, we want to have enough time for the order to be delivered and tested again.  We had a concern regarding the Jetson because it could potentially use more power than what a standard car is able to provide, so we have been looking at possible alternatives if in practice the Jetson actually cannot work properly in a car.  We also had to refine a few design choices, such as some more edge cases to consider with our project and if some features should actually be included, since we are getting conflicting feedback from faculty and TAs about this area.  A good chunk of this week was also used to work on the ethics assignment.

Our progress was mostly on track for this past week.  Due to the delay in our hardware components being delivered and some refinements we had to make in the computer vision algorithms, we weren’t able to integrate any hardware with software yet, and we are still testing individual components to make sure everything works.  This part can’t be worked on much until the hardware is finalized and delivered, which should be able to be finished very soon due to the research and testing we did.  The hosting of the website is also yet to be done, but that is something that should take a very short amount of time.

In the next week, we hope to have the eye tracking algorithms complete with the ability to detect the direction that the eye is looking in, the website should be hosted, and the hardware components should all be delivered and individually tested by then.

The majority of the time this past week was spent on the design report and additional research as we determined certain changes needed to be made since we refined some aspects of the project.  For the design report, I worked on the abstract, the design trade studies, parts of the system implementation, test, verification, and validation, project management, and related work, in addition to assisting my teammates with their sections as needed and reviewing and revising the entire report at the end.  Throughout working on the design report, we all needed to conduct some more research in order to be thorough with our descriptions.  I also needed to review all of the work we had done up until this report to make sure everyone was on the same page about what we decided for our project and any changes that were made.  While working on the design report, we also started to individually test some of our hardware components to make sure they all work, but we are still waiting on some components to be delivered since some changes with our software algorithm decisions necessitated a change in the hardware we needed.  I also continued working on the website and implementing our UI designs from last week.

According to the Gantt chart, I was supposed to have finished any work with the device schematics and part selection, component testing, refining the design documentation, and setting up the initial webapp structure.  The work with testing components needs to be pushed because we had to change some of the hardware we needed to order.  Also, because of how much time the design report was taking up last week, not as much time was allocated to continuing the initial webapp structure.  These can be alleviated by next week because we have a lighter week according to the Gantt chart.

By next week, I aim to finish the initial webapp setup and assist the others in their work so we can start integration.

I spent the majority of the time at the beginning of this week working on the design presentation slides.  Specifically, I contributed to almost all of the topics for the slides required in the design presentation guidance page aside from the information that my teammates are more familiar with.  I also helped with the speaker notes so that we’d all be equally familiar with the content of the presentation.  In addition, I did more research into the remaining hardware components we need for the device and had them ordered.  I also continued brainstorming and designing the UI for the web app with Elinora.  We discussed exactly what components we wanted in the web app and what kind of information each page would display.  We already had a few pages planned out from last week, but now we have all of the pages planned out to display the  charts tracking a user’s driving history and feedback and a log of all of the feedback.  I also started doing research into using React.js and created the initial React app.  I also did some more research into how wifi will be integrated into the device in the car, but it’s been a bit challenging so far to exactly figure out how to implement this component.

According to the Gantt chart, by the end of the week, I was supposed to have completed working on the design presentation and initial design documentation, working on the device schematic and part selection, and refining the web app design.  The design presentation was completed, the parts selection have been done, and the web app design is completed as well.  The design documentation has been started, but I think at this stage, there should be a little more fleshed out.  To catch up on this part, I will dedicate more time to it now that my schedule has cleared up more this week.

By next week, the design documentation should be worked on and completed as a team.  We also hope to have all of our hardware components acquired so that we can test them all individually to make sure that they work.  I will also be working with Elinora to start creating the pages in the React app and implementing our UI designs.  I also want to continue figuring out how to incorporate the wifi with the device in the car.

This week, I worked on researching and ordering the appropriate components for the embedded device and helped with designing the device schematic.  I also started working on the design for the UI of the web app, but after the weekly meeting with our advisor, the plans for the app might need to change depending on how the device will be able to access wifi in the car.  Since our design presentation is next week, I also worked on the slides.  Our progress is mostly on track according to the Gantt chart; however, due to the potential change in plans with how the data will be stored and tracked, we might need longer for designing the UI.  Next week, we hope to have all of our hardware components and aim to test them all individually to make sure they are working, and if not, we will have enough time to purchase more.  I also hope to refine the design for the UI and start work on the design report.