Given that integration into our entire system went well, I spent the last week tidying things up and getting some user reviews. The following details some of the tasks I did this past week and over the break:
- We decided to reduce the emotion count from 6 to 4. This was because surprise and disgust had much lower accuracy than the other emotions. Additionally, replicating and presenting a disgusted or surprised face is pretty difficult and these faces come up pretty rarely over the course of a typical conversation.
- Increased the threshold to present an emotion.
- Assume a neutral state if not confident in the emotion detected.
- Added a function to output a new state indicating if a face is even present within the bounds.
- Updates website to indicate if no face is found.
- Also helps for debugging purposes.
- Updates website to indicate if no face is found.
- Did some more research on how to move our model to Onyx & Tensorflow which might speed up computation and allow us to load a more complex model.
- During the break, I spoke to a few of my autistic brother’s teachers from my highschool regarding what they thought about the project and what could be better.
- Overall, they really liked and appreciated the project and liked the easy integration with the IPads already present in the school
I am ahead of schedule, and my part of the project is done for the most part. I’ve hit MVP and will continue making slight improvements where I see fit.
Goals for next week:
- Add averaging of the detected emotions over the past 3 seconds which should increase the confidence in our predictions.
- Keep looking into compiling the CV model onto Onyx – a Jetson-specific way of running models – which would lower latency and allow us to use our slightly better model.
- Help Mason conduct some more user tests and get feedback so that we can make small changes.
- Writing the final report and getting ready for the final presentation.Additional Question: What new tools or new knowledge did you find it necessary to learn to be able to accomplish these tasks? What learning strategies did you use to acquire this new knowledge?
- The main tools I needed to learn for this project were Pytorch, CUDA, Onyx, and the Jetson.
- I already had a lot of experience in Pytorch; however, each new model I make teaches me new things. For emotion recognition models, I watched a few YouTube videos to get a sense of how existing models already work.
- Additionally, I read a few research papers to get a sense of what top-of-the-line models are doing.
- For CUDA and Jetson, I relied mostly on the existing Nvidia and Jetson documentation as well as ChatGPT to help me pick compatible softwares.
- Public forums are also super helpful here.
- Learning Onyx has been much more difficult and I’ve mainly looked at the existing documentation. I imagine that ChatGPT and YouTube might be pretty helpful here going forward.
- I already had a lot of experience in Pytorch; however, each new model I make teaches me new things. For emotion recognition models, I watched a few YouTube videos to get a sense of how existing models already work.