Look who's talking!
Last week! This week Gauri worked on improving the conditions under which the classifier will work well by training the classifier on images with heads behind the hand gestures. However, this proved to reduce the overall accuracy of the classifier and did not provide the …
Last status report! I’m actually going to miss working on capstone 🙂 This week I worked on putting final touches on the animation such as making changes that we talked about last week and adding functionality for mode switches, text, arrows and more clarity to …
This week was finalizing everything and filming the video!
Finalizing everything mainly involved making the overall control loop and connecting it to the animation, as though it was connected to an actual device/motors in this case. We had created all of the separate pieces in a way that it was easy to put together (ex: animation taking keystroke input –> taking prediction input, predictions from mic or video printing –> outputting that prediction to the animation), so we didn’t run into any errors here.
We made some fixes to the animation, and I messed with the baffles a little more (and made them look better). We filmed a lot of the demo video and are in the process of putting it together and voicing over it. We don’t expect to have any problems here – once the video is done, the last thing we have to do is the report!
Last status report! This week we wrapped up all loose ends of our project and finished integrating it all! 🙂 We also worked on our final presentation slides, I prepared for the presentation since it was my turn this time. We think it went well. …
This week we finished all of the independent part of our projects, tested it, and worked on getting everything working to the requirements we had set for ourselves (like accuracy for the classifier, sound, …). Our classifier was working at 95% accuracy for the 2 …
We finished our classifier this week (it works!!!) and tested it on the pi with the pi camera. We realized a lot of little things, like how much better the classifier works with a bright light on the person compared to a dim light. I also set up the microphones and made baffles (worked really well!!) and played with the frequencies so that we got it to process sound pretty quickly. I’ve been working on a little bit of everything, since at this stage most things require testing and I have all of the hardware. Now we mostly just have to finish the animation and integrate everything.
We started making a plan to film our demo video, and I think we’ll be able to show it pretty well through video. This weekend is mostly all of us working together and finishing up our final presentation for this week.
We basically finished all the independent parts of our project this week!  I worked on a few things: Our classifier had a 95%+ accuracy on just identifying left vs right. However, I realized earlier this week that it would classify everything it saw as left …
This week I worked on the animation that will be the output for our final project. The animation will represent how the actual device would have worked if the circumstances had allowed us to work with the motors, motor hats and 3D printers etc. Working …
This week we all continued to work on the areas of the project we began working on last week!
I worked on collecting, labeling and organizing real images for the classifier. I tweaked some parameters for the classifier and ran it many times with different types and numbers of images. We are continuing to add more images to the dataset and run the classifier on this data, the accuracy has significantly improved!
Gauri worked on code to get frames from the RPi camera module and feed it into the classifier, she also helped collect many of the images that are a part of the dataset and most importantly she made a couple of changes that boosted the classifier accuracy 10% from 80% to 90%! She discovered an architecture called VGG16. However, it is very large and time consuming and is not possible to run on a normal laptop/RPi. She found that using a subset of the VGG16 layers and reducing some node counts in the dense layers but following a similar pattern was effective enough for our purpose. Using this model and skindetector output (without otsu binarization which didn’t help much) helped to really ramp up the accuracy!
Shrutika worked on the microphones this week as well as setting up the camera modules with the RPis to get video input. She was planning on using Audacity to check mic input but ran into roadblocks when she found that Audacity does not work with Mac OS Catalina. She decided to give up on Audacity for the time being and is figuring out how to use pyaudio to measure mic input and has been plotting wav files on matlab to compare the mic inputs and has noticed that mics in different directions provide different levels of audio input! 🙂 She is also in the process of creating baffles for the mics to improve the difference in audio picked up my mics in different directions.
Overall, we have made steady progress in different areas of the project and hope to continue to do so and quickly begin to integrate the different parts!
This week I spent a lot of time working more on the classifier. Throughout the week we received hundreds of images from our posts on various social media platforms requesting images as well as from family and friends. We have now collected over 1200 images …