Look who's talking!
This week we finished all of the independent part of our projects, tested it, and worked on getting everything working to the requirements we had set for ourselves (like accuracy for the classifier, sound, …). Our classifier was working at 95% accuracy for the 2 …
We finished our classifier this week (it works!!!) and tested it on the pi with the pi camera. We realized a lot of little things, like how much better the classifier works with a bright light on the person compared to a dim light. I …
We basically finished all the independent parts of our project this week! I worked on a few things:
Overall we have almost wrapped up this project very well 🙂 Looking forward to finishing this up successfully!
This week I worked on the animation that will be the output for our final project. The animation will represent how the actual device would have worked if the circumstances had allowed us to work with the motors, motor hats and 3D printers etc. Working …
This week we all continued to work on the areas of the project we began working on last week! I worked on collecting, labeling and organizing real images for the classifier. I tweaked some parameters for the classifier and ran it many times with different …
This week I spent a lot of time working more on the classifier.
Throughout the week we received hundreds of images from our posts on various social media platforms requesting images as well as from family and friends. We have now collected over 1200 images and continue to get more everyday. The images are of four kinds – the ok symbol for right, the L shape for left, and both these symbols in front of the person’s face. I spent a lot of time collecting, labeling and organizing these images. When I tested the classifier with these images (no matter the number of images) I noticed that the classifier when run with just the images of the hands (no heads) had a consistently higher accuracy than when I ran the classifier with all the images. Thus, we made the decision to run the classifieron and henceforth collect data of only hands in front of a background. The following has been the progress of the classifier:
200 images -> 0.54 accuracy
300 images -> 0.75 accuracy
500 images -> 0.80 accuracy with 50 epochs and 10% test and train split
700 images -> 0.83 accuracy
1000 images -> 0.87 accuracy with binarization, skin detection algorithm and parameter tweaking
This week I worked on many things – getting frames from the RPi camera video stream to pass to the gesture classifier, collecting more images (through crowdsourcing on social media which was a very interesting experience – we have a total of around 1200 real …
I have been playing with the microphones this week. Turns out that macOS Catalina doesn’t support Audacity, and then I tried to connect to an old laptop but the USB ports were broken, and had some other adaptor issues… also it’s super unclear if the …
This week was productive – as usual, everything took longer than expected, but we have been making significant progress.
Neeti and Gauri spent a lot of time this week on our gesture classifier. We realized that it’s more difficult to recognize gestures with the same number of fingers (thumbs up/thumbs down, point left/right), so we switched our gestures to the ‘ok’ symbol for right (middle finger, ring finger, pinky up), and the ‘L’ or ‘loser’ symbol for left (thumb and pointer) since these are clear different fingers. These gestures seemed to be better, but another problem was we were training with a Kaggle/LeapMotion dataset we found – however these images were taken with an infrared camera and were all of the same person, so they looked fairly different from the real images of a person that we want our device to respond. We tried:
… but these were unsuccessful. We are now creating our own set of data from images our friends and family, of which we have collected a pretty large set of images already (I guess everyone is really bored in quarantine…). If this doesn’t work we may have to change the CNN model we’re using.
On the hardware side of manual mode, Shrutika set up the raspberry pis on her home network and connected the raspberry pi cameras. We are able to capture images using OpenCV and the pi camera, and are working on getting a video stream frame by frame from the camera to feed the frames into the classifier. The output of the classifier and the animation are already connecting and working, so once the classifier works we should be done with manual mode.
Shrutika is working on setting up the microphones and reading from them today – we should have a clearer picture of where we are on automatic mode by tonight/tomorrow.
This week we spent a lot of time getting our manual mode ready for the demo! This meant that I was primarily working on the hand gesture classifier, Shrutika was working on animation, and Gauri was working on planning and integration. I spent Monday (04/06) …