Neeti’s Status Report for 04/11

This week we spent a lot of time getting our manual mode ready for the demo! This meant that I was primarily working on the hand gesture classifier, Shrutika was working on animation, and Gauri was working on planning and integration.

I spent Monday (04/06) and Tuesday (04/07)  writing the pipeline to process real images captured on my laptop and feed it into the CNN. However, late in the evening on Tuesday, Python decided to stop working on my laptop and after spending a couple of hours trying to reinstall the right version, dependencies and packages, we decided to run the code on Gauri’s laptop. Thus, we spent Tuesday night moving and setting up the code on Gauri’s system and then realized that the classifier did not work well on real images when trained with the Leap motion sensor dataset that we were using. We played around with the processing pipeline, the separation algorithm, changing gestures,  and augmenting the existing dataset with real images. However, none of these solutions were successful in recognizing the correct gesture.

On Wednesday (04/08), we worked on combining Shrutika’s python script that utilized pygame to animate the rotation of the device and the result of the classifier trained on the existing dataset when tested on the existing dataset. We met with Jens and professor Sullivan to demo this integrated pipeline and discussed the improvements that needed to be made to the manual subsystem – namely, creating a more realistic and stylistically representative animated device as well as creating a new dataset with real images.

On Thursday (04/09), I worked on finally reinstalling Python and the necessary packages and getting them to work on my laptop again.

On Friday (04/10), Shrutika and I met to work parallelly on different parts of the project. We began collecting images for our dataset on Wednesday, which I worked on collating – we now have close to 400 images that I have labeled and organized. Shrutika worked on setting up the Pis on her home network and setting up the camera modules in conjunction with the Pi.

Today (04/11), I completed the ethics assignment and will continue to work on the classifier and get it working at a reasonable accuracy with the real image dataset. We hope to have integrated and completed the manual mode subsystem by tomorrow night (04/12)!