Young’s Status Update for 12/6/20
This week I set up AWS and got it running code for the first time. Struggled a lot initially with memory issues and figuring out how to set up instances, but was very satisfied with the result. I got to test out the different models I had prepared on the Kaggle ASL dataset with different parameters and found that the best model returned a surprising 95% validation accuracy rate. Fine tuning the end layers after freezing the model from transfer learning resulted in some accuracy problems, but since most of the models did not take too long to run, I decided to train all of the model parameters over the night and found much better results. I also found that since the size of the dataset was so large (100,000 total images I subset from), training with even a few epochs returned very strong results. Having saved these model weights and model json files, I just need to export them into the web app and be able to run the classifier there. For the first time, there’s light at the end of the tunnel.
Aaron’s Status Update for 12/6/2020
This week I have been working on the trying to get a working prototype of our project up. I have updated the UI, integrated background subtraction and face removal into the video stream from the tablet. Most just getting things ready for the integration of the the neural network. I have also added a form that I am working on, in order to tell the app what sign we are trying to interpret as a part of the descoped version of the project. At this point we are just trying to get something that works and is presentable.
Malcolm’s Status Report 12/6/20
This week contained a lot of progress with background subtraction, bounding box experimentation and general debugging. Much of the progress dealing with the background subtraction was in terms of the implementation. Using background subtraction itself and image thresholding gave different results. However since background subtraction just captures the edges of a profile, the resulting image will have a smaller space necessary to represent it. So this method is helping our NN grow. We have also now decided to use a dedicated space for the “bounding box” on our hand, since the spatial tracking of the hand proved near impossible. I made attempts at it by using structural similarity and color profile-based searching in the image but the results were less than ideal. Now I am working towards fully implementing our pieces of the project and debugging.
Malcolm’s Status Report 11/28/20
This week was not a heavily productive week, as I had many assignments due, an interview and it was Thanksgiving. However I was still able to get work done on the Amazon database. I made progress on moving more of our testing and training data to the cloud as well as working on developing architectures for the neural network. We need to start fully implementing it soon, so will meet as a group and discuss the details of this.
Team Status Update for 11/21/2020
This week has been a lot of work trying to figure out how to combine the individual component of the project that we are all working on. We think we have figured out a way to integrate the neural network into the web app, so on that front it is just a matter of finishing the training on our final dataset, testing and then integration. the same goes for aws integration as well. Hand tracking is also a crucial component of the project that is still unimplemented but is being researched and worked on now. Once we figure out the next step of finishing our components and combining them I think we will be in a good place, but for now its going to be a lot of work to mitigate risks and finish the project.
Aaron’s Status Update for 11/21/2020
This week we met up to try to figure out how to consolidate our individual component for asli. I did some research into integrating Young’s work on the neural net we are training for gesture recognition into our web app. I think we were able to find a way to do so without to much hassle so now its just a matter of getting it up and running so we can integrate it. I also created to new branches for the development of hand tracking into the web app and that’s current what I’m working on figuring out how to do, the other branch is for a different method of capturing the tablet webcam video stream with lower latency. So next step for me are to get them working with priority on hand tracking as that feature is part of our core functionality. I believe we on schedule but we are on a tight deadline.
Malcolm’s Status Update for 11/21/20
This week i have been playing a lot of catch-up with my part of the data management of the project. One of our big challenges is efficient learning of the neural network, so having the data in a place to do this is of major importance. As such, we are using AWS to host both this and the network. So with this in mind I have made progress towards building up the repositories and systems we need in order to host both of these services. There are still parts of it I am unclear on and need to figure out. Otherwise I need to take time to keep understanding possible architectures for our neural network, as this is still a large part of the project that needs to be implemented. I believe our plan is to work together over the coming weeks to put our individual parts of the project together and then use them to configure and finish designing the network, as well as completely train it.
Team Status Update for 11/14/2020
This was mostly spent preparing for our interim demo that occurred this week. We had had brief meeting to see where we stand after receiving the feedback and plan to meet again this weekend to plan to solidify our schedule going forward to hopefully build a working prototype of our project so we can refine what we have. We have also uploaded the web app code to github so that we can all contribute and start combining our components. In our demo we clarified that we feel a little behind so our will be to help get back on track and mitigate any risk to the project, as well and scheduling next step for the coming weeks.
Aaron’s Status Update for 11/14/2020
This week was our interim demo so I spent most of my time making sure my component of the project was up and running to be presented. Apart from that I created a github repo for the web app, uploaded the code and we all have access as contributors. I also may have figured out and easier and better way to get the feed from the tablet camera that would reduce latency, so I’ll be looking into that. Next steps are to work on the feedback we received from the interim demo and and start the work of combining our components so we can develop a working prototype.