Tag: Status Report

This week we made a break through with the Django app. We are now able to stream video from the Fire tablets camera to the Django web app using an app called IP Webcam and OpenCV to connect the camera to the web app. It still needs polishing and we still need to test if it can run on the tablet itself but it is decent progress. Our next task after polishing should be to start developing the image processing through OpenCV so we can reduce information in images to be compare to the neural network.

We also went over our Design report feedback and clarified some parts of our design moving forward. Many of the techniques in the report like background subtraction, bounding box, joint extraction, binary pixel->integer encoding aren’t all going to be implemented in combination in the final product, they’re more previously explored feature extraction methods. They were included in the report because we unsure if we would use them due to efficiency tradeoffs. Some of them like joint extraction will take significant effort to build from scratch but there are existing module calls that assist with that, so we can use those for the time being to assess tradeoff and then maybe reimplement on our own if we actually find it useful. So although it sounds like a lot, we think its manageable.

This week was midterm week for most of us so not much work has been completed this week. There have been some difficulties in the development of the web app that pertain to having a live feed of the camera capture that have setback that portion of the project a bit. Another concern we are having is the tradeoff between writing computer vision algorithms such as hand detection and using existing modules and their functions. We are concerned that if we write most of the functionality from scratch, our implementation won’t beat the existing optimized module functions and cause our projects performance to suffer, such as increasing the response time beyond the real time constraints. A quick look at our Design report feedback shows that we still need to structure our design better and it seems like we need to do some thinking about our current approach to Machine Learning as currently our approach contains a lot of work to be done. We also need to be clear about what portions of the project are facts from experiment or hypothesis to validate. To mitigate risks we need to more deliberate with how we work on the project and we probably need to come together more to combine our individual components. Our next steps should be to take our Design report feedback to heart and refine it. As well as trying to get the project back on schedule.

OpenCV has proved to be more of a learning process than expected so web app integration has not been as fast as we had hoped but we are still on track. One thing that many other teams made apparent, that we did not consider, during the design reviews this week was that we could simplify our design by hosting the web app on AWS. We don’t think will fundamentally change our approach, it just means that aside from testing we will not have to run our own server to deploy the web app on our tablets which should save us some headaches in the future.

This week we all started working on different components of the project, the web app, edge detection and looking at the datasets to start building the neural network. Work on the web app had some issues that set us back but the server is now accessible on our tablets which is progress. We’ve done some experimentation with the OpenCV api in Django but we have fully implemented it into the app yet. The app is a crucial part of our project so it is important to get right as it ties together our deep learning and computer vision work. We also still need to create a github repository for synchronization across the different modules we are working on.

The progress with the neural net and image processing are also coming along, at a pace where we will be able to test and debug them in sync. Currently, we still need to have a team meeting about the neural net architecture and design. The main points of contention or inquiry are about specific libraries, the high-level layering of the net, and input data formatting. We also want to look into physically testing the edge detection we are writing, so we are looking into methods or ideas for this. However, the edge detection is just to speed up our net’s learning speed so we aren’t stressing over creating test cases for it specifically.

This week as we are still in the preliminary stages of our project we decided to take the week to familiarize ourselves with Machine Learning/Deep Learning before we attempt to start implement our own neural networks for ASL recognition. We decided to go through the lectures from 10-601 Introduction to Machine Learning and go through the lectures for Regularization, Neural Networks, Backpropagation, and Deep Learning. Next week we hope to begin looking at Open CV and see how we can get it running on our tablets as well getting the ball rolling by looking through our data sets we found for our ASL recognition model and deciding which one to tackle first. We are currently on track with our project schedule and how to start working on the open cv and data set aspects of the project. Some risks or issues we foresee in the project is us spending a lot of time learning the concept required to implement our project so we’re keeping a close eye on the amount of time we dedicate to studying concepts and we hope to be able to put what we learned into practice with a relatively short turnaround. So far no significant changes have been made to the project and we don’t have any issues currently.

Lectures page: (https://scs.hosted.panopto.com/Panopto/Pages/Sessions/List.aspx#folderID=%229044a1d8-bf2d-4593-b478-a9d100e8a09f%22)