Category: Status Report

This week we have been gearing up to combine our individual components into a working demo, we are a little behind in the integration as we have had to do a lot of learning and troubleshoot but we seem to be making progress nonetheless. The web app is able to run and accessed from the amazon tablet and is currently running a basic face tracking algorithm for testing. The goal is to transition to hand tracking and feature extraction and moving forward from that; integrate the neural network and get basic recognition working. To mitigate the risk of not having a viable product in the end, we may have to cut features we originally planned on having such as voice recognition and perhaps reduce the number dynamic signs interpreted while we get the static signs working. Hopefully it doesn’t come to that but we reach the point where we have to stat thinking about these decisions.

This week we made a break through with the Django app. We are now able to stream video from the Fire tablets camera to the Django web app using an app called IP Webcam and OpenCV to connect the camera to the web app. It still needs polishing and we still need to test if it can run on the tablet itself but it is decent progress. Our next task after polishing should be to start developing the image processing through OpenCV so we can reduce information in images to be compare to the neural network.

We also went over our Design report feedback and clarified some parts of our design moving forward. Many of the techniques in the report like background subtraction, bounding box, joint extraction, binary pixel->integer encoding aren’t all going to be implemented in combination in the final product, they’re more previously explored feature extraction methods. They were included in the report because we unsure if we would use them due to efficiency tradeoffs. Some of them like joint extraction will take significant effort to build from scratch but there are existing module calls that assist with that, so we can use those for the time being to assess tradeoff and then maybe reimplement on our own if we actually find it useful. So although it sounds like a lot, we think its manageable.

This week was midterm week for most of us so not much work has been completed this week. There have been some difficulties in the development of the web app that pertain to having a live feed of the camera capture that have setback that portion of the project a bit. Another concern we are having is the tradeoff between writing computer vision algorithms such as hand detection and using existing modules and their functions. We are concerned that if we write most of the functionality from scratch, our implementation won’t beat the existing optimized module functions and cause our projects performance to suffer, such as increasing the response time beyond the real time constraints. A quick look at our Design report feedback shows that we still need to structure our design better and it seems like we need to do some thinking about our current approach to Machine Learning as currently our approach contains a lot of work to be done. We also need to be clear about what portions of the project are facts from experiment or hypothesis to validate. To mitigate risks we need to more deliberate with how we work on the project and we probably need to come together more to combine our individual components. Our next steps should be to take our Design report feedback to heart and refine it. As well as trying to get the project back on schedule.

OpenCV has proved to be more of a learning process than expected so web app integration has not been as fast as we had hoped but we are still on track. One thing that many other teams made apparent, that we did not consider, during the design reviews this week was that we could simplify our design by hosting the web app on AWS. We don’t think will fundamentally change our approach, it just means that aside from testing we will not have to run our own server to deploy the web app on our tablets which should save us some headaches in the future.