What are the most significant risks that could jeopardize the success of the project? How are these risks being managed? What contingency plans are ready?
Some of the significant risks that could jeapordize the success of the project are edge cases if the user presses inbetween two keys. How are ties broken? What if they are closer to one but still on the other? We mitigated this risk by simply taking the higher note (in pitch) to break ties.
Were any changes made to the existing design of the system (requirements, block diagram, system spec, etc)? Why was this change necessary, what costs does the change incur, and how will these costs be mitigated going forward?
We made several changes to the existing design our system requirements. The first being identifying a battery to charge the gloves + micro-controller. We decided to settle on using AAA batteries in order to power this since they are small and lightweight and can provide enough to power the device at 3.3v for about roughly 3 hours.
Another change we made to the system is coloring the white keys of the keyboard layout to differentiate between octaves. We know that the camera field of view cannot see the entire length of the keyboard, so if it is only fixed on viewing two octaves at once (our current requirement), distinguising which octaves via different colors will significantly aid the CV algorithm.
For identifying fingers, we dicussed that we would add another color to the finger by printing a color out and attaching it to the top of the gloves in the view of the camera. This would help the CV algorithm identify what in the image are fingers. In addition, we are considering attaching fiducials of that color to these fingers, so the algorithm can further differentiate the fingers from one another, which will be integral when it gets the data from the micro-controller about which finger it is that is pressing a key from the pressure sensor.
As you’ve now established a set of sub-systems necessary to implement your project, what new tools have your team determined will be necessary for you to learn to be able to accomplish these tasks?
We’ve established that the iphone or android camera will need to interface with the app. This sometimes poses a technical challenge since making our product for both apple and android phones is tricky. We will be using and need to learn to use Xcode since it will be used as a wrapper over the python kivy interface to accomondate for apple products.
For interfacing the cameras of apple phones to Lee’s windows laptop for testing and streaming was also a challenge. He learned to use a tool called Iriun webcam to help accomplish having a streaming video feed from his iphone camera to work on his windows laptop. Android phones have an easier time doing this since the apple app store has may have more restrictions.