Category: Team Status Reports

What are the most significant risks that could jeopardize the success of the project? How are these risks being managed? What contingency plans are ready? 

Our project is working now. There are no major risks other than some part breaking in the last moment. There is no obvious thing that would break but with 10-20 parts any individual part has a small chance of failure. To mitigate for demo (and contingency), we have some backups parts that could theoretically be swapped.

Were any changes made to the existing design of the system (requirements, block diagram, system spec, etc)? Why was this change necessary, what costs does the change incur, and how will these costs be mitigated going forward?

No design changes other than making glasses into a glasses mask because we couldn’t get the hinges etc.. required to make glasses one size fit all.

Provide an updated schedule if changes have occurred

No changes have been made to schedule.

Testing

Unit tests: distance, angle, speed.

System tests: latency, usability, sensitivity, accuracy

Our overall finding from our tests is that the portions of the system that we developed perform very well and are very accurate. Most of our system limitations are due to the vosk V2t. For instance, our system losses in accuracy are fully from vosk as once a word is translated, our transmission and receiving process has perfect accuracy. Training our own V2T is incredibly difficult and would likely be worse than vosk anyways so there is not much we can do in regards to this. However, Vosk is still decently accurate and even when it makes mistakes, its mistakes are to similar sounding words that a user can often tell what was supposed to be said. In terms of our speed test, we have known for a while that our system was sufficiently fast (and that the limitations on speed were due to Vosk). As a result, we knew we could maintain hamming correction even though our accuracy is ~100% in transmission. For usability tests, switching to glasses seemed to make our device more usable and easy. People can now hold it up and take it down when they dont want to deal with it which is probably good for demo purposes. We had some friends try it and score it and they agreed. Finally, with system sensitivity, we discovered that our system is most suceptale to strong background noises. This is a vosk issues and the best thing we can do to mitigate it is to be careful with microphone placement. We made our microphone set up in a way that users can put it quite close to them which seems to be the best mitigation of sound sensitivity.

What are the most significant risks that could jeopardize the success of the project? How are these risks being managed? What contingency plans are ready? 

The main risk to our project at the moment is the failure of the electronics in the mounting of our design. However, since we have successfully mounted one of the glasses, we are confident that the other should go even smoother. One other things that could prove difficult is making sure that everything is clean and readily available for a user to use that may not be as familiar with the project as we are. This involved cleaning up wiring and making sure that nothing will break. We already have plans in place to remedy this as well as contingency plants to simplify the glasses design so that anyone can sue them without much issue. This could be not mounting all of the parts of the glasses because they may not fit everyone’s face.

Were any changes made to the existing design of the system (requirements, block diagram, system spec, etc)? Why was this change necessary, what costs does the change incur, and how will these costs be mitigated going forward?

The only thing that may have changed I have described above, where we may simplify the design for the demo for easy of use, while having one complete system to display what the completed product will look like. This does not have a large cost to the project other than not having a totally completed product like we envisioned for the demo, but makes the demo much ore accessible without having to change the physical design.

Provide an updated schedule if changes have occurred

No changes have been made to the design or our schedule.

The biggest risk to the project is still all of the logistics that come with mounting our system to the glasses. We need to ensure that everything is mounted securely while still keeping a minimal footprint with all of our wiring, and ensuring that our LCD will be readable. This risk is being managed mainly with our being ahead of schedule leaving us plenty of time to work out any difficulties that come with mounting. We are also continuing to keep in mind the suggestion of pivoting to a handheld system as a last resort.

No changes have been made to the design or our schedule.

Both our verification and validation will be described following, because our subsystems basically fully integrated and so all of our forms of testing will be collaborative within in the team in some way.

In terms of verification, the main metrics we need to test are end-to-end accuracy and latency, as well as connection angle and distance. Based on some initial testing, we don’t believe angle and distance will have a significant impact on accuracy and latency, it appears to be an all or nothing connection, so our tests for those will be separate.

For our accuracy and latency, we will be using around 5-10 spoken and computer generated passages of vary accents and speeds to measure both metrics. Each passage will be repeated around 3 times with averages of different wpm and overall averages taken for our reporting. During this, we will also perform stress tests involving disrupting the line of sight to ensure full functionality. These reported metrics will map directly back to the matching use case and design requirements.

For our angle and distance, we will be transmitting simpler passages compared to above to simply check for a connection. We will do this ranging from 3-15 ft and a 40 degree range and constructing a graph of where connection is possible. This will allow us to provide a visual representation of where our system is usable.

For validation we will be utilizing our friends and hopefully volunteers who are hard of hearing or deaf to provide feedback on the usability of our system. We will be creating a survey that captures our more qualitative requirements, such as how it feels to wear and how obstructive it is.

What are the most significant risks that could jeopardize the success of the project? How are these risks being managed? What contingency plans are ready? 

Our most significant risks now come from debugging bidirectional communication. Our bi directional works well in a normal conversation flow, but if two people talk relatively slowly and overlapping, the system has trouble determining who to transmit and gets stuck. This risk is being managed by coming up with code debugging and testing strategies to try and determine what is specifically happening on the backend. We are also coming up with contingency plans to just hardcode a ‘priority’ that can guarantee only one person is sending/receiving. There are a few other contingency plans we have including using a reserved EOP character to signal a switch in priority. Other than that, our only other main thing left is mounting. We dont foresee this causing many risks however we could just end up with some messy wires. We will try to avoid this but also only have so much time to iterate our mount design.

Were any changes made to the existing design of the system (requirements, block diagram, system spec, etc)? Why was this change necessary, what costs does the change incur, and how will these costs be mitigated going forward?

We have some small changes to our mounting plan. We now have a lens on one eye so that you can focus onto the screen. However, this lens is only on the bottom of the eye-hole so that you can still focus your eye on the background. This change was necessary as if your eyes are only looking through lens for too long then can become strained. There is not too much of a cost associated with this, aside from potential for eye straining which we mitigated by allowing you to focus on and off the lens. As mentioned earlier, no changes yet for bidirectional protocol but we are considering contingency changes.

Provide an updated schedule if changes have occurred

There are no changes to our schedule as we are all either on schedule or ahead.

We have some pictures and videos similar to last week but for 2-way communication. We will show them at demo if needed 🙂 (hopefully not )

What are the most significant risks that could jeopardize the success of the project? How are these risks being managed? What contingency plans are ready? 

As of now, there are no major risks to the success of the project. Our work is going smoothly and things are working as intended with minimal trouble and wasted time. We were able to demonstrate full one way communication including decoding, lcd screen, and v2t with our testing setup, This is a huge milestone and means a lot of the software work can move on to developing the 2-way communication. Our main bottleneck at this time may involve the integration of every part of the system into one device, which will involve a decent amount of design work, but shouldn’t be too much trouble. However, if we cannot achieve integration by the interim demo, we are still able to do a makeshift mounting of the system onto existing glasses.

Were any changes made to the existing design of the system (requirements, block diagram, system spec, etc)? Why was this change necessary, what costs does the change incur, and how will these costs be mitigated going forward?

No changes were made to the design this week other than small tweaks to how we are displaying the words on our display during testing. This will be an ongoing process to determine what enhances readability the most. Otherwise, this week was mainly spent testing our existing design.

Provide an updated schedule if changes have occurred

There are no changes to our schedule as we are all either on schedule or ahead.

Here is a video demonstrating our full one way communication: https://youtu.be/-NNWBw4ArGU

Here is a photo of the full testing setup with he receiver and display on the left and transmitter on the right side:

We believe that at this point there are no major risks to the project, the different parts on the project are coming together nicely. The most significant risk at this point is ensuring that our LCD will be readable from a reasonable distance, as well as finalizing how many words will be displayed on it at a time. We want the LCD to be mounted fairly close to the glasses, so we will be trying to adapt to using a lens as suggested to make the LCD more readable from the short distance. This will likely require some trial and error to make it work well.

We are mostly managing this risk by ensuring we have plenty of slack time to allow for this trial and error to happen. If all else fails we have our contingency plan of switching from glasses mounted to phone mounted to give us an easier screen for the user to read from.

As mentioned above, we will be adding a lens to our design to improve the readability of our LCD. The lenses only cost around $15 for 2 so this is well within our budget to add.

No changes have occurred to our schedule.

Below we have pictures of our signal generation on the oscilloscope.

What are the most significant risks that could jeopardize the success of the project? How are these risks being managed? What contingency plans are ready? At this point, the one main risk for our project success is the ability to get the NEC IR modulation scheme working on a pico. While seemingly (easily) possible as there are many videos and documents about how to do similar things, this is a very important part of our system. This risk is being managed by starting early and giving ourselves a lot of time in schedule + slack to work through any issues. Now that we have picked up the pico, we will be mainly focused on this portion of our system for a while. We have already drafted out code which seems plausible and are hoping that HW timed pwm + SW timed pulsing is accurate enough for our needs (which I think it probably is fingers crossed). If this absolutely cannot work, we can switch our transmission scheme timing requirements and make it a little slower (not terribly slow, just a bit slower) to get within the accuracy range for SW timing. This would hopefully make our system work.

Were any changes made to the existing design of the system (requirements, block diagram, system spec, etc)? Why was this change necessary, what costs does the change incur, and how will these costs be mitigated going forward? No changes were made to the design of the system this week. We are still working through implementing our change from last week (switching GPIO to a pico instead of directly on a Pi5). 

Provide an updated schedule if changes have occurred. No schedule updated. One might come in a week if we are having large pico difficulties.

A was written by David, B was written by Todd and C was written by DJ.

Part A: … with consideration of global factors. Global factors are world-wide contexts and factors, rather than only local ones. They do not necessarily represent geographic concerns. Global factors do not need to concern every single person in the entire world. Rather, these factors affect people outside of Pittsburgh, or those who are not in an academic environment, or those who are not technologically savvy, etc.

Deafness and hard of hearing are not disabilities constrained to the United States. In many regions of the world, these conditions can have even greater social and economic impact due to limited access to first world healthcare, disability services, hearing aids, or cochlear implants. In lower income countries, assistive technologies are often entirely unavailable, and public accommodations such as interpreters may not exist. As a result, accessibility needs for deaf and hard of hearing individuals vary significantly across countries and economic conditions. By enabling fully on-device speech-to-text processing without requiring cloud services / Wi-Fi connectivity, eyeR supports communication in environments with limited technological infrastructure. This makes the system relevant not only in technologically advanced regions, but also in underdeveloped, rural, or low infrastructure communities worldwide. Our design also reduces dependence on stable internet access, ensuring reliable functionality in areas where connectivity may be unstable or entirely unavailable.

In addition, technological literacy varies significantly across the world. Not all users are familiar with setting up software, working with the cloud, or troubleshooting devices. EyeR addresses this by prioritizing simplicity; the system is designed to function with no real setup, allowing users to simply wear the glasses and begin speaking (plug and play). The discreet glasses-based form factor integrates naturally into everyday life, making it socially acceptable across the globe. Furthermore, by relying on low-cost, widely available hardware and open-source software, the design is cheap and can be more easily distributed in reduced economic regions. Our approach ensures that eyeR is not limited to highly developed American markets, but instead remains accessible and scalable across a global population.

Part B: … with consideration of cultural factors. Cultural factors encompass the set of beliefs, moral values, traditions, language, and laws (or rules of behavior) held in common by a nation, a community, or other defined group of people.

Our device addresses the needs of the hearing impaired and hard of hearing community, and offers an alternative to bridge the gap between those who can hear and those who cannot. Sign language is not universal to the entire HoH community across many cultures and especially not within the hearing population. EyeR facilitates real-time, seamless communication that is accessible to all communities without severely altering modes of natural language. EyeR also facilitates eye contact and face-to-face communication, which is a universal mode of connection across cultures. In addition, with a transparent screen, natural conversational pacing and social interaction can be maintained without the use of a phone or other device.

EyeR is also designed with privacy in mind. Privacy is very important to many people and, across cultures and society at large, 1 on 1 conversations are meant to be private. By offering a mode of communication (infrared) that cannot be easily intercepted, eyeR allows privacy to be maintained. All data processing of the conversation, like the voice to text model, is done locally, with nothing stored in the cloud or put on the internet, keeping conversations private and secure between those that are communicating. This upholds the ethical standard of keeping conversations private when desired, and making sure that users feel safe to discuss personal and private matters without fear. This device offers a way for those in the HoH community to have an accessible and functional method that strengthens natural human interaction without totally replacing it.

Part C: … with consideration of environmental factors. Environmental factors are concerned with the environment as it relates to living organisms and natural resources.

Our first consideration with regards to environmental factors is that we don’t want our solution to require a lot of resources to make, or to be wasteful. This aligns well with our use case requirement that our solution should be lightweight and low profile as well as low cost. With our use of IR light for our serial communication, we’re able to keep the amount of hardware we need for our solution fairly minimal. In addition to this, we’re using industry standard LEDs and sensors similar to those used in TV remotes, so they have good longevity. Finally, with our use of a rechargeable battery pack to power our whole system, we will have no waste in terms of single use items like batteries.

Secondly, an important aspect of our solution is that it is meant for use outdoors for all types of social outings. Because of this, we need to consider how our technology could impact wildlife. The only disturbances that our technology would introduce to the environment is based on our IR serial communication. As we don’t anticipate our glasses being used in super close proximity to animals ( < 10 ft) the safety concerns associated with our users should be a nonissue in terms of wildlife. The main concern with the impact to wildlife would be the IR light itself, since some animals are able to see IR light we have to consider if our product would present any negative visual stimuli. In terms of the intensity of our IR light, it is less powerful than the ambient IR light due to the sunlight. We believe that it is unlikely that our 38kHz carrier frequency would make it discernable to animals in anyway, but either way our LED only has a range of 10-20 ft straight ahead and isn’t powered on unless the user is actively in conversation with another user, meaning there is for all purposes no impact on the wildlife.

We believe that the biggest risk at the moment is making sure that we are able to cleanly interface with the IR LEDs and the receivers. We had some trouble getting the LED to work with the RPi 5, so we are pivoting to using PICO boards that should be easier to interface. So right now on that end of the project, we are waiting for those to come in to continue to test the communication and IR signal generation.

However, the RPi 5 is working very well with the voice to text software and we were able to get it working with the I2S MEMS microphone that we have, which is a big step.

The only change to the design right now, as I mentioned before, was switching to RPi PICO boards for the IR LED component of the design. This was necessary due to the GPIO difficulty on the RPi 5 that we have. The costs to do this aren’t too high due to the affordability of the PICO boards.

Our schedule remains the same since we are already ahead of schedule and we were still able to get other parts of the device working well.

Here is a short video example of the V2T on the RPi using our MEMS mic:https://youtube.com/shorts/DJ7xVhHPy48

We believe the most significant risk to the project is still the level of environmental noise and interference that will occur with the IR communication. Since our parts have only just arrived at the end of this week, we have so far been mitigating the risk with plenty of research and design into error detection + correction to strengthen the end-to-end accuracy of our solution. In the coming weeks we plan on integrating the LED/Receivers early on with our signal processing code to determine the actual risk and if different parts / more design will be needed to mitigate.

 

The only minor change to our design is we will likely be using 940nm wavelength IR light instead of our proposed ~1400nm due to higher availability. This will likely be an improvement for the project in general, the 940nm parts are cheaper, have better directivity, and are more suited to IR serial communication due to their commercial use in remotes. This choice will likely incur a small additional cost of us needing to purchase clear IR filters to place on the glass lenses as a safety backup, since the smaller wavelength is technically less safe (although still low enough power to be completely harmless) it would not hurt to include the filters.

No changes have occurred to the schedule, we are well on track and likely a bit ahead of schedule for now.