Team Status Report for 3/7

What are the most significant risks that could jeopardize the success of the project? How are these risks being managed? What contingency plans are ready? At this point, the one main risk for our project success is the ability to get the NEC IR modulation scheme working on a pico. While seemingly (easily) possible as there are many videos and documents about how to do similar things, this is a very important part of our system. This risk is being managed by starting early and giving ourselves a lot of time in schedule + slack to work through any issues. Now that we have picked up the pico, we will be mainly focused on this portion of our system for a while. We have already drafted out code which seems plausible and are hoping that HW timed pwm + SW timed pulsing is accurate enough for our needs (which I think it probably is fingers crossed). If this absolutely cannot work, we can switch our transmission scheme timing requirements and make it a little slower (not terribly slow, just a bit slower) to get within the accuracy range for SW timing. This would hopefully make our system work.

Were any changes made to the existing design of the system (requirements, block diagram, system spec, etc)? Why was this change necessary, what costs does the change incur, and how will these costs be mitigated going forward? No changes were made to the design of the system this week. We are still working through implementing our change from last week (switching GPIO to a pico instead of directly on a Pi5). 

Provide an updated schedule if changes have occurred. No schedule updated. One might come in a week if we are having large pico difficulties.

A was written by David, B was written by Todd and C was written by DJ.

Part A: … with consideration of global factors. Global factors are world-wide contexts and factors, rather than only local ones. They do not necessarily represent geographic concerns. Global factors do not need to concern every single person in the entire world. Rather, these factors affect people outside of Pittsburgh, or those who are not in an academic environment, or those who are not technologically savvy, etc.

Deafness and hard of hearing are not disabilities constrained to the United States. In many regions of the world, these conditions can have even greater social and economic impact due to limited access to first world healthcare, disability services, hearing aids, or cochlear implants. In lower income countries, assistive technologies are often entirely unavailable, and public accommodations such as interpreters may not exist. As a result, accessibility needs for deaf and hard of hearing individuals vary significantly across countries and economic conditions. By enabling fully on-device speech-to-text processing without requiring cloud services / Wi-Fi connectivity, eyeR supports communication in environments with limited technological infrastructure. This makes the system relevant not only in technologically advanced regions, but also in underdeveloped, rural, or low infrastructure communities worldwide. Our design also reduces dependence on stable internet access, ensuring reliable functionality in areas where connectivity may be unstable or entirely unavailable.

In addition, technological literacy varies significantly across the world. Not all users are familiar with setting up software, working with the cloud, or troubleshooting devices. EyeR addresses this by prioritizing simplicity; the system is designed to function with no real setup, allowing users to simply wear the glasses and begin speaking (plug and play). The discreet glasses-based form factor integrates naturally into everyday life, making it socially acceptable across the globe. Furthermore, by relying on low-cost, widely available hardware and open-source software, the design is cheap and can be more easily distributed in reduced economic regions. Our approach ensures that eyeR is not limited to highly developed American markets, but instead remains accessible and scalable across a global population.

Part B: … with consideration of cultural factors. Cultural factors encompass the set of beliefs, moral values, traditions, language, and laws (or rules of behavior) held in common by a nation, a community, or other defined group of people.

Our device addresses the needs of the hearing impaired and hard of hearing community, and offers an alternative to bridge the gap between those who can hear and those who cannot. Sign language is not universal to the entire HoH community across many cultures and especially not within the hearing population. EyeR facilitates real-time, seamless communication that is accessible to all communities without severely altering modes of natural language. EyeR also facilitates eye contact and face-to-face communication, which is a universal mode of connection across cultures. In addition, with a transparent screen, natural conversational pacing and social interaction can be maintained without the use of a phone or other device.

EyeR is also designed with privacy in mind. Privacy is very important to many people and, across cultures and society at large, 1 on 1 conversations are meant to be private. By offering a mode of communication (infrared) that cannot be easily intercepted, eyeR allows privacy to be maintained. All data processing of the conversation, like the voice to text model, is done locally, with nothing stored in the cloud or put on the internet, keeping conversations private and secure between those that are communicating. This upholds the ethical standard of keeping conversations private when desired, and making sure that users feel safe to discuss personal and private matters without fear. This device offers a way for those in the HoH community to have an accessible and functional method that strengthens natural human interaction without totally replacing it.

Part C: … with consideration of environmental factors. Environmental factors are concerned with the environment as it relates to living organisms and natural resources.

Our first consideration with regards to environmental factors is that we don’t want our solution to require a lot of resources to make, or to be wasteful. This aligns well with our use case requirement that our solution should be lightweight and low profile as well as low cost. With our use of IR light for our serial communication, we’re able to keep the amount of hardware we need for our solution fairly minimal. In addition to this, we’re using industry standard LEDs and sensors similar to those used in TV remotes, so they have good longevity. Finally, with our use of a rechargeable battery pack to power our whole system, we will have no waste in terms of single use items like batteries.

Secondly, an important aspect of our solution is that it is meant for use outdoors for all types of social outings. Because of this, we need to consider how our technology could impact wildlife. The only disturbances that our technology would introduce to the environment is based on our IR serial communication. As we don’t anticipate our glasses being used in super close proximity to animals ( < 10 ft) the safety concerns associated with our users should be a nonissue in terms of wildlife. The main concern with the impact to wildlife would be the IR light itself, since some animals are able to see IR light we have to consider if our product would present any negative visual stimuli. In terms of the intensity of our IR light, it is less powerful than the ambient IR light due to the sunlight. We believe that it is unlikely that our 38kHz carrier frequency would make it discernable to animals in anyway, but either way our LED only has a range of 10-20 ft straight ahead and isn’t powered on unless the user is actively in conversation with another user, meaning there is for all purposes no impact on the wildlife.


David’s Status Report for 3/7

What did you personally accomplish this week on the project?

I spent probably 15 hours on design report so evidently some of my project work is a little lighter this week. However, even though we got the pico later than expected I was able to begin some research and code for that. I drafted out some code for PWM and for duty cycle modulation. I made 2 versions of this, one for SW timed and one for HW timed. Because the pico wasn’t picked up until after I was already on break, I was unable to test yet but hopefully not a difficult process.

Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?

I am nicely on schedule. Much of our work is now dependent on getting the pico code to work on board. Hopefully this isn’t a terrible process because once its done, then we can shift over to LCD and receiver decoding subsystem.

What deliverables do you hope to complete in the next week?

Next week my main focus is putting and debugging the pico code on the pico. After that, me DJ and Todd will likely spend an afternoon (or two) just working through bugs to get the light modulation working strongly. Again, hoping no huge issues as this is a large part of the project.

DJ Status Report 3/7

Naturally, a lot of my time this week was spent working on the design report. Most of my focus was on the system implementation section, as well as some time spent on design requirements, architecture, and tradeoffs.

In terms of additional progress on getting physical things working for our project, I was fairly limited this week due to waiting on the Raspberry Pi Picos to arrive for most of the week. I was able to due a lot of research on how to implement our signal generation on the Pico and its GPIO capabilities. There appears to be options for hardware timed PWM and software timed overall pulses (calls to sleep on the Pico are much more accurate making this a viable option) as well as hardware timing for our entire signal generation. This puts me in a good place for next week to get our signal generation working on the Pico.

I am still nicely on schedule for the project. As mentioned last week we had to shuffle some things around on the schedule, but I am on track with our new order of implementation.

Next week I will definitely be able to get the signal generation on the Pico working to the point where we can view sent words on the oscilloscope. Hopefully I’ll have time to work a bit ahead as well and help out with the signal detection and decoding side of the program as well.

Todd’s Status Report for 2/21

Personal Accomplishments this Week

I mainly worked alongside DJ and David testing and validating some of the hardware for our device. Earlier in the week, we worked on trying to understand the signal coming out the RPi GPIO for the IR LED with an oscilloscope. This led us to pivot to using the PICOs since we had trouble getting a signal in the first place. Then later in the week, I worked with DJ on validating the voice to text model on the RPi with our MEMS mic, which worked out really well, and that part of the project is ready to move forward.

On Schedule?

I am on schedule alongside the rest of the team for this week, and we are still staying slightly ahead of where we want to be which is good.

Next Week Deliverables

I am going to keep supporting my other groupmates with getting the hardware tested and functional, and will be ready to work with the PICO boards once they come in. I will also be helping the David and DJ on completing the design report due at the end of the week.

Team Status Report 2/21

We believe that the biggest risk at the moment is making sure that we are able to cleanly interface with the IR LEDs and the receivers. We had some trouble getting the LED to work with the RPi 5, so we are pivoting to using PICO boards that should be easier to interface. So right now on that end of the project, we are waiting for those to come in to continue to test the communication and IR signal generation.

However, the RPi 5 is working very well with the voice to text software and we were able to get it working with the I2S MEMS microphone that we have, which is a big step.

The only change to the design right now, as I mentioned before, was switching to RPi PICO boards for the IR LED component of the design. This was necessary due to the GPIO difficulty on the RPi 5 that we have. The costs to do this aren’t too high due to the affordability of the PICO boards.

Our schedule remains the same since we are already ahead of schedule and we were still able to get other parts of the device working well.

Here is a short video example of the V2T on the RPi using our MEMS mic:https://youtube.com/shorts/DJ7xVhHPy48

DJ Status Report for 2/21

At the beginning of this week I spent a bit of time doing some final preparation for my design presentation. Also towards the start of the week I did some work with David with signal generation on the Pi. Because of the change in how the GPIO pins work on the Pi 5, none of the libraries seemed to have the functionality that we needed (hardware timed and able to create a wave of multiple pulses at once) to achieve the accuracy we want, so we were testing a method using a Linux kernel driver for IR that had mixed reviews on the Pi forums. This didn’t work so we are migrating our design to use a Pico to handle the IR hardware interfacing.
At the end of the week I pivoted and got the Vosk speech to text working on the Pi with our breakout mic.

I am still on schedule if not slightly ahead again. The Vosk took less time to integrate with the Pi than I had anticipated, and I’ll be able to get back to signal generation when the Picos arrive next week so the schedule is looking good.

Next week i’ll be spending some time on the design report document. I’ll also be working (along with David) on the signal generation and decoding code on the Pico as well as the code to accurately query Vosk.

David’s Status Report for 2/21

What did you personally accomplish this week on the project?

I spent a decent amount of time in the early week with DJ on beginning the software for our project. We setup our RPis, but our V2T onto the Pis and began attempting GPIO modulation code. Unfortunately, the Pi 5 has a much more difficult time with hardware level GPIO than past Pis. As a result, I also spent time reformulating some design to use a pico for led control. This should be a far far easier way to actually get code to modulate the light. Aside from that, I did some work on getting the LCD screen interface coded. I have also spent about 5 hours beginning design report.

Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?

We are still ahead of schedule. Along with DJ we have packet encoding and decoding programs made. With our switch to pico, our scheduling may definitely be shifting a little bit in the next week or 2 depending on how difficult that is to work with.

What deliverables do you hope to complete in the next week?

Next week my main focus is working on pico integration. I (we) would like to get the modularized light working so that we can begin integrating with the V2T and our encoding / decoding software. Based on research this week, we think the pico should be easy to work with for this as it is designed more as a hardware-level microcontroller. Aside from this, I am also working on design report which we will complete by Friday.

Todd’s Status Report for 2/14

Personal Accomplishments this Week

Along with David, I was out of town for this week so thew work that I as well a the team got done was mostly related to the design review slides. I worked alongside the team to finish the slides, working on the diagrams and slides related to hardware. Other than that I did some further parts research on the parts that we ordered that we can begin work as soon as we can with the components.

On Schedule?

I am personally on schedule for what we wanted to accomplish this week and will be ready to begin the parts testing that we are planning on doing next week.

Next Week Deliverables

When we are doing testing next week with the RPI and LEDS, I want to keep running notes and documentation so that we have all of the information that we need so we can look back at how things performed and don’t need to do any redundant testing later.

DJ Status Report 2/14

This week, most of my time was spent working on the design presentation and preparing to present, as I am presenting this presentation. Other than that, I finalized my design research into our IR communication protocol (pictured below) and worked out the encoding of letters and the timing of the protocol to ensure that we meet our use case requirements for speed and latency (200-250WPM and 500ms latency), while making sure that we’ll have enough leeway to handle errors. At the end of the week, I also worked on the code for our Hamming encoding, testing to make sure that it would fix single bit errors and detect 2 bit errors for our custom letter encoding.

I am well on schedule with my portion of the project, slightly ahead as I hadn’t planned on beginning coding until next week.

Next week I believe that a good chunk of my time will be spent working on. the design report. I’d like to also decide on a Raspberry Pi library to use for signal generation and detection through the GPIO. Changes with how the GPIO work from the Pi 4 to 5 have apparently rendered a lot of GPIO libraries incompatible with the Pi 5, so this will require some research to determine a good fit for our project. Hopefully I’ll be able to test some simple signal generation with Todd and the waveform generator at the end of next week.

 

David’s Status Report for Feb 14th

What did you personally accomplish this week on the project?

I was out of town for much of the week so most of my accomplishments were remote accomplishments (plus we were waiting on parts anyways). I spent a decent portion of time creating and then going through batches of edits for the slides so that DJ is ready to present next week. I was largely ahead of my gantt schedule but I went ahead and continued research into signal Rpi protocol libraries. I have researched the gpiozero / libgpio library and went through example codes that proves that it should be able to hit our design requirements of light modularization.

Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?

I am ahead of schedule, we got through V2T relatively fast and can now start transitioning to hardware components as all of our parts should now be here to pick up!

What deliverables do you hope to complete in the next week?

Next week my main focus is working on packet protocol. I would like to use the GPIO library(s) to actually write the code for the led modularization and be able to code our encoding scheme into the RPi. Along with todd, we would also start some hardware implementations such as putting the V2T onto the Rpi and beginning to test light modularizations.