Davids Status Report for 3/21

What did you personally accomplish this week on the project?

I wrote code for pwm decoding and lcd screen. I then tested this code and did some debugging until it worked on the pico / breadboards we have set up. Then, I combined my codebase with DJs code that connected the Pi V2T with the pico and we were able to setup a full line of communication. ie Voice to text to pi to pico to LED to receiver to pico to LCD screen. This was more or less one directional tech mvp.

Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?

I am solidly ahead of schedule. We now have one directional transmission with high accuracy. There are hopefully some photos of this in team report.

What deliverables do you hope to complete in the next week?

Next week our main focus is on the second direction of communication. This includes some protocol handling for if 2 people are speaking at the same time. We will also spend some time on accuracy debugging. It seems like our accuracy is dependent on our V2T software (which we dont have control over), however this accuracy can be fine tuned with microphone placement. We will be testing this more soon so we know how to properly mount our devices.

Todd’s Status Report for 3/14

Personal Accomplishments this Week

This week, I mainly assisted DJ and David as they worked to get the communication between IR transmitter and receiver up and running. this involved working with the oscilloscope and conducting tests with all 3 of us to test distance constraints  and directionality for the IR components. I also spent a good bit of time working on the individual portion of the ethics assignment this week as well.

On Schedule?

I and the rest of the team are on or ahead of schedule,  but as the semester gets closer to the end, my work will begin to get more involved. So, for the next week, I want to try and get a step ahead of some of my tasks so that I am not feeling as much of a time crunch as my workload gets more heavy on the project.

Next Week Deliverables

Next week I am planning on continuing to work with David and DJ on getting all the hardware up and running to where we want it to be. I also want to begin thinking more about the physical integration of all the parts of our project, since we may want to add a lens as well.  Thinking more about integration now will let us work out the problems and potential issues so that components fit together properly and we can shift focus on more complex parts of the project if needed.

Team Status Report for 3/14

We believe that at this point there are no major risks to the project, the different parts on the project are coming together nicely. The most significant risk at this point is ensuring that our LCD will be readable from a reasonable distance, as well as finalizing how many words will be displayed on it at a time. We want the LCD to be mounted fairly close to the glasses, so we will be trying to adapt to using a lens as suggested to make the LCD more readable from the short distance. This will likely require some trial and error to make it work well.

We are mostly managing this risk by ensuring we have plenty of slack time to allow for this trial and error to happen. If all else fails we have our contingency plan of switching from glasses mounted to phone mounted to give us an easier screen for the user to read from.

As mentioned above, we will be adding a lens to our design to improve the readability of our LCD. The lenses only cost around $15 for 2 so this is well within our budget to add.

No changes have occurred to our schedule.

Below we have pictures of our signal generation on the oscilloscope.

DJ’s Status Report for 3/14

I spent the beginning of this week working together with David to get our signal generation working on the Pico. We spent a lot of time working with the oscilloscope in Techspark to ensure that both are carrier frequency and our PWM were being generated accurately enough for our protocol. We did some initial testing around with the IR receiver hooked up to the oscilloscope to get an idea of the distance and directivity, both of which are looking good for our purposes.

Later in the week I started working on our code for parsing the output of the V2T. The V2T software simply returns a string of the ongoing message, so we need a way to accurately and efficiently parse that string to determine the newly added words that we need to transmit. We also need to do this in a way that is resilient to the V2T model updating its prediction for previous words.

I am right on schedule, if not a little bit ahead of schedule.

Next week I will be continuing to work on our V2T processing and getting it to a point where the V2T and the Pi can be connected to the Pico side of our system. I also plan to start working a little bit with David on our signal decoding software as well.

Davids Status Report for 3/14

What did you personally accomplish this week on the project?

I wrote more code for pico pwm encoding. Along with DJ, we worked with multimeter for a decent amount in tech spark to ensure it was working correctly. I then continued and wrote more code for specific bit sending, ie sending a sync and an entire packet following. I then spent a few hours in lab working through this and debugging to the point that it worked! I then spent some time drafting decoding code. Some photos of our lab progress should be in the team report.

Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?

I am ahead of schedule now. Getting the encoding software working is a main part of our project and a part that I thought could be subject to delays. Now that I am through this I (we) are in a very good state to get our entire integration complete in coming weeks.

What deliverables do you hope to complete in the next week?

Next week my main focus is debugging the decoding software. This will complete our IR  and encoding submodules which means all we have left in theory is integrating with LCD and V2T submodule. Hopefully the decoding goes relatively smoothly as well in which case we are in a very good spot!

Todd’s Status Report for 3/7

Personal Accomplishments this Week

Work on the physical project was lighter this week due to class time being taken up by design presentations and ad the design report being due on Friday. For the design report I mainly worked in assisting both David and DJ on ideas and supporting them when they needed information about the intended hardware and such, and I also made a couple of the diagrams sin the report, and citing sources. We were also waiting on our RPi PICO boards to come in so some of my work was also limited in that regard, but we should be in a good place to hit the ground running when we get back.

On Schedule?

We are still on schedule with our updated plan and also personally on schedule with my individual tasks as well.  I have also started thinking and researching more on how we are going to integrate all of the parts of our project for the final product, which I will continue to work on next week.

Next Week Deliverables

I am planning on helping DJ get the PICOs up and running and I would like to get some information transmission through IR working. As I mentioned earlier, I also want to get a draft of how we are going to integrate all of the electronics into an ergonomic and usable form factor so that we don’t lose the user experience in the search for functionality.

Team Status Report for 3/7

What are the most significant risks that could jeopardize the success of the project? How are these risks being managed? What contingency plans are ready? At this point, the one main risk for our project success is the ability to get the NEC IR modulation scheme working on a pico. While seemingly (easily) possible as there are many videos and documents about how to do similar things, this is a very important part of our system. This risk is being managed by starting early and giving ourselves a lot of time in schedule + slack to work through any issues. Now that we have picked up the pico, we will be mainly focused on this portion of our system for a while. We have already drafted out code which seems plausible and are hoping that HW timed pwm + SW timed pulsing is accurate enough for our needs (which I think it probably is fingers crossed). If this absolutely cannot work, we can switch our transmission scheme timing requirements and make it a little slower (not terribly slow, just a bit slower) to get within the accuracy range for SW timing. This would hopefully make our system work.

Were any changes made to the existing design of the system (requirements, block diagram, system spec, etc)? Why was this change necessary, what costs does the change incur, and how will these costs be mitigated going forward? No changes were made to the design of the system this week. We are still working through implementing our change from last week (switching GPIO to a pico instead of directly on a Pi5). 

Provide an updated schedule if changes have occurred. No schedule updated. One might come in a week if we are having large pico difficulties.

A was written by David, B was written by Todd and C was written by DJ.

Part A: … with consideration of global factors. Global factors are world-wide contexts and factors, rather than only local ones. They do not necessarily represent geographic concerns. Global factors do not need to concern every single person in the entire world. Rather, these factors affect people outside of Pittsburgh, or those who are not in an academic environment, or those who are not technologically savvy, etc.

Deafness and hard of hearing are not disabilities constrained to the United States. In many regions of the world, these conditions can have even greater social and economic impact due to limited access to first world healthcare, disability services, hearing aids, or cochlear implants. In lower income countries, assistive technologies are often entirely unavailable, and public accommodations such as interpreters may not exist. As a result, accessibility needs for deaf and hard of hearing individuals vary significantly across countries and economic conditions. By enabling fully on-device speech-to-text processing without requiring cloud services / Wi-Fi connectivity, eyeR supports communication in environments with limited technological infrastructure. This makes the system relevant not only in technologically advanced regions, but also in underdeveloped, rural, or low infrastructure communities worldwide. Our design also reduces dependence on stable internet access, ensuring reliable functionality in areas where connectivity may be unstable or entirely unavailable.

In addition, technological literacy varies significantly across the world. Not all users are familiar with setting up software, working with the cloud, or troubleshooting devices. EyeR addresses this by prioritizing simplicity; the system is designed to function with no real setup, allowing users to simply wear the glasses and begin speaking (plug and play). The discreet glasses-based form factor integrates naturally into everyday life, making it socially acceptable across the globe. Furthermore, by relying on low-cost, widely available hardware and open-source software, the design is cheap and can be more easily distributed in reduced economic regions. Our approach ensures that eyeR is not limited to highly developed American markets, but instead remains accessible and scalable across a global population.

Part B: … with consideration of cultural factors. Cultural factors encompass the set of beliefs, moral values, traditions, language, and laws (or rules of behavior) held in common by a nation, a community, or other defined group of people.

Our device addresses the needs of the hearing impaired and hard of hearing community, and offers an alternative to bridge the gap between those who can hear and those who cannot. Sign language is not universal to the entire HoH community across many cultures and especially not within the hearing population. EyeR facilitates real-time, seamless communication that is accessible to all communities without severely altering modes of natural language. EyeR also facilitates eye contact and face-to-face communication, which is a universal mode of connection across cultures. In addition, with a transparent screen, natural conversational pacing and social interaction can be maintained without the use of a phone or other device.

EyeR is also designed with privacy in mind. Privacy is very important to many people and, across cultures and society at large, 1 on 1 conversations are meant to be private. By offering a mode of communication (infrared) that cannot be easily intercepted, eyeR allows privacy to be maintained. All data processing of the conversation, like the voice to text model, is done locally, with nothing stored in the cloud or put on the internet, keeping conversations private and secure between those that are communicating. This upholds the ethical standard of keeping conversations private when desired, and making sure that users feel safe to discuss personal and private matters without fear. This device offers a way for those in the HoH community to have an accessible and functional method that strengthens natural human interaction without totally replacing it.

Part C: … with consideration of environmental factors. Environmental factors are concerned with the environment as it relates to living organisms and natural resources.

Our first consideration with regards to environmental factors is that we don’t want our solution to require a lot of resources to make, or to be wasteful. This aligns well with our use case requirement that our solution should be lightweight and low profile as well as low cost. With our use of IR light for our serial communication, we’re able to keep the amount of hardware we need for our solution fairly minimal. In addition to this, we’re using industry standard LEDs and sensors similar to those used in TV remotes, so they have good longevity. Finally, with our use of a rechargeable battery pack to power our whole system, we will have no waste in terms of single use items like batteries.

Secondly, an important aspect of our solution is that it is meant for use outdoors for all types of social outings. Because of this, we need to consider how our technology could impact wildlife. The only disturbances that our technology would introduce to the environment is based on our IR serial communication. As we don’t anticipate our glasses being used in super close proximity to animals ( < 10 ft) the safety concerns associated with our users should be a nonissue in terms of wildlife. The main concern with the impact to wildlife would be the IR light itself, since some animals are able to see IR light we have to consider if our product would present any negative visual stimuli. In terms of the intensity of our IR light, it is less powerful than the ambient IR light due to the sunlight. We believe that it is unlikely that our 38kHz carrier frequency would make it discernable to animals in anyway, but either way our LED only has a range of 10-20 ft straight ahead and isn’t powered on unless the user is actively in conversation with another user, meaning there is for all purposes no impact on the wildlife.


David’s Status Report for 3/7

What did you personally accomplish this week on the project?

I spent probably 15 hours on design report so evidently some of my project work is a little lighter this week. However, even though we got the pico later than expected I was able to begin some research and code for that. I drafted out some code for PWM and for duty cycle modulation. I made 2 versions of this, one for SW timed and one for HW timed. Because the pico wasn’t picked up until after I was already on break, I was unable to test yet but hopefully not a difficult process.

Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?

I am nicely on schedule. Much of our work is now dependent on getting the pico code to work on board. Hopefully this isn’t a terrible process because once its done, then we can shift over to LCD and receiver decoding subsystem.

What deliverables do you hope to complete in the next week?

Next week my main focus is putting and debugging the pico code on the pico. After that, me DJ and Todd will likely spend an afternoon (or two) just working through bugs to get the light modulation working strongly. Again, hoping no huge issues as this is a large part of the project.

DJ Status Report 3/7

Naturally, a lot of my time this week was spent working on the design report. Most of my focus was on the system implementation section, as well as some time spent on design requirements, architecture, and tradeoffs.

In terms of additional progress on getting physical things working for our project, I was fairly limited this week due to waiting on the Raspberry Pi Picos to arrive for most of the week. I was able to due a lot of research on how to implement our signal generation on the Pico and its GPIO capabilities. There appears to be options for hardware timed PWM and software timed overall pulses (calls to sleep on the Pico are much more accurate making this a viable option) as well as hardware timing for our entire signal generation. This puts me in a good place for next week to get our signal generation working on the Pico.

I am still nicely on schedule for the project. As mentioned last week we had to shuffle some things around on the schedule, but I am on track with our new order of implementation.

Next week I will definitely be able to get the signal generation on the Pico working to the point where we can view sent words on the oscilloscope. Hopefully I’ll have time to work a bit ahead as well and help out with the signal detection and decoding side of the program as well.

Todd’s Status Report for 2/21

Personal Accomplishments this Week

I mainly worked alongside DJ and David testing and validating some of the hardware for our device. Earlier in the week, we worked on trying to understand the signal coming out the RPi GPIO for the IR LED with an oscilloscope. This led us to pivot to using the PICOs since we had trouble getting a signal in the first place. Then later in the week, I worked with DJ on validating the voice to text model on the RPi with our MEMS mic, which worked out really well, and that part of the project is ready to move forward.

On Schedule?

I am on schedule alongside the rest of the team for this week, and we are still staying slightly ahead of where we want to be which is good.

Next Week Deliverables

I am going to keep supporting my other groupmates with getting the hardware tested and functional, and will be ready to work with the PICO boards once they come in. I will also be helping the David and DJ on completing the design report due at the end of the week.