Carnegie Mellon ECE Capstone, Spring 2026 – Todd Achuff, DJ Lloyd, David Zoota
This week I worked with David to finish assembling the glasses. This involved mostly getting our battery packs set up and getting the microphones fully attached.
Later in the week I also worked with the rest of the group to run some final tests of our system and constructs some graphs for use in our poster and report.
The project is pretty much finished at this point so we are very on schedule and there isn’t really anything left to complete next week.
This week my main focus was mounting our system onto our printed glasses. I worked with Todd this week to move one set of hardware onto our printed glasses and testing its functionality. This involved attaching the LED, Receiver, LCD, and lens to the glasses as well as attaching and organizing all the respective wiring. We also attached and hooked up the Pico and the two arms of the glasses to the frame. Finally we confirmed that the system does work by testing it with our other half still on the breadboard, and we confirmed that our lens does allow for good readability of the screen.
Because of the limited flexibility of our glasses arms, we also decided to shift to a single handle attached to the frame for our second pair of glasses to allow for easier compatibility with multiple users at the final demo.
We are still on schedule.
Next week we intend to finish mounting the second half of the system onto the other pair of glasses, and we plan on continuing our tests for our use case and other requirements.
For this project I had to learn a lot of things related to working with the Raspberry Pi and Pico. This includes learning to interface with the GPIO pins and deciding how to transmit data over USB between the two components. This involved reading through a lot of documentation for different libraries to determine if they would meet our speed and accuracy requirements, as well as looking through forum posts to see other people’s feedbacks on libraries performances.
I also had to learn about serial communication and transmission protocols when I was designing that aspect of our system. To do this, I read through several sites and articles talking about IR communication in general and specific protocols like NEC to pick the best aspects to incorporate into our protocol.
The biggest risk to the project is still all of the logistics that come with mounting our system to the glasses. We need to ensure that everything is mounted securely while still keeping a minimal footprint with all of our wiring, and ensuring that our LCD will be readable. This risk is being managed mainly with our being ahead of schedule leaving us plenty of time to work out any difficulties that come with mounting. We are also continuing to keep in mind the suggestion of pivoting to a handheld system as a last resort.
No changes have been made to the design or our schedule.
Both our verification and validation will be described following, because our subsystems basically fully integrated and so all of our forms of testing will be collaborative within in the team in some way.
In terms of verification, the main metrics we need to test are end-to-end accuracy and latency, as well as connection angle and distance. Based on some initial testing, we don’t believe angle and distance will have a significant impact on accuracy and latency, it appears to be an all or nothing connection, so our tests for those will be separate.
For our accuracy and latency, we will be using around 5-10 spoken and computer generated passages of vary accents and speeds to measure both metrics. Each passage will be repeated around 3 times with averages of different wpm and overall averages taken for our reporting. During this, we will also perform stress tests involving disrupting the line of sight to ensure full functionality. These reported metrics will map directly back to the matching use case and design requirements.
For our angle and distance, we will be transmitting simpler passages compared to above to simply check for a connection. We will do this ranging from 3-15 ft and a 40 degree range and constructing a graph of where connection is possible. This will allow us to provide a visual representation of where our system is usable.
For validation we will be utilizing our friends and hopefully volunteers who are hard of hearing or deaf to provide feedback on the usability of our system. We will be creating a survey that captures our more qualitative requirements, such as how it feels to wear and how obstructive it is.
This week my focus was on bug fixing in preparation for the demo. I continued working on a bug that we had found last week involving hanging when two people are talking at once. I was eventually able to fix the bug and get the bidirectional system ready for the demo.
I spent the rest of the week brainstorming with the rest of the team about logistics for the mounting.
I am ahead of schedule, and will have plenty of time for testing in the next month.
Next week I will work with the rest of the team on mounting everything onto the glasses and beginning some stress testing of the mounted system.
This week I worked on the code for making our system bidirectional. This started with adding the call and response for the acknowledgement pulses, and then I expanded to add both sending and receiving capabilities to each side. Throughout testing this week, we’ve gotten the system to a point where it works well as long as only one person is talking at a time, but there are still some bugs when both users overlap talking. This is likely to do with how we are deciding who gets priority to send initially. The code currently randomly tries sending and receiving a letter initially, I chose this randomized priority because I believe it is the best way to ensure compatibility with all users in the hypothetical case that many people with this product exist. However, we will need to do more debugging of this area of the code to ensure that it handles overlapping talking correctly.
The project is ahead of schedule.
This gives us plenty of time next week to continue debugging the system. I’d like to also work more on our full system correctness testing so that within the next week or two we can start working on our use case and design requirement testing.
I started this week by doing some final preparation for the signal generation side of our system. I did some debugging on the Pico signal code as well as adding code to transmit letters from the Pi to the Pico over USB. We currently have the Pi sending letters from a queue in parallel to reduce the latency from the Vosk computation and output parsing. The Pi also sends one letter at a time, and waits for a response from the Pico before sending the next letter to avoid losing data if the Pico isn’t accepting more input.
In the middle of the week our project was ready to test the full system integration for one way communication. I worked with the rest of the group to get all of our subsystem integrated and perform some cursory tests to ensure the systems were integrated correctly.
Finally, I started the planning / making flow charts for implementing the two way communication.
We are still slightly ahead of schedule. We are in a very good place for our one way system to be presentable at the interim demo.
Next week I’ll finalize the plan for our two-way implementation and begin writing the code for that. I also want to spend some time experimenting with the Pico’s PIO for use with our signal detection which would allow us to parallelize the signal detection and would possibly simplify as well as improve the performance of the high end integration of the two way communication
We believe that at this point there are no major risks to the project, the different parts on the project are coming together nicely. The most significant risk at this point is ensuring that our LCD will be readable from a reasonable distance, as well as finalizing how many words will be displayed on it at a time. We want the LCD to be mounted fairly close to the glasses, so we will be trying to adapt to using a lens as suggested to make the LCD more readable from the short distance. This will likely require some trial and error to make it work well.
We are mostly managing this risk by ensuring we have plenty of slack time to allow for this trial and error to happen. If all else fails we have our contingency plan of switching from glasses mounted to phone mounted to give us an easier screen for the user to read from.
As mentioned above, we will be adding a lens to our design to improve the readability of our LCD. The lenses only cost around $15 for 2 so this is well within our budget to add.
No changes have occurred to our schedule.
Below we have pictures of our signal generation on the oscilloscope.
I spent the beginning of this week working together with David to get our signal generation working on the Pico. We spent a lot of time working with the oscilloscope in Techspark to ensure that both are carrier frequency and our PWM were being generated accurately enough for our protocol. We did some initial testing around with the IR receiver hooked up to the oscilloscope to get an idea of the distance and directivity, both of which are looking good for our purposes.
Later in the week I started working on our code for parsing the output of the V2T. The V2T software simply returns a string of the ongoing message, so we need a way to accurately and efficiently parse that string to determine the newly added words that we need to transmit. We also need to do this in a way that is resilient to the V2T model updating its prediction for previous words.
I am right on schedule, if not a little bit ahead of schedule.
Next week I will be continuing to work on our V2T processing and getting it to a point where the V2T and the Pi can be connected to the Pico side of our system. I also plan to start working a little bit with David on our signal decoding software as well.
Naturally, a lot of my time this week was spent working on the design report. Most of my focus was on the system implementation section, as well as some time spent on design requirements, architecture, and tradeoffs.
In terms of additional progress on getting physical things working for our project, I was fairly limited this week due to waiting on the Raspberry Pi Picos to arrive for most of the week. I was able to due a lot of research on how to implement our signal generation on the Pico and its GPIO capabilities. There appears to be options for hardware timed PWM and software timed overall pulses (calls to sleep on the Pico are much more accurate making this a viable option) as well as hardware timing for our entire signal generation. This puts me in a good place for next week to get our signal generation working on the Pico.
I am still nicely on schedule for the project. As mentioned last week we had to shuffle some things around on the schedule, but I am on track with our new order of implementation.
Next week I will definitely be able to get the signal generation on the Pico working to the point where we can view sent words on the oscilloscope. Hopefully I’ll have time to work a bit ahead as well and help out with the signal detection and decoding side of the program as well.