Category: DJ’s Status Reports

This week I worked with David to finish assembling the glasses. This involved mostly getting our battery packs set up and getting the microphones fully attached.
Later in the week I also worked with the rest of the group to run some final tests of our system and constructs some graphs for use in our poster and report.

The project is pretty much finished at this point so we are very on schedule and there isn’t really anything left to complete next week.

This week my main focus was mounting our system onto our printed glasses. I worked with Todd this week to move one set of hardware onto our printed glasses and testing its functionality. This involved attaching the LED, Receiver, LCD, and lens to the glasses as well as attaching and organizing all the respective wiring. We also attached and hooked up the Pico and the two arms of the glasses to the frame. Finally we confirmed that the system does work by testing it with our other half still on the breadboard, and we confirmed that our lens does allow for good readability of the screen.

Because of the limited flexibility of our glasses arms, we also decided to shift to a single handle attached to the frame for our second pair of glasses to allow for easier compatibility with multiple users at the final demo.

We are still on schedule.

Next week we intend to finish mounting the second half of the system onto the other pair of glasses, and we plan on continuing our tests for our use case and other requirements.

For this project I had to learn a lot of things related to working with the Raspberry Pi and Pico. This includes learning to interface with the GPIO pins and deciding how to transmit data over USB between the two components. This involved reading through a lot of documentation for different libraries to determine if they would meet our speed and accuracy requirements, as well as looking through forum posts to see other people’s feedbacks on libraries performances.

I also had to learn about serial communication and transmission protocols when I was designing that aspect of our system. To do this, I read through several sites and articles talking about IR communication in general and specific protocols like NEC to pick the best aspects to incorporate into our protocol.

This week my focus was on bug fixing in preparation for the demo. I continued working on a bug that we had found last week involving hanging when two people are talking at once. I was eventually able to fix the bug and get the bidirectional system ready for the demo.
I spent the rest of the week brainstorming with the rest of the team about logistics for the mounting.

I am ahead of schedule, and will have plenty of time for testing in the next month.

Next week I will work with the rest of the team on mounting everything onto the glasses and beginning some stress testing of the mounted system.

This week I worked on the code for making our system bidirectional. This started with adding the call and response for the acknowledgement pulses, and then I expanded to add both sending and receiving capabilities to each side. Throughout testing this week, we’ve gotten the system to a point where it works well as long as only one person is talking at a time, but there are still some bugs when both users overlap talking. This is likely to do with how we are deciding who gets priority to send initially. The code currently randomly tries sending and receiving a letter initially, I chose this randomized priority because I believe it is the best way to ensure compatibility with all users in the hypothetical case that many people with this product exist. However, we will need to do more debugging of this area of the code to ensure that it handles overlapping talking correctly.

The project is ahead of schedule.

This gives us plenty of time next week to continue debugging the system. I’d like to also work more on our full system correctness testing so that within the next week or two we can start working on our use case and design requirement testing.

I started this week by doing some final preparation for the signal generation side of our system. I did some debugging on the Pico signal code as well as adding code to transmit letters from the Pi to the Pico over USB. We currently have the Pi sending letters from a queue in parallel to reduce the latency from the Vosk computation and output parsing. The Pi also sends one letter at a time, and waits for a response from the Pico before sending the next letter to avoid losing data if the Pico isn’t accepting more input.

In the middle of the week our project was ready to test the full system integration for one way communication. I worked with the rest of the group to get all of our subsystem integrated and perform some cursory tests to ensure the systems were integrated correctly.

Finally, I started the planning / making flow charts for implementing the two way communication.

We are still slightly ahead of schedule. We are in a very good place for our one way system to be presentable at the interim demo.

Next week I’ll finalize the plan for our two-way implementation and begin writing the code for that. I also want to spend some time experimenting with the Pico’s PIO for use with our signal detection which would allow us to parallelize the signal detection and would possibly simplify as well as improve the performance of the high end integration of the two way communication

I spent the beginning of this week working together with David to get our signal generation working on the Pico. We spent a lot of time working with the oscilloscope in Techspark to ensure that both are carrier frequency and our PWM were being generated accurately enough for our protocol. We did some initial testing around with the IR receiver hooked up to the oscilloscope to get an idea of the distance and directivity, both of which are looking good for our purposes.

Later in the week I started working on our code for parsing the output of the V2T. The V2T software simply returns a string of the ongoing message, so we need a way to accurately and efficiently parse that string to determine the newly added words that we need to transmit. We also need to do this in a way that is resilient to the V2T model updating its prediction for previous words.

I am right on schedule, if not a little bit ahead of schedule.

Next week I will be continuing to work on our V2T processing and getting it to a point where the V2T and the Pi can be connected to the Pico side of our system. I also plan to start working a little bit with David on our signal decoding software as well.

Naturally, a lot of my time this week was spent working on the design report. Most of my focus was on the system implementation section, as well as some time spent on design requirements, architecture, and tradeoffs.

In terms of additional progress on getting physical things working for our project, I was fairly limited this week due to waiting on the Raspberry Pi Picos to arrive for most of the week. I was able to due a lot of research on how to implement our signal generation on the Pico and its GPIO capabilities. There appears to be options for hardware timed PWM and software timed overall pulses (calls to sleep on the Pico are much more accurate making this a viable option) as well as hardware timing for our entire signal generation. This puts me in a good place for next week to get our signal generation working on the Pico.

I am still nicely on schedule for the project. As mentioned last week we had to shuffle some things around on the schedule, but I am on track with our new order of implementation.

Next week I will definitely be able to get the signal generation on the Pico working to the point where we can view sent words on the oscilloscope. Hopefully I’ll have time to work a bit ahead as well and help out with the signal detection and decoding side of the program as well.

At the beginning of this week I spent a bit of time doing some final preparation for my design presentation. Also towards the start of the week I did some work with David with signal generation on the Pi. Because of the change in how the GPIO pins work on the Pi 5, none of the libraries seemed to have the functionality that we needed (hardware timed and able to create a wave of multiple pulses at once) to achieve the accuracy we want, so we were testing a method using a Linux kernel driver for IR that had mixed reviews on the Pi forums. This didn’t work so we are migrating our design to use a Pico to handle the IR hardware interfacing.
At the end of the week I pivoted and got the Vosk speech to text working on the Pi with our breakout mic.

I am still on schedule if not slightly ahead again. The Vosk took less time to integrate with the Pi than I had anticipated, and I’ll be able to get back to signal generation when the Picos arrive next week so the schedule is looking good.

Next week i’ll be spending some time on the design report document. I’ll also be working (along with David) on the signal generation and decoding code on the Pico as well as the code to accurately query Vosk.

This week, most of my time was spent working on the design presentation and preparing to present, as I am presenting this presentation. Other than that, I finalized my design research into our IR communication protocol (pictured below) and worked out the encoding of letters and the timing of the protocol to ensure that we meet our use case requirements for speed and latency (200-250WPM and 500ms latency), while making sure that we’ll have enough leeway to handle errors. At the end of the week, I also worked on the code for our Hamming encoding, testing to make sure that it would fix single bit errors and detect 2 bit errors for our custom letter encoding.

I am well on schedule with my portion of the project, slightly ahead as I hadn’t planned on beginning coding until next week.

Next week I believe that a good chunk of my time will be spent working on. the design report. I’d like to also decide on a Raspberry Pi library to use for signal generation and detection through the GPIO. Changes with how the GPIO work from the Pi 4 to 5 have apparently rendered a lot of GPIO libraries incompatible with the Pi 5, so this will require some research to determine a good fit for our project. Hopefully I’ll be able to test some simple signal generation with Todd and the waveform generator at the end of next week.

 

At the beginning of this week I did some initial research into serial communication protocols as well as open source speech to text to determine a rough estimate of the specs needed for our parts. I assisted in making a rough budget sheet and determining vendors for all of our parts. Then I completed the order forms for all of the components needed for our MVP.

Later in the week I experimented with Vosk speech to text, which we’ve decided to use for our project because it provides accurate, low-latency, live transcription. It also offers a variety of models of different sizes.

I also continued my research into serial communication protocols, and IR communication protocols specifically. I’ve determined some rough design ideas, specifically using a 38kHz carrier frequency combined with the ideas underlying the NEC infrared protocol (PWM) and Hamming error correction (likely 10,5) to achieve our goal of low latency, low error transmission that can support 200-250 WPM.

We are on schedule, and having been able to put in the order for all of our needed parts, as well as having the Raspberry Pis arrive already at the end of this week, puts us in a good place to continue well on track next week.

Next week a lot of my attention will be focused on the Design Review slides as well as planning for the Design Report. In terms of deliverables I plan to implement the groundwork for our encoding/signal generation and get the code running & testable on the Raspberry Pi.