Month: December 2024

Posted on

Alena’s Status Report for 12/7

This week I was able to figure out how to use Bluetooth classic with the pico w.  I was able to configure the pico as an A2DP source and connect it to the Raspberry Pi which acted as the sink, and was able to transmit audio from one to the other. However, I am having trouble integrating the microphone code with the bluetooth code, since the PCM to SBC encoding is messing up somewhere along the line. The output on the Raspberry Pi is very noisy, but you can vaguely make out what the voice is saying, it just sounds very robotic. I think there’s some more filtering that I’m going to have to do.

I also researched a GPT integration with the voice command algorithm and got it to work in conjunction with my existing algorithm, so I am looking in to making the voice command output more “human” instead of just hardcoding responses based on the recipe steps. I also was able to integrate the pyttsx3 with Raspberry Pi so that the outputs can be heard on a bluetooth speaker/headset. Overall, I made a lot of progress this week, and there are multiple options we can consider for the final project. Unfortunately the wireless mic needs to be tuned a lot more, so depending on how much time we have for that, the final product may need to have a wired mic.

Posted on

Team Status Report for 12/7

What are the most significant risks that could jeopardize the success of theproject? How are these risks being managed? What contingency plans are ready?

  • Currently, the most significant risk is still going wireless.
  • The contingency plan is to keep the wired connection between this subsystem, while improving upon the subsystem integration (sending recipe steps to the LCD, etc.)

Were any changes made to the existing design of the system (requirements,block diagram, system spec, etc)? Why was this change necessary, what costsdoes the change incur, and how will these costs be mitigated going forward?

  • We may potentially need to stay wired in terms of audio input, which would change the CAD a bit to account for that.
  • Requirements have stayed the same otherwise, as keeping the subsystem wired reduces latency and improves accuracy greatly.
Posted on

Michelle’s Status Report for 12/7

What did you personally accomplish this week on the project? Give files or photos that demonstrate your progress. Prove to the reader that you put sufficient effort into the project over the course of the week (12+ hours).

  • This week, Alena and I worked on finalizing our list of hardware for the final design.
  • This involved selecting a new speaker for the right-hand side of the glasses.
  • Using this information, I worked on designing the final casing for both sides (with battery holders)

Is your progress on schedule or behind. If you are behind, what actions will betaken to catch up to the project schedule?

  • I am currently waiting for the final verdict on some of the hardware related to the microphone + speaker subsystem, and once I have the information, I will be able to design the finalized version of the glasses.

What deliverables do you hope to complete in the next week?

  • I will have all hardware completed and integrated.
Posted on

Kayla’s Status Report for 12/7

This week, I mainly worked on testing for the web app. This includes making api requests for the search, new recipe, and recipe recommendation features and checking the functionality of these apis as well as the latency of these calls as well as providing various ingredient inputs and validating the recipe recommendations that were generated. I also researched how Alena and I would implement the feature where a user selects a recipe within the web app and this recipe gets sent to the glasses and audio output. The approach I am currently looking to use is to set up a server on Alena’s end, running with Flask where this server can serve HTTP requests which will come from the web app when a recipe is selected in the web app. We have not yet implemented this, so that is behind our schedule, but I was able to do the web app testing instead this week so we will work on this feature next week instead of doing testing.

Posted on

Alena’s Status Report for 11/30

The past two weeks I ironed out some issues that showed up with the wireless communication between the pico and the Raspberry Pi. I added functionality to the voice command algorithm such that now recipe steps are sent to the glasses when needed, and so the steps are displayed when the user is in the cooking state. I looked into making the pico wireless using BLE instead of TCP, but my code is still buggy and it doesn’t work as intended, so I will spend the next week figuring that out. In terms of integration, Michelle and I decided to use two Picos instead of 1 for the glasses to balance out, so that makes the LCD and mic/speaker code integration a lot easier. I also have been working on implementing and bringing up the speaker to take in output data from the Pi through TCP. There are a couple small issues with the Pyttsx3 library on the Raspberry Pi, but they are mainly just warnings that aren’t affecting the main functionality yet (hopefully never…). I also did the latency and accuracy testing for the voice command algorithm, including how long it takes to take in input, process it, and give the output. 

Progress is on track so far, I am hoping to get the speaker working by the end of the week and also figuring out what is causing issues for the bluetooth mic integration with the pico. I want to have some more testing done on the “integration” side as well, so for example, how long it takes to have stuff show up on the LCD after being sent from the Pi, and also any potential latency that is introduced as a result of the BLE implementation.

Posted on

Team Status Report for 11/30

What are the most significant risks that could jeopardize the success of theproject? How are these risks being managed? What contingency plans are ready?

  • Currently, the most significant risk is going wireless.
  • Currently, all of our subsystems are integrated, and the connection between the microphone, voice commands, and web app is wired between a Raspberry Pi and a Raspberry Pi Pico W.
  • However, our goal is to go entirely wireless, which is likely the most difficult part so far.
  • The contingency plan is to keep the wired connection between this subsystem.

Were any changes made to the existing design of the system (requirements,block diagram, system spec, etc)? Why was this change necessary, what costsdoes the change incur, and how will these costs be mitigated going forward?

  • A recent change to the existing design is the introduction of a second Raspberry Pi Pico W. To enhance comfort while wearing the glasses, we found that users preferred balanced designs. So, we are moving our microphone and speaker to a separate Pico W on the opposite side of the glasses.
  • There are no significant risks in this choice, however, it is likely that this change will add some weight to the final design. We will carefully consider ergonomics in upcoming user tests.
Posted on

Michelle’s Status Report for 11/30

What did you personally accomplish this week on the project? Give files or photos that demonstrate your progress. Prove to the reader that you put sufficient effort into the project over the course of the week (12+ hours).

  • This week, based on the information I gathered from last week regarding the printed design, I made modifications as needed.
  • This involved adjusting the position of the LCD and designing the other side of the glasses.
  • Based on user feedback, we determined that to balance out the glasses, it would be better to separate our components onto two sides. Hence, the microphone and speaker will be on the right side of the user’s face, and the LCD will be on the left side of the user’s face.
  • With these iterations, this was the model:
  • Because I do not have access to a 3D printer, I was not able to print this final model.
  • With these adjustments, the glasses should be fully battery-powered and have all components.

Is your progress on schedule or behind. If you are behind, what actions will betaken to catch up to the project schedule?

  • I am mostly on-time. There are a few things I need to prototype once I have all tools accessible again:
  • I will need to find out where the short is occurring on my Raspberry Pi Pico W when the Power SHIM is directly attached (plates are contacting). This will likely require some modifications in the physical design.
  • I will also need to implement the sliding mechanism for the LCD which has proven difficult due to space limitations.

What deliverables do you hope to complete in the next week?

  • I will have all hardware completed and integrated.

     

    As you’ve designed, implemented and debugged your project, what new tools or new knowledge did you find it necessary to learn to be able to accomplish these tasks? What learning strategies did you use to acquire this new knowledge?

    • There were a few things I had to learn throughout this project.
    • Going into capstone, my weakest area was probably circuits, and because I chose to focus on the physical design of the glasses, I had to familiarize myself with some hardware basics and some embedded systems.
    • I mostly relied on online guides, including datasheets, official documentation, and even some YouTube tutorials when it came to schematics.
    • I also had to dive a bit deeper into the LCD implementation (for the SSD1306), which relies on being familiar with the I2C/SPI protocol.
    • My general learning strategy was to do research first, collect resources + information, then attempt implementation, which seemed to work.