This week I primarily worked on researching the mechanisms that we are going to use for text-to-speech/speech-to-text. Python has many established libraries for this purpose, but we have the added nuance that whatever model we use, will have to run directly on the Raspberry PI 4. I was able to find a lightweight, open-source model that was actually developed by CMU researchers called PocketSphinx. This will likely work well for our use case because the model is small enough to run locally on limited hardware. We are currently on schedule and for the upcoming week I plan to finish the python code for the Raspberry PI so we can start utilizing the speech-text on the device.
Team Status Report for 2/8
The most significant risk is that the latency of the whole system is too large. We want it to be <5 second from the person finishing their statement to our device responding. If we cannot meet this requirement, we will have to redesign the data flow or move the model location sure the latency is under control. No changes were made to the existing design of the system. Below are some pictures of our frontend UI that we setup this week that will allow users to customize LLM models (not yet functional). We also finalized our order of parts this week.
Justin Ankrom’s Status Report for 2/8
This week, I worked on getting the frontend UI setup for our project. I setup a Next.js application and worked on the home page. I got the basic UI and feel for the home page going. The home page allows users to pick between a set of our LLMs if they want a easy and convenient solution, or they can host their own LLM. The frontend has no functionality yet besides the UI. A picture is attached. Progress is on schedule for what I want/need to get done. Next week, I hope to implement authentication so we can differentiate between users. Users are important so we can update the correct settings based on the user.
David’s Status Report for 2/8
I looked through the inventory to get the three main components we need: speaker, microphone, and raspberry Pi 4. The inventory had both the raspberry Pi and the microphone which I ordered, but no speaker. I did some research on a speaker that would be able to connect to a raspberry pi 4 and ordered that as well. Our project is on schedule. I hope to be able to connect the speaker, board, and microphone next week to have basic IO.