Jade’s Status Update for Mar 28
On Monday I cleaned up my audio input/output code in order to work with Ashika on integration. I turned my code into easy to call modules, documented it, and documented all the installs needed to run it. I also started writing code that will setup sockets to allow to separate computers to communicate with each other (provided they are connected to the same network). This code is preliminary code for how the raspberry pi and laptop will communicate with one another. I got the socket code working between two terminals on the same device and still need to try it on the pi and a laptop.
On Wednesday I worked with Ashika to integrate our two parts of the project so that they would run on one laptop. We spent time combining our code and eventually found ways for her ML storytelling program to call audio functions in my code. We tested it, and it seemed to work for basic words as well as for timeouts.
On Friday I worked on getting the audio input/output code running on the pi. I ran into some issues where the text-to-speech generated mp3 files with a sample rate of 24 kHz, however the pi would only play out audio files with sample rates of 44.1 kHz. In addition, the audio played back from file sounds choppy, probably due to a chunk parameter that I had to feed into the sound-playing function. I also received the conference microphone and started testing that out.
Currently, the pi can read the story to the user, but is struggling to decode user inputs correctly. The program performs better when I use the microphone on my headphones over the conference table microphone. I’m hoping to work on adjusting the microphone sensitivity levels within the speech recognition program to hopefully improve the speech recognition rates.
My schedule is a bit behind, I was expecting to get the audio input/output code working on the pi, but, I keep getting stuck on bits and pieces of getting the code working on the pi. Granted, the audio input and audio output code do work fine separately, but when the system is run together the speech recognition code seems to have issues picking up words. To fix this issue I will focus time at the beginning of the next week testing out the microphone systems and tuning microphone sensitivity levels.
In the next week I want to get the socket code running which will allow the laptop and pi to communicate. I will be tabling phase vocoding becuase it is less important for developing a working MVP.