Progress
This week I worked with Yuxuan to finish implementing all functionalities of the application. We first connected all the models trained by each teammate together and constructed a complete helper function that allows users to render different pages based on the input verb. Then we designed the VUI for entering a new entry, modifying existing entries, and deleting an entry. The user could record the item they purchased by simply saying the item name and price. The word2vec model would assign a category based on the item name, and the created entry will be saved to the database. As for modifying an entry, the user could give a command with the entry number and a modifying page would be rendered. The user could then provide specific changing requests to a field and save the changes by saying “confirm”. After the functions were implemented, we conducted several groups of tests to figure out potential edge cases for each action.
As for testing, I finished unit testing for the speech recognition model and calculated the accuracy by counting the number of misparsing words. The outcome is about 98.3% accuracy, which is much greater than the expected performance earlier in the design requirement.
Schedule
I am on schedule now.
Next step
I will work with my team to finish all the remaining work for the final demo including the poster, video, and report.
New tools and knowledge
Since I am responsible for the speech recognition part of our project, I first learned how to use the Python libraries we selected including PyAudio, noisereduce, SpeechRecognition, and gTTS to implement the audio-to-text pipeline. To understand the basic functions of these libraries, I read through the documentation and searched for sample usage of the methods.
After receiving the raspberry pi and the monitor, our team spent a lot of time setting up the system and downloading all the required libraries. Since the Linux system on raspberry pi is different from the ordinary Windows or MacOS we use on our laptops, unseen errors may take place at any component of the whole process. For example, we used “sudo apt install” for most of the libraries, but this command did not work when we were trying to download gTTS. We searched on forums and spent quite some time trying different possible solutions. In the end, we succeeded in downloading and running the library in our web application.