Pareekshith’s Status Report for 4/18

This week, I mostly worked on the opera portion of our collaboration with the School of Music and was debugging issues with the tempo of the pieces. After several attempts, I figured out the problem and was able to obtain the correct tempo for the piece that Dr. Dueck will be playing for our demo. There’s one more piece that I need to work on that is more difficult because of certain changes in tempo throughout the piece, but I’ll try to get that to work using specific cuing mechanisms. On the other hand, I am also working on testing the speech portion and using the WER metric to see the accuracy of the ASR. I don’t need the WER of the entire dialogue by an actor, simply the trigger phrase that’s used to pivot between actors. This shouldn’t take too long to do, and I should have it completed in about a day or so. This upcoming week, I plan to complete the validation and testing for the speech side, set up the cues in Accomplice, and help my team with any final debugging that needs to be done. While working on this project, I learned a lot about speech technology, fuzzy matching, and music technology. Interestingly, I learned a majority of the first two from my Speech Technology for Conversational AI course that I’m taking this semester, and that’s where I learned about WER, Vosk, and the other components of the speech side. On the other hand, learning about Accomplice, how it works, and how to use it needed me to get on a call with Professor Dannenberg, who was extremely kind and helpful. I was able to get most of the information I needed through our chat, and he’s been nice enough to be available for any questions and concerns that I might have.