This week, I made progress on the speech as well as the musical portion of the project. I’ve decided to follow 2 tracks: one which focuses on opera and getting the piano music (as part of the collaboration with the School of Music) and the other which is based on plays and uses Vosk for ASR. For the demo, I’ll be showing the latter track and display the Lavalier microphone connected to the wearable node that’ll indicate the parsing of the speech and the switching of the pivots on a UI. Regarding the other path, we had a call with Professor Dannenberg during our meeting with Dr. Dueck and the School of Music, and he introduced an off-the-shelf solution that relies on MIDI files and software; this solution may be exactly what we need for our project. This solution has the ability to send information and commands through a robust network protocol developed by Professor Dannenberg, so we’d be able to send information to the central Pi and make it send the commands over to the motor controller. I’m currently on schedule with my work; I’ll be meeting with Professor Dannenberg next week to set up his solution locally and make sure it’s all working, and I’ll be further enhancing the speech solution and verifying that it works accurately when there are multiple microphones involved in a close setting. I hope to be able to get both systems to work for our final project presentation, as it’d be cool to demonstrate the different capabilities our system has and the various use cases we support.
Ted’s Status Report for 03/21/2026
This week, I worked on connecting the VESC motor controller to the BLDC motors and power source. I mainly focused on CADing (is that the right acronym?) the 3d driver pulleys, idler pulleys, motor plates, and other miscellaneous things and printed them before our meeting Saturday. Then, I hooked up the motors to the VESC, power source, and RPi and tested them to see if all the connections had been soldered correctly and were functioning. Bearings have been delivered, but I wasn’t able to pick them up before the ECE office closed. I’m hoping to pick them up and finish constructing the rail monday, and then begin testing with our completed UWB sensor/node system. In terms of schedule, we’re developing most of our subsystems at the same time, and once the rail is finished and all parts are received, we should be at a good pace. I need to reprint some of the pulleys for a better fit on the timing belt, and I hope to get that done by next week as well. I want to reduce wear and tear as much as possible. The open source configuration software for the motor controller is a bit finicky to work with, and I’m trying to get experienced with it as fast as possible.
Pareekshith’s Status Report for 3/21
This week, I did research into open-source libraries that we could use to do real-time music tracking. The one library that I found that works really well with our use case was Matchmaker, which was released last year and is able to output the current position in the script based on the sheet music and the audio data. With this, we might have to pivot from our Lavalier microphones to a Samson microphone that isn’t omnidirectional and can focus properly on the piano music and not the surrounding operatic dialogue. The only concern I have with this is that it works specifically for piano; other instruments and orchestras would most likely mess up the alignment. Because of this, I want to check with Dr. Dueck if using this library is feasible. We might need to slightly pivot with this, especially because I am unable to find any other library that works on orchestral music. During our group’s work session this Saturday, I also helped with the taping of the anchors and the battery packs. I’m currently on schedule, and I plan to experiment with the Samson microphone that one of my partners might have. There are some technical details that need to be honed out, such as the conversion of the sheet music into specific MusicXML files and the exact audio input format. These will be covered and explored next week, and I plan on doing on-device testing and integration with the Pis.
Ted’s Status Report for 03/14/2026
This week, we met back up with Dr. Dueck and received some feedback on the design of our wearable nodes, showing them a rough prototype to give them an estimate of what type of device they’d have on their person. I’ve also received some of the parts and have been working on interfacing the BLDC motors with the VESC motor controller. The VESC motor controller 3-phase wires come preequipped with MR-30 connectors, but the BLDC motors just come with rough wires. I’m thinking of just chopping off the connectors and simply just soldering the phase wires directly to the motor controller instead of attempting to install any MR-30 Connectors. One of the micro-usb connections on the VESC is a little loose but it shouldn’t be a source of any issues unless under heavy vibration. I’ve also finished designing the GT-2 timing pulley that will be used with our motor and timing belt, and plan to print it soon. I’ll be printing multiple copies incase they shatter or break during testing so we won’t have to wait an extended period of time to replace it. Schedule is still on track, and by next week once ball bearings have been delivered I hope to begin construction of the rail itself.
Pareekshith’s Status Report for 3/14
This past week, I received the Lavalier microphones from the ECE Receiving office and thus, got a better idea of how to integrate them into our system. When we met with Dr. Dueck and her students, we discussed placements of the wearable nodes and the mics under the costumes, and we will be in contact figuring out how exactly to hide them discreetly. The good news is that the microphones are 6 meters long, so if there is a safe spot on an actor, the microphone can easily clip on and work properly. I also brought up the dilemma that we have with Vox, namely that it won’t work as well as expected with an operatic voice. Because of this, we are considering switching gears and using the sheet music as the script and then indicating where the cues should happen. Although our original plan was to use speech technology, given the collaboration with the School of Music, it makes sense to go with this approach; if we have time near the end, we might also try to incorporate speech technology into our system for drama productions, which typically have more conversational speech than operas and musicals. I’m still on track with my schedule, although I might need some more time to research libraries that can track music in real time; we’ll also have to discuss with Dr. Dueck the situation regarding an orchestra, where multiple instruments will be playing and could cause interference. This upcoming week, I hope to do just that, and I also want to find a way to keep the microphones incorporated so that we don’t waste our budget for no reason; we might end up using the microphones as microphones, and these can then be used to simply store the audio from each actor and be used for post-production processing.
Ted’s Status Report for 03/07/2026
This week, we were mainly focused on finishing the design report before break. I mainly worked on the testing requirements and other miscellaneous sections, while collaborating with the rest of my group mates for the system implementation and architecture sections. I also ordered most of the parts needed for the physical rail assembly, as I’m hoping to buy the plywood needed from Techspark. I finished researching what type of motor specifications we would need, and also met with Dr. Dueck and our SOM collaborators to get some important feedback that would be helpful for our design. The feedback from our collaborators included questions about ideal placements of our UWB nodes, languages used in singing, possible coordination in script and camera movements, preference over camera focus, and preference on camera shift count. I’ve also begun designing some pulley and gears we might need for our belt design in CAD, and should be ready to print them out according to schedule. We are still on schedule, and I hope to begin the physical construction of our rail before next week once all parts are ordered and delivered. We are slightly behind schedule, but that’s mainly due to the fact that we forgot about spring break when designing our schedule. Our slack time makes up for this, and we should still be on track with some extra slack time left in case there are any other delays.
Pareekshith’s Status Report for 3/7
The week before spring break, my group and I were focused heavily on completing the design report. Along with detailed audio subsystem explanations and certain design tradeoffs, I specifically took charge of the Design Requirements and Project Management portions of the report. I also ordered the Lavalier microphones that were critical for the audio component. Along with this, the team also met Dr. Dueck and her students in the 57584 course from the School of Music, and we had a great conversation about the potential role of our project in opera productions. We pored over different details, explored optimizations, and voiced our opinions about potential challenges that could arise through the course of the collaboration. I’m in charge of keeping the group updated with our team’s schedule, and we plan to meet weekly to show our latest progress, get feedback, and have actual testers of our product. I’m currently on track with my schedule, and next week, I hope to receive my microphones (along with our team’s parts) and start setting up our wearable node and its corresponding software. The only thing I’m a bit worried about with the School of Music collaboration is that I don’t know if Vosk, our offline ASR library, will be able to handle opera voices that aren’t spoken English. We’ll have to see if it’s viable for that, and if not, we’d either need to search for a model that is compatible or abandon the audio component specifically for opera (it would still work for stage plays though).
Ted’s Status Report for 02/21/2026
This week, I kept working on the physical design of the rail and helped develop our design presentation, as well as any final major design decisions we planned on using. I’m still waiting to meet with Dr. Dueck and the students we’ll be collaborating with on Wednesday, so I can finalize the dimensions for whatever studio we plan on testing with. We are still on schedule, only thing is that we are slightly delaying our ordering schedule so that we can first meet with Professor Qing Li and Dueck to ensure that our design is satisfactory. As we are going to be waiting a little longer to order parts, but it shouldn’t affect our team operations too much, as we are working on other aspects of the design, like my CAD model, in parallel to any plans we had for development with physical parts. I’m beginning to finalize power calculations, as well as determine what type of motor reductions we might need to include to ensure the motors don’t burn out after 3-4 hours of continuous use. In the next week, I hope to get feedback from our collaborators and advising professors/TAs to adjust CAD designs accordingly, as well as finalizing component orders so we can begin physical implementation.
Team Status Report for 02/21/2026
Some risks we noticed were the RPis we were planning to use with our UWB node weren’t fully compatible with our design. Additionally, a risk that also came to light was the possibility that RPi 5s wouldn’t be consistent in their compatibility with other aspects of our design, such as the UWB nodes and Microphones due to port configuration. Furthermore, what we expect for our project is the fact that we have so many moving parts and modules that depend and interact with each other. If we get slightly delayed in development of one part, the delay could cascade down our schedule and cause pretty bad problems in terms of development down the line. We plan on mitigating these risks by first testing with the RPi 5 to ensure that they work with our modules, as well as including slack time in our schedule to ensure that if we are delayed, we have sufficient time to catch up and finish before any deadlines. As of right now, the only potential change to our design might be the shift from using the RPi 5s to the RPi 4s because of the powering/communication issue that could happen with the Pi 5s and the UWB tags. We do want to test both and see if there is genuinely an issue with the Pi 5. If the Pi 5 works, we’ll stick with that because we already got several from the inventory, and we wouldn’t have to use a portion of our budget on purchasing 4 different Pi 4s. Additionally, we switched to using BLDC motors instead of stepper motors, as we realized the VESC motor controller we’re borrowing from inventory doesn’t support stepper motor operation. Not many costs are incurred, as connections all stay the same.We slightly delayed our ordering schedule to wait for the design review from our TA and Professor Qing. This way we can adjust the bill of materials accordingly within specification. We are now planning to order the necessary parts before spring break starts on 02/28. Doing so will allow us to have the necessary parts and initialize our first prototype.
Pareekshith’s Status Report for 2/21
This week, I was the speaker for our group’s design presentation, so I had to prepare and review the content of the slides and our design to make sure I was ready for the presentation as well as any further questions. During the Q&A portion, Tamal mentioned that using Raspberry Pi 5s for the wearable node component could be risky due to potential inconsistency, so he recommended using earlier models like the Raspberry Pi 4. In light of this, my group and I decided to look further into the differences and the causes for concern, and we plan to discuss this as well with our TA Rhea and Professor Qing Li. Also, as the speech lead, I’m looking into finalizing the design and pipeline for the audio component, making sure the right data is getting processed by the right libraries to maximize accuracy and minimize latency. Next week, I plan to finish the design report with my team, order the necessary components (especially for the audio system), and begin the software implementation of the system.