This week, I did research into open-source libraries that we could use to do real-time music tracking. The one library that I found that works really well with our use case was Matchmaker, which was released last year and is able to output the current position in the script based on the sheet music and the audio data. With this, we might have to pivot from our Lavalier microphones to a Samson microphone that isn’t omnidirectional and can focus properly on the piano music and not the surrounding operatic dialogue. The only concern I have with this is that it works specifically for piano; other instruments and orchestras would most likely mess up the alignment. Because of this, I want to check with Dr. Dueck if using this library is feasible. We might need to slightly pivot with this, especially because I am unable to find any other library that works on orchestral music. During our group’s work session this Saturday, I also helped with the taping of the anchors and the battery packs. I’m currently on schedule, and I plan to experiment with the Samson microphone that one of my partners might have. There are some technical details that need to be honed out, such as the conversion of the sheet music into specific MusicXML files and the exact audio input format. These will be covered and explored next week, and I plan on doing on-device testing and integration with the Pis.
Ted’s Status Report for 03/14/2026
This week, we met back up with Dr. Dueck and received some feedback on the design of our wearable nodes, showing them a rough prototype to give them an estimate of what type of device they’d have on their person. I’ve also received some of the parts and have been working on interfacing the BLDC motors with the VESC motor controller. The VESC motor controller 3-phase wires come preequipped with MR-30 connectors, but the BLDC motors just come with rough wires. I’m thinking of just chopping off the connectors and simply just soldering the phase wires directly to the motor controller instead of attempting to install any MR-30 Connectors. One of the micro-usb connections on the VESC is a little loose but it shouldn’t be a source of any issues unless under heavy vibration. I’ve also finished designing the GT-2 timing pulley that will be used with our motor and timing belt, and plan to print it soon. I’ll be printing multiple copies incase they shatter or break during testing so we won’t have to wait an extended period of time to replace it. Schedule is still on track, and by next week once ball bearings have been delivered I hope to begin construction of the rail itself.
Ahmad’s Status Report 3/14
This week I was able to demonstrate the design behind the wearable node to the opera performers including the configuration of the UWB sensor with the RPI 5, battery pack, and lavalier mic. This demonstration provided the opera performers some insight into the sizing of the equipment to help them with the outfit coordination. We also discussed script cues for future reference including, “FS,” which signals full stage view for our system. Then throughout the week I worked on configuring the DWM3001CDK UWB development boards, flashing appropriate firmware, and establishing reliable ranging between two nodes. I tested one board as an initiator and another as a standalone anchor, while verifying that ranging data could be produced and read through serial interface. Additionally, I explored different power configurations that can be utilized to allow the anchors to function in parallel with the battery pack. These steps confirmed that the hardware stack required for wireless position tracking is functioning!
Team Status Report for 03/14/2026
This week productivity revolved around the team receiving the equipment we needed and meeting again with Dr. Dueck. Our meeting this week with Dr. Dueck where we showcased the design behind the tracking sensors. This demonstration provided the opera performers some insight into the sizing of the equipment to help them with outfit coordination. The meeting was overall successful with continued discussion on script cues and camera operation. Additionally, a novel proposition was introduced to utilize the piano’s audio as a way for the AutoCam to follow script position. Pareekshith received the Lavalier microphones, and after discussion with Dr. Dueck decided that it might be better to track the sheet music rather than the performer’s speech, especially given that it might be hard for Vosk to understand operatic dialogue. The design might change for this reason, and so we might end up having a single microphone next to the piano to get clear audio of the sheet music rather than having each actor with a clip-on microphone. Ahmad’s UWB sensors, battery packs, and RPI 5s have also arrived this week and he has begun testing and integration. Due to the auto shutoff from the battery packs, the UWB sensors will have to be manually connected with additional resistors to draw more current. The software is working well on the other hand with proper anchor and node integration. Ted received the BLDC motors, timing belt, and other various parts for the physical rail. The BLDC motors don’t come attached to connectors, so the VESC motor phase wires will need to be stripped and soldered manually. Additionally, the bearings still haven’t arrived, so physical rail construction is a little delayed.
Pareekshith’s Status Report for 3/14
This past week, I received the Lavalier microphones from the ECE Receiving office and thus, got a better idea of how to integrate them into our system. When we met with Dr. Dueck and her students, we discussed placements of the wearable nodes and the mics under the costumes, and we will be in contact figuring out how exactly to hide them discreetly. The good news is that the microphones are 6 meters long, so if there is a safe spot on an actor, the microphone can easily clip on and work properly. I also brought up the dilemma that we have with Vox, namely that it won’t work as well as expected with an operatic voice. Because of this, we are considering switching gears and using the sheet music as the script and then indicating where the cues should happen. Although our original plan was to use speech technology, given the collaboration with the School of Music, it makes sense to go with this approach; if we have time near the end, we might also try to incorporate speech technology into our system for drama productions, which typically have more conversational speech than operas and musicals. I’m still on track with my schedule, although I might need some more time to research libraries that can track music in real time; we’ll also have to discuss with Dr. Dueck the situation regarding an orchestra, where multiple instruments will be playing and could cause interference. This upcoming week, I hope to do just that, and I also want to find a way to keep the microphones incorporated so that we don’t waste our budget for no reason; we might end up using the microphones as microphones, and these can then be used to simply store the audio from each actor and be used for post-production processing.
Team Status Report for 03/07/2026
Team: The week prior to Spring Break, we focused on 3 actions: completing the design report, ordering the correct parts, and beginning our collaboration with Dr. Dueck and the School of Music. Based on our first meeting with Dr. Dueck, the most significant risk we encountered was regarding the audio subsystem of the project. The library we plan to use, Vosk, is meant for conversational speech, not operatic dialogue, and so this could pose a difficult problem for us. We will definitely test our system and audio pipeline with opera and check if it works under a workload like that, and our contingency plan will be to use the remote-controlled method for opera and other similar musical productions. The good news is that Vosk is compatible with multiple languages, which is great for international productions, but the issue about speech recognition when the speech is operatic still persists, and we can only determine the exact performance once we receive the parts, assemble them, complete the software, and integrate both sides. No changes have been made to our design, and as a reminder, the remote-controlled operation is our MVP, so that will be something we develop anyways and can use to confirm proper performance under all conditions. Our schedule is technically one week behind, but that’s mainly because we forgot to account for spring break when formulating our schedule. Our two weeks of slack time is able to make up for this loss.
A was written by Pareekshith, B was written by Ted, and C was written by Ahmad.
A: The design of AutoCam was made keeping in mind the fact that the productions utilizing our product probably wouldn’t be very technologically savvy, and so we’re trying to make it easy to install, easy to use, and easy to get great results. As mentioned before, we are aiming for a smartphone-agnostic design, so anyone from around the world could use AutoCam for their own purposes with whatever phone they have, and the potential use cases are vast. We want to keep options open for users, so we’ll be looking to have a feature where the system is remote-controlled, giving productions the hands-on experience that they might desire, and we’ll also be using speech technology to process dialogue and enable cueing. Different use cases will call for different implementations, and we want to give that freedom to potential users. Finally, we are designing AutoCam keeping budgets in mind, as we want to democratize filmmaking and cinematography for aspiring directors and filmmakers around the world.
B: Theater and stageplay is inherently a cultural experience. Stageplay, theater, and even oral history all share deep cultural roots in many societies around the world. Stage performances are especially governed by anything from audience expectations and behavior norms within a theater environment. In most performances, silence and minimal distractions during important scenes is generally expected. Because of this, we designed AutoCam with these norms in mind, and wanted to ensure that our system would output as little noise pollution as possible. Our design utilizes brushless DC motors in order to disrupt the audience experience as little as possible. Additionally, we’re hoping that our system will accommodate different languages by using speech recognition libraries that can be tuned for different languages and styles of performance, like spoken word vs opera. The wearable components are also intended to be lightweight and discreet so that they won’t interfere with performers’ costumes, stage moment, or cultural performance practices.
C: Environmental factors are considerable as an engineer, and placing these factors in high regard can help us better understand the beneficial and detrimental effects of our product. In this case, AutoCam poses little to no direct threat to the natural environment, since it is a small scale system intended for indoor use in stage productions. It does not require hazardous chemicals, produce emissions during operation, or consume large amounts of energy compared to traditional large scale filming infrastructure. Most of the system’s components, such as the Raspberry Pis, UWB sensors, microphones, and camera hardware, are low power electronic devices, and the physical rail itself is being constructed primarily from lightweight materials like plywood and copper piping. By using a smartphone as the recording device rather than a specialized high power camera system, AutoCam also helps reduce the need for more resource intensive equipment. On the other hand, environmental considerations still matter in the design of AutoCam because electronic systems contribute to material waste and energy consumption over time. To address this, we desgined the system to be durable, reusable, and modular so that individual components can be repaired, replaced, or upgraded without discarding the entire product. This is especially important for parts such as the wearable nodes, sensors, and rail hardware, which may experience repeated use across multiple performances.
Ahmad’s Status Report 3/7
The week before spring break, my group and I were focused heavily on completing the design report. Along with working through several sections such as testing and validation requirements, system implementation details, and other technical documentation, I helped guide the team through organizing and finalizing the overall report structure. I spent a lot of time coordinating with teammates to make sure our sections aligned and that the technical decisions we described were consistent across the document. I also ordered the battery packs and UWB sensors right before spring break, and they’ve already arrived, so I’ll just need to pick them up once I’m back on campus so we can begin assembling and testing the hardware. Along with this, we’ve continued working with Dr. Dueck and her students, which has been really helpful for thinking about how we’ll approach testing and getting deeper insights into how the system should behave in real-world scenarios. These conversations have helped us better understand what kinds of testing setups and feedback will be most valuable once our system is operational. I’m excited to get back to Pittsburgh and start connecting all of the components together and beginning our initial testing phase. We also plan to begin building the physical rail system once we return, which will allow us to start integrating the mechanical components with the sensing hardware. Once everything is assembled and connected, we’ll be able to begin validating the system and iterating based on feedback from Dr. Dueck and her students as we move further into development.
Ted’s Status Report for 03/07/2026
This week, we were mainly focused on finishing the design report before break. I mainly worked on the testing requirements and other miscellaneous sections, while collaborating with the rest of my group mates for the system implementation and architecture sections. I also ordered most of the parts needed for the physical rail assembly, as I’m hoping to buy the plywood needed from Techspark. I finished researching what type of motor specifications we would need, and also met with Dr. Dueck and our SOM collaborators to get some important feedback that would be helpful for our design. The feedback from our collaborators included questions about ideal placements of our UWB nodes, languages used in singing, possible coordination in script and camera movements, preference over camera focus, and preference on camera shift count. I’ve also begun designing some pulley and gears we might need for our belt design in CAD, and should be ready to print them out according to schedule. We are still on schedule, and I hope to begin the physical construction of our rail before next week once all parts are ordered and delivered. We are slightly behind schedule, but that’s mainly due to the fact that we forgot about spring break when designing our schedule. Our slack time makes up for this, and we should still be on track with some extra slack time left in case there are any other delays.
Pareekshith’s Status Report for 3/7
The week before spring break, my group and I were focused heavily on completing the design report. Along with detailed audio subsystem explanations and certain design tradeoffs, I specifically took charge of the Design Requirements and Project Management portions of the report. I also ordered the Lavalier microphones that were critical for the audio component. Along with this, the team also met Dr. Dueck and her students in the 57584 course from the School of Music, and we had a great conversation about the potential role of our project in opera productions. We pored over different details, explored optimizations, and voiced our opinions about potential challenges that could arise through the course of the collaboration. I’m in charge of keeping the group updated with our team’s schedule, and we plan to meet weekly to show our latest progress, get feedback, and have actual testers of our product. I’m currently on track with my schedule, and next week, I hope to receive my microphones (along with our team’s parts) and start setting up our wearable node and its corresponding software. The only thing I’m a bit worried about with the School of Music collaboration is that I don’t know if Vosk, our offline ASR library, will be able to handle opera voices that aren’t spoken English. We’ll have to see if it’s viable for that, and if not, we’d either need to search for a model that is compatible or abandon the audio component specifically for opera (it would still work for stage plays though).
Ted’s Status Report for 02/21/2026
This week, I kept working on the physical design of the rail and helped develop our design presentation, as well as any final major design decisions we planned on using. I’m still waiting to meet with Dr. Dueck and the students we’ll be collaborating with on Wednesday, so I can finalize the dimensions for whatever studio we plan on testing with. We are still on schedule, only thing is that we are slightly delaying our ordering schedule so that we can first meet with Professor Qing Li and Dueck to ensure that our design is satisfactory. As we are going to be waiting a little longer to order parts, but it shouldn’t affect our team operations too much, as we are working on other aspects of the design, like my CAD model, in parallel to any plans we had for development with physical parts. I’m beginning to finalize power calculations, as well as determine what type of motor reductions we might need to include to ensure the motors don’t burn out after 3-4 hours of continuous use. In the next week, I hope to get feedback from our collaborators and advising professors/TAs to adjust CAD designs accordingly, as well as finalizing component orders so we can begin physical implementation.