Category: Nasir’s Status Reports

This week I completed intelligibility testing for the Focus Field system using our defined speech-to-text (STT) comparison method. I recorded baseline audio using a standard headphone microphone and compared it against audio processed through our system, then ran both through the same STT tool. I measured performance using word-level error metrics (missed words, extra words, and substitutions) and computed the overall error rate and relative improvement. Based on these results, our system meets the ≥20% ASR error reduction requirement.

At this point, we have completed testing across all major system requirements and have validated that the system meets our design goals. This week I plan to finalize contributions to the project poster and complete my sections of the final report. I also want to ensure that all testing results are clearly documented and presented in a way that supports our design requirements and overall system performance.

This week I focused on system testing and was able to fully complete testing for steering error, latency, target lock accuracy, and interferer suppression. I ran structured trials and verified that most of our system requirements are being met, with the exception of latency, which is currently slightly above the ≤100 ms target (currently at 104ms). I also spent time refining and debugging the testing process itself. Overall, this week was focused on collecting reliable performance data and ensuring that our testing methods are accurate and meaningful.

My progress is on schedule in terms of completing testing and validating system performance, but slightly behind on meeting the latency requirement. The next step is to optimize the system pipeline and reduce unnecessary processing delays. I am in a good position to quickly iterate and improve performance.

Next week I plan to run intelligibility testing. This will allow us to complete testing across all major system requirements. I also plan to continue refining system performance, particularly focusing on reducing latency to meet the ≤100 ms target.

Throughout this process, I had to learn more about system-level testing, especially how to correctly measure and interpret performance metrics like latency. One key realization was that designing a test is just as important as running it, since incorrect measurement methods can lead to misleading results. To learn these concepts, I relied on the strategy of troubleshooting through trial and error. This hands-on approach, combined with iterative debugging, helped me quickly identify mistakes and improve both my testing methods and overall understanding.

 

This week I focused on running system tests and continuing development of our testing infrastructure for the Focus Field project. I conducted 67 human trials for target lock accuracy with 1–4 users and recorded results, achieving a 77.6% success rate (52/67), which shows the system is functional but still needs improvement. I also performed steering error testing by physically measuring angular offsets using a protractor in a 2D horizontal plane at 1 meter, aligning with our intended use case. In addition, I continued working on our testing suite, including debugging and fixing an issue in one of the automated test files. I also prepared the system for upcoming tests, including interferer suppression and latency, which already have implemented testing scripts.

My progress is on schedule in terms of development and testing, but slightly behind in meeting performance targets, specifically target lock accuracy. However, this is expected during the testing phase as we begin identifying system limitations. Moving forward, I plan to use the collected data to refine the system, focusing on improving accuracy through better alignment and potential algorithm adjustments.

Next week I plan to run interferer suppression and latency tests using the testing suites that are already implemented. I also want to begin intelligibility testing using our defined speech-to-text comparison approach. The goal is to collect full performance data across all requirements and continue improving the system based on those results.

At this stage, I have focused on verifying that individual components of the system meet their defined requirements through structured testing. For target lock accuracy, I verified system performance by running 67 trials and comparing the measured success rate (77.6%) directly against the ≥90% requirement. For steering error, I verified the system’s directional accuracy by measuring angular deviation using a protractor-based setup in a 2D horizontal plane at 1 meter and comparing results to the ≤15° threshold. I have also prepared verification tests for interferer suppression and latency, including implementing and debugging the corresponding testing scripts, although full results have not yet been collected. Additionally, I have defined the verification approach for intelligibility using ASR error comparison, which will quantify performance once executed. Overall, my verification work ensures that each subsystem can be quantitatively evaluated against its specific engineering requirements.

This week I focused on system testing for our Focus Field prototype, specifically evaluating our core performance requirements using the fully assembled setup. For target lock accuracy, I conducted 67 human trials with 1–4 users in view and recorded system performance, resulting in a 77.6% success rate (52/67), which is currently below our ≥90% requirement. I also ran steering error tests by physically measuring angular offsets using a protractor on a flat plane at 1 meter, aligning with our system assumption of horizontal-only (2D) measurement. These tests helped validate our testing methodology while also identifying areas where system performance needs improvement. In addition, I continued working with our testing infrastructure and debugging, including identifying and fixing an issue in one of our test files related to automated evaluation. Overall, this week was focused on collecting real performance data and validating our testing pipeline.

Progress may be slightly behind in terms of meeting performance requirements, particularly with target lock accuracy currently below our threshold. However, this is expected at this stage since we are now transitioning from setup into real testing and identifying system limitations. The next step is to analyze the causes of the lower accuracy and refine our approach, whether through improved alignment, algorithm adjustments, or system tuning. Since our testing framework is now working, we are in a good position to iterate quickly and improve results.

Next week I plan to run additional system tests for interferer suppression and latency using the testing suites that have already been implemented. I also plan to begin intelligibility testing using our defined approach with speech-to-text comparison to measure ASR error reduction. The goal is to collect complete performance data across all major requirements and continue refining the system based on those results.

This week I spent my project time preparing for our interim demo and helping the team work through the final blocker preventing full system assembly. A major focus was the chassis, since the 3D printing process has taken longer than expected and this has delayed mounting and aligning key hardware components. I stayed involved in discussions around the chassis design, fit, and readiness so that once the printed parts are complete, we can move quickly into final assembly and demo preparation. I have remained actively engaged in coordination, troubleshooting, and preparation work to keep the project moving forward.

Our progress is slightly behind because the chassis has taken longer to complete than expected, and this is currently the last major blocker before we are ready for the interim demo. To catch up, we plan to move immediately into assembly, alignment, and confirmation testing as soon as the printed chassis is available.

Next week I hope to help complete final chassis integration, mount the cameras and microphone array onto the system, and support the team in getting the full prototype ready for the interim demo. I also want to begin or continue confirmation testing once the hardware is assembled so we can validate that the system is operating as expected. The main deliverable is a working, demo-ready prototype that shows the core functionality of Focus Field.

This week I spent most of my time working on the ethics assignment. While completing the assignment, I revisited our system requirements and design decisions to make sure they were aligned with ethical considerations such as user privacy, accessibility, and responsible deployment. This review helped confirm that many of our requirements such as accuracy thresholds, latency limits, and cost constraints; support broader societal goals like accessibility and equitable access to communication technology.

In addition to the ethics work, I continued developing the digital testing suite for the project. I refined portions of the code related to our automated testing infrastructure so that the system can more reliably measure performance metrics as we begin integrating hardware. This included improving parts of the testing logic and making sure the framework can properly capture, log, and compare relevant metrics when the system is running. The goal of this is to ensure that once the full system is operational, we have a consistent and repeatable way to validate whether our design meets its defined requirements.

My progress is still on schedule. We will finally be able to test once we get the system fully mounted on its chassis.

Next week I plan to continue working on the testing suite and begin verifying that the existing tests operate correctly when the system is running in real time.

This week I focused on developing the testing infrastructure and benchmarking approach for evaluating whether our Focus Field system meets its defined performance requirements. I designed the overall testing methodology for each requirement and implemented several testing scripts directly into the project codebase.

I created a test for interferer suppression (tests/test_interferer_suppression_60deg.py) that measures signal levels in dB at both the input and output of the system and logs the difference to evaluate whether the system achieves the required ≥ 8 dB suppression at a 60° off-axis angle.

I also implemented an end-to-end latency test (tests/test_end_to_end_latency.py) that measures the delay between audio input and processed output to verify the ≤ 100 ms latency requirement.

In addition to writing these tests, I outlined the experimental procedures for evaluating target lock accuracy and steering error once the cameras are mounted on the chassis, which will involve repeated human trials with one to four speakers and recording the resulting accuracy and angular error. I also developed the evaluation approach for intelligibility improvements using automatic speech recognition, where baseline audio and processed audio will be transcribed using speech-to-text (STT) tools and compared to calculate relative error reduction. Overall, my work this week focused on ensuring we have a structured and measurable way to validate system performance once hardware integration begins.

My progress is currently on schedule. Establishing the testing framework allows us to immediately evaluate system performance as components are integrated, which reduces the risk of delays later in the project.

Next week I plan to verify that the testing scripts function correctly when the system is running in real time, as this has not yet been fully validated. I also plan to refine logging and output formats so that results from repeated trials can be easily analyzed and compared against our requirements. Once the system is operational with hardware components, I will begin running preliminary confirmation tests.

This week, my primary focus was leading the writing of our Design Review Report (85% of time) while the team finalized hardware acquisition. As the testing, integration, and QA lead, my role becomes most active once subsystems are operational, so being that we just received our components I don’t have any pressing physical work, as expected and drafted in our Gantt Chart. Since we only recently received the microphone array, cameras, and supporting hardware in the mail, there was limited hands-on integration work available at this stage. I used this time productively by organizing the performance validation plan and working on the Design Review. This preparation will allow us to move efficiently into the integration phase without losing time defining metrics or procedures.

Now that components are in hand, my next steps are to coordinate subsystem bring-up and begin staged integration testing. My focus will shift toward synchronizing the audio and visual pipelines, verifying real-time performance, and ensuring that early subsystem outputs align with our use-case requirements.

This week I was traveling and did not complete hands-on development work on the project. However, this time away was planned in advance and communicated clearly with my team. During this period, I stayed in consistent contact with teammates to keep up with development, design discussions, and any decisions being made. I reviewed updates and provided input when needed. I do not have new files but I maintained active involvement through virtual communication.

My progress is still on schedule overall because this travel period was anticipated and built into my planning with the team. There is minimal ramp-up needed to return to development tasks. To stay aligned, I will prioritize requirement follow-ups and prototyping support when I return.

Next week, I plan to re-engage with active development by solidifying our devices performance to make sure we are hitting our requirement benchmarks. I also aim to rescope any requirements so the team can move forward confidently into our implementation.

This week I have spent most of my efforts on the Proposal Presentation. I have also been working on finding the exact components that we will need use that are not in inventory. During this search I have found some optimization in certain products that make our hardware requirements very feasible. In some cases I even have pushed our requirements further in hopes of getting better performance. I have also spent time during my status report and the team status report.

I think I am still tracking fine but the mishap this week with group presentations did cause more time to be spent focusing on the presentation slides than expected. This in turn has taken about 2.5 work hours that could have been spent elsewhere but no extreme impedance.

By this weeks end, I want to have a list of all components that we have from inventory and all the exact components we plan to purchase so we can move past the part of questioning which hardware components are being used.