Author: ncrownie

This week I completed intelligibility testing for the Focus Field system using our defined speech-to-text (STT) comparison method. I recorded baseline audio using a standard headphone microphone and compared it against audio processed through our system, then ran both through the same STT tool. I measured performance using word-level error metrics (missed words, extra words, and substitutions) and computed the overall error rate and relative improvement. Based on these results, our system meets the ≥20% ASR error reduction requirement.

At this point, we have completed testing across all major system requirements and have validated that the system meets our design goals. This week I plan to finalize contributions to the project poster and complete my sections of the final report. I also want to ensure that all testing results are clearly documented and presented in a way that supports our design requirements and overall system performance.

This week I focused on system testing and was able to fully complete testing for steering error, latency, target lock accuracy, and interferer suppression. I ran structured trials and verified that most of our system requirements are being met, with the exception of latency, which is currently slightly above the ≤100 ms target (currently at 104ms). I also spent time refining and debugging the testing process itself. Overall, this week was focused on collecting reliable performance data and ensuring that our testing methods are accurate and meaningful.

My progress is on schedule in terms of completing testing and validating system performance, but slightly behind on meeting the latency requirement. The next step is to optimize the system pipeline and reduce unnecessary processing delays. I am in a good position to quickly iterate and improve performance.

Next week I plan to run intelligibility testing. This will allow us to complete testing across all major system requirements. I also plan to continue refining system performance, particularly focusing on reducing latency to meet the ≤100 ms target.

Throughout this process, I had to learn more about system-level testing, especially how to correctly measure and interpret performance metrics like latency. One key realization was that designing a test is just as important as running it, since incorrect measurement methods can lead to misleading results. To learn these concepts, I relied on the strategy of troubleshooting through trial and error. This hands-on approach, combined with iterative debugging, helped me quickly identify mistakes and improve both my testing methods and overall understanding.

This week I focused on running system tests and continuing development of our testing infrastructure for the Focus Field project. I conducted 67 human trials for target lock accuracy with 1–4 users and recorded results, achieving a 77.6% success rate (52/67), which shows the system is functional but still needs improvement. I also performed steering error testing by physically measuring angular offsets using a protractor in a 2D horizontal plane at 1 meter, aligning with our intended use case. In addition, I continued working on our testing suite, including debugging and fixing an issue in one of the automated test files. I also prepared the system for upcoming tests, including interferer suppression and latency, which already have implemented testing scripts.

My progress is on schedule in terms of development and testing, but slightly behind in meeting performance targets, specifically target lock accuracy. However, this is expected during the testing phase as we begin identifying system limitations. Moving forward, I plan to use the collected data to refine the system, focusing on improving accuracy through better alignment and potential algorithm adjustments.

Next week I plan to run interferer suppression and latency tests using the testing suites that are already implemented. I also want to begin intelligibility testing using our defined speech-to-text comparison approach. The goal is to collect full performance data across all requirements and continue improving the system based on those results.

At this stage, I have focused on verifying that individual components of the system meet their defined requirements through structured testing. For target lock accuracy, I verified system performance by running 67 trials and comparing the measured success rate (77.6%) directly against the ≥90% requirement. For steering error, I verified the system’s directional accuracy by measuring angular deviation using a protractor-based setup in a 2D horizontal plane at 1 meter and comparing results to the ≤15° threshold. I have also prepared verification tests for interferer suppression and latency, including implementing and debugging the corresponding testing scripts, although full results have not yet been collected. Additionally, I have defined the verification approach for intelligibility using ASR error comparison, which will quantify performance once executed. Overall, my verification work ensures that each subsystem can be quantitatively evaluated against its specific engineering requirements.

This week I focused on system testing for our Focus Field prototype, specifically evaluating our core performance requirements using the fully assembled setup. For target lock accuracy, I conducted 67 human trials with 1–4 users in view and recorded system performance, resulting in a 77.6% success rate (52/67), which is currently below our ≥90% requirement. I also ran steering error tests by physically measuring angular offsets using a protractor on a flat plane at 1 meter, aligning with our system assumption of horizontal-only (2D) measurement. These tests helped validate our testing methodology while also identifying areas where system performance needs improvement. In addition, I continued working with our testing infrastructure and debugging, including identifying and fixing an issue in one of our test files related to automated evaluation. Overall, this week was focused on collecting real performance data and validating our testing pipeline.

Progress may be slightly behind in terms of meeting performance requirements, particularly with target lock accuracy currently below our threshold. However, this is expected at this stage since we are now transitioning from setup into real testing and identifying system limitations. The next step is to analyze the causes of the lower accuracy and refine our approach, whether through improved alignment, algorithm adjustments, or system tuning. Since our testing framework is now working, we are in a good position to iterate quickly and improve results.

Next week I plan to run additional system tests for interferer suppression and latency using the testing suites that have already been implemented. I also plan to begin intelligibility testing using our defined approach with speech-to-text comparison to measure ASR error reduction. The goal is to collect complete performance data across all major requirements and continue refining the system based on those results.

This week I spent my project time preparing for our interim demo and helping the team work through the final blocker preventing full system assembly. A major focus was the chassis, since the 3D printing process has taken longer than expected and this has delayed mounting and aligning key hardware components. I stayed involved in discussions around the chassis design, fit, and readiness so that once the printed parts are complete, we can move quickly into final assembly and demo preparation. I have remained actively engaged in coordination, troubleshooting, and preparation work to keep the project moving forward.

Our progress is slightly behind because the chassis has taken longer to complete than expected, and this is currently the last major blocker before we are ready for the interim demo. To catch up, we plan to move immediately into assembly, alignment, and confirmation testing as soon as the printed chassis is available.

Next week I hope to help complete final chassis integration, mount the cameras and microphone array onto the system, and support the team in getting the full prototype ready for the interim demo. I also want to begin or continue confirmation testing once the hardware is assembled so we can validate that the system is operating as expected. The main deliverable is a working, demo-ready prototype that shows the core functionality of Focus Field.

This week I spent most of my time working on the ethics assignment. While completing the assignment, I revisited our system requirements and design decisions to make sure they were aligned with ethical considerations such as user privacy, accessibility, and responsible deployment. This review helped confirm that many of our requirements such as accuracy thresholds, latency limits, and cost constraints; support broader societal goals like accessibility and equitable access to communication technology.

In addition to the ethics work, I continued developing the digital testing suite for the project. I refined portions of the code related to our automated testing infrastructure so that the system can more reliably measure performance metrics as we begin integrating hardware. This included improving parts of the testing logic and making sure the framework can properly capture, log, and compare relevant metrics when the system is running. The goal of this is to ensure that once the full system is operational, we have a consistent and repeatable way to validate whether our design meets its defined requirements.

My progress is still on schedule. We will finally be able to test once we get the system fully mounted on its chassis.

Next week I plan to continue working on the testing suite and begin verifying that the existing tests operate correctly when the system is running in real time.