This week I focused on developing the testing infrastructure and benchmarking approach for evaluating whether our Focus Field system meets its defined performance requirements. I designed the overall testing methodology for each requirement and implemented several testing scripts directly into the project codebase.
I created a test for interferer suppression (tests/test_interferer_suppression_60deg.py) that measures signal levels in dB at both the input and output of the system and logs the difference to evaluate whether the system achieves the required ≥ 8 dB suppression at a 60° off-axis angle.
I also implemented an end-to-end latency test (tests/test_end_to_end_latency.py) that measures the delay between audio input and processed output to verify the ≤ 100 ms latency requirement.
In addition to writing these tests, I outlined the experimental procedures for evaluating target lock accuracy and steering error once the cameras are mounted on the chassis, which will involve repeated human trials with one to four speakers and recording the resulting accuracy and angular error. I also developed the evaluation approach for intelligibility improvements using automatic speech recognition, where baseline audio and processed audio will be transcribed using speech-to-text (STT) tools and compared to calculate relative error reduction. Overall, my work this week focused on ensuring we have a structured and measurable way to validate system performance once hardware integration begins.
My progress is currently on schedule. Establishing the testing framework allows us to immediately evaluate system performance as components are integrated, which reduces the risk of delays later in the project.
Next week I plan to verify that the testing scripts function correctly when the system is running in real time, as this has not yet been fully validated. I also plan to refine logging and output formats so that results from repeated trials can be easily analyzed and compared against our requirements. Once the system is operational with hardware components, I will begin running preliminary confirmation tests.
