Posted on

Cole’s Status Report for 03/14/2026

This week I focused on finalizing the first draft of the 3D printed enclosure for our system. I initially started by first researching similar projects to see if there was any open source CAD work for the specific hardware components that we are using. I was able to find a very solid enclosure for the mic array that seemed to fit into my thought out design. 

Camera Mount

The main CAD design that I worked on was the mount for the cameras. These cameras came with preattached mounts, so getting the exact dimensions of the raw cameras was quite difficult. I was able to estimate using 3D models for the cameras and importing them into the design. The idea behind the camera mount is that each camera will be attached to one side of the triangle mount, and the wires from the cameras will flow through the gap on the bottom and into the middle of the pillar. On top of the mount is where the microphone array sits. This was very straightforward in my opinion since I already found an open source enclosure for the microphone array. Thus, the microphone array easily fits on top of the camera mount and the output wire will fold underneath and into the middle of the camera mount. Another aspect of the camera mount is the prongs on the bottom. There are so the camera mount can simply be placed onto the shaft to connect the parts. 

 

Full Design Draft

As of right now this is a solid first draft and has promising looks for the rest of the design

Schedule

Currently on schedule.

Next Steps

Next week I plan on 3D printing each component for the first time in order to start making changes and tweaks to the designs. This will ensure that the design is robust before starting to build upon it.

 

Posted on

Nasir’s Status Report for 03/14/2026

This week I spent most of my time working on the ethics assignment. While completing the assignment, I revisited our system requirements and design decisions to make sure they were aligned with ethical considerations such as user privacy, accessibility, and responsible deployment. This review helped confirm that many of our requirements such as accuracy thresholds, latency limits, and cost constraints; support broader societal goals like accessibility and equitable access to communication technology.

In addition to the ethics work, I continued developing the digital testing suite for the project. I refined portions of the code related to our automated testing infrastructure so that the system can more reliably measure performance metrics as we begin integrating hardware. This included improving parts of the testing logic and making sure the framework can properly capture, log, and compare relevant metrics when the system is running. The goal of this is to ensure that once the full system is operational, we have a consistent and repeatable way to validate whether our design meets its defined requirements.

My progress is still on schedule. We will finally be able to test once we get the system fully mounted on its chassis.

Next week I plan to continue working on the testing suite and begin verifying that the existing tests operate correctly when the system is running in real time.

Posted on

Omar’s Status Report for 03/07/2026

Executive summary

This week I took the AV stack from a prototype pipeline toward a deployable, Pi-ready meeting-room front end, and I personally built out the new live operator UI. The result is a production-style ML runtime on main plus a much stronger monitoring/dashboard surface waiting on a feature branch.

Shipped on origin/main – ML/Pi runtime and perception

  • Implemented a multiprocessing runtime that splits audio and vision into separate processes, with OS tuning hooks for affinity, scheduler, memory locking, and thread caps to make the Pi profile realistic under load.

  • Added mic health analysis and a reliability-aware active-speaker posterior that fuses audio speech probability, visual speaking, DOA agreement, temporal continuity, and mic health instead of relying on simple heuristics.

  • Upgraded VAD, denoise, visual speaking, and face tracking paths to model-backed/model-aware implementations, and introduced a beamforming fallback ladder (MVDR → subset MVDR → delay-and-sum → best-mic) plus refreshed telemetry, config validation, recording, and tests.

In progress – UI / operator experience (my work, not yet on main)

  • Overhauled the streaming UI into a WebSocket-based live operator dashboard on the rilical/debug-ml-pi-pipeline branch, replacing slow polling with a proper telemetry stream.

  • Built out a new dashboard surface: camera tiles, DOA compass, mic health bars, audio meters, lock timeline, event log, and a calibration panel so an operator can actually monitor and tune the system live.

  • Added a camera config API with sidecar JSON persistence and camera auto‑reconnect, publishing status on vision.camera_status.{cam_id}, so cameras can be configured and recovered without code edits.

Validation and test foundations

  • Integrated and extended regression tests for interferer suppression and end-to-end latency, so the new runtime and MVDR path are backed by measurable timing and off‑axis rejection checks instead of anecdotal performance.

  • Used these tests to validate that the new Pi profile and multiprocess architecture behave as intended before we rely on them in real meeting-room scenarios.

Posted on

Cole’s Status Report for 03/07/2026

My primary focus was on the physical enclosure design for the system. I determined how all the hardware components would be physically integrated into a single structure that can support the microphone, cameras, and Raspberry Pi.

I began by creating brief sketches of how I imagined the design would look. After looking at a few different designs I started to gather 3D models of components and take measurements where needed. Once the component dimensions were defined, I began designing an enclosure layout that could securely hold all of the components while maintaining proper spacing and alignment. This helped me determine where the components will fit best. However, I spent a lot of time determining where each component will function the best so that it can capture audio effectively without obstruction from the enclosure walls. Additionally, I wanted to ensure that the fan from the Raspberry Pi cooling system does not interfere with the function of the microphone array.

Schedule

Currently on schedule.

Next Steps

Next week I will finalize the exact placement and mounting of all components so that I can generate an initial 3D print of the enclosure. This will put our team in a great position for testing purposes and leave room for hardware integration.

Posted on

Team Status Report for 03/07/2026

One of the most significant risks at this stage of the project is designing a chassis that securely mounts the cameras while maintaining proper alignment with the microphone array. Because the system relies on visual detection and directional audio working together, even small shifts in camera positioning or misalignment between the cameras and microphones could negatively affect speaker identification accuracy and steering performance. Ensuring the physical stability and precision of the hardware layout is therefore critical to achieving reliable system behavior during testing and operation.

To manage this risk, the team plans to carefully evaluate multiple design options for the device’s chassis before finalizing a solution. We will use an iterative prototyping approach, gradually refining the physical design to ensure the cameras remain stable and correctly aligned with the microphone array. This process will involve testing different mounting approaches and making adjustments based on observed performance and stability.

At this time, no contingency plan is necessary because the risk is manageable through design iteration and careful prototyping. No changes have been made to the system requirements, block diagram, or overall system specifications in the past couple of weeks.

A was written by Cole Franklin, B was written by Nasir Crownie and C was written by Omar Ghabayen. Nasir reviewed and edited for unity.

A. Global Factors

The Focus Field project considers global factors by addressing a communication challenge that affects people in many different regions and work environments around the world. As remote work, virtual meetings, and online collaboration become more common globally, many people participate in calls from noisy homes, shared offices, or public spaces. By designing a low-cost system that improves speech clarity without requiring specialized infrastructure, the project aims to support clearer communication for users across different countries, professions, and levels of technical experience.

B. Cultural Factors

The project also considers cultural factors by recognizing that communication technologies should support speakers of many different languages and backgrounds. People participate in meetings, classes, and collaborations using a wide variety of languages and accents, so the system is designed to improve speech clarity without relying on language-specific processing. By focusing on isolating the active speaker’s voice rather than interpreting the language itself, the device can support clearer communication for users regardless of the language they speak, helping make the technology more inclusive and accessible across diverse cultural communities.

C. Environmental Factors

The Focus Field project has minimal environmental impact because it is designed as a small, USB-powered desktop device. It does not require large infrastructure, high energy consumption, or specialized installations, and it operates using standard computing hardware that is already common in many work environments. As a result, the system does not significantly affect natural resources or the surrounding environment beyond typical low-power electronic device usage.

Posted on

Nasir’s Status Report for 03/07/2026

This week I focused on developing the testing infrastructure and benchmarking approach for evaluating whether our Focus Field system meets its defined performance requirements. I designed the overall testing methodology for each requirement and implemented several testing scripts directly into the project codebase.

I created a test for interferer suppression (tests/test_interferer_suppression_60deg.py) that measures signal levels in dB at both the input and output of the system and logs the difference to evaluate whether the system achieves the required ≥ 8 dB suppression at a 60° off-axis angle.

I also implemented an end-to-end latency test (tests/test_end_to_end_latency.py) that measures the delay between audio input and processed output to verify the ≤ 100 ms latency requirement.

In addition to writing these tests, I outlined the experimental procedures for evaluating target lock accuracy and steering error once the cameras are mounted on the chassis, which will involve repeated human trials with one to four speakers and recording the resulting accuracy and angular error. I also developed the evaluation approach for intelligibility improvements using automatic speech recognition, where baseline audio and processed audio will be transcribed using speech-to-text (STT) tools and compared to calculate relative error reduction. Overall, my work this week focused on ensuring we have a structured and measurable way to validate system performance once hardware integration begins.

My progress is currently on schedule. Establishing the testing framework allows us to immediately evaluate system performance as components are integrated, which reduces the risk of delays later in the project.

Next week I plan to verify that the testing scripts function correctly when the system is running in real time, as this has not yet been fully validated. I also plan to refine logging and output formats so that results from repeated trials can be easily analyzed and compared against our requirements. Once the system is operational with hardware components, I will begin running preliminary confirmation tests.

Posted on

Cole’s Status Report for 02/21/2026

This week I focused primarily on hardware integration and initial system validation. This week all of our components were delivered, so I have begun to work on the initial system build and verification of each component.

Hardware Integration

Once all of the hardware components arrived I was able to pick them up and ensure they were functioning properly. For the cameras I ensured that the device was UVC compliant and operates in a stable mode for 1080p streaming. After working with the components for quite a while I was able to confirm that there were no immediate frame drops or device disconnects under normal load. I spent less time working with the microphone array, but I was able to confirm basic audio input capture from the component. Overall this has ensured that all of our components function as expected before we begin to integrate the components to the entire system.

This phase ensured that all components function as expected before deeper system integration.

Enclosure Design Planning & CAD Development 

This week I also began brainstorming and designing the enclosure for our system. Since this project requires three cameras for 360 degree FoV, the enclosure is critical for arranging the cameras and ensuring they are locked into place.

Once the hardware components were received I was able to take detailed measurements using a digital caliper. This gave me a rough idea of the design we were going to need. Second I measured all of the cable ports and connections that our system requires. This is slightly more complex since there are going to be ports that are not going to be used within our system. 

A couple of things that still need to be considered for this 3D printed enclosure are:

  • Potential Sound Interference of the Microphone Array
  • Airflow for heat dissipation
  • Ease of Assembly

 

Schedule

My progress is currently on schedule.

 

Next Steps

Next week I plan on creating an initial CAD draft for the enclosure of the system. This will allow me to get some feedback and thoughts on the design and make any changes before attempting to print. Outside of the enclosure of the system, next week I plan on collecting more quantitative performance data of the system operating under real loads.

Posted on

Omar’s Status Report for 02/21/2026

This week focused on hardening the Raspberry Pi deployment pipeline and enforcing strict hardware contracts to eliminate silent degradation paths. The system now enforces a fail-fast startup model requiring 3+ USB capture cameras and 8+ microphone channels before runtime initialization. This significantly improves deployment reliability and operational predictability.

Major Accomplishments

1. Strict Hardware Contract Enforcement

  • Implemented hard-stop behavior when minimum hardware requirements are not met.

  • Integrated strict validation into configuration generation, preflight checks, and runtime startup.

  • Added regression tests (test_pi_contract.py) to lock behavior into CI.

  • Updated runbook documentation to reflect enforced requirements and expected failure semantics.

2. Camera Topology and Discovery Reliability

  • Filtered non-capture V4L2 nodes to eliminate false-positive detections.

  • Tightened V4L2 probing logic for improved capture-capable device identification.

  • Standardized use of stable numeric /dev/videoN indices to reduce symlink ambiguity.

  • Improved explicit 3-camera index generation behavior for safer configuration.

  • Reduced noisy probing and unnecessary backend fallbacks.

  • Added CAP_ANY fallback selectively to improve backend resilience where appropriate.

3. Preflight and Diagnostics Improvements

  • Elevated pi_preflight.py into a formalized deployment gate.

  • Improved diagnostic clarity around camera open failures.

  • Hardened audio device resolution and operator-facing guidance.

  • Documented face cascade fallback behavior and detection validation steps.

  • Reduced setup fragility by removing unavailable wheel dependencies.

4. Runtime and System Stability Enhancements

  • Enforced fail-fast runtime startup when hardware contract is violated.

  • Improved device probing determinism and reduced ambiguous behavior.

  • Strengthened configuration generation workflows.

  • Added structured fallback profiles where explicitly allowed.

Current System State

The system has transitioned from foundational architecture build-out into operational hardening. The camera discovery stack is now deterministic and filtered to capture-only devices. Startup validation ensures that incorrect hardware configurations fail immediately rather than running in degraded states.

This materially improves deployment reliability and reduces debugging ambiguity in field environments.

Next Focus Areas

  • Continued validation under varied USB topologies.

  • Long-duration stability testing on Pi hardware.

  • Performance measurement under full 3-camera + 8-mic load.

  • Additional telemetry improvements for early anomaly detection.

Overall, the system now enforces a strict hardware contract before pipeline execution, significantly increasing reliability and deployment confidence.

Posted on

Team Status Report for 02/21/2026

This week our team received all of our ordered components. These components include the 8-channel USB mic-array and the three USB cameras. This was a huge week for our team since we were finally able to put our work together and test it on some real components. The main work we focused on this week was setting up and testing out the components to ensure they were working properly. Additionally, our group has been finalizing the details of our design report. 

Most Significant Risks & Mitigation Plans

Risk: Processing Overload

This remains our biggest concern in our design. The simultaneous use of the 3 USB cameras with the 8-channel mic-array requires a lot of processing and may overload the CPU and USB bandwidth of the Raspberry Pi 4.

 

The risks associated with this processing overload can lead to potential frame drop, increased audio latency, and overall system instability.

Risk Management:

  • Switched from CSI camera plan to UVC USB cameras for easier driver support.
  • Reduced camera resolution target from 1080p at 30fps to 720p at 30fps
  • Raspberry Pi Cooling system

Contingencies:

  • Adding an additional Pi for vision processing and communicate via Ethernet
  • Lower camera frame rate (20–24 fps).
  • Using hardware-accelerated vision libraries

Design Changes Made Since Last Report

None.

Schedule Changes

None.

 

Posted on

Nasir’s Status Report for 02/21/2026

This week, my primary focus was leading the writing of our Design Review Report (85% of time) while the team finalized hardware acquisition. As the testing, integration, and QA lead, my role becomes most active once subsystems are operational, so being that we just received our components I don’t have any pressing physical work, as expected and drafted in our Gantt Chart. Since we only recently received the microphone array, cameras, and supporting hardware in the mail, there was limited hands-on integration work available at this stage. I used this time productively by organizing the performance validation plan and working on the Design Review. This preparation will allow us to move efficiently into the integration phase without losing time defining metrics or procedures.

Now that components are in hand, my next steps are to coordinate subsystem bring-up and begin staged integration testing. My focus will shift toward synchronizing the audio and visual pipelines, verifying real-time performance, and ensuring that early subsystem outputs align with our use-case requirements.