Category: Team Status Reports

Team Status Report for 4/25

System Progress
This week we completed our final presentation and transitioned into the final testing and validation phase of TrashDash. The system is fully integrated, and we have verified that core functionalities including voice activation, gesture recognition, motor control, and obstacle avoidance—work together in a complete pipeline. We are now focusing on robustness and demo readiness. I am primarily responsible for the demo poster and edge-case testing to ensure reliability under non-ideal conditions.

System Design & Changes
The overall system architecture remains unchanged, but we identified a key limitation in the power subsystem. The current onboard battery cannot reliably support simultaneous operation of the camera and ToF sensors, preventing fully untethered operation. In response, we are exploring higher-capacity battery options and software-level optimizations such as reducing sensor load and managing duty cycles. A fallback option using external power is also prepared for demo stability.

Testing & Verification
We conducted both unit testing and system-level testing:

  • Unit Tests
    • Voice Module: Verified wake word and stop command detection across varying distances and noise levels (met latency and accuracy targets in controlled settings).
    • Gesture Recognition: Tested detection of thumbs-up (move) and fist (stop) under different lighting conditions; performance degrades slightly in low light.
    • Motor Subsystem: Validated movement commands (forward, backward, turning, rotation) and confirmed stable response from the motor driver.
    • ToF Sensors: Tested obstacle detection at different distances; confirmed correct triggering of avoidance logic.
  • System-Level Tests
    • End-to-end pipeline: voice → gesture → motion → obstacle avoidance → stop/resume
    • Edge-case scenarios: low lighting, delayed commands, cluttered environments, and continuous operation

Findings & Design Changes

  • Power limitation is the main bottleneck → investigating stronger batteries and power optimization.
  • Slight degradation in vision performance under low light → considering parameter tuning and controlled demo environment.
  • System latency and responsiveness meet requirements under normal conditions → no major redesign needed.
  • Added fallback strategies (external power, controlled demo setup) to ensure reliability.

Risk & Mitigation
The primary risk is system instability due to insufficient power during full operation. Mitigation includes upgrading the power supply, optimizing component usage, and preparing a wired fallback for the demo. Additional risks include edge-case failures in perception, which are being addressed through targeted testing and controlled demo conditions.

Next Week’s Plan
Next week, we will finalize testing, stabilize the power system, and complete the demo setup. I will finalize the demo poster and continue edge-case validation. The goal is to deliver a smooth, reliable demonstration that clearly showcases all core features of TrashDash.

Team Status Report for 4/18

System Progress This week 

We completed the full end-to-end system pipeline. The voice module receives a wake word and command, triggering the robot to rotate in place scanning for the user’s hand gesture. Upon detecting a Thumbs Up, the robot calibrates the user’s distance using stereo depth, then navigates forward with lateral-shift obstacle avoidance. We also resolved the voice module I2C issue caused by a firmware reflash by analyzing the firmware source code and reverting to the original firmware. A 25-test verification plan mapped to our five use-case requirements has been compiled.

System Design & Changes 

We added a SCANNING state to the state machine and used direct wheel control (SetSpeed) to bypass velocity ramping for smooth rotation. We decoupled voice and gesture activation using a dedicated ROS2 topic (/voice/start_scan) to avoid race conditions during node startup. The ToF node now uses strict four-field validation to filter out EMI-corrupted serial data. Use-case requirements were rewritten to be more user-focused based on professor feedback.

Risk & Mitigation 

The front ToF sensor occasionally reads false low values, possibly due to chassis obstruction, which we need to verify by checking the mounting position. Serial data corruption from motor EMI is handled by validation but still causes occasional data gaps. The current robot navigation speed is set conservatively low to ensure reliability during development; we plan to increase the speed next week and evaluate whether obstacle avoidance and distance tracking remain accurate at higher speeds.

Next Week’s Plan 

We will continue to validate all verification tests and collect more quantitative results for the final demo. We also plan to increase the robot’s movement speed and conduct more user experience testing to evaluate whether the system feels responsive and intuitive in a realistic dorm setting.

Team Status Report for 4/4

System Progress & Design

This week we finished the mid-term demo, where we showed the current state of the TrashDash system including voice activation, gesture detection, and chassis motion. From the feedback several improvements were identified and discussed. First, to reduce the risk of unintentional gesture triggers, switching from Thumbs Up to a less commonly made gesture such as Thumbs Down was considered, as it is less likely to be accidentally activated in everyday scenarios. Second, the obstacle detection threshold for the ToF sensors should be increased to give the robot more time to decelerate before reaching an obstacle, as the current threshold is too short to allow safe braking at operating speed. We also finished ordering all of our hardware devices that we are going to use this week. 

Risk and Mitigation

For this project, one of the potential risks is that the depth camera can not give an accurate distance measurement for the user’s position. The mitigation strategy is to use depth readings from several consecutive frames and average them to smooth out noise and reduce the impact of individual bad readings. Or we can make use of the tof sensors to detect if we already reached the user. Also, if the voice module is not able to detect the command in a noisy environment, it’s also possible to add the function of triggering the robot with http response directly from the computer. 

Plan for Next Week

Our next step will be making the robot actively move toward the user as well as testing the whole system.

Validation

End-to-end validation trials are conducted with users seated at distances of 1 m, 2.5 m, and 4 m in realistic dormitory environments. Each trial evaluates successful navigation to the user’s hand, obstacle-free path execution, trash collection completion, and return to idle state. For a 4 m distance, total runtime must remain below 10 seconds. In addition to quantitative performance metrics, qualitative user feedback is collected to assess perceived responsiveness, safety, and convenience. This validation ensures that TrashDash not only meets technical specifications but also fulfills its intended purpose of improving hygiene and accessibility in shared living environments.

Team Status Report for 3/28

System Progress
This week we successfully tested all major subsystems on the Raspberry Pi and achieved initial system integration. The motor driver is fully functional, supporting forward/backward movement, lateral shifting, rotation, and turning. The OAK-D depth camera can reliably detect different hand gestures, and the voice recognition module successfully detects the wake word “hi trashdash.” We integrated the car chassis, camera, and voice module into a working pipeline: after the wake word and confirmation (“throw trash”), the system activates, moves forward upon detecting a thumbs-up gesture, and stops when a fist gesture is recognized. Additionally, preliminary obstacle avoidance logic using ToF sensors has been validated in a simulated setup.

System Design & Changes
There are no major changes to the overall system architecture. We continue to use the Raspberry Pi as the central controller coordinating voice, vision, and motor subsystems. One practical adjustment is prioritizing early integration of gesture-based control (thumbs-up to move, fist to stop) to demonstrate an end-to-end workflow for the interim demo, before fully implementing continuous hand tracking and navigation.

Risk & Mitigation
The main risks currently are hardware integration and system latency. Mounting all components (camera, ToF sensors, and wiring) onto the chassis may introduce instability or connection issues; this will be mitigated through secure mounting and incremental testing after each component is installed. Another risk is latency between voice input, gesture recognition, and motor response, which could affect responsiveness; we plan to optimize processing pipelines and reduce unnecessary computation. Finally, obstacle avoidance is not fully validated yet since ToF sensors are not mounted; we will mitigate this by conducting real-world testing once installation is complete and refining thresholds for reliable detection.

Next Week’s Plan
Next week, we will focus on physically mounting and securing all components onto the chassis and ensuring stable system operation. After full hardware integration, we will implement and test hand-following behavior so the robot can continuously track and move toward the user’s hand instead of only responding to discrete gestures. In parallel, we will prepare for the interim demo: Siying will cover the introduction and voice module, Qimeng will present the camera and ToF sensing, and Yilu will present the motor system and outline plans for the next three weeks.

Team Status Report for 3/21

System Progress

This week, we worked on both software and hardware development. The vision subsystem can now run MediaPipe for hand detection, and the voice module has been integrated with hardware control using Python and I2C. We also started assembling the motor chassis and testing basic motor control and ToF sensing.

System Design & Changes

There are no major changes to our system design this week. We continue to use the Raspberry Pi as the main controller and keep STM32 as a backup option. The system is still developed in separate modules before integration.

Risk & Mitigation

One risk is hardware integration delay, since some components arrived later and testing is still in progress. We also observed some instability in sensors like the ToF module. To address this, we are testing each subsystem independently and debugging hardware connections before full integration.

Next Week’s Plan

Next week, we will set up the Raspberry Pi environment and connect all subsystems together. We will also continue improving the stability of the vision and sensing modules. Our goal is to start system-level integration and prepare for the interim demo.

Team Status Report for 3/14

System Progress

This week we worked on the ethnic homework including both individual part and team section. We also worked on setting up the voice recognition module in our robot system. For example, instead of using the default command work, we modified the wake-up word and the command words (“throw trash”) so that the system can respond to our customized voice inputs. And we explored how to connect these signals to Rpi to be part of the control flow in our system. Furthermore, we started working on the computer vision subsystem for the TrashDash project, and set up the camera pipeline and began experimenting with MediaPipe for hand detection. We also explored how to combine the RGB image with depth information from the camera in order to estimate the 3D position of the user’s hand. For the motor part, we assembled the car chassis platform and brought up the motors to verify that they work properly.

System Design

To make it easier for users to empty the trash bin, we plan to redesign the physical structure by adopting a nested double-bin structure. The space between the two bins will accommodate our hardware devices, and this design will also simplify the process of removing and cleaning the trash bins. This approach enhances both usability and maintenance convenience.

Risk and Mitigation

For this project, there is a potential risk of conflict when both camera vision and ToF sensors simultaneously detect the target, as their signals might disagree on the object’s position. To mitigate this, the system should prioritize ToF sensor data when identifying obstacles, allowing the ToF signal to override the camera’s input. This ensures that obstacle detection is more reliable and safe. Also, anomaly detection and fault tolerance mechanisms should be implemented, when the outputs of the two sensors differ significantly, the system triggers a safe mode or performs resampling to avoid incorrect navigation.

Plan for Next Week

Our next step will be to continue working on the subsystems and then we will integrate each part together to meet the full control logic. 

 

Team Status Report for 3/7

System Progress

This week we completed the design presentation and finalized the design report for the TrashDash project. Siying delivered the design presentation, which went really well! The team collaborated to complete different sections of the report based on our responsibilities. Yilu wrote the Abstract and Introduction, Project Management, Ethical Issues, and Related Work, and also worked on the Mobility subsystem implementation, which includes the Raspberry Pi control, motor controller design, and overall software integration for the movement system. Siying was responsible for the Use-case Requirements, Design Requirements, Voice subsystem, and Test, Verification, and Validation sections. Qimeng contributed to the Architecture / Principle of Operation, Design Trade Studies, and Summary sections. The System Implementation section was written collaboratively, covering the main subsystems: the vision subsystem (hand detection and localization), the voice subsystem, and the mobility subsystem.

System Design

No significant changes were made to the existing system design, requirements, block diagram, or specifications this week. We confirmed that the Raspberry Pi will serve as the main computation unit to coordinate all subsystems in TrashDash, while the STM32 microcontroller will act as a backup controller for low-level motor control if needed. This decision was made because the Raspberry Pi simplifies integration of vision and voice models, while STM32 provides a reliable fallback option for motor control and hardware interfacing.

Risk and Mitigation

One potential risk for the TrashDash project is delays in hardware delivery, particularly the motor platform and other components required for testing. If these parts arrive later than expected, it may delay system bring-up and integration. To mitigate this risk, we are preparing the software components in advance, such as developing the motor control driver and reviewing the chassis documentation, so that testing can begin immediately once the hardware arrives.

Another risk is related to system reliability and integration, including potential issues with voice recognition in noisy environments and coordination between the vision, voice, and mobility subsystems. To mitigate this, we plan to test each subsystem independently before integration and evaluate the voice module under different noise conditions. Incremental testing and clearly defined interfaces between modules will help reduce integration issues and improve overall system stability.

Plan for Next Week

Next week we plan to begin initial hardware testing. We will test the voice recognition module to evaluate its usability and responsiveness for triggering TrashDash. In addition, we will bring up the motor platform once it arrives and test the custom motor controller driver to verify that the motors can be properly controlled through our system. These tests will help us validate key subsystems before moving forward with full system integration.

Part A (Yilu): TrashDash also needs to be considered in the context of global factors, particularly accessibility, hygiene practices, and differences in living environments. While the project is designed for university dorm rooms, similar needs exist in many settings worldwide where convenience and sanitation are important, such as small apartments, elderly care facilities, and shared living spaces. In regions where waste management and hygiene practices are critical to public health, technologies that make trash disposal easier may help reduce littering and improve cleanliness. At the same time, the system assumes access to certain resources such as reliable electricity, voice interaction in supported languages, and familiarity with technology, which may limit usability in some global contexts. Considering these factors is important to ensure that systems like TrashDash are designed with broader accessibility in mind, including support for different languages, varying noise environments, and users with different levels of technological experience.

Part B (Siying): Cultural factors are also considered in the design of this system. In many student communities, especially in shared dormitory environments, maintaining cleanliness and respecting shared spaces are important social expectations. TrashDash supports these values by making proper waste disposal easier and more accessible. In addition, the system uses natural interaction methods such as voice commands and hand gestures, which accommodate different language preferences and levels of technological familiarity among users. The design also considers rules of behavior in indoor environments, such as minimizing noise and ensuring safe navigation around people and furniture. By aligning with these cultural expectations, TrashDash aims to provide a practical solution that fits naturally into everyday student life.

Part C (Qimeng): Environmental factors are also important to consider in the design of TrashDash. The system aims to encourage more consistent and convenient waste disposal, which can help reduce litter and improve cleanliness in indoor living environments such as dormitories or small apartments. By making it easier for users to throw away trash without leaving their workspace, TrashDash may help prevent waste from accumulating in shared spaces and reduce potential hygiene issues. In addition, the design considers energy efficiency and hardware usage. The system uses low-power embedded components such as a Raspberry Pi and microcontroller-based motor control, and it only activates mobility when necessary, which helps limit unnecessary power consumption. The project also relies on commercially available electronic components and a reusable robotic platform, which reduces the need for custom hardware manufacturing and helps minimize material waste. Considering these environmental aspects helps ensure that TrashDash not only improves convenience for users but also aligns with broader goals of sustainability and responsible use of resources.

Team Status Report for 2/21

System Progress

This week, we completed all purchase orders for our main system components. The remaining items to be ordered include the trash bin, a buck converter for voltage regulation, and a mounting clipper/bracket to secure the camera on top of the trash bin.

With the core components already ordered, we are now positioned to begin physical integration once the deliveries arrive.

System Design & Changes

No changes were made to the existing system design, requirements, block diagram, or specifications this week. The architecture remains consistent with our previous proposal, and all component selections align with our defined system requirements.

Risk & mitigation: 

The primary risk at this stage is that our total spending is approaching the $600 budget limit. Although we have already purchased the major components, the remaining purchases leave little margin for unexpected expenses. To manage this risk, we have prioritized essential components and avoided optional upgrades. If additional costs arise during integration, we are prepared to substitute non-critical parts with lower-cost alternatives to remain within budget. 

Another concern raised this week, particularly mentioned by Professor Mukherjee, is the relatively high power consumption of the Raspberry Pi 5. While it provides strong computational capability for our computer vision implementation, its power demands may affect battery life and thermal performance. Since we have already requested the Raspberry Pi 5 from inventory, we plan to proceed with it for initial development while carefully managing power through the use of a buck converter and software optimization. If power consumption becomes a significant issue during testing, we will evaluate lower-power embedded alternatives as a contingency plan.

Next Week’s Plan

Next week, our primary focus will be on writing and finalizing the design report. At the same time, since most components are expected to arrive next week, we will begin assembling the hardware and conducting initial integration tests.

Team Status Report for 2/14

This week, we mainly focused on the design of our whole system and finalized all the components we need to purchase.

System design

We divided our system into three main parts: the motor platform, the navigation and vision sensing, and the voice recognition module. And for each part, we selected the components based on our quantitative user requirements. 

motor platform:

We selected some motor platforms that would fit our user requirements, as well as considered the compatibility problem of the platform with our Raspberry Pi system. We finally decided to use Hiwonder Large Metal 4WD Vehicle Chassis, which can be used on both Rpi and stm32. 

Voice recognition:

The voice recognition module requires relatively high-speed voice detection response, as well as high detection accuracy. Based on these requirements, we chose the WonderEcho AI Voice Recognition Module, which has an integrated neural network processor for offline voice recognition and has an accuracy that reaches up to 98%. 

Navigation & camera:

Based on the provided inventory list, we finally decided to use the provided Oak-D Pro Robotics Camera, since the camera’s stereoscopic depth perception capabilities would help us in detecting the hand gesture of the user with reasonable speed, as well as navigate to the user.  

Risk & mitigation:

Our risk right now is the potential difficulty of running the motor platform on our Raspberry Pi system. Therefore, to address this issue, we decided to order another stm32 chip for our backup plan. 

Next week’s plan:

After we receive these components, we will first test these components individually to ensure all parts are functioning properly. And we will also test if the motor control will work as we expected, and test if the speed and load of the motor platform meet our requirements. 

 

Team Status Report for 2/7

Project Overview and Scope Refinement

Based on feedback from our last meeting with Professor Tamal Mukherjee and Kyle, we refined the project scope to improve feasibility. The project was revised from an autonomous trash can designed to track and intercept trash mid-air to one that responds to voice commands and navigates to users to collect trash. This change significantly reduces mechanical and perception complexity while preserving the project’s core goals of autonomy, user interaction, and practical utility in a dorm or office environment.

Preliminary System Structure

We began outlining a high-level system architecture for the autonomous trash bin. The system will consist of sensing modules (camera, UWB, ToF), a perception layer for voice recognition and environmental understanding, a navigation and planning module for path generation and obstacle avoidance, and a low-level control layer for motor actuation. The Raspberry Pi will serve as the primary controller, coordinating sensor input, machine learning inference, and motion control.

Risk Assessment and Mitigation

Our biggest concern so far is the complexity of integrating multiple sensors, machine learning models, and control systems. Hardware incompatibilities, driver issues, or unexpected sensor behavior could slow development. To reduce this risk, we will validate each subsystem independently before full system integration. Early prototyping and staged testing will allow us to identify and resolve issues before they propagate through the system. 

Progress This Week

This week, we completed the proposal presentation slides, and Yilu delivered the presentation on Wednesday. Additionally , the team began researching machine learning models and hardware components.

  • Siying investigated candidate voice recognition and hand-detection models. 
  • Qimeng evaluated potential hardware components, including cameras, ultra-wideband (UWB) sensors, and time-of-flight (ToF) sensors. 
  • Yilu focused on identifying motors and a mobile chassis compatible with the Raspberry Pi platform.

Plans for next week

In the coming week, we plan on finalizing component selections and placing orders for required hardware. We will also complete the detailed system design, including sensor placement and control flow, and begin preparing materials for the design presentation. Initial setup and bring-up of selected components will start as soon as hardware becomes available.