Team E7: The Wand Makers

Carnegie Mellon ECE Capstone, Spring 2025 — Sharon Lai, Olina Zhang, Nadia Palar

Weekly Status Reports

Team Status Report for 2/22/2025
Posted by : yunjiaz Team Status Reports

Risks and Contingency Plans

The most significant risk right now is that our model evaluation is behind schedule, which may delay integration with the IR wand system. Since we need to compare CNN, RNN, and LSFM models before finalizing one for deployment, delays in testing could impact our further implementation and optimization.

To mitigate this risk, Olina will prioritize completing model evaluations next week and continue working during spring break weekend to ensure we stay on track.

Design Changes

No changes to our design at this point.

Schedule

We are behind schedule due to delays in model testing. However, we are working to catch up next week and will use spring break to ensure we meet our project milestones.

Olina’s Status Report for 2/22/2025
Posted by : yunjiaz Olina's Status Reports

This week, I focused on setting up and refining the model evaluation process. I continued working with the CNN, RNN, and LSFM models, preparing them for a detailed comparison in terms of accuracy, speed, and computational efficiency.

To ensure a fair comparison, I refined data preprocessing steps, standardized input formats. I also set up evaluation scripts to measure key performance metrics such as accuracy, precision, recall, and inference time. While initial testing is in progress, I have yet to complete a full analysis.

To get back on track, I plan to complete all model evaluations next week by running full-scale tests and comparing the architectures based on their efficiency and accuracy. Since I am behind schedule, I will continue working during spring break to ensure that I meet the project goals.

Team Status Report for 2/15/2025
Posted by : npalar Team Status Reports

Risks and Contingency Plans
The most significant risk right now is if our ordered PCB doesn’t work after being ordered because there may not be sufficient slack to redesign and order another PCB after receiving our V1. As such, we are spending substantial time verifying the PCB design before ordering it. Additionally, there are multiple test points on the boards in case hotfixes are required.

Design Changes
No changes to our design at this point.

Schedule
We remain on schedule.

Part A (Olina Zhang):
Our wand-based gesture detection and infrared control technology benefit public health, safety, and welfare in a variety of ways. Our technology eliminates the need for physical switches, which improves safety. It offers a touch-free interaction technique, which reduces the spread of viruses and impurities on frequently handled objects like light switches and TV remotes.This is ideal for persons with restricted mobility, disabilities, or the elderly, who may struggle with traditional remote-controlled devices. The technology allows people to easily control their surroundings with simple gestures, reducing the risk of falls or accidents caused by reaching for physical controls.

In terms of welfare, our product strives to improve people’s quality of life. We want to create a wand that is small, simple to use, and provides consistent results. Our technology provides convenience to a wide spectrum of consumers. This promotes independent living for people with physical disabilities and enhances comfort and usability in smart homes.

Part B (Nadia Palar):
The wand provides a unique and expressive experience for individuals to interact with their technological devices. The wand also gives individuals a way to interact with devices in a more meaningful way as well as express an interest in magic. One such social context in which this might be particularly of note is in the classroom, where the wand could be used to make lessons and engaging with educational material more interactive and appealing for some students.

Part C (Sharon Lai):
One of the primary economic considerations is the final cost of manufacturing the wand. The components involved—IMUs, IR transmitters, and CNN-powered systems—are becoming increasingly affordable due to advances in sensor technology, machine learning, and miniaturization. Our approach allows us to leverage these innovations to create a cost-effective and economically viable option for individuals looking for a new and more expressive way to interact with their technological devices.

Nadia’s Status Report for 2/15/2025
Posted by : npalar Nadia's Status Reports

Initial iterations of the device drivers for the IR receiver and external flash memory have been finished. The codebase is less messy, but there are some restructuring changes that will be finished. Since Sharon has began testing some of the initial firmware, I started a new branch to begin integrating FreeRTOS for task management; this will ideally help with some of the contention and aid with overall robustness and handling. This involves additional restructuring of the code so that various blocks can be more cleanly organized into tasks.

For deliverables, I created more polished versions of our block diagrams for the design presentation this week and started work on the design report document.

I remain on schedule. Next week, I will work with Sharon to complete the first hardware order of the semester so we can get the PCB fabricated as well as begin more thorough development testing for our drivers and overall use-case flow.

Sharon’s Status Report for 02/15/2025
Posted by : hsuanjul Sharon's Status Reports

Accomplishments

  • Continued update firmware, the updated firmware with HAL library is compilable right now:

IR Module Updates — checked the IR library for efficiency, and deleted redundant functions

Wand Module Adjustments — removed all modules except IR to discard the struct types that we no longer need. Noted that TIM2 configuration may be misaligned with the system frequency, potentially causing timing issues.

Deleted ADC & USART — Removed the ADC and USART modules entirelyand merged them into main.c.

Update memory allocation for CNN — Implemented a new weight_update mode to store constants in external flash rather than the microcontroller’s limited internal flash. Also wrote the corresponding script that updates weight.h. The static buffer in main.c was reduced from 1024 * 8 to 1024 * 7 to avoid a runtime overflow. Note that this change still needs verification through stress testing to confirm that the RAM usage is stabilized and no new overflow condition arises.

  • Updated PCB, connected with Professor Bain, and confirmed to use JLC for manufacturing. Still trying to connect with Quinn to get informed about the purchase procedure.

Please check the project GitHub for updated details.

Schedule Update

The firmware schedule remains on track. For the PCB manufacturing, I am slightly behind schedule. I’ll start going through the purchasing process as soon as I connect with Quinn next week.

Plans for Next Week

  • Confirm the RAM usage fix (reduced static buffer) is stable.
  • Test updated firmware on board and debug.
  • Get the first PCB version manufactured ASAP.

Olina’s Status Report for 2/15/2025
Posted by : yunjiaz Olina's Status Reports

This week, I worked on model building, training optimization, and dataset processing this week. In order to begin multi-class classification, I first cleaned and prepared the motion data, applied sequence padding, and converted gesture labels into one-hot encoding.

I experimented with three distinct architectures for model development: CNN, RNN, and LSFM. While the RNN (Bidirectional LSTM) assisted in tracking motion across time, the CNN was utilized to record spatial patterns. Both strategies were used in the LSFM model to increase precision and effectiveness.

I make progress to adjust hyperparameters such as learning rates, dropout rates, epoch counts, and batch size. I implemented model checkpointing and early halting to avoid overfitting.

I am on track with the project timeline. Next week, I will fine-tune the models by adjusting kernel sizes, units, and regularization. I will also run more evaluations and start real-time testing to ensure the model works smoothly with the wand system.

Nadia’s Status Report for 2/8/2025
Posted by : npalar Nadia's Status Reports

Since Sharon finished sourcing all of the hardware, we were able to begin setting up the codebase with some simple code using the HAL libraries with the STM32 Cube IDE. This included ensuring the chosen MCU had sufficient peripherals for all of the functionality we wanted to implement as well as allocating these peripherals. As of now, the codebase builds, but is still somewhat messy as we were trying to get a minimum working example up and running.

On the deliverables side, I began work on the slides for the design presentation to reflect the hardware choices we made.

Next steps include making architectural changes for quality-of-life to make integration easier, as well as ensuring that all of the functions will interface correctly with the hardware for easier integration. Additionally, the team will need to finalize our design slides for the presentation the following week.

Team Status Report for 2/8/2025
Posted by : hsuanjul Team Status Reports

Risks and Contingency Plans

  • Firmware Stability & Compatibility Issues The transition from the STM32 standard library to the HAL library could introduce unexpected issues in peripheral functionality, particularly in SPI flash, I2C communication, etc. We will test each module incrementally and verify functionality before proceeding to new updates. We are also keeping backups of the standard library implementation in case of severe issues. If the HAL library causes critical instability, we will explore using a hybrid approach—keeping some standard library components for specific modules while migrating the rest to HAL.
  • Gesture Recognition Model Accuracy Data preprocessing steps (e.g., normalization, filtering) need to be refined. And we plan to do more research into CNN and other architectures.

Design Changes & Impact

  • Watchdog Timer Removal & Power Management Adjustments During firmware debugging, it was confirmed that all four MCU timers (TIM1-TIM4) are already allocated for other tasks, meaning the watchdog timer cannot be used for power optimization. this means that the system will need to rely more on hardware-based power management rather than software-based sleep modes. However, since the MCU is already low-power, this change should not significantly impact energy efficiency. We will still optimize the circuit design to minimize unnecessary power consumption.

Schedule Updates

No major changes were made and the schedule remains as is.

Progress / Photos

Please see the project GitHub.

Sharon’s Status Report for 2/8/2025
Posted by : hsuanjul Sharon's Status Reports

Accomplishments

  • Firmware update: updated firmware from stm32 standard library to hal library based. This will make it easier for us to add further features and make updates. Also, this will allow us to compile on both MacOS and Windows systems. Below is my update note:

STM32CubeMX ReconfigurationCleaned up the project structure, deleted unnecessary modules, and moved key configurations into main.h.

SPI & External Flash UpdatesUpdated external flash driver and verified SPI pin configurations in CubeMX.

I2C & MPU6050 IntegrationDownloaded and integrated the MPULib folder. Updated the I2C.c and I2C.h files, ensuring MPU6050 initializes after I2C initialization.

GPIO, LED, & Button HandlingEnsured LED initialization follows GPIO setup and tested LED toggling. Integrated button handling logic, though I still need to validate edge cases more thoroughly.

  • Project Proposal Presentation: delivered a project proposal presentation about the general idea and plans for the project

Please check the project GitHub for updated details.

Schedule Update

I remain on track. The firmware update was going smoothly. During code update, I confirmed that all 4 timers in the MCU (TIM1 to TIM4) are occupied for other usages, so we will not be able to run the watchdog function. In this case, energy considerations will rely more on circuit design. This won’t be too much of a worry because the MCU itself actually does not consume a lot of power given that it is a rather low-level chip.

Plans for Next Week

Firmware Debugging & Refinement

  • IR module integration
  • Update USART
  • Investigate and refine button handling logic for better reliability.
  • Test firmware under different power conditions and check for stability issues.
  • Identify and fix any remaining edge cases in sensor readings or communication protocols.

Ordering the PCB

  • Generate and verify Gerber files for manufacturing.
  • Place the PCB order and confirm lead times.

By the end of next week, I plan to have a HAL library-based firmware version that is able to sense IMU data and run CNN. Also, the PCB design should be submitted for manufacturing.

Olina’s Status Report for 2/8/2025
Posted by : yunjiaz Olina's Status Reports

This week, I focused on data collection for the gesture recognition model in our wand project. Specifically, I collected data for six different gestures. Specifically,  I recorded 50 samples for each gesture. I manually recorded and labeled each set to ensure the data is consistent. In addition to data collection, I began preliminary data preprocessing, including normalizing the motion data inputs and organizing the dataset into a format compatible with our CNN training pipeline. I also spent time reviewing related research on gesture recognition to identify potential improvements for our model architecture.

My progress is on schedule.

Next week, I plan to work on

  1. Develop and implement the initial version of the CNN model for gesture recognition.
  2. Conduct initial training and testing of the model to evaluate baseline performance.