Jenny Han, Bradley Zhou, Jade Wang. Carnegie Mellon ECE Capstone, Spring 2021
This week, I worked with Bradley and Jenny to improve our pen tracking algorithm. We tried several methods to ensure the accuracy of the device. However, we ran into issues with the double integration and found that the error was magnified with integration. This week, …
This week we worked on the distance module and tried to ensure the accuracy of our own device. We tried to use double integration to calculate the displacement. However, using double integration caused a lot of error which made it challenging to use the pen …
This week I worked on the calibration module and tried to get a reasonable accuracy with it. However, my team ran into the issue that double integration over acceleration caused issues in the calibration because the integration emphasized the inaccuracy. To deal with this, we looked into other ways to deal with the distance calculation. This included a backup software approach with open CV or a hardware approach with a GPS.
Other than the pen module, I worked on the computer macros. Currently, the computer can click on coordinates on the Windows computer screen and click any point based on pixel coordinates. Also, I was able to figure out how to use features such as control Z or refresh buttons for our terminal remote control.
Even though we had some issues with the distance module, I think we are still on track. We will attempt the distance calculation this week and continue to work on the gesture recognition this week.
This week I worked on the hand gesture model and the pen. For the hand gestures, I marked the dataset and cleaned it: After that, I trained and tested the model. Here are the results: For the pen, I helped wire the accelerometer and …
This week, we made significant progress in our pen hardware. We were able to read in data from the accelerometer/gyroscope and send it to our Python code via the HC-05 Bluetooth module. Due to some library issues, we had to pivot to using Windows as …
This week, I met with Bradley and Jenny several times to make progress in our pen hardware after meeting with Professor Kim on Monday. After Bradley assembled the initial pen hardware, we all met up to help debug his assembly on the breadboard. We explored several Arduino MPU6050 libraries to get our code to read in the accelerometer/gyroscope data. After finishing that, we wired the Bluetooth piece to try and read in the data via Python code. The Bluetooth libraries available for MacOS are fairly outdated so we had to pivot several times. PyBluez, Sockets, and BLE libraries all did not work. Eventually, we resorted to pivoting to a different operating system and decided to use Windows.
Additionally, I created some initial wireframes for our GUI design using Figma:
This week, I worked on the hardware and software prototyping of the pen. First, I constructed the circuit on a breadboard, connecting the arduino, accelerometer, and bluetooth module. After ensuring the accelerometer worked and accurately reported data through the serial monitor, I explored various options …
This week, we met up multiple times to finish the design report and move forward in building our project. We got some good feedback from the professor and TA in details that our project was missing. We are doing our best to turn this into …
This week I worked on the design report and helped write the software implementation, introduction, software testing plan, schedule, and task breakdown sections. I also helped to revise and edit our report.
In terms of the project, I worked on the gesture recognizer. I got OpenCV running on my computer, found and marked a dataset I could use, and started to work on the gesture recognizer model.
Here are images of the dataset I marked.
This week, I worked on finishing our Design Report Document and turning that. Additionally, I worked on the Python script to read data from Bluetooth data from the HC-05 module. I experimented with the PyBluez library and found several complications with the library and its …