Jenny Han, Bradley Zhou, Jade Wang. Carnegie Mellon ECE Capstone, Spring 2021
This week, I spent toying with the new IMU we received, the BNO055. The benefit to this IMU is it features a magnetometer and more accurate orientation readings. We had originally planned on using this new IMU alongside a software library – scikit kinematics, to …
This week we worked on the pen location tracking and the GUI. For the pen location tracking, we got a new IMU and were testing out different software elements to get more accurate readings as we were having issues with accuracy and orientation. We attempted …
This week I worked on the GUI. I finished the calibration page, main page, and add the gesture page. I am currently working on making it so users can automatically add gestures through the GUI and plan on finishing that feature by the end of the weekend. I was debating how to best attempt this feature and I think that if the GUI leads directly to the training mode and there was a set amount of time for the gestures to train for and save that would be the best GUI for the user.
Also, I worked on looking at the pen software functionality and familiarized myself with SciKit Kinematics. I am exploring the new IMU and how to best integrate it with the system.
This week I will finish automating the add gesture page and work on the pen ergonomics and location tracking software.
This week, I worked on automating the hand gesture recognizer so that users can add new gestures and map them to user macros. The program now will take in the new gesture readings and automatically train the model immediately. The program then restarts automatically so …
This week I worked on the GUI for the calibration and gesture recognizing page using Tkinter. I also researched more into SciKit Kinematics to understand how to integrate the new IMU with the library functions. This week I will finish the GUI and link the …
This week, I worked more on the pen tracking algorithm as well as integration with mouse movement. After observations that the kalman filter was quite slow and didn’t result in a noticeably more accurate result, I removed that for now. I also observed that the …
This, week I worked on both the pen tracking algorithm and the hand gesture recognition algorithm. For the pen tracking algorithm, I tried out a Kalman filter for the acceleration readings from the MPU6050. I tried various parameters but it seemed like the Kalman filter …
This week we worked on the pen tracking software, the gesture recognition, the GUI for the project, and linking the gestures with macros.
For the pen tracking software, we updated the double integration so it would integrate over a window of past values. This wasn’t too accurate either, so we ordered a new IMU with more information and more accurate data. This came today, so hopefully, we will be able to get a more accurate reading with the new IMU. Another thing we were considering was a software boost for accuracy. We are still trying to find the best way to track the pen with OpenCV, but we are planning on having more input (OpenCV + IMU) so the reading is more accurate.
For the gesture recognition, we were able to accurately decipher between different hand gestures and trigger computer macros for them. Next week we will work on saving new gestures.
For the GUI, we began to create the interface with which the user will interact with. This includes the home screen and calibration screen. Next week we will work on the calibration of the pen with the GUI (but first the accuracy of the pen tracking)
This week I worked on the pen tracking algorithm. I found an error in our implementation and updated it so it would integrate over the last 10 data points collected by the IMU. This still had some inaccuracy, so we decided to get a new …
