Jenny Han, Bradley Zhou, Jade Wang. Carnegie Mellon ECE Capstone, Spring 2021
This week we refined our pen tracking algorithm, finalized the GUI, and completed the calibration code. For the pen tracking algorithm, we were able to combine acceleration data, orientation data, and hand tracking from OpenCV to get a reasonable mouse position. After meeting with our …
This week I worked on finishing automating the hand gestures with the GUI, the pen tracking with OpenCV, fine-tuning the GUI, and the accuracy of the pen. We ended up changing our plan for the pen tracking to include OpenCV hand tracking to improve the accuracy of our mouse tracking algorithm.
Here are a few demos:
Demo for pen clicking: https://drive.google.com/file/d/1qb6ZgsfIvp1ET2bqQAoe1I6CgzHnUK-n/view?usp=sharing
Demo for hand tracking: https://drive.google.com/file/d/1LxP3QMM8vi1FtG5JySy1Php_KzaJSKU7/view?usp=sharing
Next week I will work on finishing the calibration page (for mouse position and orientation), the final presentation, and improve the accuracy of the pen with a weighted input (OpenCV hand tracking, IMU orientation data, and IMU acceleration data)
This week we worked on the pen location tracking and the GUI. For the pen location tracking, we got a new IMU and were testing out different software elements to get more accurate readings as we were having issues with accuracy and orientation. We attempted …
This week I worked on the GUI. I finished the calibration page, main page, and add the gesture page. I am currently working on making it so users can automatically add gestures through the GUI and plan on finishing that feature by the end of …
This week I worked on the GUI for the calibration and gesture recognizing page using Tkinter. I also researched more into SciKit Kinematics to understand how to integrate the new IMU with the library functions.
This week I will finish the GUI and link the calibration code to the GUI and projector
This week we worked on the pen tracking software, the gesture recognition, the GUI for the project, and linking the gestures with macros. For the pen tracking software, we updated the double integration so it would integrate over a window of past values. This wasn’t …
This week I worked on the pen tracking algorithm. I found an error in our implementation and updated it so it would integrate over the last 10 data points collected by the IMU. This still had some inaccuracy, so we decided to get a new IMU (arrived today) and will attempt to work on that more the next week.
Secondly, I worked on finding an OpenCV solution for the pen tracking. I think with the unreliablity of the pen accelerometer data, if we combined a software element it would make the overall implementation more reliable (since there are more inputs). However, I was having some issue finding a reliable open source for it. I looked into the MediaPipe software as well as OpenCV software. I think if I cannot find a reliable source for object tracking, we could use a laser instead of a pen against the wall for a similar effect.
Finally, I worked on the GUI of the project and am using TKinter for the calibration, about, home and help pages.
This week, I worked with Bradley and Jenny to improve our pen tracking algorithm. We tried several methods to ensure the accuracy of the device. However, we ran into issues with the double integration and found that the error was magnified with integration. This week, …