Author: mjwang

Jade’s Status Report for 5/8/2021

Jade’s Status Report for 5/8/2021

This week, I worked with Bradley and Jenny to refine our pen tracking algorithm such that it incorporated the accelerometer data. We decided to use a weighted approach since the accelerometer data exaggerates any errors in distance calculation. I also worked with Jenny to add 

Team Status Report for 5/8/2021

Team Status Report for 5/8/2021

This week we improved our pen tracking algorithm by incorporating acceleration data into pipeline. Professor Kim suggested using a weighted algorithm to do this and thus, we were able to implement acceleration data in our algorithm by playing with the weights of the acceleration vs. 

Jade Wang’s Status Report for 5/1/2021

Jade Wang’s Status Report for 5/1/2021

This week, I finished fully automating the process of adding a new gesture with the GUI. The GUI page now refreshes automatically to display the newly added hand gesture and the user macro that it maps to.

Additionally, we decided to alter our approach for the pen input due to the pen sensor readings not being accurate enough, despite our several attempts at refining the data via different methods of integration, matrices, filters, etc. We decided on using an OpenCV approach in which we track the hand via camera to help with getting a more precise position of the hand holding the pen. Thus, this week I used OpenCV to get the position of the hand and get the center coordinates. In this way, we could use the coordinates along with the gyroscope/orientation data to get an even more precise measurement of where the tip of the pen is.

Next week, we will try to incorporate accelerometer data into the pen + OpenCV data to hopefully get even more precise coordinates. Please look at Jenny’s status report for a demo of the hand tracking and gyroscope data mapping to mouse movement.

Jade Wang’s Status Report For 4/24/2020

Jade Wang’s Status Report For 4/24/2020

This week, I worked on automating the hand gesture recognizer so that users can add new gestures and map them to user macros. The program now will take in the new gesture readings and automatically train the model immediately. The program then restarts automatically so 

Jade’s Status Report for 4/10/2021

Jade’s Status Report for 4/10/2021

This, week I worked on both the pen tracking algorithm and the hand gesture recognition algorithm. For the pen tracking algorithm, I tried out a Kalman filter for the acceleration readings from the MPU6050. I tried various parameters but it seemed like the Kalman filter 

Team Status Report For 3/27/2021

Team Status Report For 3/27/2021

This week, we made significant progress in our pen hardware. We were able to read in data from the accelerometer/gyroscope and send it to our Python code via the HC-05 Bluetooth module. Due to some library issues, we had to pivot to using Windows as an operating system. Additionally, we created initial wireframes for our Python GUI for pen calibration. A photo of our hardware wired up is shown below:

Jade Wang’s Status Report For 3/27/2021

Jade Wang’s Status Report For 3/27/2021

This week, I met with Bradley and Jenny several times to make progress in our pen hardware after meeting with Professor Kim on Monday. After Bradley assembled the initial pen hardware, we all met up to help debug his assembly on the breadboard. We explored 

Team Status Report for 3/20/21

Team Status Report for 3/20/21

This week, we met up multiple times to finish the design report and move forward in building our project. We got some good feedback from the professor and TA in details that our project was missing. We are doing our best to turn this into 

Jade Wang’s Status Report for 3/20/21

Jade Wang’s Status Report for 3/20/21

This week, I worked on finishing our Design Report Document and turning that.  Additionally, I worked on the Python script to read data from Bluetooth data from the HC-05 module. I experimented with the PyBluez library and found several complications with the library and its dependencies. I was having a lot of trouble with these dependencies so I looked into other libraries to work with. In doing so, I discovered other libraries that make use of “Bluetooth Low Energy” technology and found that this actually might be a better technology to use in comparison to just regular Bluetooth. I found that BLE is actually used for applications that do not need to send large amounts of data — which may fit our purpose since we will just be sending accelerometer and gyroscope data points, and some inputs for button clicks. Thus, I found a different library called “Bleak” that makes use of BLE technology and got it to work in a script. Here’s a terminal output of scanned Bluetooth devices using the Bleak library: 

For now, I’ll be using this as our Python Bluetooth API library. If this doesn’t workout, I will switch back to PyBluez and try to get it working again but I’m fairly confident Bleak will work and consume less power simultaneously.

I also met with Jenny and Bradley again to talk about our progress and next steps. Bradley is in the middle of working on connecting the Arduino Micro, HC-05, and the sensors. Once he gets that working, I’ll be able to try and read in bluetooth data for the HC-05 module.

I’ll continue working on writing an Arduino script as well to read in the sensor data and send it via Bluetooth so we can immediately test the hardware once it is assembled.

Given the professor’s feedback, I think we are a little behind in our project but we are working hard to get back on track, assembling parts and writing actual code to make sure our planned implementation will work and made adjustments if not.

 

 

Jade Wang’s Status Report for 3/13/21

Jade Wang’s Status Report for 3/13/21

This week, I finalized the Design Presentation Slides, wrote speaker notes, and practiced/rehearsed my presentation. In doing so, I flushed out more details regarding metrics and validation with Jenny and Bradley. We received a lot of in-depth feedback for our presentation regarding content. Something important