This week, I had fixed the bluetooth issue and connection up as well as the data encoding/decoding algorithm to work. The problem was that the bluetooth connection on Mac is always shown as disconnected but by setting the connection up manually through python code, it worked. I also have written code to distinguish between the left/right EMG data flow and combined it with the calibration code that I had before. We discussed integrating the EMG into the EEG package that is up running and will be using the similar method of triggering events from the EEG ML classification. I think I a back on track, the plan for this upcoming week is to test and integrate this into the whole system. I have discussed with Jonathan about using the FFT features but he thought our ML model classification is accurate enough, so I think we will probably add the FFT after depending on the time we have after getting the whole system up.
Jonathan’s Status Report for 04/16/2022
I continued to work on doing more modeling with the double blink signal and implemented the proper logic to determine a double blink signal. I trained a model with about 15% error in detecting a double blink but included a single blink detection model that will process user signals before the double blink detection model to ensure better classification of double blink signals. This way, the double blink signal can only be detected once we already see a blink and some hard-coded factors within the decision model will ensure the double blinking doesn’t falsely trigger too often. To do this, if a single blink is detected, we switch to using the double blink detection since we should no longer be attempting to detect winks, and the double blink detection loop is run with a timeout of 1 second. Then the system goes back to detecting simpler signals and we get a coherent detection of double blinks that can be sent to the front end. I also worked with Jean to integrate the EMG code into the master application so that EMG can be run concurrently with EEG. Our system now operates with 4 software threads: 2 for I/O, 1 for ML model prediction, and 1 for managing the frontend application. We are now on target to do testing on our final integrated product next week. This is inline with our schedule as we are now within the slack period, and we hope to begin polishing our product next week.
Team Status Report for 04/16/2022
This week, we resolved the Bluetooth problem. Our group integrated the EMG sensing system with the EEG sensing system to allow user input from both devices to the same front end. We have not tested the left and right shoulder events with the keyboard, but we will be doing that this week. We are almost good for our MVP.
Wendy’s Status Report for 04/09/2022
This week, I still mostly completed my research on how to connect the Python application with the OS so it can be accessed on any Mac application. There doesn’t seem to be a way to do this easily so if I am unable to do this, I will look into building out a system that won’t be on the OS but is local to the computer. Currently, I am behind schedule because I did not work on what I planned to accomplish this week due to Carnival. However, I already started looking into connecting the EMG events with the keyboard interface and with the new sensor having arrived, it will be done early this week.
Jean’s Status Report for 04/09/2022
This week, I had been working on a bluetooth module. Earlier in the week I found a problem with the instability in the connection between the bluetooth module and MacBook bluetooth. I couldn’t find a solution online but people have been reported to have similar issues. I recently found the solution to maintain the connection of the bluetooth to Mac so I have written the python code to test on sending and reading data from the bluetooth. However, the data is not being sent nor received on the python and arduino side. I am planning to test the pin transmission using an oscilloscope on Monday. If the problem is not resolved I will go ask for advice from the Arduino class professor for some guidance by the middle of this week. EMG sensor has arrived so I plan to test on it with the interface integration this week. For this week’s plan, after getting the basic bluetooth setup to work, we have to figure out data packaging for the incoming left vs right arm that will be sent through the bluetooth module. I also thought of trying out FFT data analysis with Jonathan’s model after this.
Jonathan’s Status Report for 04/09/2022
This week, I worked on differentiating a double winking signal from other signals. I tried different kinds of features obtained from the raw input data to accomplish this. I mainly am trying to find a “second peak” within an input signal and need a convenient statistic to measure this attribute. I attempted to take the input samples and try to find maximums over smaller periods of time, like 0.2 seconds, and observe the top two values obtained from this procedure to use as predicting whether a signal contains a double blink or not. However, it has proven difficult to create linearly separable data that can generalize to a live system using this method. Due to Carnival, I did not have the opportunity to continue troubleshooting this problem and this will be my primary focus next week. Whatever statistical features I test next week can generalize to detecting a triple blink occurrence, also, which should also be projected to be complete next week. I am currently aligned with the schedule as we originally did not intend to do much work during Carnival weekend in our plan.
Team Status Report for 04/09/2022
This week, our group was able to demo a keyboard application with the left wink mapping to the left key and right wink mapping to the right key. Unfortunately, we were unable to show that our EMG sensor could be connected to the interface because the capacitor on the sensor blew out during testing. It is pretty difficult to find the sensors we are using and shipping takes a few weeks; however, we have one coming in the coming week so we can retry the integration. We also spoke to Professor Fedder about this mishap and after taking a look and running some quick tests, it seems like we can just re-solder a capacitor to the sensor and have it working again. Because of this (and with Carnival), we are slightly behind with testing the EMG with the Arduino and integration from last week, but as the EMG sensor has arrived now we will test on it this week.
Updated Schedule
We have made some changes to our schedule. This is what we plan on achieving the last three weeks of this course.
For more clear viewing, our schedule can be found in the under the sheet labeled ‘Gantt.’
Jonathan’s Status Report for 04/02/2022
My main project this week included speeding up the EEG prediction model to work in real time and refactoring the production process of the ML modeling process so we can get models to operate quickly and correctly with the right set of inputs feeding into the model for the real-time system. Now it is very quick and efficient to integrate different ML models into the decisioning process in the real-time system. This included many refactoring processes and reorganizing our file system layout. I also added additional features that plot all the inputs against one another so we can observe on the sample data how well a model is able to perform given how linearly separable or inseparable the input features are. For example, attached below is a plot of two features. The feature on the x-axis is the maximum reading of the AF4 electrode for each sample taken and the feature on the y-axis is the ratio of the maximums of AF3 and AF4 subtracted by the mean. These features are selected because they probably correlate with whether a right wink occurred, where we would see a lower maximum ratio, corresponding to a lot of activity in the right node and little in the left node (hence, minimizing the ratio of AF3 and AF4) and this generated plot gives us a rough idea that using the ratio to separate and predict right winking is pretty closely linearly separable. Hence, we have better confidence our model is predicting using good features. I am on track with our team’s progress goals, and plan to implement the double blink and triple blink detection models and a finite state machine to control how and when to apply a wink detection model we have already trained versus applying a double blink detection model. Furthermore, we need some data samples of plain noise to be able to alert our application if it should attempt to classify real-time data, as to not create a bunch of false predictions due to a bad connection.
Jean’s Status Report for 04/02/2022
This week I have been focusing on two things, first is reading the research paper to help find extra classification parameters to feed into our ML classification model for Jonathan. I have found a couple of potential features/ parameters from our data that we can extract the details can be found here: https://docs.google.com/document/d/1DWQb7xx60W7LE4RCIq6Q3eSEznv9ImYWsoFrphL3Hy8/edit?usp=sharing
In the ideas column are what I had asked Jonathan to implement. We tried the kurtosis one for this week and will move on to trying FFT features next week. The result that Jonathan said was that the kurtosis brought our accuracy up. I will try out more features in this upcoming week/the week after. The other part was that since I got pyfirmata to work out, I had been implementing the bluetooth transfers and some calibration for EMG. I hope to get this ready by next week. Since there are multiple components that I am in charge of, I think I am on track for everything except for FFT which Jonathan and I planned to do after the Demo.