Kevin
Accomplishments
-
- This week I reviewed the last PCB revision. I placed the order for the PCB through PCBway, same as the previous boards. I also ordered more parts so we have enough to build out 3 of the new PCBs. This should be our last order. We won’t have enough time to buy new parts so I ordered extras of some components that could easily get damaged or lost.
- I worked on improving the keystroke detection using the delta method from earlier. I was able to tune the parameters to get a slightly improved result. However, we switched to using a thresholding method as it seems to perform better.
- I worked with James on collecting longer samples of data. In order to test our clustering performance, we collected about 30 samples from each key on the keyboard. We then experimented with different clustering techniques and features. Kmeans with euclidean distance gave us the best results.
- While collecting the data, we used two boards so that we can also experiment with TDoA data. The TDoA appeared to be working until about one third into the audio clip. At that point the keystrokes moved to 85ms apart from each other, which does not make sense. There may be an issue of adding or dropping samples while transmitting.
- We believe using Cepstral features and TDoA will give us decent clustering results.
Upcoming Work
-
- Next week I will be focusing on supporting the effort of training and clustering. This will involve collecting data tuning parameters.
- The new PCB should arrive this week, so once that arrives I will be putting it together and verifying its functionality.
James
Accomplishments
-
- This week, we worked on collecting long samples of 3-way TDoA data. TDoA between the PCB boards was found have high degrees of separation for non-adjacent keys.
- However, when moving to a much longer audio recording (6 minutes), we found that the audio signal between the two microphones were becoming misaligned, with the same keystroke appearing on one microphone over 80ms before the other. This should not be possible, as that would require a distance difference of 27 meters based on the speed of sound through air. We suspect that one or both of the sensor packages is dropping samples.
- We found a more faster, more noise-resistant method of cracking the substitution cipher problem, using quadgram probability data from http://practicalcryptography.com/. This method was able to decipher a 5500 word cipher within 10 minutes. Noise and unknown word boundaries had minimal effect.
Upcoming work
- I will need to fine tune the clustering parameters to improve clustering accuracy.
Ronit
Accomplishments
- In order to increase the resolution in the TDOA data, we increased the sampling rate from 40kHz to 60kHz. There is an average separation of about 1cm between each key switch on the keyboard. With a higher sampling rate, the number of samples.
- We were able to get good separation of keystrokes using tdoa and cepstral features.
- We tried using 3-way tdoa, however we were not able to collect good data. The ESP32 dev board with the external mic was not properly impedance matched.
- We found an efficient means of solving the substitution cipher using ngrams. We are now able to crack substitution ciphers in less than 10 mins for passages under 400 words.
Upcoming work
- We need to collect 3 way tdoa data to get better separation. So we need to build a 3rd PCB.
- We need to start integration.
Team Status
Accomplishments
-
- This week put out the order for our final PCB.
- We found a way to efficiently decoded the substitution cipher.
- We were able to get good clustering using cepstral and tdoa data.
Upcoming work
In the following weeks, we will need to divert more attention to the machine learning aspects of the project, as well as to fine tune much of the signal processing algorithms in order to be more robust and effective.