For our fans,

We’ve finished the final design report, which you can check out here!

• Wrote final report document.
• Experimented with Apple HomePod and was unsuccessful in jamming it with our program.
• Experimented with the directional mic and our computers are unable to play from the speaker while having an output device (the directional mic) attached to the headphone jack.
• Measured the final volume of our exploit’s output.

• Finished up our final design report
• Tested directional microphone, which doesn’t have correct driver configurations to let us read in and output sound
• Tested HomePod, which is much more resilient to our exploit. We chalk this up to a much more robust system of speakers designed to more effectively parse out sound than a low-power, single-mic, always-on system.

Last week, Eugene delivered our final presentation. Find the slides here!

This is late, but here’s the work that I put in last week:

• Reimplemented the python workflow into MATLAB using the volume threshold loop
• Tweaked parameters for performance
• Investigated cost tradeoffs with sound() v. audioplayer() in MATLAB to meet latency bounds

• Met with Prof. Stern wrt how to proceed in our attack
• Helped Spencer a little on the Matlab rewriting of the code.
• Developed a testing framework in Matlab. Laid out a clear design for the metrics and validation component for our project.
• Worked on the final presentation slides.

• Met with Prof. Stern wrt how to proceed in our attack
• Rewrote Matlab version of attack code
• Finished various dtw + mfcc implementations
  • Constantly polling in 30ms frames vs. triggering only at a given threshold
  • DTW incorporation causes much lower false positive rate. Can play music or some verbal sounds without triggering automatically, which is far better than our midpoint demo.
  • Seems a little bit too slow running in practice for jamming an iPhone. Could be due to my hardware being older than Eugene’s since the attack seems to work on his computer.

• Met with Stern on Wednesday to try to fix issues with our MFCC + DTW code.
• The librosa library we were using in Python does not provide enough options for tweaking the algorithm to our specific application, and hence we decided to switch to MATLAB after a second meeting with Professor Stern.
• Verification of our algorithm requires the use of spectrograms, and we are currently having trouble replicating spectrograms in MATLAB for audio samples whose spectrograms are already known.

 

• Met with Prof. Stern multiple times to try resolving the issues we had with our pipeline.
• Verification of our results with the Librosa MFCC + FastDTW chain is difficult due to the untransparent nature of the code.
• We switched to Matlab to try making a more transparent, verifiable, and well supported pipeline.
• Currently we are having trouble creating spectrograms that verify the results that we care about – the warped data should have a similar spectrogram to the reference sample to be warped, with the utterances in the same locations.
• Will continue to explore fixes with this pipeline.

• After meeting with Stern on Wednesday, I spent time trying to apply DTW to MFCC coefficients to identify how to map between two signals. Initial results proved unfruitful.
• Following Spencer’s and Cyrus’ meeting with Stern on Friday, I tried to re-implement our code in MATLAB for more robust tools, and more support + documentation.
• Barring installation errors with MATLAB, we are currently blocked on interpreting spectrogram results so will need to meet with Stern (again) to resolve.