Bobbie

Done this week

I successfully reduced the distortion in the SDW-MWF implementation. During lab on Monday, Professor Sullivan helped identify the problem as a windowing issue: previously there was no overlap between windows, which caused boundary effects at the edge of each window. Using a 50% window overlap drastically reduced the distortion. This is at the tradeoff of now needing to process the next (overlapping) frame before producing output, which adds 5-9 ms delay to the output. This is within our timing budget, so it is okay. The new output is below.

For the wearable mic array mount, I ordered this phone chest mount from Amazon: https://www.amazon.com/Samsung-Holder-Record-Awesome-Action/dp/B00MYN0CGI/ref=cm_cr_arp_d_product_top?ie=UTF8

This will serve as the basic model of our wearable mount. The main problem we anticipate from this mount is rotation about the central mounting point. Ideally there should be two solid pieces upon which the mics are mounted, with straps connecting as well. This will consist of:

1. Two rigid mount holders (metal, laser-cut acrylic, or similar)
2. Webbing
3. Buckles/straps

I also wrote code to generate the HINT test recordings. However, this is not completely practical to use yet, because there is not an interface for quickly switching between various SNRs – this is needed because the HINT is an adaptive test, switching SNR levels based on the subject’s  success rates.

Goals for next week

• Build mount for microphones to attach to the purchased chest harness
• Create interface for running HINT given recordings

Kevin

Done this week

One of my goals for this week was to come up with a finalize an initial physical setup for the microphone array.  I decided that for a linear array of n omnidirectional microphones with equal inter element spacing, the distance between each microphone element should be d=c/(2*f0)where c is the speed of sound and f0 is the midband frequency.  Because the human voice frequency in telephony ranges from approximately 300 Hz to 3400 Hz, I chose 1800 Hz to be the midband frequency.  As a result, the distance between each microphone was chosen to be 9.5cm apart.  This means with a 4 microphone setup, we would require an array of 38 cm in length.

Additionally, regarding how each microphone input would be processed, the delay Tn between the desired signal arriving at the nth microphone and the signal arriving at the first microphone (at the origin of the coordinate system) is Tn =(n-1)*d*sin(ø)/c where d=9.5 cm and c is the speed of sound (343 m/s).

For signal processing, I have an initial MATLAB program which implements Frost Beamforming to try on our microphone setup.  I have not done extensive testing yet on the adaptive beamforming code. This following week will be spent testing and modifying the beamforming code with (hopefully) real microphone input signals.

On the hardware side of things, I have worked in lab with Sean to create a temporary microphone setup for testing using microphones that were available from the past. The microphones we used for now are PM: WM-55D103.  The specific circuit we made for each condenser microphone is shown in the figure below for testing.

Goals for next week

• Start building a setup rig with the harness that Bobbie purchased to create wearable microphone array
• Test adaptive beamforming algorithm on ideal generated signals
  • Possibly move to real-input signals

 

Sean

Done this Week

This week, we put together a circuit for the microphones Professor Sullivan gave us and successfully got an analog signal read on the Teensy board. I just used simple analog read functions from the Arduino library to read sound levels. I also starting writing code using the PDM library to process the input from our surface-mount microphones

We decided to go forward with a rechargeable battery pack for external power, pictured below. In order to prevent damaging the board from supplying USB power and external power, I cut the trace between the two pads, to isolate the voltages. I will also cut the power line of the USB cord, just in case the trace isn’t completely cut.

I found tutorials for outputting high speed clocks from the Teensy, and will talk to Professor Sullivan more about these methods.

Goals for Next Week

• Read digital signal on board
• Decide on mic driver clock method based on usability, jitter, and accuracy

Bobbie

Done this week

This week I implemented speech-distortion weighted multi-channel Wiener filtering as described in “Multichannel Filtering For Optimum Noise Reduction In Microphone Arrays” (Florêncio and Malvar, 2001). The results are very strong in reducing noise, but also introduce a very noticeable distortion in the output signal. See the attached audio for an example.

This implementation processes the audio frame-by-frame, writing results to a buffer. For now, the buffer is simply written to file at script completion, but in an actual program it would be read from continuously for output. The 38 seconds of input were processed in ~3 seconds including file reading and writing; this implies that processing a frame takes at most about 70 milliseconds, which is within our time budget. The result shown involved some tweaking of the parameter P as well as the parameters for the least mean squares adaptive filter. I don’t see any glaring errors in my implementation of the algorithm, but the distortion levels are currently quite high and distracting. I will continue looking at this and trying to reduce the distortion, because the noise is reduced by about 24 dB.

Original input:

Post-MWF output:

I also generated some working C code using Matlab Coder. This was quite straightforward to do; Matlab Coder was able to automatically detect the types based on a script which used the relevant functions. We have the option of using Matlab Embedded Coder to generate directly for a ARM Cortex-M target, which may run faster because it will optimize for the particular processor we are using. However, we are not over on our timing budget yet, and it is useful to be able to see the C source code, so we will just use source code generation for now.

 

Goals for next week

• Reduce the distortion in the SDW-MWF implementation to a more manageable level
• Generate one HINT test to be run on the test environment
• Draft a parts list and diagram for the wearable mic array mount

Kevin

Done this week

This week I looked into how our team would go from our basic Matlab implementation, which consisted of LMS noise cancellation using a reference and primary signal, to using a microphone array.  After looking through research papers, adaptive beamforming seemed to be a very promising.

The basic principle of adaptive beamforming is as follows.  Beamforming microphone arrays are spatial filters that take multiple microphone signals as input and combine them to a single output signal. Usually the combined output is calculated by filtering each microphone signal through a digital FIR filter and summing the output of all filters as shown in the figure. The filters are designed so that their output add constructively when sound is coming from a specific direction (main lobe) and add destructively when sound is coming from all other directions. This creates the spatial filtering effect of focusing at the sound that is coming from the main lobe direction while attenuating sounds coming from all other directions.

Goals for next week

• Have a working implementation of beamforming
• Build temporary microphone circuit for testing (soldering microphones and designing circuit)
  • Test on microphones that we received in lab
• Design actual microphone array for wearable
  • (distance between microphones, what materials for mounting, etc.)

Sean

Done this Week

This week I fixed the PCB board so that we can order it through PCBway. This included manually adding layers that weren’t generated by the Eagle software and changing file encodings. I will order that as soon as I hear back from Quinn.

I have starting writing code for the Teensy to get simple inputs from the digital pins, where we will soon be inputting a signal from the microphones given to us by Professor Sullivan.

I did research on outputting a low-noise clock from the Teensy board, which can be done by clock-dividing the 16MHz I2S master clock. I also researched pulse density modulation on the Teensy board, which I found is included in the Audio library for the Teensy. Finally, I looked into power options for the Teensy. For now, we will use USB power, but the final product will need to have a portable battery pack, which we can set up with 2 or 3 AA batteries. Ideally, we’ll use a lower-profile, rechargeable battery for convenience and wearability.

Goals for Next Week

• Get signal through board using larger microphones
• Generate clock output on Teensy
• Begin using PDM library on Teensy
• Connect Teensy Audio Shield for audio output to headphones
• Figure out what method to use for external power so as not to damage the board when it is hooked up to the USB cable and an external power source.

Schedule

We have just missed a couple milestones on our schedule.

We have not sent a signal through the mics into the hardware system yet. The main delay here has been the very small mics we initially ordered and follow-on issues with PCB design. The PCB issues have been fixed and we should be able to order this week (waiting to hear back from Quinn). In the meantime, on Monday Sean will put together the larger microphones that Professor Sullivan lent us and get an actual input signal by then.

We also have not yet created a HINT Test which we can run on our test environment yet. This is mainly because there is not much point in creating one when the microphone-to-hardware connection to receive the signal does not exist yet. Bobbie will do this over the coming week.

Although these milestones were missed, we are not far behind in the project overall. This is because other non-blocked tasks in signal processing made progress when the tasks for the milestones above were blocked.

As another schedule-related concern, Sean will be travelling most of this week for interviews, and that may put his work slightly behind schedule. He will work diligently while he is gone to try to keep the project moving forward.

Bobbie

Done this week

This week I generated test audio samples which would simulate the delays involved in sound arriving at our microphone array, using the Matlab script I wrote last week.

 

I also explored alternative signal processing techniques beyond adaptive nulling which could also be useful in our project. The most promising of these were Speech-distortion weighted multi-channel Wiener filtering (Ngo 2011), and the acoustic rake receiver (Dokmanic, Scheibler, and Vetterli 2015).

Over the course of this research I identified a few common useful utilities and prepared them for use in our project. This includes implementing the short-time Fourier transform and its inverse in Matlab, and finding the documentation for the Matlab “snr” function.

Goals for next week

• Implement SDW-MWF in Matlab and evaluate its performance relative to our existing Matlab code.
• Generate a working C implementation of the better-performing noise cancellation algorithm using Matlab Coder.

Kevin

Done this week

This week I explored the specific use case of white-noise adaptive cancellation with the implementation currently in place.  More specifically, trying to pinpoint the conditions that cause the filtering to work well vs. not well.

 

Additionally, I worked with Bobbie to come up with a quantitative way to analyze our results/ANC output and compare it to our desired signal.  We settled on using the Matlab “snr” function to evaluate the signal-to-noise ratio in our algorithm output relative to the original signal.

 

Also, I emailed professor Cameron Riviere to meet as a group and discuss his previous experience with adaptive noise cancelling.  He is a professor from the Robotics department with whom I am currently taking a sensors and sensing class with. We believe he will be a good resource for creating an effective actual microphone array as well as being able to receive feedback on our ANC implementation and figuring out how to integrate our current basic MATLAB implementation with a microphone array.  His previous ANC experience does not involve audio-signal inputs, however we should still be able to receive helpful feedback.

Goals for next week

• Create a concrete geometric layout for how we want the microphone array with justification
• Meet with professor Riviere to get a clearer direction on future endeavors
  • Clear concept on how to integrate our basic ANC code with multiple reference microphones.  (i.e. how to apply ANC to multiple reference signals)
  • Feedback on current ANC implementation
  • Feedback on our current testing environment

Questions to explore

• How will multiple different noise signals affect our reference signals?
• What do we do if our primary signal has a noise input that is much greater in magnitude (10+ dB) than the signal desired?
  • Our guess would be that this is not as important of a use-case because even a person without hearing impairments would have a difficult time in this scenario

Sean

Done this week

This week I noticed some crossing over on the routing microphone surface-mount PCB that I designed in Eagle. It looks like it might not be necessary, so I want to take a look at these and change the routing to avoid crossing over if possible.

I also did some research into audio on the Teensy board and found the Teensy Audio System Design Tool, which provides a GUI for drawing audio block diagrams and exporting Arduino code.

While our signal processing is very likely to be too complex to model graphically using these blocks, this will still be a useful tool that we can use to generate code and simplify getting audio in and out of the Teensy board.

Goals for next week

• Finalize and send out the microphone mount PCB
• Get simple input and output through the Teensy Audio Shield

 

Bobbie

Done this week

This week we received the parts for the test environment (minus the mannequin) and got the test environment set up and outputting. This required some research into getting IEC958 (S/PDIF) devices working with Ubuntu, but I got it after a little trial and error (images on next page).

 

In the images you can see that the speakers are quite close to each other. We might need some extension cables to get the speakers farther apart, but I think we should wait until we see if that’s a problem or not. Until we get the mic hardware set up, we can use the Matlab script I wrote last week to simulate sounds from this speaker setup.

Goals for next week

1. Generate some test files using Matlab speaker geometry script (this should be very fast)
2. Start working on signal processing side of things

 

Kevin

Done this week

This week we started by collecting speech files to test in a basic MATLAB simulation.  We now have approximately 100 test files of different people speaking syllables/words/phrases that we can use for not only our MATLAB simulation but also our final hearing test.  Additionally, we have started with our basic MATLAB simulations to narrow down the specific adaptive noise cancellation methods we will use. After attempting and tweaking multiple different variations of ANC, we decided to start with a simple Least-Means-Squared (LMS) ANC algorithm that would be both fast and simple to implement.  

 

Below are two examples of our algorithm on two different test signals.  

We found that this algorithm works much better when noise (reference) does not greatly affect desired signal (pretty obvious).  Additionally, the two greatest factors in accuracy of our cancelled signal depended on both magnitude and period of the noise signal (ex. If reference signal has much smaller period than desired, you get cleaner cancelled signal).  

Goals for next week

1. Refine current ANC algorithm to handle more use cases
2. Start considering how to implement ANC algorithm with multiple microphones

Sean

Done this Week

This week we received our surface-mount microphones. They are extremely small, so we need to create a PCB that will allow us to attach wires to it for testing. This week I designed a first draft of the board in Eagle with a premade solder footprint of the INMP621 microphone and attached it to screw-in wire pins. This will make it easy for us to try different wirings in the building phase. Eventually, I will design a similar PCB with solder pads so we can make the device smaller.

 

Goals for this Week

Sam said he could get me an Altium license, and I would like to try designing the PCB on there. With Eagle, I can’t rearrange the pins in the design, which is fine for the build testing phase, but may become a problem when we try to design a low-profile device.

 

I also would like to start programming on the Teensy board with the Arduino IDE. I will start out with simple audio input/output through the Teensy Audio Shield. I will also start researching libraries we can use to make audio processing easier and faster on the Teensy. Kevin and I will work together to implement an algorithm that is a good balance of effective and efficient. The LMS algorithm he is currently testing is fast, but may have some trouble working in our desired testing environment, a loud room, since the frequencies of the desired signal and the noise will be similar, since both will be human voices.