Month: November 2019

Spent the week finishing up the implementation of Wad.js functions and helping test the noise filter so that it can be added to the main pipeline.

Still need to integrate the digital signal processing part with the backend.

Have slack coming up so we should be able to catch up,over thanksgiving break.

Since we decided to not use the MATLAB engine, I have been converting my code over to Python. I initially ran into some trouble getting all the libraries set up for some of the helper modules, but all of the functionality of the MATLAB algorithm has been reimplemented with NumPy, SciPy, Matplotlib, and Pandas in Python. The next step is to integrate it with the backend and get the web app working. I will be in Pittsburgh over break so I plan to use that time to make sure everything runs smoothly before our initial demo and presentation.

• Spent the week building multiple passive high pass filters, built high pass filters with the same cutoff frequencies and had the output of one cascade into the input of another. There are four high pass filters connected each filter was designed to have a 40Hz high pass filter cutoff frequency. Then connected the microphone with the low pass filter that was already built, then sent the output of the low pass filter to the laptop. The output sent to the laptop sounded very low when played on the laptop.
• The schedule is on progress
• The plan for the upcoming weeks is to work on amplifying the output of the circuit with an op-amp and connecting the high pass filter with the rest of the circuit. After the low pass filter, the output can be amplified. Then by using the 3.5mm breakout, the 3.5mm 2-sided cable, and the 3.5mm to USB adapter the output can be sent to the laptop.

This week I tested if the filtered mic works with the laptop/webapp interface properly, as part of combining all our components. I did not get to a stage where I could combine the DSP part into the application due to problems from last week, a team member being sick, and having a big project(USB) due for 341. As per the schedule there is still a couple of days left to finish combining it but realistically it is a goal for next week, and I have adjusted the schedule by 2 days.

Other than that I did work on the frontend by implementing Wad.js functions and we now have direct mic access.

Plan for next week is to combine the DSP part and store the microphone data into the database in an efficient manner.

• Improved on the filter circuit by using four of the same low pass filters with the same cutoff frequency of 14kHZ and cascading the output of one filter into the input of another filter, the system of low pass filters reduced the peak to peak voltage of the output signal of the system to less than 10 percent of the peak to peak of the input signal to the system. Also, tested the integration of the microphone, two 3.5mm breakouts, two-sided aux cable, aux to USB adapter, and laptop. The components were able to properly integrate and get sound from the mic onto the laptop.
• The schedule is on progress.
• Plans for the upcoming week is to now integrate the microphone with the filter circuit as an input and then integrate the output of the filter circuit with the other components like the laptop. Also, plan to work on the high pass filter part of the circuit.

Got beat detection to work better with higher thresholds. This made the detected onsets less frequent, but more clear and easier to distinguish. However, we ran into a problem with the MATLAB Engine on the servers. Due to some documentation errors, my partner was not able to use it. Therefore, I am going to be switching to Python. As of now, I have converted the main voice tempo detector function to Python and am working on getting the helper modules converted. This will probably take me until next week to finish. Once done, I plan to begin implementing this on the servers and get it running on the web app. Even though this conversion will take extra time, I have the beat detection done early, and the addition of built-in slack means I can finish before the final demo.

Was behind schedule due to trouble with MATLAB Engine API (discrepancy between official documentation and implementation).  To resolve this the MATLAB code is being converted to Python and I am working on converting the environment to a previous Python version as well. It is worth pursuing both avenues as the Python version not only acts as a safeguard but also allows us to benchmark the performance between the two, to get some quantative metrics.

I have also updated my schedule with more detail to focus efforts in these last 4 weeks, based on the demo date.

• Spent time testing the filter circuit. After testing founding that the filter was starting to cut off around 60kHz, but the desire is to cut off at 14kHz. Started to build the same resistor and capacitor configuration as the current low pass filter. The plan is that with multiple low pass filters with the same configuration, the output of one low pass filter can be feed into the input of another low pass filter in order to achieve the desired frequency cut off.
• The progress is still on schedule
• The plan for the coming week is to continue working on the filters and solder the other 3.5mm jack breakout. Also, connect the mic, the two breakouts, the 2 sided cable, the 3.5mm jack to the usb converter, and the laptop just to see if the parts allow the mic to properly connect to the laptop

This week I tested the MATLAB Python Engine API, which will be used to integrate the DSP portion into the webapp. I am behind schedule on the whole GUI for the app but I aim to catch up this week.

This week I wrapped up the voice tempo detection. There are still areas to optimize, but for the most part it is able to detect on-beat rapping. I rectify and smooth the audio before running it through an onset detection algorithm. Then, I calculate where beats should be based on the bpm given by the backing track beat detector. Here is an example of amateur rap where it is able to detect on-beat rapping.

 

The magenta lines indicate hit beats and green lines indicate missed beats. You can see gaps in hits where the rapper is pausing and taking breaks. In the coming weeks I plan to begin integrating this into the backend and have it working on live-audio in the web app.