Author: ewirthap

What did you personally accomplish this week on the project? Give files or
photos that demonstrate your progress. Prove to the reader that you put sufficient effort into the project over the course of the week (12+ hours).

I implemented gaussian filtering in my test audio generation script. It works on simple sine waves, still need to test with guitar audio samples.

I have decided on using short time Fourier transforms (stft) and rms amplitude detection for pitch and velocity detection respectively.

Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?

I was hoping to accomplish more over break, but did not end up having as much time as I thought I would. I am still on track, slightly behind. There are no unknowns in reading in guitar audio and doing pre-processing, so I do not expect that wrapping up this deliverable will take any significant amount of time.
What deliverables do you hope to complete in the next week?

Pre-processing script reading in guitar audio finalized. Start to implement stft with constant q transform. Implement single stft window as first step, then implement binning stft.

What did you personally accomplish this week on the project? Give files or
photos that demonstrate your progress. Prove to the reader that you put sufficient effort into the project over the course of the week (12+ hours).

I have started using PyAudio to read/write audio files and do some experimentation with sound generation just to see what its capabilities are.

I referenced this article:

https://plainenglish.io/blog/making-a-synth-with-python-oscillators-2cb8e68e9c3b

To see what simple signal processing/signal manipulation techniques I can use to create interesting sounds. I have the capability to read in guitar audio. I still need to ensure my preprocessing steps can be efficiently accomplished, but I do not foresee any issues with preprocessing.

Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?

I have given myself a week for preprocessing audio and a week for testing preprocessing. This means I am not too far behind, but I am hoping to spend much more time next week and have preprocessing and preprocessing testing done early next week so I can start on actual pitch detection. I did not accomplish as much I was hoping to this week (due to other work and sickness). 
What deliverables do you hope to complete in the next week?

Fully working and tested script to read in guitar audio, perform any noise reduction if necessary, and return the signal with note onset detection.

Please list the particular ECE courses (if any) that covered the engineering science and mathematics principles your team used to develop your design? If there are no courses please tells us how you learned those principles over the last week?

I am currently taking Digital Signal Processing (18491). Along with an understanding of the basic signal processing ideas I already have, I researched common ways to do pitch detection. My research is below.

What did you personally accomplish this week on the project? Give files or
photos that demonstrate your progress. Prove to the reader that you put sufficient effort into the project over the course of the week (12+ hours).

I researched my component of transcribing a monophonic guitar into midi. I researched two avenues of this, real time conversion of guitar audio to midi and doing the transcription from a recording (non-real time).

Real time pitch tracking methods and analysis:

http://www.music.mcgill.ca/~ich/classes/mumt611_05/Presentations.old/mumt614pitchtrack.pdf

https://www.cycfi.com/2017/10/fast-and-efficient-pitch-detection/

https://ccrma.stanford.edu/~pdelac/research/MyPublishedPapers/icmc_2001-pitch_best.pdf

I also spent time looking into how to make midi more expressive. Namely supplying meaningful velocity data. Roger Dannenberg has a great paper on this with comparisons of different synth’s velocity and a breakdown of a technical approach for velocity detection.

https://www.cs.cmu.edu/~rbd/papers/velocity-icmc2006.pdf

I also found an interesting master’s project which turns the voice into a midi controller. It implements this via a VST plugin. This seems like an interesting route, but does not apply for non-real time audio to midi conversion:

https://scholarworks.uark.edu/cgi/viewcontent.cgi?article=1013&context=eleguht

and a guide for VSTs:

https://integraudio.com/best-tools-to-develop-vst-plugins/

Another PyAudio midi resource:

https://python.plainenglish.io/build-your-own-python-synthesizer-part-3-162796b7d351

Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?

I am on track, I have a good idea of the use case/design requirements and how I will attempt to meet them in creation of a non-real time audio to midi transcription program using the resources I have collected above.
What deliverables do you hope to complete in the next week?

I hope to order an amp and adc and start development of a PyAudio program which can record a guitar input (keeping in mind the program will need to run on a rasberry pi.

What did you personally accomplish this week on the project? Give files or
photos that demonstrate your progress. Prove to the reader that you put sufficient effort into the project over the course of the week (12+ hours).

Developed project proposal with group. Presented draft project proposal. Met as a team today and decided on new direction for project. Following discussion, I researched options for audio to midi conversion to be able to better redefine user group and scope of project when group is next able to meet:

Found a good source for midi input to Raspberry PI:

https://www.samplerbox.org/makeitelectronic

https://github.com/josephernest/SamplerBox

Audio to MIDI Products:

https://www.sweetwater.com/store/detail/G2MV3–sonuus-g2m-v3-universal-guitar-to-midi-converter/reviews

https://www.sweetwater.com/store/detail/i2M–sonuus-i2m-musicport/reviews

https://www.perfectcircuit.com/kenton-pro-cv-to-midi.html?gclid=CjwKCAiAlp2fBhBPEiwA2Q10Dz8AVSQs6e9ATXALiwCIpGEk7iqlD8WwO-GY1Y1E6YM8pyXWVEK6FhoCIPQQAvD_BwE

Audio to MIDI Software resources:

https://migic.com/

https://www.jamorigin.com/

https://github.com/tiagoft/audio_to_midi

Also found and read the following paper on audio to MIDI conversion:

http://amber.feld.cvut.cz/user/vass/papers/Vass_Poster04.pdf

Which references the following paper which is a very very good resource for signal processing if I end up writing custom audio to MIDI code:

https://www.ee.columbia.edu/~dpwe/papers/Brown91-cqt.pdf

Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?

I am behind schedule. We are coming together as a group and almost have a clear collective idea of what the project will be. Once we nail down specifics of who the user group will be and what the components of the project will be, I will have the ability to start actually prototyping/developing my component. Meeting with the group to nail down the user group and define user/design requirements needs to be happening this coming week.

What deliverables do you hope to complete in the next week?

Hope to work with group to define precise design/user requirements. Hopefully this will happen early enough to give me time to research my component. Minimum goals for this week: precise design/user requirements. My component should be well researched, ideally so I am in a good position for outlining how my component will function/prototyping the following week.