Principles of engineering, science and mathematics
- Short-Time Fourier Transform of a time domain signal. (18-290)
- Utilization of frequency domain vs time domain to analyze different signals for different purposes (18-290)
- Time domain for rhythm processor: we can analyze the start of a new note by detecting a sudden drop and rise in energy. This will allow us to detect the different length of each note and when a note ends leading to the start of a new note.
- Frequency domain for frequency processor. Thanks to the Short-Time Fourier Transform we will be able to detect the average frequency over a specific interval (of the length of our box size in the STFT), and with this frequency we can detect what note it is being played.
- Signal-to-noise ratio (18-100, 18-290)
- Our system will reject audios that have a poor low SNR.
- Object Oriented Programming (15-112)
- We will need to have different objects in our code for the characteristics of the fourier transforms, notes, audio files, etc.
- Web Applications (17-437)
- Our system will require a web application for the users to access the system from their laptops.
Significant Risks
Unfortunately this week we have fallen a little bit off-schedule. One of our team members was sick for the whole week, therefore they were unable to make progress on the project. We also had issues with searching for software libraries that will allow us to test our frequency processor due to having installation issues which took longer than expected. For example, we tried installing a library called CREPE in python that would give us the frequencies given an audio sample. We were planning on using this to compare it with our frequency processor output. The problem is to install this it required tensorflow and installing tensorflow was giving us a lot of issues. After unsuccessfully trying to install CREPE, we decided to move away from CREPE and found another library called Parselmouth which was very easy to install. Therefore, we will be using Parselmouth to test our frequency processor. This library allows us to get all the frequencies of a specific audio, therefore we could compare the output of this library to the output of our own processor.
Therefore the risk that I see is us not being able to finish everything on-time. However, I think that by putting in more work in the upcoming weeks we should be able to get back on track. Working on the Design Review helped us reorganize ourselves and adjust our plans based on the setbacks of this week.
Changes made to design
We have not made any changes to our overall design, but we have worked more on developing the specifics of the sub-processors and the transcription engine. We have determined a method of isolating the pitches at specific segments of the signal, a binary mapping system to determine when a new key-press occurs, and a design structure that contains this information in a way that is easily comprehensible by the VexFlow library.
Updated schedule
We have updated our schedule a little bit, to make the process of building the sub-processors more concurrent and adding some sections for the user-testing part of our project.
Photos about progress
Parselmouth manually examining/testing of pitch over time compared to SciPy (previous method). Parselmouth is more accurate and comprehensible.
