This week I wrapped up the voice tempo detection. There are still areas to optimize, but for the most part it is able to detect on-beat rapping. I rectify and smooth the audio before running it through an onset detection algorithm. Then, I calculate where beats should be based on the bpm given by the backing track beat detector. Here is an example of amateur rap where it is able to detect on-beat rapping.
The magenta lines indicate hit beats and green lines indicate missed beats. You can see gaps in hits where the rapper is pausing and taking breaks. In the coming weeks I plan to begin integrating this into the backend and have it working on live-audio in the web app.