This week I gathered more formal piano playing videos from Professor Dueck’s students and did some analysis on them. More specifically, I took tense/ non tense recordings with the pianists, then post processed the recordings to gather wrist angle data that will help inform future iterations of the tension algorithm.
During the session, I recorded the same tense and non-tense technique playing as usual. I also recorded a side angle for extra data in addition to our current above angle in case it provided a clearer difference between tense and non-tense playing. I then trimmed videos to be split by exercise and sorted them into tense/ non tense groups.
After taking and processing the recordings, I wrote a script that would record wrist angle over the course of the entire clip and output the data as a text file to an output directory. I then automated it to run on all of the recordings. Finally, I graphed the wrist angle data for the tense/non tense playing for each exercise and compared them to one another.
Tense vs. Non tense wrist angles for different exercises
From the graphs, I was able to confirm that tense playing correlates to less change in wrist deviation over time. I was also able to confirm what I suspected earlier—that playing with hand span is smaller means smaller differences in wrist deviation between tense and non tense playing. This explains why the previous algorithm worked for exercises involving larger hand spans (arpeggios & chords), but didn’t work for exercises involving smaller hand spans (scales & Hanon).
I do have some concerns that these findings aren’t 100% sound. For some of the recordings, the built in Mediapipe hand labeling would sometimes mislabel the hands, causing angle data between hands to mix. While I don’t think this has a large impact on the metrics I’m looking for, I do think having a better way to label hands would improve the overall robustness of our system. Additionally, only two students were able to make our session last Wednesday, meaning that these findings may be specific to only these two students. I’ll gather more data with the remaining students this coming Wednesday to verify if this is true or not.
I’ll look more into how to use these findings to write a new tension algorithm/ modify our old one to work for smaller hand span exercises tomorrow. I’ll also analyze the side view recordings more in depth. The remainder of my time will be spent preparing for the demo the following week and helping out with integration.