This week I finished the basic proof of concept for the posture detector. Using Google’s BlazeNet pose estimation model, I was able to implement some basic thresholding on a single shoulder landmark to determine when the user is slouching. The video below shows myself sitting up straight briefly so the program can take a “snapshot” of good posture, then whenever I slouch I receive a message at the terminal telling me to fix my posture. If I don’t fix it, I keep receiving messages every 2 seconds. Once I fix my posture, the messages stop.
The link to the video on google drive is here.
I simply calculated the distance between the shoulder landmark in each frame and the “reference” frame showing myself having good posture which was captured at the start of the video. Some problems with this is that it will produce many false positives. If I shift left to right then I will get notified even if my posture remains good. For this reason, next week I will begin training a neural network for posture detection. The ability of a neural network to learn a rich and complex decision boundary makes it suitable for this application. When classifying posture in a person, many types of movement of landmarks on the body do not correspond to bad posture. I hope that the neural network will be able to learn to distinguish between movement that hints at bad posture and movement that is ok. I will be using a google co-lab notebook to train a simple neural network.
Another benefit of the neural network implementation is that our team simply has to learn from posture research what good posture is, take many photos of ourselves with that good posture, and do the same for bad posture. The neural network will do the hard work of learning what the difference is.
I am on schedule.