Month: March 2020

Progress

I’m waiting on processing the data we took with openpose, which takes a while on my laptop. Chris said he would use the Xavier to do this, which should be faster.

Deliverables for next week

By next week I want to have all the data processed so Chris and I can test our algorithms on it.

Progress

I am still behind schedule, but catching up slowly.

Updated Gantt Chart and risk management plan

Since Mark has been able to order his glove components and a soldering kit to assemble them, and Chris and I can work on the algorithm remotely, the risks remain the same as in the design document. We anticipate no further problems around fabrication or shipping of parts.

Progress

I’ve begun working on a classifier for the hit gesture by setting thresholds at the angle at the elbow. Other than that, I have not made much progress since we have had to transition to remote work and rescope our project.

Deliverables for next week

By next week, I want to have working algorithms for stomp and hit that can achieve reasonable accuracy on our training samples.

Schedule

I am slowly getting back onto schedule, since I am making progress on the classification algorithms.

Progress Made

Over Spring break, I did not do any work on the project. After returning to Pittsburgh, most of my time was spent on trying to get my life in order so not too much time was spent on the project, other than organizing the data that we collected before break.

Work for this week

I hope to be able to get the LSTM classifier working with reasonable accuracy on the Xavier board by the end of the week. I will work with OpenCV to do hand detection then running the classifier on the images of the hands for these gestures.

Schedule

Currently, I am behind on my schedule by about two weeks. This hopefully will be made up by the fact that I am spending most of my time at home and also that we have a decent amount of slack built into our schedule.

Plan moving forward

I have uploaded my 1 pager explaining what I plan on working on for the rest of the semester.

Progress

We have collected all of the data that we should need for the training and classification of both music creation gestures as well as editing gestures. We collected about 50 samples of each gesture, with multiple people performing these gestures.

Work for this week

We will be working during class time on each of our sections with a Zoom meeting going as our “face to face” time. As travel plans to return are uncertain for Chris as well as Mark currently, we will try to do as much as we can in Pittsburgh. Chris will hand off the Xavier Board to Tony soon.

Chris and Tony will work on their respective classifiers and try to get them to work on the Xavier board hopefully by the week’s end with reasonable accuracy.

Mark will continue to be working on the glove and on the Raspberry Pi.

Schedule

We are still behind schedule trying to figure out how to coordinate our work as well as needing to work on building the classification for the gestures. We still have a decent amount of slack for the rest of the semester (hopefully integration will not take too long) that we could use to make up for the lost time. However, it will be difficult to work as quickly as when we are all together so we may be losing a bit of time there as well.

Progress

Prior to leaving for spring break, I took training samples with Chris and Mark. We took at least 50 samples of each gesture, along with some negative look-alike examples to test the robustness of our algorithms.

Deliverables next week

Chris will give me the Xavier sometime before leaving Pittsburgh, and I want to be able to run the Kinect on the Xavier. We have also split up our responsibilities for the remainder of the semester, so my task is to develop the music creation gestures (stomp, clap, hit). I want to have algorithms for those that can achieve at least 70% accuracy on our training examples.

Schedule

I am still behind schedule, since I need to start the algorithms for the music creation gestures.

Progress

I got the Kinect to work! Using libfreenect2, I could get a video stream working. However, libfreenect does not have joint detection features, so we will still have to use OpenPose or another library for that.

Deliverables next week

By next week I want to figure out how to use the Kinect with the Xavier and either do joint detection with OpenPose or another library.

Schedule

I am running more and more behind schedule. Figuring out how to use all the hardware is taking a while, which is preventing us from developing our action recognition algorithms.

Progress:

I have been able to set up OpenPose on the Xavier Board and it seems to run around 17 fps. Still working on the action recognition algorithm. I was working on trying to test our current algorithms for clap and stomp as well and try to refine the thresholds we are using. Furthermore, I spent lots of time working on the design document which took a bit longer than I hoped.

Schedule:

I believe that I have fallen about a week behind the original schedule with other classes taking up a bit more time than I anticipated this week as well as working on the design document taking a little more time than we had originally planned on the schedule. I should be able to make this up using the first planned slack time during the end of Spring break when I won’t have any other work.