Posted on

Brian Lane’s Weekly Status Report for 10/23

I spent this week setting up some preliminary pytorch scripts for training our model and adapting the pre-built model we have slated to use. Because of a shift in our planning, I now need to assist Andrew in the creation of our pose estimation software, because the hand pose data will now be used when training our model, rather than raw image data.

Following this quick pivot, I will spend next week creating and running a script to apply the hand pose estimation to our data set while training our pre-built model in preparation for our upcoming interim demo.

Posted on

Andrew’s Status Report

Last week, we had mid-semester break so the team did not submit a status report. Our primary focus that week was working on the design review. This past week, we each completed the ethics assignment and discussed our work in class with both the guest speaker as well as other teams in 18-500 as well as 18-555. As for our actual project, the webcam we ordered arrived this week, so I had to dedicate a bit to setting it up on my computer. I’m still finishing writing up the code for pose estimation, but that is nearly complete. I have started writing the calibration process for our range of motion algorithm. I should have that finished within the next couple of days. The main difficulty now is going to be porting the individual code I have written onto an interface that connects with the OS Interface library, but this is taken into account in our integration phase. As a personal update, I have been injured the past two days and have communicated this with the professor. As such, I’ll be a bit stalled these upcoming next few days, so I’ll be a bit more communicative on my end with regards to that with the team and the teaching staff.

Posted on

Team Status Report for 10/23

The past 2 weeks, the team focused on completing the Design Review Report and also on working on our individual portions of the project. There may be some individual circumstances for team members (Andrew’s injury) that require a change of schedule and responsibilities. This is a risk that we did not plan for but we plan on dealing with by seeing if Alan can help contribute to Andrew’s project responsibility in the immediate future. There are no design choices so far but the schedule will be updated for next week’s status report depending on how things proceed this coming week.

Posted on

Alan’s Status Report for 10/23

Status Report for 10/16:

Last week, I mainly worked on writing the Design Review Report. I wrote the abstract and sections I, II, III, VI,  and VII. I also wrote the OS Interface portions of sections IV and V. I formatted the report and added in my teammate’s contributions and then submitted.

Status Report for 10/23:

This week, I had a meeting with Tamal early in the week to discuss the midsemester quantitative feedback that I received. We talked about the Design Review Report, where I learned that the biggest issue was the lack of figures, which will be something we add for the final report. Additionally, we talked about the future of the project and how I could adjust my own personal contributions. We agreed that the OS Interface portion of the project was not too big of a challenge by itself, and that I should instead contribute more to the other two parts of the project (hand detection and gesture recognition). I was also tasked with creating an OS Interface document illustrating the finite state machine as well as the interface for each functional module in the OS Interface. This includes the calibration step and the running system state. I foresee some scheduling changes that will most likely be included in the next week’s status report due to the adjustment of my responsibilities.

Posted on

Brian Lane’s Status Update 10/11

I spent the last week drafting, refining, and presenting our final design presentation, as well as preparing to give said presentation before the class.

Further, I did some more research into gesture recognition and found that many more studies used a pose estimation algorithm to identify hand landmarks and run the estimated pose through their model for gesture detection. This was in contrast to my initial approach, which was to feed in camera data directly. Presented with this new paradigm and upon further consideration this makes sense, as this method eliminates much of the noise of background colors and images and reduces the number of features the model will need to learn.

This week the adaptation of the pre-trained model will begin, as well as work on the final design documents/report. We are still waiting on AWS access to GPUs, which will aid in the speedy training of our model.

Posted on

Andrew’s Status Report 10/10

This week, we came off the design review report presentation and god some good feedback from TAs. For me personally, I am ahead of schedule on MediaPipe handtracking research and am in the process of building visualization tools using Matplotlib so I can more easily work on the pose estimation algorithm. I should be finishing this up by the end of this week as well as finishing up my part in the design report.

Posted on

Team Status Report for 10/9

As a team, our biggest focus this week was learning from the Design Review presentations and reflecting on the feedback we received for our own presentation. The team discussed the feedback and how we would incorporate it as we began writing our Design Review report. Currently, as pointed out in our feedback, we do not have a lot of preliminary results so a lot of initial parameters such as distance from the camera and having people in the background will likely be the biggest risk that we would have to work around. As we wrap up our Design Review report in the coming week, we will also begin with creating the hand detection and gesture recognition components and testing them, as we should have our needed AWS credits and camera by then. The design and schedule are still the same although we will provide more details in the Design Review report.

Posted on

Alan’s Status Report for 10/9

This week, since I was ahead of my personal schedule and with deadlines for the Design Review Report coming up, most of my time was focused on contributing to writing parts of the Design Review Report. I learned a lot from all of the different groups’ presentations and from the instructor feedback and have been working on incorporating suggestions and clarifying points about our project in the Design Review Report. In the next week, I will finish writing my portion of the Design Review Report and then start working with Andrew or Brian on getting their components of the project up and running before continuing with my own OS interface portion.

Posted on

Andrew’s Status Report 10/3

I spent most of this week researching more into the mediapipe library. I tested more with my web cam, and have a better understanding on how the actual hand landmark is parsed and implemented within the cv pipeline. I explored more on how you could use the hand landmark information as well, detecting distance between fingers, distance between hands if multiple are detected on screen, etc. While I may not use this as a feature enhancement dataset for our gesture detection algorithm as we’re going to make that more traditionally with just the image datasets we found online. We also worked as a team on the design review sides as our presentation is coming up this week. We’re polishing up our presentation flow and the main parts we wanted to address from the project proposal presentation that we got feedback on. I am on schedule with my work.

Posted on

Team Status Report for 10/2

This week, the team as a whole mostly worked on reflecting on the meeting where we reviewed our Proposal and developing our Design Review slides. Individually, we continued to do our own research into picking specific tools we would use to work on our assigned big sections of the project. Most of our work is independent and mostly done to test out different software or models for when we start developing the actual product. A new risk that was brought up was the amount of time it might take to train our model for gesture recognition as well as the amount of overhead needed to get this connected to our cursor API. This risk may impact our requirement for latency in the long term, but as we have not started connecting components we cannot make preliminary measurements yet. The system design is still the same but we are refining it in our Design Review. The schedule is also fine as it is, but we foresee that after the Design Review we may need to make some changes as we actually start getting into the real production work.