Category: Team’s Status Reports

This week and last week, the team started doing testing, verification, and metric collection alongside consistent integration and improvement of our system. All of the risks and challenges throughout the semester have been handled and at this point we are just working on refining our product to function as well as possible for the final demo. We are mostly doing testing and making small system changes and adjustments to meet/surpass our requirements. The schedule and design for our project is still the same, and we will be focusing on finishing the final deliverables in the coming week.

This week, the team reflected on the feedback for our Interim Demo, most of which was incorporating some quality of life and smoothness changes into our design as well as being better prepared to explain our system and responsibilities for the final demo. The team has also transitioned from pure implementation to begin focusing on observing tradeoffs and collecting metrics before the final presentation, and moreso for the final report. The biggest risk our team has is gathering enough information to present as tradeoffs, specifically for creating a tradeoff graph. The development of our system is still proceeding nicely as we continue to improve our gesture model accuracy and implement smoothness changes to our mouse functions, but we have yet to see how smoothly our metric collection progresses. Our design and schedule are still the same and we are on track to present some tradeoffs and metrics during the final presentation after Thanksgiving break.

This week, the team finished a successful Interim Demo! We were able to show off a large chunk of our system’s functionality, namely mouse movement and left mouse clicking/dragging, and received great feedback and suggestions with which to proceed. With this deadline out of the way and looking forward towards the final presentation and final report, the biggest risk in our path is dealing with testing and verification. Now that we have a working system, we can start with gathering testing metrics and preparing user stories for future testing.

Our system has no design changes although we are all now making new small changes to improve the smoothness and functionality of the system. We did however discover a bug in our schedule thanks to the professors! Here is an updated and fixed version of our schedule.

Schedule

This week, the whole team continued to work on the deliverables that we plan to show during the Interim Demo. There was more collaboration and discussion between team members this week as we started to integrate our components together. The integration of pose estimation and mouse movement is already functional but can still be fine tuned. Training of the gesture recognition model using pose estimation has also begun and is progressing smoothly. At this point, the biggest risks are if our implementation that we have committed to can meet the requirements and quantitative metrics that we set for ourselves. Hopefully through the Interim Demo we can receive feedback about if any aspect of our project needs to be rescoped or if there should be other considerations we have to make. Currently the system design and schedule are the same and we are working towards preparing a successful Interim Demo.

This week, the team came together to discuss our individual progress and to make plans going forward towards future deadlines, especially the Interim Demo. As mentioned last week, now that the team has had time to work on the actual implementation of the project, we decided to update our schedule to more accurately reflect the tasks and timeframes.

Here is the Updated Schedule.

Additionally, we also decided on a design change for our project. Originally, we planned on feeding full image data directly from our input camera into the gesture recognition model. However, since our approach for hand detection involved using pose estimation, which put landmark coordinates onto the detected hand in each image, we decided to instead use these landmark coordinates to train our machine learning gesture recognition model instead. All of the image data in the model dataset would first be put through our pose estimation module to obtain landmark coordinates on the hands of each image, and these coordinates would be passed into the model for training and testing. This should allow for a simpler model that can be trained quicker and produce more accurate results, since a set of landmark coordinates is much simpler than pure image data. This updated design choice is reflected in our schedule with an earlier pose estimation integration task that we are all involved in.

As we near the end of our project, integration does not seem to be as daunting of a risk and instead we need to plan ahead for how we will carry out our testing and verification. A new bigger risk now is for us to consider how to measure the metrics we outlined in the project requirements. For now, we will focus on finishing our planned product for the Interim Demo and start with testing on this interim product as we continue towards our final product.

The past 2 weeks, the team focused on completing the Design Review Report and also on working on our individual portions of the project. There may be some individual circumstances for team members (Andrew’s injury) that require a change of schedule and responsibilities. This is a risk that we did not plan for but we plan on dealing with by seeing if Alan can help contribute to Andrew’s project responsibility in the immediate future. There are no design choices so far but the schedule will be updated for next week’s status report depending on how things proceed this coming week.

As a team, our biggest focus this week was learning from the Design Review presentations and reflecting on the feedback we received for our own presentation. The team discussed the feedback and how we would incorporate it as we began writing our Design Review report. Currently, as pointed out in our feedback, we do not have a lot of preliminary results so a lot of initial parameters such as distance from the camera and having people in the background will likely be the biggest risk that we would have to work around. As we wrap up our Design Review report in the coming week, we will also begin with creating the hand detection and gesture recognition components and testing them, as we should have our needed AWS credits and camera by then. The design and schedule are still the same although we will provide more details in the Design Review report.

This week, the team as a whole mostly worked on reflecting on the meeting where we reviewed our Proposal and developing our Design Review slides. Individually, we continued to do our own research into picking specific tools we would use to work on our assigned big sections of the project. Most of our work is independent and mostly done to test out different software or models for when we start developing the actual product. A new risk that was brought up was the amount of time it might take to train our model for gesture recognition as well as the amount of overhead needed to get this connected to our cursor API. This risk may impact our requirement for latency in the long term, but as we have not started connecting components we cannot make preliminary measurements yet. The system design is still the same but we are refining it in our Design Review. The schedule is also fine as it is, but we foresee that after the Design Review we may need to make some changes as we actually start getting into the real production work.

This week, our team gave our proposal presentation and reviewed the feedback later on in the week. We started on our individual responsibilities outlined in the schedule with testing out functionality of the hand detection library, exploring datasets for the gesture detection, and looking through documentation for the integration with cursor. It seems like the hand detection library should work very well for us and give us all the information we need to encode hand position as well as potentially gestures. Currently, the biggest risk is still the future integration we are foreseeing between all of our components. There have been no changes to the design and no changes to the schedule as of yet, and we are still comfortably on schedule.

This week, our team finalized our project idea, the Virtual Whiteboard. After submitting the abstract, we met with Professor Mukherjee and TA Tao Jin to discuss our project further. We talked about narrowing down the use case, including more detailed requirements, and the technology we could use to support our design. We discussed the possibilities of using IMU, IR sensors, and CV. After that our team started working on our Proposal slides for the upcoming presentation. We looked into further details for specific equipment and datasets to use for our CV approach. The biggest risk in our project right now is making sure that the approach we chose (CV) ends up working. To ensure things go smoothly, we are continuing to plan out the development of our project through the proposal. The project is currently on schedule.