Team Status Report for 04/29/23

What are the most significant risks that could jeopardize the success of the project? How are these risks being managed? What contingency plans are ready?

The biggest risk right now is making sure the whole system works. There were a couple of bugs in the HID driver that require fixing. For example, the compute module was successfully able to classify the input sensor data and attempted to dispatch it to the computer, but sometimes the host computer was not able to receive it. We also experimented with debouncing times so that random gestures aren’t picked up but the action isn’t too slow.

Were any changes made to the existing design of the system (requirements, block diagram, system spec, etc)? Why was this change necessary, what costs does the change incur, and how will these costs be mitigated going forward?

None since last week.

Provide an updated schedule if changes have occurred

No changes.

Team work adjustments

No teamwork adjustments have been made. We are currently on track to accomplish our tasks and are currently all just working on them.

List all unit tests and overall system test carried out for experimentation of the system. List any findings and design changes made from your analysis of test results and other data obtained from the experimentation.

  1. Latency tests – averaged over 20 trials. The finding we found is that using a time-series-based approach is pretty speedy compared to neural networks, which was stated in the presentation.
Measurement Actual Value
Oscilloscope measurement of Pi 4 GPIO pin and sensor detection output 

(overall measurement)

~ 112ms
Sensor data collection ~ 100ms
Model classification time ~ 5ms
Wi-Fi transmission delay with ping tests ~ 5ms
Dispatch time TBD

2. User experience tests. The finding we found is that our initial goals were overestimates as we were able to significantly exceed the target metrics we had, with the exception of the range. This is because we felt like for the best user experience, the configuration should be as hands-free as possible, which is why we switched the communication protocol from local wifi to using the Raspberry Pi as a hotspot. That way, we would not need to reconfigure both the glove and Pi when the user uses the glove in a different location. Even though this came with a trade-off of lesser range, we decided it was alright considering that it still worked at least 10 meters away, which is far enough in a classroom setting.

Requirement Measurement Procedure Target Metric Actual Value
Usability Ease-of-use: User study measuring comfort, setup time, and responsiveness 90% user satisfaction based on survey

Setup time < 5 mins

~4 minutes
Portability Measure weight with a scale.  < 500 g 57g + 34g battery
= 91g
Range Test wireless communication across measured distances Up to 50 m between glove and compute 10 m
Battery Life Run glove module with all sensors and communication running and measure total power consumption up to 1 hour of continuous use ~40 hours

3. Accuracy tests – We tested the zoom-in and out gestures, however, the zoom-in wasn’t as accurate as we had hoped. We found out that the sensor of the index finger wasn’t that responsive to flex, which is particularly important for our zoom-in gesture. Xuan will replace the sensor in hopes that the algorithm will achieve higher accuracy.

Train/test split evaluation

Accuracy of dispatched HID controls

> 90% gesture recognition accuracy 95% for rotation/pan

Zoom in – 65%

Zoom out – 90%

Enable-disable – TBD

Dispatch accuracy TBD

Team Status Report for 04/22/23

What are the most significant risks that could jeopardize the success of the project? How are these risks being managed? What contingency plans are ready?

The biggest risk right now is the setup experience becoming more tedious than anticipated. We are currently exploring different ways to attach flex sensors to the glove that don’t affect the setup experience too significantly.

Were any changes made to the existing design of the system (requirements, block diagram, system spec, etc)? Why was this change necessary, what costs does the change incur, and how will these costs be mitigated going forward?

None since last week.

Provide an updated schedule if changes have occurred

No changes.

Team work adjustments

No teamwork adjustments have been made. We are currently on track to accomplish our tasks and are currently all just working on them.

Team Status Report for 04/08/2023

What are the most significant risks that could jeopardize the success of the project? How are these risks being managed? What contingency plans are ready?

Much like last week, the most significant risk right now is still just the integration between the individual subsystems. We did finish basic system integration as we showed in the interim demo, as we got a full dataflow working from sensor reading to HID dispatch command. However, we still need to work on continuing this integration, but it should be going along as the glove itself was finished this week. We’re also conducting tests in our individual subsystems during the integration process, and more tests are to follow.

Were any changes made to the existing design of the system (requirements, block diagram, system spec, etc)? Why was this change necessary, what costs does the change incur, and how will these costs be mitigated going forward?

None since last week.

Provide an updated schedule if changes have occurred

No changes.

Team work adjustments

No teamwork adjustments have been made. We are currently on track to accomplish our tasks and are currently all just working on them.

Now that you are entering into the verification and validation phase of your project, provide a comprehensive update on what tests you have you run or are planning to run. In particular, how will you analyze the anticipated measured results to verify your contribution to the project meets the engineering design requirements or the use case requirements?

Accuracy will be measured by collecting data from different runs of gestures and comparing the number of samples that are correctly classified.

Performance will be measured in the classification layer by measuring (in Python) how long it takes to classify each window of data that comes in from the glove module.

Team Status Report for 04/01/23

What are the most significant risks that could jeopardize the success of the project? How are these risks being managed? What contingency plans are ready?

The most significant risk right now is still just the integration between the individual subsystems. We have some subsystems integrated together (HID + gesture model). We plan to finish the glove integration tomorrow in time for the interim demo.

Were any changes made to the existing design of the system (requirements, block diagram, system spec, etc)? Why was this change necessary, what costs does the change incur, and how will these costs be mitigated going forward?

None since last week.

Provide an updated schedule if changes have occurred

No changes.

Team work adjustments

No teamwork adjustments have been made. We are currently on track to accomplish our tasks and are currently all just working on them.

Team Status Report for 03/25/23

What are the most significant risks that could jeopardize the success of the project? How are these risks being managed? What contingency plans are ready?

The most significant risk right now is not being able to integrate all the systems in time. We are working to make sure that all our individual subsytems get to a testable state as quickly as possible (aiming for this week) so we can work on integration in time for the interim demo.

Were any changes made to the existing design of the system (requirements, block diagram, system spec, etc)? Why was this change necessary, what costs does the change incur, and how will these costs be mitigated going forward?

None since last week.

Provide an updated schedule if changes have occurred

No changes.

Team work adjustments

No teamwork adjustments have been made. We are currently on track to accomplish our tasks and are currently all just working on them.

Team Status Report for 03/18/23

What are the most significant risks that could jeopardize the success of the project? How are these risks being managed? What contingency plans are ready?

The most significant risk right now remains the same — if the windowed time series analysis turns out to be inaccurate. As described last week, if this doesn’t work, we plan to incorporate more rigorous calibration into the system or possibly restructure the gestures so that they are more distinct from each other. Additionally, we could resort to the neural network approach, although this is the less ideal case because it will likely have a performance penalty and may not necessarily have better accuracy.

Were any changes made to the existing design of the system (requirements, block diagram, system spec, etc)? Why was this change necessary, what costs does the change incur, and how will these costs be mitigated going forward?

None since last week.

Provide an updated schedule if changes have occurred

Team work adjustments

No teamwork adjustments have been made. We are currently on track to accomplish our tasks and are currently all just working on them.

Team Status Report for 03/11/23

What are the most significant risks that could jeopardize the success of the project? How are these risks being managed? What contingency plans are ready?

The most significant risk right now is if the windowed time series analysis turns out to be inaccurate. In case this doesn’t work, we plan to incorporate more rigorous calibration into the system or possibly restructure the gestures so that they are more distinct from each other. Additionally, we could resort to the neural network approach, although this is the less ideal case because it will likely have a performance penalty and may not necessarily have better accuracy.

Were any changes made to the existing design of the system (requirements, block diagram, system spec, etc)? Why was this change necessary, what costs does the change incur, and how will these costs be mitigated going forward?

None since last week.

Provide an updated schedule if changes have occurred

None

Team work adjustments

No teamwork adjustments have been made. We are currently on track to accomplish our tasks, and are currently all just working on them.

Team Status Report for 02/25/23

What are the most significant risks that could jeopardize the success of the project? How are these risks being managed? What contingency plans are ready?

The most significant risk that we have identified right now that could jeopardize the success of the project is if the machine learning turns out to be highly inaccurate or insufficiently performant. To manage this risk, we identified ways to perform the gesture detection using basic time series analysis and a decision tree which will be much faster because it doesn’t depend on a deep neural network and can even be implemented directly on the edge. This gesture detection would simply examine trends (increasing/decreasing) of the different sensor data and match them with the corresponding gestures we identified.

Were any changes made to the existing design of the system (requirements, block diagram, system spec, etc)? Why was this change necessary, what costs does the change incur, and how will these costs be mitigated going forward?

None since last week.

Provide an updated schedule if changes have occurred

The schedule was updated to account for the design report and to shorten the amount of time allocated for smaller tasks such as soldering the circuit.

Team work adjustments

We did not make any reassignments to our list of tasks yet since it currently seems well-balanced. However, we adjusted our schedule to make more of our tasks happen in parallel when they weren’t dependent on each other and also give us more time this week to focus on the design report which we did not originally account for in our schedule.

 

Team Status Report for 02/18/23

Risk Management

The most significant risks that could jeopardize the success of the project are if it turns out that the ML model we use is too slow or inaccurate to create a usable product. In case this happens, our backup plan is to replace the ML model with a heuristic algorithm that analyzes the time-series data as they come in. This would allow us to make certain gesture predictions using a basic decision tree that can easily run on the compute module quickly.

Changes/Updates

The block diagram was updated to include more detail about what specific sensors and communication protocols we will be using. This is still in line with our original proposed plan, just with more detail.

Gestures

While brainstorming what gestures we would support, we thought about how we might enable or disable activation of the gesture recognition/transmission. We originally proposed having another switch alongside the power switch to control transmission, however with further discussions with the professor and TA, we realized that it would be too unwieldy for a user to do that. Instead, we opted to come up with a custom gesture that could be used to enable or disable the transmission.

Further details on the gestures are described here: Gestures

Principles Used

  • Maintainability – we drafted our design docs to be easy to read so that if something breaks down the line, we can quickly refer to it to identify the reasoning behind certain design decisions
  • Modularity – Our device was designed with modularity in mind so that if a single part breaks, it will be easy to quickly swap it out with a working component
  • Testability – our system was designed so that it would be easy to test individual units as well as the system as a whole
  • Integration – we designed our system to integrate with existing infrastructure in the environment of our intended users. Examples of this are leveraging Wi-Fi and USB as communication protocols between our systems and with peripherals
  • Usability – usability is at the core of the design choices we made. This was evident in how we opted to go for a gesture to enable or disable transmission as opposed to a physical switch

Team Status Report for 02/11/23

This week, we worked on preparing for the proposal presentation slides as well as presenting it during class. In line with this, we planned out the schedule and timeline to

Later in the week, we finalized discussions related to the components of the Compute Module. Afterwards, we were able to refine our block diagram, replacing the originally proposed Jetson with a Raspberry Pi 4 and Coral TPU combination.

After finalizing the list of materials, we put in our orders through the request form for all the components except for the Coral TPU. Currently, it appears that the Coral TPU is out of stock everywhere, so we will postpone the order for this part until later in the semester.

Our project includes considerations for society. By tailoring this product for educators and presenters, we can help improve the effectiveness of education around the world which will have downstream effects across all aspects of society.