Team Status Report for 03/11/23

What are the most significant risks that could jeopardize the success of the project? How are these risks being managed? What contingency plans are ready?

The most significant risk right now is if the windowed time series analysis turns out to be inaccurate. In case this doesn’t work, we plan to incorporate more rigorous calibration into the system or possibly restructure the gestures so that they are more distinct from each other. Additionally, we could resort to the neural network approach, although this is the less ideal case because it will likely have a performance penalty and may not necessarily have better accuracy.

Were any changes made to the existing design of the system (requirements, block diagram, system spec, etc)? Why was this change necessary, what costs does the change incur, and how will these costs be mitigated going forward?

None since last week.

Provide an updated schedule if changes have occurred

None

Team work adjustments

No teamwork adjustments have been made. We are currently on track to accomplish our tasks, and are currently all just working on them.

Gram’s Status Report for 03/11/23

What did you personally accomplish this week on the project? Give files orphotos that demonstrate your progress. Prove to the reader that you put sufficient effort into the project over the course of the week (12+ hours).

The past week, I worked on writing the design report. Specifically, I worked on the Introduction, Use Case Requirements, parts of the Design Trade Studies, System Implementation, and Project Management sections, Related Work, and Summary.

I also began writing the controller code for the compute module. I setup the Mosquitto (MQTT) broker on the Raspberry Pi so that it can listen for incoming messages from a connected glove. I also wrote a Python program that subscribed to the data packets that would be broadcasted by the glove. Since the glove MQTT client is still in progress, I tested this functionality by writing a dummy MQTT client in Python to publish messages.

Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?

Progress is still on schedule.

What deliverables do you hope to complete in the next week?

I plan to finish writing the controller code this week. I will need to write the logic for aggregating received data packets into windows that will be fed into the decision tree. Depending on how quickly the system can process incoming packets, I may also need to implement some sort of buffer queue to handle the case of data packets arriving faster than the system can process them.

Furthermore, I will also finish implementing the decision tree. I will try to implement the logic for classifying what gesture a user is doing without necessarily correctly identifying the degree/intensity of the action yet. If the decision tree logic is finished quickly, I hope to implement the degree/intensity determination logic as well.

As you’ve now established a set of sub-systems necessary to implement your project, what new tools have your team determined will be necessary for you to learn to be able to accomplish these tasks?

I will need to learn more about windowed analysis algorithms over time series data because I need to identify what optimal window size I should use. Moreover, I will likely need to learn more about data smoothing algorithms to prevent jittering. Furthermore, I will need to learn how to effectively remove the gravity vector from the accelerometer data. This will be challenging since the gravity vector may be split up across the three axes, depending on the orientation of the hand.

Team Status Report for 02/25/23

What are the most significant risks that could jeopardize the success of the project? How are these risks being managed? What contingency plans are ready?

The most significant risk that we have identified right now that could jeopardize the success of the project is if the machine learning turns out to be highly inaccurate or insufficiently performant. To manage this risk, we identified ways to perform the gesture detection using basic time series analysis and a decision tree which will be much faster because it doesn’t depend on a deep neural network and can even be implemented directly on the edge. This gesture detection would simply examine trends (increasing/decreasing) of the different sensor data and match them with the corresponding gestures we identified.

Were any changes made to the existing design of the system (requirements, block diagram, system spec, etc)? Why was this change necessary, what costs does the change incur, and how will these costs be mitigated going forward?

None since last week.

Provide an updated schedule if changes have occurred

The schedule was updated to account for the design report and to shorten the amount of time allocated for smaller tasks such as soldering the circuit.

Team work adjustments

We did not make any reassignments to our list of tasks yet since it currently seems well-balanced. However, we adjusted our schedule to make more of our tasks happen in parallel when they weren’t dependent on each other and also give us more time this week to focus on the design report which we did not originally account for in our schedule.

 

Gram’s Status Report for 02/25/23

What did you personally accomplish this week on the project? Give files or photos that demonstrate your progress. Prove to the reader that you put sufficient effort into the project over the course of the week (12+ hours).

This week, I helped prepare the slides for the design presentation and started working on the design report, incorporating some of the feedback we received from the design presentation.

Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?

We’re slightly behind based on our original schedule, but also a lot of the tasks we had originally listed were allotted more time than they actually needed. We updated our Gantt chart to reflect this which puts us back on schedule.

What deliverables do you hope to complete in the next week?

I will continue working on the design report with the rest of my team and finish it by then. Additionally, I’ll work on assembling the ML pipeline so we can start testing and collecting training data.

Team Status Report for 02/18/23

Risk Management

The most significant risks that could jeopardize the success of the project are if it turns out that the ML model we use is too slow or inaccurate to create a usable product. In case this happens, our backup plan is to replace the ML model with a heuristic algorithm that analyzes the time-series data as they come in. This would allow us to make certain gesture predictions using a basic decision tree that can easily run on the compute module quickly.

Changes/Updates

The block diagram was updated to include more detail about what specific sensors and communication protocols we will be using. This is still in line with our original proposed plan, just with more detail.

Gestures

While brainstorming what gestures we would support, we thought about how we might enable or disable activation of the gesture recognition/transmission. We originally proposed having another switch alongside the power switch to control transmission, however with further discussions with the professor and TA, we realized that it would be too unwieldy for a user to do that. Instead, we opted to come up with a custom gesture that could be used to enable or disable the transmission.

Further details on the gestures are described here: Gestures

Principles Used

  • Maintainability – we drafted our design docs to be easy to read so that if something breaks down the line, we can quickly refer to it to identify the reasoning behind certain design decisions
  • Modularity – Our device was designed with modularity in mind so that if a single part breaks, it will be easy to quickly swap it out with a working component
  • Testability – our system was designed so that it would be easy to test individual units as well as the system as a whole
  • Integration – we designed our system to integrate with existing infrastructure in the environment of our intended users. Examples of this are leveraging Wi-Fi and USB as communication protocols between our systems and with peripherals
  • Usability – usability is at the core of the design choices we made. This was evident in how we opted to go for a gesture to enable or disable transmission as opposed to a physical switch

Gram’s Status Report for 02/18/23

Updates

Block diagram updates

This week, I identified more specifically what sensors and communication protocols we will be using for our device. I also worked on the wiring between the different components and microcontrollers — illustrating how many wires we would actually need between components. I updated our block diagram to reflect this:

Communication Protocols

For the accelerometer, we found from the MPU-6050 IMU datasheet that it communicates using the I2C protocol. For communication between the ESP32 (gloves) and the Raspberry Pi (compute module), we originally only said that we were communicating over Wi-Fi but we had not yet thought about what protocol we would be using to communicate. I researched into what protocols were common for high-frequency updates from IoT devices and found the MQTT protocol to be the best fit for our setup. MQTT works similar to various other pub-sub systems while minimizing communication overhead. Moreover, there are existing MQTT libraries we can use for the ESP which can connect to an MQTT (Mosquitto) broker on the Raspberry Pi server.

Gestures

This week, I also worked on refining the exact gestures we would recognize. I worked to help break down our conceptual actions (zoom, pan, and rotation) into concrete gestures that we could recognize. We managed to break this down into 5 specific gestures:

  • Enable transmission
  • Disable transmission
  • Zoom in
  • Zoom out
  • Rotate
  • Pan

The gestures are explained in more detail here: Gestures

Schedule

Currently, we are still on schedule to finish the project in a timely manner.

Upcoming Deliverables

This coming week, I will be working on formalizing these design decisions in the design report. Specifically, I will work on finding resources to back up our design choices and, if necessary, modify them if resources state otherwise.

Relevant ECE Courses

  • 18-100, 18-220, 18-240, 18-794,

Team Status Report for 02/11/23

This week, we worked on preparing for the proposal presentation slides as well as presenting it during class. In line with this, we planned out the schedule and timeline to

Later in the week, we finalized discussions related to the components of the Compute Module. Afterwards, we were able to refine our block diagram, replacing the originally proposed Jetson with a Raspberry Pi 4 and Coral TPU combination.

After finalizing the list of materials, we put in our orders through the request form for all the components except for the Coral TPU. Currently, it appears that the Coral TPU is out of stock everywhere, so we will postpone the order for this part until later in the semester.

Our project includes considerations for society. By tailoring this product for educators and presenters, we can help improve the effectiveness of education around the world which will have downstream effects across all aspects of society.

Gram’s Status Report for 02/11/23

At the start of the week, I worked with my other groupmates to prepare slides for the proposal presentation presented by Xuan. We refined our requirements based on feedback provided from our initial proposal submission.

I also refined our block diagram this week.

After discussion with my group, we decided to modify our original design. Instead of using a Jetson as the controller for the compute module, we decided to make use of a Raspberry Pi 4 with a Coral TPU accessory instead. This works better because of many reasons:

  • Raspberry Pi + Coral TPU is significantly cheaper than a Jetson
  • Raspberry Pi supports Wi-Fi out-of-the-box
  • Jetson doesn’t appear to support HID behavior

Later in the week, after finalizing our block diagram, I helped draft our Bill of Materials. After we finalized our list, I put in the orders on the Google Form.