Team Status Report for 11/30

Summary

Our group completed:

• We have a working voice pipeline. This includes;
  • The speech recognition code that will eventually run on the raspberry pi.
  • Word2vec pipeline to find the term that most closely matches the subject of the input sentence.
  • OpenAI prompt engineering + GPT vision integration for coming up with natural voice responses.
• Iterated on CAD designs for raspberry pi cases.
• Created new inference server with the ability to serve GroundingDINO inferences to the Raspberry Pi. This allows the Raspberry Pi, which itself is not powerful enough to run GroundingDINO to have aceccss to GroundingDINO inferences.

What are the most significant risks that could jeopardize the success of the project? How are these risks being managed? What contingency plans are ready?

At this point, our project is coming together relatively cohesively, and we will definitely have a functional final product by demo day, but there are a few areas which could use more focus. Before demo day, work must be done regarding assembling our final hardware product. Our speakers and microphone seemingly have yet to arrive, and we must still print out a few of the final copies of our case. Additionally, our group needs to focus more of its energy towards coming up with tests that show the effectiveness of our project as a product, as that is one of the primary focuses of the final presentation and report.

Were any changes made to the existing design of the system (requirements, block diagram, system spec, etc)? Why was this change necessary, what costs does the change incur, and how will these costs be mitigated going forward?

Ironically, we were able to add much of the planned functionality back to our system by having the microphone frontend implemented.

Additionally, with the creation of the GroundingDINO inference server, we have extended the functionality of our project by enabling the Raspberry Pi to have access to additional computing power, and a more powerful model. This will allow users who are either less paranoid about our service, and/or are willing to pay a small subscription to have access to a better product.

Provide an updated schedule if changes have occurred.

No significant changes have occurred since the last few reports. We will still need to implement the additional features listed above, and will have to focus on creating a good final presentation and report.