Posted on

Team Status Report for 02/24

Risk Mitigation

According to both the feedback from the design review presentation and our evaluation, the most significant risk is the NLP process. To further guarantee usability for visually impaired groups, we would add detailed features including audio instruction that could help the customers to use the product more efficiently and a “start recording” and “end recording” audio notification when pressing the button to let the users know when they could start giving commands. 

After the design review presentation, our group met again and went over the feedback together. Based on the feedback, we distributed the tasks by assigning each part of the design report to different group members. 

Design Changes

An audio instruction will be added to the web app to guarantee that the users can know the standard commands in advance. A “help” command is introduced in all pages of the web app so that the users could trigger the audio helper at any time. 

A “start recording” and “end recording” audio notification should be added each time the button is pressed. This ensures that the visually impaired groups could know when they can input vocal commands and when to stop. More details of the implementation will be included in the design report. 

Updated Schedule

The primary focus for next week is the design report. Each section of the report is assigned to one or more group members and should be reviewed before submission.

Posted on

Lynn’s Status Report for 02/17

Progress

After the weekly meeting with the instructor and TA, our team started to work on finalizing the design details. Our team discussed a lot about further design choices and tried to answer the “why” question about those choices. My primary focus was on the completed version of the front-end UI and the finite state machine diagram of the user interaction flow chart. After that, Yuxuan and I worked together to specify the MVC design of the web application from both an ordinary user’s and a visually impaired user’s perspective. 

To prepare for the design review presentation next week, I read through the documentation of spaCy and word2vec to gain a further understanding of the NLP process. 

A copy of the finalized UI and FSM is attached.

 

Schedule

I am on schedule.

Next Step

As the presenter of the design review, I would first finish the presentation during Monday’s or Wednesday’s class. After that, I will start working on the primary microphone setup since the hardware components ordered last week have arrived.

Posted on

Lynn’s Status Report for 02/10

Progress

The primary focus for this week is the proposal presentation. Although I am not the presenter this time, research on the technical challenges and corresponding solution approaches is still my major effort. For the audio input section of our design, signal processing libraries including PyAudio, PyPi noisereduce module, and SpeechRecognition should be utilized to reduce noise and transfer audio to text strings. Natural language processing models such as spaCy and pre-trained BERT model are then considered to recognize user commands and categorize the purchased items. I designed a general block diagram from user input to all expected components and the various forms of output financial reports.

After the proposal presentation, I started to design the layout front-end UI of our web application. A primary draft is attached. 

 I also researched the usability of the spaCy library to identify and match the user commands to specific functionalities. The Tokenizer and Matcher in the library should be utilized, and I wrote several Python scripts to further familiarize myself with spaCy. 

Schedule

I am on schedule.

Next Step

I will first focus on finalizing the design of front-end UI the next week, and our team will design the MVC framework of the web application together. I will also continue working on researching and implementing the NLP models.