Posted on

Nanxi’s Status Report for 03/06/2021

This week I worked on the material list of our project and everything is ordered. I refined the design of our project after reading the feedback for our proposal. I designed the recipe delivery method to be step-by-step instruction with the option to skip or repeat. Specific user interface and implementation details have been decided. I also took the chance of preparing my presentation to review our design and specify requirements on a finer level. (Please refer to our slide for my effort.)

We are on schedule for the project. Next week our hardware will arrive and I want to test them with the external library we will use. I will also need to start designing and making the LED grid.

Posted on

Yang’s Project Report for 03/06/2021

This week I worked with some starter code in NeMo, the speech recognition package we are hoping to use. The pretrained models seem to work fairly well in terms of recognizing the actual input waveform, although I had some technical difficulties with getting it to capture the audio and run speech recognition at the same time on my desktop.

Additionally, I worked on my presentation and also submitted our parts list.

Posted on

Yang’s Status Report for 2/27/2021

This week I looked at some libraries from nvidia for image inference, and automatic speech recognition. I found that jetson-inference https://github.com/dusty-nv/jetson-inference has pretrained object recognition and image recognition that we could use for on-board image inference. Additionally, I found the NeMo library for Automatic Speech Recognition (ASR). It is pretrained and would work well for our use case and is also tailored for the NVIDIA platform.

Overall we are on schedule and had a successful presentation and for the next week, I plan on building the required components for speech recognition using the nemo api. Additionally, I will be working with my team to select out the final hardware components.

 

Posted on

Team Status Report for 02/27/2021

This week our team worked on the bill of materials. We will make the decision by this weekend in order to get the items placed next Tuesday. From the available items list, we noticed that there are RPi and NVIDIA Jetson, so we have to decide if we want to borrow the equipment instead of buying a new one. The schedule is a little bit behind because we haven’t made the decision, but we will have more discussion during the weekend.

On the other hand, after receiving feedback from other teams and answering questions in class, we noticed that our estimation of latency might not be accurate and very difficult to meet. The risk will be managed by investigating the pipeline and better measurement of the approximate time each step will take next week. We will refine our latency expectation and accuracy expectation for the coming presentation with the support of related works and materials. The time and effort that we need to put into these researches might affect our existing schedule, but we allotted time in our plan beforehand, so we will still be able to finish the work on time.

 

Posted on

Elena’s Status Report for 02/27/2021

This week I looked into the Google Cloud Vision API that we planned to use for the image recognition part. By testing on the API with different images of perishables, I believe that the API will be reliable on photos that we take for real food, as it provides high accuracy on the type of food and precision on the amount. Some example results of oranges are the following.

I think the progress is on schedule, and for the next week, I am planning to research more about the past works on recipe recommendation systems or other recommendation systems based on few features(restriction on recipes, number of people).  Also, I will work with my team on the design review presentation.

Posted on

Nanxi’s Status Report for 02/27/2021

I searched for the hardwares we need. Since we are more likely to use Jetson Nano, I made a list of corresponding hardwares to purchase and their cost. This can be easily changed to target RPi since most of the hardware are compatible. With Amazon, we can receive most of our hardware in 3 days of order. Additionally, I looked into the LED grids that we need. I did not find LED grids that fits our requirements. We need to build the grid from LED strips by ourselves.

Additionally, I watched the demo of many object recognition project targeting Jetson Nano and read the user manual. I think we have a good grasp on what it could do for us.

 

Posted on

Team Status Report for 02/20/2021

This week we had discussions on the topic of the project and a plan for implementation details. At the beginning of the week, we were unclear about the focus, but after brainstorming and good communications of ideas, we all agreed on our current plan.

Before getting started with the proposal, we met with Prof. Sullivan and received advice on the scope of the project. After consideration, we decided to narrow the scope of our original plans, because we agree that limiting the number of items and recipes will be more practical, and we will still keep our original plan as a stretch if we have time.

Posted on

Nanxi’s Status Report for 02/20/2021

I collaborated with Yang and Elena to further define the scope of the project and the overview of our design while putting together the presentation slides for our  proposal presentation. I worked on the design overview flowchart, and specified our finer design modules under the entire project. The flowchart is in the presentation slide. For the slides, additionally, I was in charge of pages for Use case, Requirements: Accuracy of Recognition Systems and Quality of Recipe Recommendations and Design overview. (Please refer to our slides to see how much effort I put in.)

Our progress is on schedule. For next week, I hope to start getting our hardware peripherals ready and testing out the external library we will use.

Posted on

Elena’s Status Report for 02/20/2021

For this week I worked on the proposal slides with my teammates. I worked on user flow diagrams, proposed schedules, and solution approaches. After meeting with Prof. Sullivan this week, I made some researches on sensors that may help us recognize the items better. I think our team have good communication and for next week I will look into more approaches that can help us build a strong recommendation system.

Posted on

Yang’s Project Report for 02/20/2021

This week, I put significant effort into figuring out our team’s project plan. I, along with Nanxi and Elena, spent time looking into what hardware we are interested in using (Jetson Nano), what software we need for Speech Recognition/Generation, LED controls, Image Recognition, for our project proposal presentation. I believe that we are on track and have a good idea of how to proceed with the rest of our project.