Category: Team Status Reports

This week our team finished the setup of the system in the mini-fridge and tested the integrated system. The most significant risk that could jeopardize the success of the project is how to showcase the entire functionality in the demo video. Since the live demo is only one part of the entire demo video, and our project plays the role in the daily routine of the user, we need to find a proper expression that demonstrates our product in few minutes. This risk is managed by having good communication in our team meeting to design a plan with specific steps and content we want to add to the video. We will add annotations to the live demo part so that we can make sure each step is explained to the audience.

Since it is already the last week of class, and we finished the implementation part of our project, we are not making changes to the existing design, and we don’t change the schedule as the things left are to work on the report, demo, and poster.

An image of the mini-fridge is attached.

 

 

The most significant risk that could jeopardize the success of the project is the integration of the system in the mini-fridge. At the beginning of the semester and for the most time of the semester, we are using the clear acrylic board as an imitation of one shelf in the fridge. But we noticed that it might be better to just set up our system in the mini-fridge for a good demo result. Therefore, our team is currently focusing on building up the system in the mini-fridge. Since we set up the camera separately to take the picture of the grid, we have to figure out how to do this in the fridge. The risks are managed by mounting/ taping the camera on the wall of the fridge; we might need to adjust for the best angle and height, but we will try to fix the camera at one point for the best image results.

There are no changes made to the existing design of the system. Since we are coming to the end of the semester, we believe that sticking to our current design and implementation is the best idea.

There are no significant changes to the current schedule. We updated the demo and report timeline according to the course schedule.

 

Currently, we are working on the integration of the entire project as a team and refining each part/subsystem separately. The most significant risk that could jeopardize the success of the project is the migration of code from PC to the Jetson nano board. Also, since we are working separately and cannot meet for the integrating work, we need some methods to connect things together.

In the past weeks, we experienced problems with importing some Machine Learning libraries on Jetson Nano, and after communications with other teams and research on the internet, we decided to switch to Google Speech API. In addition, we are trying to control the Jetson Nano through SSH remotely so that we can work on the same piece. We also used Github to control the workflow and keep track of the progress. Therefore, we are able to manage these risks through these ways.

In general, there are no changes to the design of the system, except that we mentioned above about the probable change of Speech API. The usage of speech API and user interface will not be modified due to this change.

Our team is going according to the plan.

As a team, we looked into integrating the speech recognition with the image recognition to be able to do a basic request and response for the demo. We ran into some issues with the LEDs, since we had to get replacements. It is currently working out well and we expect to be able to demo a fully functional pipeline, albeit without the depth of recipes and ingredients that we expect for the final demo. This should be a good midpoint for us, to have the general framework done, and can simply add in additional functionality/data.

The biggest risk for us seems to be some porting issues, with the microphone on the Jetson being much weaker than the normal desktop microphone, meaning it is harder to trigger for commands. However, tuning some parameters fixed that issue. So we are currently looking at some other issues with images as well, however, we think that this should not be a bottleneck.

There were generally no changes to our deadlines and we expect to finish on time.

As a team, we are currently aiming to integrate all of our components together for the interim demo. We are slightly behind on this aspect but we are planning on using the long weekend to catch on that regards and hold meetings to put all of our components together in time for the demo.

The biggest risk for us is putting together the image recognition to work correctly for the interim demo. to mitigate this risk, both Yang and Elena will focus on this before the demo. We believe that this will be sufficient to fix up the issues that we have in time.

There were no changes to the system design or schedule this week, and our progress has been consistent so far.

We received advice from instructors that we need to test how well the Image Recognition API performs on recognizing our ingredients. We agree that it is important to consider this because this part is very important for the entire pipeline to work. We are currently working on the testing part, and if the performance is not ideal, we will consider train the pre-trained model with our dataset or implement a CNN model that focuses on the dataset we are using to manage the risks of this part.

Another suggestion we got is that we should also consider sending the text version of the recipe to the user through email so that the users can easily check the recipe if they want to. We bought wifi adapters and received them yesterday, and we are working on the email functionality now. We are still trying to figure out how to get the wifi adapter to work on Jetson nano.

The next step of our project is to connect every part together. We will research the pipeline and start implementing the connections between sub-systems in the coming week and also continue explorations on our own parts individually. There are no changes to our planned schedule.

We have received both copies of the hardware and tested that they work as expected. Since LED grid is being made, a risk is that while working remotely, it is hard to communicate the exact specs, so that we might have 2 copies of slightly different LED grids. We will minimize the risk by assembling the LED grid together and write test files to ensure that they behave the same way as expected.

Another challenge is that our power supply didn’t arrive on time, so we cannot embed the program yet. We are testing the peripherals from the computer directly. We might need to pay more attention testing these application we wrote once it’s flashed on Jetson Nano.

We decided to change the LED gridding from 2*5 to 3*3. That is, instead of 10 food items, we aim to target 9 items instead. We made the change since it allows clearer gridding while also simplifies the logic for LED light up algorithm.

In general, we do not have any changes to our proposed schedule. This week we worked on the design review presentation and also the report together. At this stage, we are very clear with what we want as a product and how to implement it.

This week our team worked on the bill of materials. We will make the decision by this weekend in order to get the items placed next Tuesday. From the available items list, we noticed that there are RPi and NVIDIA Jetson, so we have to decide if we want to borrow the equipment instead of buying a new one. The schedule is a little bit behind because we haven’t made the decision, but we will have more discussion during the weekend.

On the other hand, after receiving feedback from other teams and answering questions in class, we noticed that our estimation of latency might not be accurate and very difficult to meet. The risk will be managed by investigating the pipeline and better measurement of the approximate time each step will take next week. We will refine our latency expectation and accuracy expectation for the coming presentation with the support of related works and materials. The time and effort that we need to put into these researches might affect our existing schedule, but we allotted time in our plan beforehand, so we will still be able to finish the work on time.

 

This week we had discussions on the topic of the project and a plan for implementation details. At the beginning of the week, we were unclear about the focus, but after brainstorming and good communications of ideas, we all agreed on our current plan.

Before getting started with the proposal, we met with Prof. Sullivan and received advice on the scope of the project. After consideration, we decided to narrow the scope of our original plans, because we agree that limiting the number of items and recipes will be more practical, and we will still keep our original plan as a stretch if we have time.