Category: Team Status Reports

Risk Identification and Mitigation Strategies
Primary Risk Concern: Our primary concern is the OCR integration on time. This module of the project was just finished this week. Yuma still needs to integrate it into the code, taking more time.

Mitigation Measures:
Yuma will be solely focused on finishing the final integration. Siyuan and Jason will work on the final documentation, including final reports, final video, and final poster.

Project Design Updates
As of the current reporting period, there have been no modifications to the initial design of the project.

Schedule and Timeline
We are a little lagging behind the schedule. However, since we only have final integration unfinished, we should have everything done by demo day.

List all unit tests and overall system test carried out for experimentation of the system. List any findings and design changes made from your analysis of test results and other data obtained from the experimentation.

Tests:

temperature and humidity sensor test

recomandation algorithm test

item registration test

scalling and system reliability test

usability test

scanning test

display test

Findings & Design Changes:

The design of our overall system did not have much changes. However, in the test of scanning items, we have some findings regarding OCR. We did not expect that there are many formats, sizes, and colors of expiration dates. This makes our model tranning harder than we thought. Also, we find that the choice of camera is really important. Our first camera was cheap and it does not have great image quality. Then, we purchased an expensive one, which do have great image quality but was not designed to focus for something at close distance. In the end, due to limited budget, we have to choose a camera with good image quality and can focus at close distance, but the focusing speed is slow and easily affected by shaky hands. All these reasons combined, make our OCR module does not have a satisfying scanning accuracy.

Risk Identification and Mitigation Strategies
Primary Risk Concern: Our primary concern currently is being able to integrate our modules together, especially the camera and the scanner module. This includes not only the coding part, but also physically putting them together. As the scanner will be apart from the rbpi module and connected via wire, we will have to see how we can put them together without making it difficult for our users.

Mitigation Measures:
We are able to tweak our physical design part of the scanner if needed. The wiring part can be extended or shortened if necessary. Other mitigation measures and validation methods will be discussed in Validation section.

Project Design Updates
As of the current reporting period, there have been no modifications to the initial design of the project.

Validation

For validation, our main aspect is putting the modules all together and seeing if they work as intended at the specs that we want. Specifically, we want to measure how fast our scanner system works, and how accurate the information is, and how easy it would be to fix any mistakes and log them into our system. These metrics will be validated by testing our module in a real life setting. We will setup our scanner module as well as our central module in a room, and try to do a test run of a real life scenario. This includes scanning, logging, and checking information. We will then debug any issues that may arise in the process, such as timing requirements as well as accuracy requirements.

Schedule and Timeline
We are on schedule, and hope to be done integrating by next week. We will dedicate the rest of the time on testing and validating our module.

Risk Identification and Mitigation Strategies
Primary Risk Concern: Our primary concern currently is the OCR accuracy. Our current camera setting and training dataset are not able to provide enough OCR accuracy.

Mitigation Measures:
We are currently trying to find another camera with autofocus and wider focus range. We have ordered another camera with autofocus and 40mm to infinit focus range. Another mitigation measure could be switching to another dataset to train our OCR model.

Project Design Updates
As of the current reporting period, there have been no modifications to the initial design of the project.

Schedule and Timeline
We are not on track with our timeline. The development for the OCR algorithm detecting expiration dates has been falling behind and we haven’t been able to integrate that portion of the scanner module yet.
The updated Gantt Chart is attached below. We still have one week of slack.

Risk Identification and Mitigation Strategies
Primary Risk Concern: Our primary concern currently is the camera image acquisition rate. We did not expect to run into a problem where the camera is not able to stream images continuously, instead it happens at the frequency that a python function is called. Since this rate limits the amount of images we are able to acquire in a single period of time, we are not able to gather images that we may want for good classification.
Mitigation Measures:
We are currently trying to find alternative methods of acquiring images, such as a screen capture or photos being dumped to disk before we read off of it. These methods add latency, so we are also experimenting with multi-threading the main python function in hopes of getting better image acquisition performance. Another mitigation measure could be strengthening the OCR algorithm, though this is a last resort due to the nature of its complexity.
Project Design Updates
As of the current reporting period, there have been no modifications to the initial design of the project.
Schedule and Timeline
We are not on track with our timeline. The development for the OCR algorithm detecting expiration dates has been falling behind and we haven’t been able to integrate that portion of the scanner module yet.

Risk Identification and Mitigation Strategies

Primary Risk Concern: Our concern remains the same throughout, however progress has been made since last week. The OCR model has been developed in a very simple way right now, therefore not reaching its 90+ accuracy rate. This should be fixed in the next coming weeks.

Mitigation Measures: Mitigation measures such as different formats for expiration dates have been put into consideration and been developed. Custom data sets to train on the OCR model is also needed for better accuracy.

Project Design Updates

As of the current reporting period, there have been no modifications to the initial design of the project. 

Schedule and Timeline

We are on track with our timeline. 

Risk Identification and Mitigation Strategies

Primary Risk Concern: Since our project did not progress much this week, our primary risk concern for this phase of the project is still being able to identify the expiration date correctly given the input image taken by our camera. There are many concerns regarding the accuracy of the data, as even the slightest error could lead to faults. For now, we will still focus on preprocessing mechanisms, as we have not implemented character recognition yet. 

Mitigation Measures: Despite the planned preprocessing procedures we mentioned in the last status report, we still need to consider some things about expiration dates. For the input, we should take into consideration what would be the most common format for expiration dates. Although most USA products are in MM/DD/YYYY format, some products from the Asian food market may have YYYY/MM/DD format. This would imply that software would have to be able to distinguish between them.

Project Design Updates

As of the current reporting period, there have been no modifications to the initial design of the project. 

Schedule and Timeline

We are on track with our timeline. Although we did not expect writing the design review report would take a week, we still have enough time for integration and testing. 

Additional Questions: 

A was written by Yuma Matsuoka, B was written by Jaesup Kim and C was written by Siyuan Li.

Part A

Our project is helping bridge the gap between those who do not have technical knowledge and those who do. In our society, people who have technical knowledge (i.e. students, young people) are able to utilize technologies such as the iPhone, applications, etc to make life easier. For example, those who are young are more likely to use mobile check-in for a flight than those who are old, who are likely to go to the airport early to print out physical passes. Our technology will not discriminate based on technical knowledge, as the entry to operation will be very low. The product will take less than 5 minutes to set up, and will be very intuitive to use. Furthermore, it will be a hard product, hopefully closing the idea gap that apps present. This hopefully will be a step in lowering the implicit accessibility barrier in using technology to simplify our lives.

Part B

Our project helps people from all cultures. Different cultures have different grocery items and foods, and also storage spaces to keep them fresh. Our scanner is able to work on any environment and any storage space, whether it be a fridge or pantry, or a chilled basement. Our product will count for all of those environmental aspects as well as condition of the items. Even products that come from a foreign origin with no barcode will work as you can manually enter the item.

Part C

Our project directly contributes to lessening the environmental impact associated with the disposal of food. Food waste contributes to a significant portion of methane emissions, a potent greenhouse gas, when it decomposes in landfills. By ensuring that families and individuals are reminded of expiration dates, IntelliStorage helps in making sure that groceries are consumed before they spoil, thereby decreasing the volume of food that ends up in landfills. 

Moreover, our project’s efficient management system encourages users to make more conscious purchasing decisions. By having a better overview of what is already stocked at home, users can avoid overbuying, leading to not only less food waste but also a reduction in the carbon footprint associated with the transportation, and packaging of excess groceries.

 

Risk Identification and Mitigation Strategies

Primary Risk Concern: Our primary risk concern for this phase of the project is being able to identify the expiration date correctly given the input image taken by our camera. We have done optimization for our camera, therefore we are now moving on to the processing phase There are many concerns regarding the accuracy of the date, as even the slightest error could lead to a totally wrong information. Therefore, a lot of preprocessing and postprocessing is required to ensure optimal character recognition. For now, we will focus on preprocessing mechanisms, as we have not implemented the character recognition yet. 

 

Mitigation Measures:

We are planning on implementing these preprocessing measures: 

1. Resizing image
2. Contrast enhancement
3. Color to grayscale adjustment
4. Contour analysis to filter out non-text regions 

 

Other than the image itself, some things to consider for expiration dates in general:

For the input we should take into consideration what would be most common for expiration dates, which would either be started by “Best by”, or “Use by”. Then the date would be divided by “.”, “/”. It would also be in MM/DD/YYYY format. We would also have to be able to take in only MM/DD inputs as well. This would also imply that software would have to keep track of either or both dates, complicating the database.

Project Design Updates

As of the current reporting period, there have been no modifications to the initial design of the project. 

Schedule and Timeline

We are on track of our timeline. Even if the OCR implementation takes longer than expected, we have ample time for integration and testing during the week after Spring Break. 

Risk Identification and Mitigation Strategies

Primary Risk Concern: One of the bigger challenges in this project is accurately detecting the expiration date of products using a camera. This primary concern is split into two sub-problems.

1. Image Clarity for Text Recognition: A concern that has somewhat materialized from last week was Image clarity. Our first camera that we acquired was not able to focus on targets <10cm, which led to blurry numbers and text. We tried to plug this image into google lens to see if a state-of-the-art algorithm could detect the text on the image, but it was unable to. We were surprised as this camera was the same one a previous capstone group used (Where’s the Barcode).
2. Capture Timing: The final main concern is when the camera acquires the image. The user will face the expiration date towards the camera, but there is no cue when the image is most clear / the expiration date is most in view, and when the camera is most in focus. 

Mitigation Measures: 

1. To have lower likelihood of roadblock on the camera hardware, we preemptively ordered multiple cameras (one ArduCam from Inventory, one from Amazon) with low focal distance and auto-focus. 
2. We have tentatively drafted an algorithm that allows for clear expiration date images. This is purely hypothetical and will not be able to be tested until the camera is integrated with the software, but we plan on syncing the image acquisition with the barcode acquisition. When the barcode is read, the images acquired from the camera +-1 second will be all scanned through an OCR algorithm and the image with the highest classification value will be used. This is based on the principle that if the barcode can be clearly read, hopefully the expiration date can be as well.

Project Design Updates

As of the current reporting period, there have been no modifications to the initial design of the project. 

Schedule and Timeline

The project was a little delayed this week due to problems setting up the RPI5. We were unable to use the lab keyboard/mouse to hook up to the RPI, and thus we were blocked from going through the setup process. This is very minor though, and we are roughly ⅓ of a week behind schedule, and we hope to get back on progress this week.

Appendix

(Original Camera Image, target = 15cm)

(Original Camera Image, target = 10cm)

(Original Camera Image, target = 5cm)

 

Additional Questions: 

A was written by Siyuan Li, B was written by Jaesup Kim and C was written by Yuma Matsuoka.

Part A

IntelliStorage is poised to significantly impact public health, safety, and welfare through its innovative approach to household storage management. By leveraging advanced technologies such as barcode scanners, microcontrollers, and a sophisticated camera and database system, IntelliStorage addresses crucial aspects of household management that directly affect the well-being of individuals.

 

From a public health perspective, the system’s ability to track expiration dates is paramount. Food safety is a major concern for households, as the consumption of expired goods can lead to health issues ranging from mild food poisoning to severe allergic reactions. IntelliStorage mitigates this risk by ensuring that consumers are aware of the expiration dates of items within their pantries and refrigerators. By providing daily summaries and consumption suggestions, the system not only promotes the efficient use of food but also significantly reduces the chances of health complications arising from the ingestion of spoiled products.

 

In terms of safety, the absence of physical harm is a critical consideration. Cluttered storage spaces can pose significant risks, including physical injury from falling objects or the growth of harmful bacteria and molds in poorly managed pantries. IntelliStorage’s item detection and management feature organizes and maintains an accurate inventory of stored items, reducing the likelihood of accidents and enhancing the overall safety of the home environment.

 

Furthermore, the welfare of individuals is intricately linked to their ability to access and efficiently manage basic needs, such as food and clothing. IntelliStorage’s comprehensive network across different storage spaces ensures that necessities are adequately stocked, accessible, and utilized before expiration. This level of organization and management not only saves time and resources but also contributes to a sense of security and well-being among users.

 

Part B

One of the social factors that we considered in designing this product solution is the environmental (green) issue of food waste. By using Intellistorage, users can avoid throwing away food that has passed its expiration date and instead consume it before it spoils or donate it to food banks or charities. This will help reduce food waste and its environmental consequences.

 

Another social factor that we considered in designing this product solution is the economic issue of rising food prices. Consumers are paying more for the same amount of food, which can affect their budget and purchasing power. By using our scanner, customers can save money by avoiding buying duplicate items that they already have at home. This will help users optimize their grocery spending and reduce unnecessary costs.

 

Part C

Intellistorage is able to alleviate a lot of unnecessary stress on the supply chain. For many, it is very common to go to the grocery store and buy something you already had at home. By using IntelliStorage, you will be able to keep track of what is already at home, so that there is no need to buy anything that has already been bought. If many adapt this technology in their household, this would lead to less spending and smarter purchases. This would alleviate the need for supermarkets to order excess items, and would decrease demand. 

 

In events of supply chain disruptions such as COVID-19, this decrease in demand would allow for more necessities to be distributed rather than excess that is bound for waste. This should have a very positive economic impact as the goods are being utilized more efficiently, as well as a positive social impact.

 

Risk Identification and Mitigation Strategies

Primary Risk Concern: The project’s foremost challenge lies in the utilization of a camera for detecting expiration dates on products. This concern bifurcates into two significant issues:

1. Compatibility with Raspberry Pi: The primary uncertainty revolves around whether the selected camera is compatible with Raspberry Pi hardware. This compatibility is crucial for the seamless integration of hardware and software components of our project.
2. Image Clarity for Text Recognition: Another pivotal concern is whether the chosen camera possesses the requisite resolution and clarity for the software to accurately recognize and interpret expiration dates.
3. Capture Timing: The final main concern is when the camera acquires the image. The user will face the expiration date towards the camera, but there is no cue when the image is most clear / the expiration date is most in view, and when the camera is most in focus. 

 

Mitigation Measures: To preemptively address these risks, we have procured two distinct cameras. The initial choice is a high-quality camera, selected for its superior image quality, pending compatibility confirmation with Raspberry Pi. Should this camera not meet our compatibility requirements, we have a contingency option—a second camera, already included in our Raspberry Pi inventory. This backup camera is guaranteed to be compatible with Raspberry Pi, ensuring our project’s progression without delays related to hardware integration.

 

Project Design Updates

As of the current reporting period, there have been no modifications to the initial design of the project. The team remains committed to executing the project as per the originally outlined specifications and design framework.

 

Schedule and Timeline

The project is progressing as planned, with no deviations from the initial schedule. We remain on track to meet our projected milestones and deliverables within the established timeframe.

Appendix