Category: Team Status Reports

• No design changes have been made

• • Partially completed UI optimizations
  • Debugged and stabilized preference/allergen comparison logic in the product card UI.
  • Initial work on NLP-based filtering has started, focusing on extracting keywords from user inputs.

• NLP-based Allergy Detection Accuracy
  • Risk
    • The NLP-based allergy detection may fail to accurately flag allergens due to variations in ingredient names, formatting inconsistencies, or missing data in product ingredient lists.
  • Risk management
    • Implement keyword based filtering as a baseline before integrating AI-driven NLP
    • Cross-check allergens with multiple data sources (Spoonacular, Open Food Facts API)
    • Display AI-identified allergens and preferences on the User Settings page, beneath the profile text input, where users describe their allergens and dietary habits in natural language, allowing users to review and confirm which items are correctly flagged as allergens or preferred products.
  • Contingency Plan
    • Allow users to manually confirm flagged items
    • Implement a highlighting feature for uncertain matches for additional user validation

• UI Enhancement for Allergy & Preference Filtering for clarification of product selection and improvements in user accessibility of managing allergens
  – Allergen flagged items now appear in red highlighted boxes for visibility
  – User preferred products based on dietary preferences are displayed in green highlighted boxes

• MQTT connection & data transmission
• Ingredient Substitution & allergy filtering UI
• Obtain product UPC value
• Spoonacular API requests

• • Risk
    • We are a bit worried that the case and hardware components will be too heavy to be equipped to the case
  • Risk Mitigation
    • Start creating prototypes of the case before the final 3D printed design
    • Test it on the shopping cart attachment clip

• • No design changes have been made

• • Updated Schedule:
    
  • So far, we have made progress on
    • Developed UI for all main pages
  • • Retrieving UPC code from barcode scanner and rasberry pi connection
    • CAD design for 3D printed case

• • Risk
    • We may not be able to implement our full list of features such as AI meal recommendations or allergen substitutions
  • Contingency Plan
    • We are prioritizing our different features. The main functionality is cart tracking and item scanning. Then, we will prioritize allergen substitutions.

• • Risk
    • Network instability may cause message loss / delays in product updates.
  • Contingency Plan
    • Implement retry mechanisms to resend messages in case of failures.
    • If the network is unstable, store scanned product data locally on the mobile app & sync when online.

• Transition from Database to Open Food Facts API and Cache
  • Originally we planned to build our own database of aldi foods and information. However, we recently found an API online that is able to access the aldi inventory and return a product through its UPC, which is the output of the barcode scanner. Therefore, instead of using memory and time to build this database, we will query for it during the shopping
  • A concern of this is latency. Therefore, we want to use a cache to retrieve frequently used item in order to still meet our quantitative requirements.

• • Started integrating LangChain to process scanned grocery items, cross-check them with planned meals, and request substitutions when needed
  • Created a schema to store scanned products, user dietary preferences, and updated meal plans, with basic CRUD for efficient data retrieval
  • Implemented MQTT client in the mobile app

• • Risk
    • The barcode scanner may fail to recognize some products or barcodes
  • Risk management
    • The app should indicate when it is unable to read the barcode or unable to find the product through alerts to the customer.
  • Contingency Plan
    • The app should allow a customer to manually enter in a product if the barcode scanner and database system is unable to recognize a product.

• Transition from Computer Vision to Barcode Scanning
  • Why: Our visit to Aldi revealed that CV might struggle with detecting stacked or partially visible items, leading to missed detection. Additionally, almost all products except some produce had barcodes, making barcode scanning a more reliable and deterministic approach. This ensures users can confidently scan each item without worrying about recognition errors. Barcode scanning also provides instant confirmation, unlike CV, which may silently fail.
  • Impact:
    • Remove YOLOv8-based object detection, eliminating the need for an RGB camera.
    • Reduced computational complexity -> Moved from Jetson Nano to Raspberry Pi for better efficiency
    • Improved reliability, as barcode scanning is more accurate for grocery identification.
  • Cost considerations:
    • Reduced cost by eliminating the need for high-powered GPU-based system.
    • Incurred new costs for barcode scanners
  • Mitigation:
    • The UI will guide users through manual product entry in case a product is unscannable.

• Since we removed CV-based object detection, the focus has shifted to barcode scanning, AI_driven recommendations, and dietary filtering.
• The revised schedule is:
  • GNATT chart
• So far, we have made progress on:
  • UI app development
  • Hardware Design
  • Database Setup

With respect to public health: SmartCart promotes both physical and mental wellbeing. On the physical side, it encourages balanced nutrition through its meal planning features. The psychological benefit comes from reducing the stress associated with meal planning and grocery shopping decisions.

With respect to safety: The system’s primary safety feature is its allergen detection and filtering capability. By automatically identifying and flagging products containing specific allergens, it helps prevent potentially dangerous allergic reactions. This creates a safety barrier for individuals with food allergies.

With respect to welfare: Smart cart helps food security by optimizing how suers spend their grocery budget and preventing food from going to waste. It makes nutritious food more accessible by providing smart recommendations tailored to each person’s needs and resources.

• Object Detection Confusion
  • Risk
    • The camera may struggle to differentiate between similar looking items (e.g. Fiji apples vs Honeycrisp apples) or misidentify certain products.
  • Risk management
    • Collect & train the YOLOv8 model with wider variety of grocery products to improve detection accuracy. Optimize confidence thresholds in detection.
  • Contingency Plan
    • Implement a User Confirmation Step: When the app detects similar items, prompt the user with a choice selection (e.g. “Was the item you just put in A, B, or C?”) for quick correction.

• Initially, we planned to populate our own dataset of Aldi grocery items to account for environmental and lighting discrepancies. However, we plan to limit our scope of the dataset to use the online Aldi database of products to prioritize functionality of our prototype. Now, it should be able to recognize a standard set of grocery items without hindering our progress if we were to spend too much time creating a brand new custom dataset.

• App setup & design
• Yolov8 and Open CV pipeline finalization
• Aldi’s dataset collection started
• Finalized  to-be-purchased hardware devices
• Designed device shell