1. Overview
Our project continues to remain on track, as we make progress in hardware development, CV modeling and system integration. For this week, we made key advancements in model prototyping, hardware selection, and data acquisition. Our prototype object detection model was trained and tested with real-world fridge images, and we continued on finalizing hardware components, including camera configurations and Raspberry Pi integration. For our next steps we will focus on expanding datasets, optimizing the inference performance and integrating the detection pipeline into the full system.
2. Key Achievements
Hardware and Embedded Systems
- Completed a top-level design framework for the project, ensuring seamless integration between hardware, computer vision and mobile components.
- Ordered two additional Raspberry Pi boards to allow parallel processing and improved scalability
- Tested the IMX219-160 camera with Raspberry Pi 4 Model B, confirming successful image capture
- Identified issues with image stitching due to excessive FOV distortion, ordered IMX219-77 cameras for improved results
- Finalized list of components required for prototype, including camera sliders for motorized scanning
Computer Vision
- Trained initial YOLOv5 model on preliminary dataset of grocery items
- Conducted real-world testing by capturing images from fridge and running inference on them
- Successful detection of grocery item within fridge
Mobile App Development
- Continued refining the React Native mobile application, improving UI elements and core inventory tracking features.
- Researched potential libraries and APIs for computer vision integration, laying the groundwork for object recognition features.
- Improved app navigation and interaction between the mobile app and backend services.
3. Next Steps
Hardware and Embedded Systems
- Order camera slider mechanism to enable motorized scanning
- Procure IMX219-77 cameras to replace previous model
- Set up real-time data transfer between camera and Raspberry Pi
- Evaluate multi-camera configurations for full fridge scanning
Computer Vision
- Collect additional dataset of grocery images/ annotate training samples
- Continue with model training to improve performance
- Adjust model parameters/ experiment with other models to improve detection accuracy
Mobile App Development
- Further refine the UI and improve responsiveness for a better user experience.
- Optimize data retrieval and improve real-time synchronization between the app and backend services.
- Begin initial work on integrating computer vision features into the app.
- Conduct additional performance testing and address any remaining compatibility issues.
4. Outlook
Our team is making good progress across all aspects. Although we’ve had to acquire new hardware due to unforeseen challenges, we remain on schedule. In the coming weeks, we will shift more towards system integration, improving model accuracy, and optimizing hardware performance to ensure a fully functional prototype by the next milestone.