Elly’s Status Report for 10/18

I worked on finishing the design report for this week and finalizing design choices and implementation details for the software. We decided to use a local database with Apple’s Core Data framework instead of the Open Food Facts API because having a database without the proper product name and price would lead to a bad user experience as certain details could be missing if we rely on an open source database. Thus we chose to create our own database with products from a local grocery store or an online store such as Amazon to simulate the database that a grocery store would have. Another detail we finalized was the pathfinding. We will be using D* Lite due to its strengths in dynamic environments and its simplicity compared to D*. The LiDAR coordinates will be converted into a cost matrix that will be fed into the algorithm as an input. In terms of resources, there is pseudocode and a few Python implementations of the algorithm that we will be referencing for this project.

I am behind schedule as last week as I planned to have the mobile app finalized, but I was unable to work on it due to finishing up the design report. To get back on track, I plan to spend this week finishing up the mobile app by creating the local database and finalizing the UI of the app. Unfortunately, this means that I will need to push back the implementation of LiDAR and obstacle detection, but since there is a slack week, I should be able to get it done by the interim demo.

Team Status Report for 10/4

Overall Progress

  • Ordered all parts for project
  • Received Raspberry Pi 5 and LiDAR Sensor
  • Barcode Scanner and OpenFoodFacts API integration into mobile app
  • Started UWB location tracking between two Apple devices
  • Set up Raspberry Pi 5 and VS Code remote SSH

Risks & Management 

One potential risk is the parts not coming in time. Since a lot of our project relies on the hardware working, if we do not have enough time to ensure accuracy and reliability within our hardware, the rest of our project’s timeline can fall behind. Thus, we hope the parts will arrive sometime next week, hopefully early next week. That being said, if parts do not show up on time, we will try to set up and test our hardware systems so that they are more reliable or ready when the parts do arrive. 

Design Changes & Justification

Earlier this week, we realized that the third-party UWB module we wanted to use would not work unless we sign up for the Apple MFI Program. The UWB module is compatible with Apple devices, but the issue is that we wanted to send information about the iPhone’s location to the Raspberry Pi. Apple does not allow you to send information from its devices to third-party devices unless you are a part of the MFI Program. This program is for engineers who are looking to mass-produce a product, which does not apply to us. After consulting with Professor Spielberg, he mentioned that we could use another iPhone in place of the UWB module since we believe that sending location information between iPhones should be allowed. He also mentioned that we should try and talk to other faculty to see if they are a part of the MFI Program. Using another iPhone does not increase our costs, but if Basket Buddy were to be a real product, it would be an issue.

After feedback and some discussion, we decided to remove the macro-tracking aspect of our mobile app. We removed this to clarify our use case and user. Overall, we realized the nutritional information didn’t add significant value to our project and made the point of our project more muddied. Thus, we pinpointed that Basket Buddy is for shoppers who want a convenient shopping experience, rather than health-conscious shoppers.

Elly’s Status Report for 10/4

I worked on implementing a barcode scanner for the mobile app and connected the barcode scanner output to the OpenFoodFacts database via their API to get the title of the product. The barcode scanner uses the AV Foundation framework, allowing me to create a ViewController that manages an AVCaptureSession. This allows the user to give permission to use their camera and scan the barcode. This outputs a string of numbers, which can be used to make a request to OpenFoodFacts to search for the product name. The product name will be populated in a list that the user can view. One issue with the barcode scanner that needs to be fixed is that the list of items disappears when the user exits the app, so I need to implement a state that allows the user to keep their current session. Another issue is that OpenFoodFacts does not provide the price of the product since this is something that varies by store. Thus, we need to decide how to set the price for each product and whether OpenFoodFacts is appropriate for our project or if we should create our own database.

I also worked on researching pathfinding algorithms for our shopping cart. Last week, I determined a potential obstacle avoidance algorithm that we could use, but in order for the cart to follow the user, we decided to use D*. This is because D* is designed for changing environments, so if the cart detects a new obstacle via LiDAR, this algorithm can replan that path without recalculating the entire path. 

Next week, I plan to fix the issues I mentioned above with the mobile app and get a fully functional mobile app. If there is additional time, I hope to start doing research on LiDAR as we have the sensor and see what kind of information we can get from it.

Barcode Scanner Resources:

https://medium.com/@jpmtech/how-to-make-a-qr-or-barcode-scanner-in-swiftui-68d8dae8e908 

https://medium.com/@ios_guru/swiftui-and-custom-barcode-scanner-f3daaeabfcea 

Barcode Scanner Demo:

https://vimeo.com/1124535079?share=copy 

D* Resources:

https://engineering.miko.ai/path-planning-algorithms-a-comparative-study-between-a-and-d-lite-01133b28b8b4 

https://www.ri.cmu.edu/pub_files/pub3/stentz_anthony__tony__1994_2/stentz_anthony__tony__1994_2.pdf 

Elly’s Status Report for 9/27

This week I worked on expanding on the design of our project in preparation for the design presentation. Since we decided that Bluetooth and GPS would not be a viable solution for location tracking, I helped research other methods of location tracking, and our team decided to use UWB. From there, I looked into Apple’s Nearby Interaction framework which supports UWB and picked out our UWB module (DWM3001CTR13) that would be placed on the cart. The module and the iPhone allow for UWB connection which can be used to ensure that the shopping cart stays within range of the shopper. 

I also researched how to integrate LiDAR and pathfinding for our cart. We decided to use a 2D 360 LiDAR sensor connected to a Raspberry Pi 5. From there, I looked into how we could get the information from that LiDAR sensor and combine it with the information from the UWB module to direct the movement of the cart. Robot Operating System (ROS) is a popular framework for building robot applications which we plan on using, but there may be issues with using it as it may not operate well on Macs. I read a research paper called “Development of Human Following Mobile Robot System Using Laser Range Scanner” which details how the group created a human following robot with LiDAR. In this paper, they mention the collision avoidance function they used, which outlines a collision detection area. If there is an obstacle in the collision detection area, the obstacle has its own circular collision avoidance area. The function then calculates the collision avoidance vector which is tangent to the circular collision avoidance area and in the same direction as the shopper’s direction. This vector is the direction in which the robot will travel. Since this is very similar to our project, we plan on using a similar function for our cart, integrating the coordinates of the shopper that we get from UWB.

We are currently on track for this week since most of the focus is on figuring out the design and design presentation. Although the function from the research article is a good option, there are more pathfinding algorithms that can be explored, so for next week, I plan on having a finalized pathfinding algorithm.

Team Status Report for 9/27

Overall Progress

  • Created our Project Design Slides
  • Mobile app initial framework created and basic UI is in place
  • Picked out parts for TA approval and review

Risks & Management 

A major risk that has come up this week is that GPS does not work accurately indoors. We have verified this feedback and have now chosen to use a different shopper-tracking mechanism: Ultra-Wideband (UWB). This is expanded upon further in our design changes. 

When searching for parts to purchase for our project, we realized that a lot of the robot chassis and other necessary components would arrive, at the earliest, after Fall break. Thus, we had to work around this and choose parts that would specifically arrive on time, primarily from Amazon, and integrate well with the rest of our systems.

A hardware risk that has emerged is that the motors in the robot kit might not have enough torque to allow the robot cart to maintain max speed with 30lbs of groceries. Since these motors are included in our robot’s kit, we will plan to use them for the basic testing and then subsequently upgrade our motors after testing, if needed. We are also looking into other motor options we’ll buy if necessary. We will have to consider how the new motors would fit into our robot’s design – for example, how to attach them to the robot if the hole patterns are different, and how to attach the motor coupler to the axle if it has a different diameter. We also need to consider if the new motor has a different amount of current or voltage needed, and if our H-Bridge can safely supply them without damaging itself.

Another risk we faced is that the robot’s chassis is limited to only supporting a load of 5-10kg (11-22lbs). This could be an issue since one of our use case requirements is that the cart can hold a maximum of 30 lbs. If needed, we will look into reinforcing the chassis, wheels, and axles to support more weight.

Design Changes & Justification

One major change we made was to use UWB instead of Bluetooth and GPS for shopper location tracking. Since iPhones already use UWB, and we plan on using an iPhone to host the mobile app, we decided to use UWB instead for more accurate indoor tracking and Apple’s Nearby Location framework. This adds another component to our project as we need to purchase a UWB module for the cart. 

Another change we made was integrating the barcode scanner into our mobile app and removing the macro tracking. We wanted to better integrate the two functionalities of our project, the autonomous following and item tracking, and decided to move the barcode scanning to the phone to add more functionality to the mobile app. Additionally, we decided to remove the macro tracking since it did not provide a strong use case for our users. Since we will no longer need to buy a barcode scanner, this helps offset the cost of buying the UWB module.

There are no major changes to our schedule.

 

Product Solution Meeting Needs

A was written by Audrey, B was written by Elly, and C was written by Rose.

Part A: Public Health, Safety, and Welfare 

Our solution meets the needs of users’ health and welfare by relieving stress on joints and back and lessening fatigue from pushing a heavy cart for a long time while grocery shopping. Instead, users can freely roam around the grocery store, and the robot will follow them. This also frees the hands of users, making shopping more accessible to those with disabilities, allowing elderly users with arthritis to avoid overextension, and allowing parents to shop while caring for children. 

Our solution meets the needs of users’ safety by using LiDAR obstacle avoidance to prevent collisions with shelves, people, or other carts. Using the UWB module, the cart will follow the assigned user and not get confused in crowded spaces. Additionally, automatic stopping when the assigned user is outside of range will ensure the safety of bystanders. 

Part B: Social Factors

Basket Buddy connects to social factors because it changes how people interact with grocery stores. By creating a simpler shopping experience where users no longer have to push the cart and have an easier way of keeping track of their items, Basket Buddy reduces the stress when grocery shopping, providing broader access to grocery shopping. This allows shoppers to focus on social interactions with other customers rather than worrying about keeping track of their shopping cart.  The built-in barcode scanner and item tracking feature support economic decision-making, reflecting the social and cultural factors around budgeting and diet. Different groups of people may use this feature to manage their shopping finances and dietary restrictions. Additionally, the mobile app feature that allows users to generate a barcode totalling all the items in their cart creates a faster checkout process, with less time spent in lines. Features like obstacle avoidance also help reduce disruptions and conflicts in crowded aisles.

Part C: Economic Factors

Basket Buddy is designed to make use of readily available components. By using parts such as a Raspberry Pi, a Teensy microcontroller, and a UWB module, it avoids the expense of custom fabrication. This approach cuts down on one-time engineering work and helps the system move from prototype to store deployment faster, which ultimately lowers the cost per unit when the cart fleet is rolled out.

Day-to-day operating expenses are also minimal. The cart handles store navigation itself, and item tracking is done on the user’s device, so the cart system doesn ot depend on costly networking or cloud services, reducing recurring data and hosting fees. Its one-hour battery life is sized to match a typical shopping trip without overspending on oversized power systems. These choices mean retailers can introduce the carts without major increases to their operating budgets, while shoppers gain a more convenient and time-saving experience. Over time, that efficiency and the improved customer experience can boost store traffic and sales, making it a worthwhile investment.

Elly’s Status Report for 9/20

Our project will contain a mobile app that allows shoppers to view a list of items and the nutritional facts of the items in the cart. This week, I researched how to develop mobile apps and made a draft of the UI. The home page will be broken up into three sections. The first shows the total number of items, calories, and price at the top. The next section shows the macro breakdown with the total grams of protein, carbs, and fat. Finally, the third section will list out each product with the price and number of calories. When the user clicks on the gear in the top right, it brings them to the controls page, giving the user the option to start, stop, and disconnect from the cart.

To develop the app, I plan to use Xcode with Swift and SwiftUI to run the app locally on a phone. Core Bluetooth will be used to connect and facilitate communication between the app and the shopping cart, and Core Data will be used to make a local database of products.

Next week, I will start coding to create the basic framework for the app’s UI, which will be on track for our schedule.