Author: shubhij

Achievements

This week I spent time on the fullness detection model, playing around with different implementations for detecting the fullness to see what would increase the accuracy. Right now the carts all seem to be detecting to be similar fullnesses, despite having different fullness. The current approach was using edge detection to count the contours, but since our system is capturing the carts at a relatively slow fps, the carts are also a little bit blurry, rendering the edge detection a little inaccurate. Due to this, another approach I was playing around with was observing colors, and the number of black pixels there are in the frame. This also didn’t seem to the have the best accuracy, so I am still in the process of figuring out what could be the best approach.

Progress

Since I am not really happy with the current accuracy of the fullness detection, so I will be spending the next two days to increase it before the final poster is due, after which we will be working on the final video and report. We also need to work on the final demo because all of our testing has been done at Salem’s, but once we have a 1-hour long video to display on a monitor for the demo, we also need to make sure that our system will work for our makeshift in house checkout system that we will be demoing in person in Wiegand.

Achievements

We went to Scotty’s Market and gathered more footage data for developing and testing purposes at Salem’s Market. We also finished integrating all the existing modules to be able to provide an output based on some footage we currently have. I also finished redoing the fullness acquisition module to use edge detection and improve accuracy for cart fullness. I used edge detection and contours to figure out the area occupied by the items within in the cart, and then after playing with the numbers I worked on increasing its accuracy. I also have been talking to Salem’s about getting permission to install our system in the market, and he brought up concerns about it impacting his business and some safety concerns, for which he wants to meet in person to talk about. I also got permission from him to borrow a few carts from Scotty’s Market for our in person demo, so we will be able to set up a realistic system for our demo in person as well.

Progress

After talking to Salem’s market, I need to schedule a meeting with the owner in person, and he requested that our advisor come as well, due to his concerns about the way this system will impact his business. His biggest concern is how cameras make people of minority groups feel, as well as if he were to give access to his pre-installed cameras, whether the security would be compromised. I hope to resolve this issue in the following week so that we can move on with more testing, but in the meantime, I am working on solidifying the modules to be more accurate, and I think based on our new daily plan we are on track.

Achievements

We got confirmation from Salem’s Market to test the system – they have 6 checkout lanes but we will only be testing out 3. Currently in the process of getting CMU approval to test at Scotty’s but no answer yet, due to legality concerns. Implemented relative fullness detection, but need to still test it out.

Progress

Since we don’t have Scotty’s approval but we have Salem’s Market approval, we will be testing at Salem’s in strip district, but a little far so not the most optimal, and will take more time. We need to still integrate all components of the system, but that should only take a few days and then we can use the rest of the week to test out the system, in time for the the demo.

Achievements

This week I worked on catching up and making progress on the project. I was able to wrap up implementing the database that all our modules will communicate with. Additionally, I went to Scotty’s Market to take pictures of carts so I could train a custom model that I could use in my implementation of identifying carts to be able to observe the relative fullness, which I also finished up. I also wrote the pseudo code for implementing the relative fullness. I reached out to Scotty’s Market, and followed up with multiple contacts, but no one has gotten back to me, so it is looking like we are going to have to makeshift a grocery store to test our system.

Progress

Since I have the module flushed out, I just need to wrap up implementation of identifying the relative fullness of the carts so that we can start testing the full system. I plan to get it done this week so that we have a week to test the system. I think we are catching up and making good progress on our system, and we should be done with implementation soon. 

Accomplishments

This week I spent a majority of time on the design report, especially since we had to make many changes to our design as I was working on object detection and realized that we can’t realistically count the number of items in each cart via the camera pipeline. Since it would be too inaccurate for our use case, we realized the better way is to look at the relative fullness of the cart and use edge detection to estimate the 2D space that is taken up by the items in the cart, and assign a quantitative number to that cart as a metric for relative fullness. Due to this design change, I went a little bit backwards in my research, as I had to switch gears and understand more of how to use edge detection to accomplish this. I spent some more time also doing research to account for this, which has set me back in progress a little bit. Additionally, since we are now not working on integrating an FPGA to our system, we have now decided to convert the repo to be on a python stack to allow for more simple OpenCV usage, which is something I reset this week. I also reached out to the director of dining services at CMU to talk about working with Scotty’s Market for our user testing, but I have not received anything back from them. There may be a chance that we have to create our own testing scenarios.

Progress

Currently, I am a little bit behind on the implementation according to the Gantt chart, but I plan on working extra this week to make up for it, and will hopefully have majority of the implementation done and preliminary tested with the camera that we ordered. To do this, I will be meeting with my group extra as well this week so we can all help each other get back on track.