Category: Team Status Reports

An overview of what we did:

• Allen and Lucky wrote the final presentation slides and Allen delivered the presentation on Wednesday
• Since integration between the RPI and the web application had not yet been completed yet, Lucky copy and pasted the people detection code onto his laptop with a webcam and tested it with his existing object detection code. Everything seemed to work well with our initial testing (people were detected successfully, which then triggered his webcam to take a picture of the shelf, the picture was analyzed by the object detection code, and results were displayed on his screen.
• Allen got the RPI to communicate with the web application and he’s in the process of doing the reverse. Once that is completed, Lucky will give Allen his code to integrate it with his web application.
• Lucky continued improving on his object detection code to be more modular and into a library for Allen to use
• Discussed what to put on the poster and will be finalizing the poster tomorrow

In terms of what we will do this upcoming week, we hope to do the following:

• Finalize integration and testing
  • Connecting the web application to the RPI
  • Integrating Lucky’s code with Allen’s
• Finish poster
• Present final demo
• Finalize research paper

The most significant risks that we know of thus far are:

• Integrating RPI with web application
  • Although Allen had a decent breakthrough today, he still has to connect from the web application to the RPI. To mitigate this, he’ll do a similar approach from how he connected the system in the reverse direction.
• Integrating object detection code with the web application
  • Although Lucky’s code is very modular and should have little/no trouble integrating with Allen’s code, we haven’t done it yet. We’ll aim to do this early in the week to give us wiggle room before the final demo
• Latency i.e. meeting user requirements
  • After implementing the sliding window technique, we realized the algorithm could potentially take longer than intended, and may face difficulty in meeting the latency standards we set during the abstract and proposal time
• Communication and storage in the integrated system

Changes To our schedule

• We don’t’ have any significant changes to our schedule
• If things continue to trend as planned we should be okay with some leeway for troubleshooting

An overview of what we did:

• Allen was working on the RPI (i.e. integrating background subtraction code and trying to integrate with the web application)
• Lucky has been turning his code into essentially a library for Allen to use (i.e. takes care of integration between the web application and the object detection)
• Lucky set up our shelf and is the process of testing items with his webcam and code

In terms of what we will do this upcoming week, we hope to do the following:

• Set up the demo shelf parameters
  • Live testing of subcomponents
  • Finalize integration and interconnectivity
  • Complete poster
  • Start outlining how the video will be

The most significant risks that we know of thus far are:

• Integrating RPI with web application
  • Connecting these two subsystems has been quite difficult thus far and for now, we have been simply trying to use HTTP post requests to keep things as simple as possible.
• Latency i.e. meeting user requirements
  • After implementing the sliding window technique, we realized the algorithm could potentially take longer than intended, and may face difficulty in meeting the latency standards we set during the abstract and proposal time
• Communication and storage in the integrated system

Changes To our schedule

• We don’t’ have any significant changes to our schedule
• If things continue to trend as planned we should be okay with some leeway for troubleshooting

An overview of what we did:

• Allen and Lucky finalized the portions of their subcomponent projects
• Tak, Allen, and Lucky are in the process of shifting to integration
  • Portiung Tak’s motion detection to the raspberry pi and camera module
  • Setting Lucky’s subcomponent with a laptop and webcam
  • Setting up the demo shelf

In terms of what we will do this upcoming week, we hope to do the following:

• Set up the demo shelf parameters
  • Get the camera setups finalized (i.e. how far away, how high, what angles, etc.)
  • Live testing of subcomponents
  • Progress to integration and interconnectivity

The most significant risks that we know of thus far are:

• Porting subcomponents to raspberry pi
  • We have already come to face difficulty in getting the motion detection onto a raspberry pi and figuring out communication
  • With delays and what not in getting the second raspberry pi, we may be forced to simply use a laptop and webcam for the object detection component and only one of the raspberry pi’s and camera’s for the motion detection
• Latency i.e. meeting user requirements
  • After implementing the sliding window technique, we realized the algorithm could potentially take longer than intended, and may face difficulty in meeting the latency standards we set during the abstract and proposal time
• Communication and storage in the integrated system

Changes To our schedule

• We don’t’ have any significant changes to our schedule
• If things continue to trend as planned we should be okay with some leeway for troubleshooting

An overview of what we did:

• Each team member presented at the interim demo, received feedback and carried that into the remaining work for our project.
• Each member worked on individual subsystems for our pre-integration project.

In terms of what we will do this upcoming week, we hope to do the following:

• Complete the purchases/manufacture required for our physical space and system (shelf, camera locations, items, etc)
• Begin the process of having our scripts recognized by and running on the raspberry pi, and also test the integration by making different requests at different times.

The most significant risks that we know of thus far are:

• Different designs of our system (Takshsheel’s idea for the camera) and also integration of the local machine work into one system. 
• Completion of our subsystems which is so far on point, however, bugs that are hard to debug could possibly cause serious troubles for completion. It’s not a guarantee, but it is likely we’ll see something of that nature, and it would be critical to solve it at the earliest when that happens. 

Changes To our schedule

• There are no new changes to our schedule since the interim demo.

An overview of what we did:

• We each attended the ethics seminar in person and that lead us to some design choice thoughts and considerations we had not gotten too on our own
  • The main one was considerations on users customizing which items are available by our system to recognize
    • Different approaches had different security concerns and user requirement concerns i.e. users updating the global data set
• We had a couple meetings and conversation to go over our design changes and considerations
  • We decided to have a global data set of images we have prescreened and tested for security / ethical issues, and have tester to meet user requirements
    • Then the user’s can select from our global data set
    • In a real world environment, we would institute a system for user’s to request additional items for us to screen and add to the global data set

In terms of what we will do this upcoming week, we hope to do the following:

• Meet to iron out the specific implementation of our integration components
  • Get the raspberry pi’s running
  • Finish sub systems so we can move to testing

The most significant risks that we know of thus far are:

• Interconnection between our subsystems
  • We have a more high level idea of how the whole system works, but now we have to programmatically begin to implement them in our code

Changes To our schedule:

• There are no changes to our integration schedule of late, we are proceeding as planned
• Takshsheel is changing his schedule where he initially wanted the counter completed by this week which is moving to next week.

An overview of what we did:

• Wrote our Ethics assignments and discussed what we wrote amongst each other
• Received feedback on the design report review and have noted those changes to be implemented for our final version of the report
• Progress was made in the background subtraction for motion / people detection
• Progress was made for the object detection as well as a potential solution to our SIFT / SURF issue
• Ordered and received the remaining parts that we need for one complete wireless system (we were missing a SD card and a camera tripod)
• Completed essentially 80% of the web application before spring break

In terms of what we will do this upcoming week, we hope to do the following:

• Determine how our set-up will look for the interim demo (i.e. how are we going to set up the shelf?)
• Set up the wireless system and attempt to send dummy data to a local machine (i.e. one of our laptops)
• Wrapping up everything on the web application that doesn’t require the input data from the detection algorithms (i.e. designing the UI for the view that shows the data to the user, linking some more URL routes, etc.)
• Continued progress on detection for both motion/people and objects

The most significant risks that we know of thus far are:

• The most significant thus far is still the potential that we will have to work with 2 different camera systems considering the scarcity of raspberry pi’s right now
  • We might have to use a laptop, webcam, and do a different control design
• We are not sure how well the ARDUCAM lenses will work with images (i.e. we believe that it is a fisheye lens and we may need to order another type of lens if that doesn’t work out with object detection)
  • We will determine how well the current fisheye lens works with object detection this week.
• Another potential risk / consideration is integration since we have made progress in detection, but we need to really think through how are system will actually use the detection in an autonomous integrated manner
• A risk we have in the background subtraction part of our openCV background subtraction part is that, there’s not been a resolution to keeping track of how many people are on the screen. Mitigation plan for this would probably involve getting help from our instructors and professors in CMU and research beyond for figuring out how to extract information from a grayscale image.

There have been no changes to our schedule, but in terms of the design, we realized through research in setting up our wireless system that we will need a SD card to run the RPI and the camera will need to be propped up with a tripod. As a result, we ordered both those parts this week and they have recently arrived. We will be setting up the system this week.

An overview of what we did:

• Finished and presented our project proposal
• Went over feedback such as reconsidering which algorithm / process to use to track people and took notes on it
• Viewed other’s research proposals and gave feedback when necessary, we also noted some tools and technologies we could consider for own project
• We continued to research independently for tools on our project
• We will meet to go over our research items and discuss next steps in the coming week

In terms of what we will do this upcoming week, we hope to do the following:

• Decide which web framework we will use
• Decide which database we will use
• Decide whether processing will be done in the cloud or a local machine
  • Decide which cloud platform we will use
• Decide which algorithm we will move forward with for object detection
• Decide how we will detect when there has been motion in the aisle and which algorithm to use for people detection
• Assign specific tasks and deadlines for preliminary implementation moving forward

The most significant risks that we know of thus far are:

• Using algorithms that demand too much computing power or take too long for the systems we run them on
  • Mitigation: comparing processing on cloud vs local machine
• Accurate trigger mechanism for when to run object detection
  • Mitigation: Researching fast motion detection processes i.e. background subtraction
• Poor understanding => poor implementation => non functioning product
  • Mitigation: through research stage, and asking questions on slack to probe the experienced minds of the professors and TA’s

There were no changes made to the existing design or schedule