Lucky’s Status Report for April 16, 2022

This week I did the following tasks:

• Deeper research into more complex segmentation approaches
  • Upon further research and designing of sliding window I found some disadvantages to the approach
    • Fixed window size or aspect ratio can result in not capturing the full objects in windows or expensive computation
  • I learned about modern techniques that were developed to tackle that issue i.e. R CNN and Fast R CNN, Faster R CNN, and more
  • Therefore, I compared K-means, Mean Shift, and Graph Based segmentation
    • K-Mean: predefined number of clusters, randomly set centroids (will be the mean(s) i.e. the average of the colors in a cluster) for each cluster, assign each point to the nearest mean thereby creating a cluster, then recompute the means and repeat until the means stop moving i.e. convergence
      • Pros:
        • Not as complex as the other two approaches
        • Very fast
        • Multiple initialization methods can be used
      • Cons:
        • Manually have to set number of clusters
        • Sensitive to initialization
        • Sensitive to outliers
    • Mean Shift: for each pixel – first compute mean of the pixels in a surrounding window, then move the pixel’s assigned centroid to the new mean location, repeat until the mean converges to the nearest mode which will be the local extrema i.e. the hill of a normalized density feature mapping. Each hill will be the centroid of the pixels around them thus separating the pixels into clusters
      • Pros:
        • Simple algorithm
        • No initialization required
        • Robust to outliers
      • Cons:
        • Computationally expensive
        • Sensitive to window size parameter
    • Graph Based Segmentation: each pixel is a vertex, an edge is produced between neighboring pixels, each edge is assigned a weight based on affinity / similarity between the pixels/vertices, then the graph is partitioned to subgraphs where pairs of pixels in the same subgraph have a high affinity, and pairs of different subgraphs have a low affinity i.e. a min-cut
      • Pros:
        • Complex segmentation capabilities
      • Cons:
        • There is a bias to cut into smaller segments
          • Requires normalizing cuts to favor larger subgraphs
            • No known polynomial time solution for minimizing the cost of a normalized cut, but there are approximations
• Upon comparing all of the solution approaches I used the following constraints:
  • Low computational expense and latency to meet user requirements
  • The camera is going to be fixed and the shelf is of a fixed size and distance from the camera
    • Results in images of fixed aspect ratio
    • Results in items having a constrained placement on shelf rows
  • We constrained the size of items for the scope of this project
• Therefore, I decided that given the constraints and scope of the project, an implementation of sliding window would be the feasible design solution
• Completed a functional sliding window matching algorithm
• Put together the shelf we bought

 

In terms of schedule, I think I am around where I expected to be. I would like to move faster this next week though to give some more leeway for hiccups as we get our project to something worthy of a decent final demo.

 

What Next

• Set up webcam and shelf with the following specifications to begin live testing such that the frame is composed of only the shelf for speed and accuracy since there would be minimal sliding window matching segments outside of shelf area
  • Distance from shelf to webcam:
  • Orientation of webcam:
    • Using SIFT matching which is supposed to be rotation invariant, but algorithm rotates test image for the sake of user visualization while testing
  • Height from floor of webcam:
    • Centered to shelf to allow for most forward facing camera angle of complete shelf
  • Figure out a set original captured Image aspect ratio and adjust sliding window accordingly