Status Report 2(9.29-10.5)

Team

  1. Created the slides for the design review presentation. The slides include new requirements that we have changed in the past two weeks.
  2. Discussed details about the design review presentation and reviewed the slides together and discussed about the risks of the project.
  3. We discussed the following potential risks:
    1. The mobile base not being able to arrive at the destination in time. The current solution is to use more powerful motors to increase the speed of the trash can. Receiving info on via bluetooth and calculating power settings takes an insignificant amount of time.
    2. We used the hough circle approach alone to detect the ball that we are going to throw into the trash can and the results are not good. The raw images cannot generate good detection of the ball. The risk would be that hough circle would not work in all situations. We are trying to remove the background to improve the detection results and if that doesn’t work, we will use FMO detection for our project.

Ruohai Ge

  1. Find suitable videos for Kalman filter testing
    • This video has a ball with very high speed
    • Same throw parabola as the normal trash throw
    • The camera angle is also similar
    • Edited it into 4 usable clips for testing
    • Original video

    • original video

  2. Use FIJI to label the coordinates
    • Zheng is still working on the coordinates tracking
    • I need to hardcode these coordinated for the Kalman filter to predict
    • I searched online and found out that FIJI is a really good software
    • One can import an image sequence and obtain all the points you marked into a CSV file
    • largely facilitate my testing pipeline
    • The following showed the FIJI  output csv, I can use the corresponding xy coordinates.
  3. Implement the Kalman filter
    • Figure out the most simple motion model of the ball
    • No noise added to the filter now since the points are hardcode without sensor errors
    • Wrote the whole pipeline to combine Kalman filter, video and result display
  4. Obtain reasonable output
        • The green one is the real ball position labeled manually
        • The red one is the Kalman filter prediction
  5. Deliverable for next weeks
    • Test on moe clipped videos with some noise added
    • Predict the whole trajectory instead of one points and update the trajectory based on update coordinates of the ball
  6. Schedule Status
    • On schedule, Finish implementing Kalman filter this week

Xingsheng Wang

  1. Created the hardware part (hardware design and hardware metrics) and item list of the slides for the design review presentation.
  2. Configured the bluetooth HC-05 module to the desired setup. (Slave mode, baud rate 115200)
  3. Used a new way (mbed + ST flash) to program the STM32 board. This new way is much more efficient and easier to debug than the complex IDE used previously.
  4. Finished implementing the bluetooth test program for the system. Using a phone, we can control an onboard LED via bluetooth. Testing is in progress and by far the response time is fast.
  5. Deliverables next week:
    1. The mobile base have arrived. I will start building the base and look into the motor controls.
    2. Start initial code setup for the motors.
  6. Schedule situation: on schedule.

Zheng Xu

  1. Accomplishments
    1. Tried different background removal algorithm.
      1. The self-implemented diff background removal does not yield good results, as there are more noise in camera frame than expected.
      2. The OpenCV MOG2 background removal has many noises in the actual test, and noise removal algorithm has not been able to remove the noise.
      3. The OpenCV GMG background removal needs some time before it settles down, but the performance is relatively good.
    2. Applied hough circle to removed background. The result is still not desirable, as objects are moving too fast with respect to the camera and the object appears to be blurry.
    3. Did literature review on new algorithms for detecting moving object in the foreground. The ideal algorithm in mind is the Fast Moving Object Detection, FMO. https://arxiv.org/pdf/1611.07889.pdf
    4. Tested camera frame latency and frame rate for data used in FMO algorithm.
    5. Wrote a preliminary code that implements the algorithm, has not yet tested on real device.
  2. Promises for next week
    1. Try Fast Moving Object Detection on actual device
    2. Capture all the moving objects in the frame.
    3. Do noise removal to find the trash we want.
    4. Put all the existing modules into pipeline and test overall speed and frame rate.