Progress
In week 11, I had progress in recording the IR Code for the IR Audio Amplifier and the IR Disco LED lights. In addition, we successfully did our interim demo, showcasing our current progress on IRMAN development. Shirley and I have also tested on image transmission and the image stitching approaches in the image preprocessing pipeline.
This marks phase 4 Entry and CV & MechE are the main focus of the team.
We have extended our research on the different approaches for the CV image pre-processing. Since the camera is mounted on a base rotating about Z-axis, in the calibration phase, the camera is going to rotate 360 degrees and take finite amount of pictures at each stop. Let the number of pictures taken be N. Granted that the camera rotates in clockwise, each image m_i , for all i in N will cover 360*i/N field of view region. Our first approach is to stitch N photos into one photo using image homography using OpenCV library. Mounted on our stepper motor, the following panorama photo is automatically generated with uniform stepping.
However, when we feed that into our trained CV model, the algorithm fails to recognize the TrusTech A/C. We speculate it is due to the image warping given the nature of the homography algorithm. Because of the different perspectives in view and imperfect camera, the photos are inevitably going to be warped to some extent. Therefore, we took the original approach, which is to perform object detection by individual photos. When we feed the following photo into the CV algorithm, it successfully classifies our target device (TrusTech A/C) as bounded by the yellow box.
The next step is finding the centroid of each device in the “absolute location” as shown in the following snapshot of our final document.
After finding the “absolute location” of the device of all target devices, we are going to return the device locations as a dictionary back to IRMAN (through RPi Client socket) and stored locally. When specific device command is invoked again on WebApp, the IRMAN can quickly search through the device list and find out the device location and plan the motor accordingly towards the target device.
Schedule
This week’s development is on schedule. Now shifting more focus to CV pipeline (image post-processing, device location search) and mechanical design and integration.
Deliverables Next Week
- Start making the IR Man figurine and all the necessary 3D printed parts.
- Start writing the multi-threading client code to receive asynchronously for the device location list
- Finish all the motion codes for the motor and think about the operating scenarios.
- New IR Devices Arrive and Device IR registration
- Mechanical Design: Integration and 3D printing rotating base
- Mechanical Design: IR MAN Figurine Design and Manufacture
- Benchmark Testing and Metrics Reports for each MVP
- Start Soldering the IR Circuits onto the Proto board.
0 Comments