Author: lmersky

This week, I focused on making some optimizations to the camera settings to reduce latency. Specifically, I worked on the following:

Improved Camera Latency

• Collaborated with the team to measure latency from user input to system output by switching to MJPEG compression for the stereo camera 
• Reduced latency to a range of 200 – 250 ms which is about a 200 ms improvement from the previous camera configuration

 

Project Status
I am currently a little behind with the project timeline. The team was busy this week because of the semester coming to a close, and we we’re not able to get an early start to our last additions such as the z-axis. However, we met over the weekend and made a plan for success next week. We have allotted sufficient time next week to finish course assignments and add the final touches.

Goals for Next Week

1. Integrate Z-axis into the gantry system and implement depth calculation in OpenCV
2. Conduct final use case tests to include in the final report (water pouring and more object manipulation

This week, I focused on finalizing and testing key system components in preparation for our final presentation. Specifically, I worked on the following:

1. System Testing and Validation
   • Collaborated with the team to conduct tests on the project, gathering critical information on use-case and design requirements for the final presentation
2. Made a New Gantt Chart
   • This outlines the final tasks we have planned and when we plan to do them
     
3. Researching Camera Latency
   • During testing this week, we measured a lot of latency from the camera capturing frames and performing OpenCV calculations. We measured latency through video taping which had some human error. I did some research online to measure CV latency directly in software

https://www.dlology.com/blog/how-to-measure-the-latency-of-a-webcam-with-opencv/

• I also found official OpenCV documentation for measuring performance and possible optimizations I could use to reduce latency

https://docs.opencv.org/4.x/dc/d71/tutorial_py_optimization.html

Project Status
I am currently on track to meet the project timeline. I have high expectations for how our final presentation will go. I’m also confident in the team completing the rest of our tasks, as we planned in the Gantt chart. 

Goals for Next Week

1. Update the current main project script to include Z-axis computations and send Z-axis commands to the Gantry system
2. Create the force lookup table. I will create the table by squeezing objects with the gripper and measuring the distance of compression of the foam on the gripper and the force the gripper is exuding on the object
3. Update the gripper control software such that the gripper will squeeze with variable force instead of binary open or close

New Knowledge and Learning Strategies

The most important knowledge I learned for the part of the project I worked on was the OpenCV library, as this is the crux of the camera system. The hardest part was working with the software that was available for the camera I picked out. The majority of time I spent on the project was with my teammates because system integration was a big challenge. I learned a lot from my teammates, as I had to understand their parts of the project to debug our system implementation.

I primarily searched online to find forums or documentation on problems I ran into or questions I had. However, I would say I learned the most through my teammates.

This week, I made significant progress by working collaboratively with team members to integrate various components of the system. Specifically:

• Gantry System Integration: Successfully integrated the camera software with Jack’s gantry system.
• Remote System Integration: Worked closely with Cary to integrate the remote system, enabling the remote to communicate effectively with the ESP32 for the gripper.
• Gripper Control Software: Completed the gripper control software, allowing the gripper to respond quickly to remote inputs.
• Camera Stand: Jack constructed a camera stand, which has greatly improved the stability and usability of the camera setup for testing and integration.

These contributions have allowed us to achieve a fully integrated system.

Currently, I am on track with the revised project timeline. The integration work completed this week ensures that we are well-positioned to achieve our MVP for the Interim Demo on 11/18.

My primary goals for the coming week are:

1. Address the issues that are presented during the Interim Demo
2. Start trials for the design and user requirement tests
3. Start integration for depth measurement

Verification for the camera system

• Measure the distance traveled in software and compare to the actual distance moved by the user
  • Record the  xy coordinates of the ArUco tag on the remote calculated by OpenCV and compare them with the actual dimensions of the visible workspace

This week, I completed the tasks related to outputting the XY location and roll orientation of ArUco tags, achieving one of my primary goals for this week. I also started cleaning up the camera code in preparation for the upcoming system integration phase, ensuring that it is optimized and ready to be implemented. I’ve been able to learn more about the camera software packages to understand it better such that I can configure it for an easier integration with the rest of the project. Here’s a link to my document detailing the updated camera code.

While I am still slightly behind the initial schedule, my recent progress has brought me closer to our revised timeline. I will be packaging the computer vision code for integration and initiating the software for PC and ESP32 communication and other important communication protocols over the weekend. This will ensure a strong foundation for the integration process that will take place all of next week.

Goals for Next Week

In the coming week (and Sunday) , I aim to:

• Complete the software for PC and ESP32 communication to support reliable data transfer during integration.
• Develop the control software for the gripper, allowing it to respond accurately to inputs from the remote
• After these are completed, the main goal is completing MVP by 11/18

These deliverables are crucial for integration, and I am committed to meeting these objectives to ensure the project is on track for a successful Interim Demo.

This week I devised a new test plan for our project, and I got a demo for ArUco tag detection working. Here is a link to my work document.

According to our previous schedule, I’m about two weeks behind in work. Originally, our team was supposed to have at least two weeks of total system integration for the Interim demo. As of now, we will have one week to integrate. That being said, we have devised a new schedule that will lead us to a successful Interim demo. I have a lot of work to do, but I have made this plan and devoted more time next week to complete the necessary tasks.

My goals for next week are as follows:

• Track Multiple ArUco Tags
• Track ArUco Tag Orientation (roll)
• Create software for PC and esp32 communication