Leland’s Status Report 12/7/24

This week, I focused on making some optimizations to the camera settings to reduce latency. Specifically, I worked on the following:

Improved Camera Latency

  • Collaborated with the team to measure latency from user input to system output by switching to MJPEG compression for the stereo camera 
  • Reduced latency to a range of 200 – 250 ms which is about a 200 ms improvement from the previous camera configuration

 

Project Status
I am currently a little behind with the project timeline. The team was busy this week because of the semester coming to a close, and we we’re not able to get an early start to our last additions such as the z-axis. However, we met over the weekend and made a plan for success next week. We have allotted sufficient time next week to finish course assignments and add the final touches.

Goals for Next Week

  1. Integrate Z-axis into the gantry system and implement depth calculation in OpenCV
  2. Conduct final use case tests to include in the final report (water pouring and more object manipulation

Leland’s Status Report 11/30/24

This week, I focused on finalizing and testing key system components in preparation for our final presentation. Specifically, I worked on the following:

  1. System Testing and Validation
    • Collaborated with the team to conduct tests on the project, gathering critical information on use-case and design requirements for the final presentation
  2. Made a New Gantt Chart
    • This outlines the final tasks we have planned and when we plan to do them
  3. Researching Camera Latency
    • During testing this week, we measured a lot of latency from the camera capturing frames and performing OpenCV calculations. We measured latency through video taping which had some human error. I did some research online to measure CV latency directly in software

https://www.dlology.com/blog/how-to-measure-the-latency-of-a-webcam-with-opencv/

  • I also found official OpenCV documentation for measuring performance and possible optimizations I could use to reduce latency

https://docs.opencv.org/4.x/dc/d71/tutorial_py_optimization.html

Project Status
I am currently on track to meet the project timeline. I have high expectations for how our final presentation will go. I’m also confident in the team completing the rest of our tasks, as we planned in the Gantt chart. 

Goals for Next Week

  1. Update the current main project script to include Z-axis computations and send Z-axis commands to the Gantry system
  2. Create the force lookup table. I will create the table by squeezing objects with the gripper and measuring the distance of compression of the foam on the gripper and the force the gripper is exuding on the object
  3. Update the gripper control software such that the gripper will squeeze with variable force instead of binary open or close

New Knowledge and Learning Strategies

The most important knowledge I learned for the part of the project I worked on was the OpenCV library, as this is the crux of the camera system. The hardest part was working with the software that was available for the camera I picked out. The majority of time I spent on the project was with my teammates because system integration was a big challenge. I learned a lot from my teammates, as I had to understand their parts of the project to debug our system implementation.

I primarily searched online to find forums or documentation on problems I ran into or questions I had. However, I would say I learned the most through my teammates.

Team Status Report 11/16/24

Our team made significant strides toward system integration in preparation for the Interim Demo on November 18th:

  • Gantry Integration: Camera and gantry systems were integrated successfully. Jack implemented a solution to reduce latency in gantry movement by breaking commands into smaller steps, improving responsiveness.
  • Remote and Gripper Systems: The remote system was integrated with the ESP32, and the gripper control software was completed, enabling object manipulation. The gripper reliably handled a 500g water bottle during testing.
  • Hardware Improvements: Jack built a new controller housing for the ESP32 and a custom camera stand, enhancing system stability for testing.

These efforts have brought us to a serviceable MVP. Here is a video of testing the MVP this Saturday.

The compressed timeline for integration remains the primary risk. To address this:

  • The team plans to meet this Sunday to ensure the MVP is ready for the Interim Demo on Monday

No major design changes were made. Minor updates include a new 3D printed case for the remote and the construction of a custom camera stand to replace a delayed delivery.

Next Steps

  • Finalize testing and debugging to ensure a smooth Interim Demo.
  • Begin post-demo performance optimizations, including force sensing, latency improvements, z-axis integration, etc.

Overall, the team is excited to deliver a successful MVP.

Here is the validation plan for the user requirements of the overall project.

Leland’s Status Report 11/16/24

This week, I made significant progress by working collaboratively with team members to integrate various components of the system. Specifically:

  • Gantry System Integration: Successfully integrated the camera software with Jack’s gantry system.
  • Remote System Integration: Worked closely with Cary to integrate the remote system, enabling the remote to communicate effectively with the ESP32 for the gripper.
  • Gripper Control Software: Completed the gripper control software, allowing the gripper to respond quickly to remote inputs.
  • Camera Stand: Jack constructed a camera stand, which has greatly improved the stability and usability of the camera setup for testing and integration.

These contributions have allowed us to achieve a fully integrated system.

Currently, I am on track with the revised project timeline. The integration work completed this week ensures that we are well-positioned to achieve our MVP for the Interim Demo on 11/18.

My primary goals for the coming week are:

  1. Address the issues that are presented during the Interim Demo
  2. Start trials for the design and user requirement tests
  3. Start integration for depth measurement

Verification for the camera system

  • Measure the distance traveled in software and compare to the actual distance moved by the user
    • Record the  xy coordinates of the ArUco tag on the remote calculated by OpenCV and compare them with the actual dimensions of the visible workspace

Team Status Report for 11/9/2024

This week, our team has been working intensively to finalize each individual subsystem in preparation for the Interim Demo on November 18th. Our goal is to complete all subsystem development by the end of this weekend, allowing us to move into system integration next week. Each team member has been dedicated to ensuring their subsystem is functional and ready for this critical deadline.

While we’ve encountered some delays and are slightly behind on the Gantt chart, we expect to accelerate our progress and return to schedule this week. The updated Gantt chart continues to guide our team, and we’re adjusting our plans to accommodate the current timeline. Recognizing the tight schedule, we have prioritized all of all next week to prepare for the demo.

The primary risk at this stage remains the time constraint for integration. With each subsystem set to complete by this weekend, we have intentionally allocated next week for integration and testing. This provides us with focused time to ensure seamless functionality across all components. While this timeframe is compressed, we believe that our focused efforts and contingency plans will allow us to mitigate integration risks effectively.

No major changes were made to the overall system design this week. Some minor adjustments within the hardware for the gantry system were made, but these did not impact our core design or budget. Jack has added and is working on more additional 3-D printed parts for the gantry system, as he has been assembling it.

The team has maintained open communication, and all members have committed additional time to ensure readiness for the Interim Demo. Although it will be a rush to complete integration, we are optimistic that we can accomplish this by the demo date. By following our structured timeline and focusing on clear communication, we remain on track to meet our project goals.

Leland’s Status Report for 11/9/2024

This week, I completed the tasks related to outputting the XY location and roll orientation of ArUco tags, achieving one of my primary goals for this week. I also started cleaning up the camera code in preparation for the upcoming system integration phase, ensuring that it is optimized and ready to be implemented. I’ve been able to learn more about the camera software packages to understand it better such that I can configure it for an easier integration with the rest of the project. Here’s a link to my document detailing the updated camera code.

While I am still slightly behind the initial schedule, my recent progress has brought me closer to our revised timeline. I will be packaging the computer vision code for integration and initiating the software for PC and ESP32 communication and other important communication protocols over the weekend. This will ensure a strong foundation for the integration process that will take place all of next week.

Goals for Next Week

In the coming week (and Sunday) , I aim to:

  • Complete the software for PC and ESP32 communication to support reliable data transfer during integration.
  • Develop the control software for the gripper, allowing it to respond accurately to inputs from the remote
  • After these are completed, the main goal is completing MVP by 11/18

These deliverables are crucial for integration, and I am committed to meeting these objectives to ensure the project is on track for a successful Interim Demo.

Leland’s Status Report for 11/2/2024

This week I devised a new test plan for our project, and I got a demo for ArUco tag detection working. Here is a link to my work document.

According to our previous schedule, I’m about two weeks behind in work. Originally, our team was supposed to have at least two weeks of total system integration for the Interim demo. As of now, we will have one week to integrate. That being said, we have devised a new schedule that will lead us to a successful Interim demo. I have a lot of work to do, but I have made this plan and devoted more time next week to complete the necessary tasks.

My goals for next week are as follows:

  • Track Multiple ArUco Tags
  • Track ArUco Tag Orientation (roll)
  • Create software for PC and esp32 communication

Team Status Report for 11/2/2024

This past week, our team has been working hard to stay on track for our upcoming Interim Demo on November 18th. Each team member has been deeply focused on their respective sub-systems, ensuring that each component meets the requirements for our MVP. Recognizing the importance of a clear roadmap, we have updated our Gantt chart to include a more detailed breakdown of the tasks required in the coming weeks, providing a structured path to meet our project goals. In addition to updating our Gantt chart, we have also developed a new test plan proposal.

Each team member has been focused on preparing their sub-system for the Interim demo. However, as our project has progressed, we’ve encountered some challenges, with time being our most pressing concern. Development of the individual sub-systems has taken longer than initially expected, leaving us with only a single week to complete system integration before the demo. This condensed timeline adds an element of risk, particularly in ensuring seamless integration of each component.

To mitigate this risk, our team has doubled down on our commitment to the project. We have collectively agreed to dedicate all available time to work on this project, prioritizing it above other commitments as we approach the demo date. The updated Gantt chart will serve as a vital tool for tracking our progress and maintaining a clear overview of tasks, allowing us to stay organized and adapt swiftly to any obstacle.

The next week is critical to the success of our project. It is absolutely essential that each team member follows the Gantt chart closely, staying aligned with the timeline we’ve established. While the time constraints are challenging, we remain optimistic and committed to a successful MVP demo. We can see a clear path forward, but achieving our goal will require our best effort and complete focus.

Leland’s Status Report for 10/26/2024

This week, I followed the directions from this GitHub repository to download the necessary eYs3D libraries, but also some nice wrapper functions. After that, I was able to run a video demo of the depth map and the normal frames. The GitHub repository included these demos to test and calibrate the camera. Here is a link to my work document for this week.

Unfortunately, my progress is not on schedule. I wanted to include the ArUco tag calculation this week, but I wasn’t able to get that done. I have developed a schedule of my total workload for the next coming weeks. This will help me keep up with my other classes and free up time to spend towards this project. This is included in my work document. Despite being behind on schedule, I’m proud of my work this week. I had a lot of other hard class assignments to complete this week, and I got those done. I’m convinced that if I keep up this pace, we’ll be back on track by next week.

I have my deliverables for next week in my work document, but I will reiterate them here. By next week, I hope to write my own python script that utilizes the software packages I downloaded. This script needs to be able to identify and locate ArUco tags. I need the relative XYZ position of the tags and their orientation.