Posted on

Team Status Report for 3/16/24

After doing a post spring break evaluation, the most significant risks that could jeopardize the success of the project revolve around the camera, both in terms of its function and the dependencies that require its outputs, such as the kinematics calculation module. Despite having a pipeline that can detect depth and display a live camera feed smoothly on my (Nathan’s) laptop, when using X11 forwarding, the resulting feed was extremely slow and laggy. Our plan to manage and mitigate this risk is to get our RPi monitor as soon as possible to test on actual usage as well as look for any opportunities to lower bandwidth and latency. The Luxonis documentation has benchmarks to test these values so we can analyze if we have any shortcomings. Additionally, another risk that stems from the first risk is the fact that we are behind on our timeline. However, we have recently placed orders for pcbs this week so for these fabrication tasks, we have controlled what we can control in terms of timing. This week saw a lot of fabrication of parts, so our next weeks will see an abundance of integration and in-person meeting time.

No changes were made to the existing design of the system and it is consistent with the change made after spring break.

Although no official changes to the schedule have been made, we are systematically cutting things out from the future and trying our best to push forward in terms of days.Here are some parts Hayden and Varun made!

Posted on

Nathan’s Status Report for 3/16/24

For my progress this week, it was threefold.  The first aspect of this week involved trying to successfully boot up the Raspberry PI and SSH into it in a headless fashion since that is how it will be operating on the rover. However, I ran into some trouble earlier this week when I broke a microSD card while attempting the first boot, so progress on this front was delayed until the end of the week. Towards the end of the week, I successfully managed to SSH into the PI and register our devices MAC address to CMU-DEVICE.

The middle half of the week saw me spend pretty much all of Thursday doing the ethics assignment, and I found both readings extremely insightful.

The last part of my week involved installing dependencies and running the depthai software on the Raspberry Pi remotely through SSH. After doing research on the ColorCamera issue as stated in previous status reports, I might have found my issue, which is that when the device is connected to the Pi, and when it reconnects back to the host after performing inference, it is using a USB2 connection instead of a USB3 connection. One potential way to solve this is to reduce the data stream, but as of now I am unsure how to do this.

Currently, I am working on two different pipelines. MonoCamera -> Object Detection Pipeline and a depth pipeline that uses a colormap where I am trying to extract coordinate data and potentially link it with the Object Detection pipeline.

Currently, progress is behind because there are still issues with the camera being laggy on the Pi side as well as continued development that needs to be done with my two pipelines. With no exams this coming week, I hope to catch up to the project schedule by utilizing the week’s free time effectively.

In the next week, I hope to develop a pipeline that takes the closest depth value in an image as well as find a method to reduce data rate by potentially editing a bandwidth or FPS setting.

Posted on

Varun’s Status Update for 3/16/24

This week’s theme is fabrication (and ethics)! I’ll break down my week in terms of the days-

Wednesday (evening) – I spent the evening finalizing the internals of the servo box. On Tuesday morning, I realized that my original plan for assembling the servo box was inefficient, which was to use a long M3 bolt with a captive nut on the other side of the box to hold it together. I realized that if I just used a heat-set insert on the top of the larger half, I could reduce the bolt size and therefore reduce the likelihood of any shear stress snapping the bolts in half.

Thursday (evening) – Most of the day was spent on the ethics assignment, but I also found the time to print and verify the fit of the servo first stage with the servo.

Friday (whole day) – During the first part of the day, I went to TechSpark and waterjetted the parts. We were intending to waterjet on both Monday and Wednesday during class, but there were people using the waterjet during the class time; the part would take too long and eat into the next class. I began waterjetting the part on Friday morning, and much to my annoyance, there were multiple hurdles within TechSpark to get the actual board cut out; however, with an hour-and-a-half delay, I finished cutting what will serve to be the base of the Rover!

Finally, Hayden and I went back to RoboClub, where our parts were 3D printing, and ironed out some imperfections with the board. (there were some holes that for some reason hadn’t been cut out by the waterjet, and the nature of the material had created some imperfections that affected the way the board sat on the table.) The rest of the day was spent printing out the rest of the parts, and one-by-one verifying that they would fit together. Here they are! (the third stage is printing as I write this). I’ll be assembling the arm tomorrow.

We are still behind, but if we finish the arm by tomorrow (as planned), we can make up time next week (by expediting Rover manufacturing and doing integration earlier than anticipated.) By next Sunday, the physical Rover needs to be done. In my downtime, I’ll be working on verifying motor encoder functionality.

Posted on

Hayden’s Status Report 3/16/24

This week I made a lot of progress. I spent Friday helping Varun fabricate parts for the robot arm. We should be able to complete the drive train parts in the next few days. I also finished the PCB design and will be able to order it ASAP once I confirm the parts are available on the website. I got a breadboard and tested my code from last week and it works; now that the code works the PCB was designed to work with this code. After we finish fabricating the rover in the next week we will be able to work on the kinematics; I have finished my research for the driving kinematics and I am currently looking into Wi-Fi communication between RPis so I can get that up and running after the rover is built.

Attached are some screenshots of the PCB and the code working for the controller:

This is the breadboard I used to confirm the design.

We are still behind on fabricating the rover, but once we have it created I believe it will be much easier for us to write meaningful code for the kinematics and debug it. We are on track to have a moving rover by the interim demo.

In order for things to be on track I need to purchase the drive train components by Wednesday and have all the other components 3D printed by then as well. I will also order the PCB ASAP because it might take a while to ship. I also need to confirm with other members of the group that we have purchased/acquired the RPis and the monitor to make sure we can get the controller side complete in the next two weeks.

Posted on

Nathan’s Status Report for 3/9/24

For the past couple of weeks, the primary focus I had towards our project was twofold: the design report and the camera. A significant part of the week leading up to spring break, around 10-12 hours worth, was spent on the design report. For my sections of the introduction, tradeoffs, testing & verification, and project management, it required a lot of time spent doing research on current statistics about individuals facing mobility challenges as well as existing solutions. Additionally, research was conducted on tradeoffs between Wi-Fi and Bluetooth as well.

For the camera aspect of the project, I read and investigated the source code of the Luxonis camera repository, specifically the depthai-viewer tool. I wanted to see why the rgb capability was possible when using their tool but not when I manually write a pipeline that utilizes it. In addition, I am also investigating an example that utilizes a pointcloud and see what useful data arises there, but the bottleneck still lies with the color camera issue. Because the depthai-viewer registers the cameras as a default stereo pair, it is possible there is a configuration that I am missing.

Now that the design report is complete, which took a majority of my time for the week, I can fully focus my energy on both setting up the Raspberry Pi as well as getting to the bottom of the camera issue. I started booting the Raspberry Pi and setting up SSH, but I accidentally broke the SD card, so progress was temporarily halted on this front. Thus, my progress is slightly behind but I hope to be able to catch up with this narrowing of focus and fixation on my tasks.

In the next week, I hope to successfully setup the Raspberry Pi in a headless fashion, set up SSH on the Pi, and be able to run my tutorial that utilizes the ColorCamera feature on the Oak D SR.

Posted on

Hayden’s Status Report 3/09/24

These past two weeks I’ve spent the majority of my time verifying the design and touching up the CAD so we will be able to hit the ground running after Spring Break with fabrication of our rover. I spent probably about 8 hours alone confirming our design and finding sources for our design proposal report. As far as progress, I have not been able to work on the physical fabrication our experimentation parts because I was ill the week prior to Spring Break and then I have been out of town. However, while working on our design report submission I confirmed the ability of our design to drive, lift items properly, and receive data from the encoder on the servos at a high enough rate to have the amount of control we would like.

Additionally, I finally had the time to write some Arduino code and I was able to get the analog button presses to translate to outputs from the Arduino. Here is the basic code I was able to write with the help of one of our design sources.

Looking at the schedule I am pretty much on task as far as design; nevertheless, I am very far behind as far as fabrication. This week Varun and I plan on using the water jet to create our bottom plate and we will be able to keep 3D printing components. Once, we order the final components of the drive train we will be able to assemble the entire rover and the only part we will have left is integration. We were very optimistic in believing that we could fabricate the entire rover before break but now that our design is set in stone I believe we will be able to get caught up in the next two weeks. Kinematics is still in the works and I need to put a lot more effort into the coding aspect of the project now that the CAD is finalized. This next week I will finish the entirety of the 3D printed drive train modules and I will finalize a PCB so I can order one and begin debugging ASAP.

Posted on

Team Status Report for 3/09/24

The biggest risk, which I (Varun) only realized while testing, was whether or not the suction cups could even hold up an iPad. Thankfully, that risk was provably mitigated. Currently, the most significant risk is the kinematics. We’ll need to update the previously thought-out kinematics to be slightly more robust (described in our design report). I hope to figure that out this coming week.

We changed the block diagram of our system a bit, to account for said kinematics. We’ll need fast encoder feedback, which can really only be done on bare metal microcontrollers, rather than the OS-stuffed Raspberry Pi 4. We updated our block diagrams to reflect this change. It adds the cost of the Raspberry Pi Pico, which is rather minimal. Varun’s status report has the progress of the suction system!

Part A – Varun: 

Our target demographic is, by design, people who are not technologically savvy. As mentioned many times prior, the inspiration for this project came through Hayden’s grandfather, whose capability to move is rather limited. Also limited is his technological ability. As such, we decided to simplify the control sequence to ensure that people whose technological ability is at the same level can easily use HomeRover. Additionally, the automatic sequence to grab an object is easily reachable by the touch of a button, so users don’t have to spend a lot of time to learn how to use our system. The factor we want to touch is one that is not talked about much, which is recovery. Though recovery is extremely important, it is a part of the process that is hard to suffer through. We aim to ease that suffering just a little bit, with HomeRover.

Part B – Hayden: 
In regards to different cultural factors our design is robust in a manner that does not require a written language. Our control center uses four buttons for controls which will be labeled with arrows and two buttons for interacting. The two buttons will be labeled with icons that can be translated in a user manual to make the design usable by all cultures. The monitor will display when the rover is in the proper range to pickup the item using colors; red and green are seemingly universal for good and bad so we are employing these colors on our display. As far as moral values in religions our design uses ethically sourced materials and we are mitigating waste by using our modular design. As far as traditions and laws, the only groups I can think of that we are violating their beliefs with our design are the Amish and Mennonite communities; these groups are not in our target demographic so our design is sufficient culturally.

Part C – Nathan:

Considering environmental factors, especially when pertaining to our design’s relationship with living organisms and natural resources, we are primarily concerned with the sourcing of our parts. Because our design utilizes LiPo batteries on both the user side and control side, we have to be cognizant on the origins of the lithium, and by extension the cobalt, that goes into these rechargeable batteries. According to Siddharth Kara, “roughly 75 percent of the world’s supply of cobalt is mined in the Congo”, and oftentimes this is by the hands of child labor. If we are not aware of the origins of the cobalt that go into our batteries, we are directly supporting these brutal mining practices that are causing both human and environmental catastrophe through the exploitation and exhaustion of the natural resources of the Congo land. This is an environmental consequence of utmost importance and it will be considered heavily when purchasing the parts for our system.

Posted on

Varun’s Status Update for 3/09/24

I was able to finish the CAD for the pumps, which is quite important for the overall project, and a lot more specifically for the design review. I also worked quite a lot this week, indeed, on the design review. I worked on the implementation side of the design review, and was able to individually place screenshots of the work that I did in the design review such that I could describe each system in a more detailed fashion. As such, we ended up being right at the edge of the max quantity of pages, but we should be under the quantity limit for each individual section!

I was also, thankfully, able to work with some physical parts this week. I was able to retrieve most of the electronic guts for the Rover, which will be important as we continue to fabricate it over the next week. The most important mechanism for me to test was the suction mechanism. I wanted to make sure that with the motorized pumps, we would be able to lift our target weight. When I got the pump in the initial part delivery, I found that it could only push air out, not pull air in. After I spent awhile inspecting the inside of the pump, I figured out how I could add a 3D printed part and essentially add a suction input. A picture:

The blue plastic part above is the 3D printed part. Though this did add some suction, it wasn’t nearly enough to hold my iPad, which is our weight goal. I was able to source an original plastic heading for the DC pump that had the suction in-built, and with just two pumps, we were able to lift an iPad, our target weight! The video.

Though this is good, we are still a bit behind on fabrication. We have cut time on other things, such as with the OAK-D SR, which should keep us on track, but we’ll have to put a bit more time this week to meet our fabrication goal. I hope by next week to have the completed Rover; if not, the Robot Arm fabricated.

Posted on

Nathan’s Status Report for 2/24/24

For the first half of this week, from Sunday to Wednesday morning, I was practicing my Design Proposal presentation. I wrote down meaningful talking points I wanted to hit on each slide, and then I made sure to get the slide transitions down smoothly and without pausing or awkwardness. In addition, I practiced my timing and in front of a live audience (my roommates). For the latter half of the week, I dove in-depth into the Luxonis DepthAI documentation to try and figure out which frameworks and functions would be useful for our application. I read numerous examples from Luxonis’ depthai-experiments repo to try and find the relevant depth processing we need. Alongside this experimentation, I was figuring out the nuances behind the dependencies needed to install the required packages. Currently, I’m facing an issue where I am unable to perform RGB Color Camera capture; it doesn’t crash only when I run MonoCamera applications, which is odd. I’ve tried troubleshooting versions and I’m still investigating this issue.  The following photos below show the depth application examples I got to work that do not involve RGB color capture.

In addition, I made a repo to get started on a hello_world.py file where they walk you through how to create a pipeline to then start image capture and image processing, and the Github link to this repo is https://github.com/njzhu/HomeRover

My progress is slightly behind because of this small issue with the rgb camera, but once I figure that out, I hope I am able to understand the examples and apply depth capture to objects in our surrounding areas and rover’s vicinity. Since the examples are made by Luxonis, it should be extremely helpful and informative.

In the next week, I hope to get my hello_world.py up and running successfully and be able to do basic depth capture using the stereo depth perception technology on the Luxonis.

Posted on

Hayden’s Status Report 2/24/24

This week was way less productive than I wanted it to be. Sunday was dedicated to finishing up the CAD for the rover and finalizing our design review presentation. While I did not accomplish much in terms of completing design tasks for the rover beyond the CAD, I spent a lot of time learning how to calculate kinematics for the motion of the rover. I should be able to start working on the software system for the drive train. Since the drive train and the design are now finalized I was able to calculate the RPM we need the motor to spin to meet our target max speed of 0.5m/s. Given the fact the wheels are 2.5 inches in diameter we need to have an RPM of 150 to be at the maximum of our driving speed. I have began to relearn and mess around with Arduino code from the following sources that are inspiring the PCB design:
Source 1
Source 2
Source 3

Unfortunately I fell ill towards the end of the week and I have not been able to fabricate parts with Varun; nor have I been super productive. However, I am finally back on schedule and the time I’ve taken to learn kinematics and review Arduino programming will definitely streamline things in the future. In the next week I will finalize the PCB design and test my Arduino code with a basic breadboarded circuit. I will also help out as much as I can with fabricating the physical rover. Once again we are very proud of our CAD so here is another angle: