Team Status Report for 12/4/21

Over the past week we have been working on the two pending tasks on the schedule: final testing and housing. We laser cut the remainder of the pieces and attached them to the 3d printed arms and brackets we made before the break. The body is pretty sturdy and awaiting drop testing, but as a form of risk mitigation we still have our prototype. The new housing unit is completed (video) so the only other task in our schedule is to finish up testing. In today’s round of testing we had four trials:

Min: 1.27 m

Max: 2.84 m

Mean: 2.40 m

Median: 2.74

We have been making adjustments to the code as we go with our trials, like adjusting binary threshold and thrust from trial to trial. We have also been able to reduce the latency by removing a delay in the function that sends data to the Arduino via serial. The movement is much smoother after we made this change. Our goal for the next week to finish drop tests by Tuesday so we can switch focus to shooting our video and working on the final report.

Lahari’s Status Report for 12/4/21

Over the past two weeks we continued to work on the final housing unit and did more drop tests. I laser cut the remainder of the pieces including the lid, and a few more of the wall pieces. I fully assembled the housing over the last few days, then we glued everything into place. Vikram and I soldered some new connectors for the new housing and we mounted a new set of motors to the arms. We plan to showcase this product instead of the prototype at the demos and in our video. We have been working on final testing as well over the last few weeks.

Before the presentation  we did several tests pertaining to our design choices. Namely, we collected data for tradeoffs related to camera lens, camera resolution, battery cell count, and propeller size. I redid the test of camera resolution against detection rate and frame rate as the results we had from the week previous were done under varying light conditions, the new results showed a more clear tradeoff between frame rate and detection rate. Since the presentation we have been working on final drop tests as we were unable to get enough trials in before the presentation. We completed 4 trials of drop testing today. The closest landing distance was around 1.25 meters. We have found testing to be pretty feasible at night as well, with a lower threshold value since the light from the bridge illuminates the target.

Vikram’s Status Report for 12/4/2021

This week, I first worked on preparing for our final presentation. I practiced for the first few days of the week memorizing and rehearsing the slide deck and what I would say to make sure that the presentation went smoothly.

We also worked further on assembling our new housing out of 3D printed arms and acrylic body panels. We also installed a hinge for the top for easier access to the inside. We glued the bottom inside the hexagon and fixed the side panels with custom 120 degree brackets to form an even hexagon. The arms are bolted on, and the walls are tacked together with glue at the top. I also made the wiring adapters that go from the 2mm bullet connectors from the motors to the 3.5mm connectors for the ESCs.

Lastly, we performed about 7 drops with our system to characterize performance. While doing this, we noticed an issue with the ability to SSH into the raspberry pi above the drop zone because of wifi, so we amended our script to wait until it receives signal over UART from the Arduino to start the detection script. This way we run the python script while indoors and then when we hit the button to start the Arduino, it triggers the python script to start without needing to SSH in and run it directly. This helps our efficiency with performing drops. Additionally, we tested in a few different lighting conditions, ranging from bright sunny to very dark and with tuning our threshold, we were able to detect the target circle in both scenarios. A future possibility is to have a dynamic tuner that changes threshold based on amount of light or another metric. After some more testing, we will be looking to create our video and write up our testing data in the report.

Daniel’s Status Report for 12/4/2021

At the beginning of this week, I mainly helped the team work on the final presentation. In addition to working together on deciding slide content and layout, I personally worked on data analysis to extract latency values of various parts of our system. We had previously recorded various slow motion videos in an attempt to measure latencies of core parts of our system; for example, to find the time from the Arduino sending a PWM value to the motor to actually seeing motor movement, we recorded a video with the motor in the frame, and an LED that would turn on when the PWM value was sent. These videos all included a stop watch in the frame to keep track of time. In order to extract the latency values for this, I imported the videos to iMovie and went frame by frame, writing down the two time stamps of when the LED turned on, and when the motor actually spun. The difference between these was the latency for this part of the system. I recorded these values on a Google Sheets and found the average of all the runs we did.

I then created a slide in the presentation to show where these latency values fit into the overall system, showing a block diagram of the major steps of the system, and the latencies between them. After this, I also helped the team record a quick demo video demonstrating the camera to propulsion pipeline.

Later that week, we started working on finalizing the laser cut and 3D printed housing. Vikram and I glued the walls to the floor on the base of the housing using hot glue, and I also soldered 16 AWG gauge 2 mm to 3.5 mm adapters for the motors to be able to connect them to the ESCs.

Lastly, we spent the end of the week testing our full system. As usual, I would hold the device and drop it as Lahari initiated it and Vikram caught it down below. Since this testing would be used in the final report, I took a picture from above after every drop, and recorded the distance the device was away from the target that Vikram measured. During our drops, we encountered an issue where the unstable WiFi connection on the Pausch bridge meant that SSHing into the Pi and running the CV script could not be done consistently, thereby lengthening the time for each drop as we had to wait for the Pi to reconnect to the WiFi. To fix this issue, we went back to the lab and I edited the Python script on the Pi to run once the Arduino sends a particular message to it over UART. This meant that the Pi’s CV portion of the script would only start once the button connected to the Arduino was pressed, thereby allowing us to stop relying on a stable SSH connection as we could run script while we had a WiFi connection, and the Pi would simply wait for the button press to begin. I also tried to make the script run automatically after the Pi booted up, such that we would never even have to SSH into the Pi; however, some issues regarding the version of Python the .bashrc file was using meant that our script could not run properly.

We dropped 7 times, and while wind was an issue at certain times, we managed to get our first drop within the target foam-core rectangle, around 50 inches from the center of the target. Additionally, as it got darker, detection got weaker. Lahari and I changed the threshold of the CV algorithm, allowing us to keep dropping in darker conditions.

We will continue testing the system in the coming week,  as well as record our final video.

Team Status Report for 11/20/21

This week we wanted to test the propulsion fixes we worked on last week, since we got the chance to charge the battery fully. With the weights also redistributed and the thrust at 75% we were able to produce some movement. It was particularly windless that day, so the motor off drop is directly below the dropping point.

Pictured: Drop with motor off, 50% thrust, 75% thrust

On Monday we did a camera test to rule out the fisheye Raspberry Pi camera against a equirectangular Logitech camera. The camera is a bit heavier but the detection is a lot better that the Pi camera. Once we chosen the Webcam, we also varied the the resolution to see the tradeoff between detection rate and fps. With 480p we had 62% and 3.9fps with 720p we had 46% and 1.7fps. We will likely repeat this test, because the change in lighting between trials was substantial which is why the detection rate is not higher like expected, but the frame rate is clearly to low with 720p. Any smaller resolutions compromise detection and larger ones compromise frame rate.

The rest of the week we spent planning and fabricating the new device housing. We made several CAD models, laser cut most of the body, but were unable to 3D print because of time. We will be 3D printing the arms on which the propellers attach, and this will be a priority next week.

Lahari’s Status Report for 11/20/21

This week in class we tweaked the computer vision and worked through our propulsion issues. The later part of the week we finalized the design for the robot and did some fabrication. On Monday, we went out to test the propulsion with the fully charged battery added thrust. These changes seemed to make all the difference and our device was able to move a sizeable distance based on the photos attached. There was a some mid-air rotation so this will be the focus going into next week.

We also did a test to compare the Raspberry Pi (fisheye) camera and the Logitech webcam to see which was better for detection at different lateral distances. With the large circle (2-meter diameter) I printed this week, there was 0% detection with the Pi camera and about 60% detection on the Webcam on average. We will be moving forward with the webcam. We also tested detection rates and frame rate against resolution and found 480p to be optimal.

From Thursday onwards we made CAD models and fabricated the final design in TechSpark. Vikram made the designs for the walls and base of the device, while I designed the lid and added some hardware cutouts using SolidWorks. We decided to use Acrylic, which I laser cut on Saturday.


Vikram’s Status Report 11/20/2021

This week I worked on the CAD models for the updated housing. We also did tests with the camera at the bridge to the performance of different cameras.

The new CAD model has the frame of the device being made out of laser cut 1/8 inch acrylic, and the arms made out of 3d printed material. We were able to cut 5 of the 6 new walls and bottom plate. Access to 3D printers has been difficult, so the new arms have not been printed yet.  The updated housing will use brackets internally to keep rigidity, as well as a hinge on the top for ease of opening/closing.

For the camera tests,  we found that the fisheye lens just wasn’t conducive to detection because it shrunk everything close to the center, making even large circle detection markers impossible to make out over video. We have switched to a larger target, and a webcam which has much better performance at distance. In addition to a straight up vertical test, we also tested the performance/detection range of the webcam horizontally, testing 3 different lateral distances from the marker, as well as 3 different resolutions.

Next, I would like to measure the latency of our system to see responsiveness to different commands. This will help us gather more data, in addition to the drops we conducted (and will conduct).
The trade studies for the camera are ongoing and for the propulsion system we hope to characterize different proportional thrust values, as well as just using constant thrust vectors.

Daniel’s Status Report for 11/20/2021

This week, I helped the team do more tests in order to see if we could replicate the results we found last week regarding our solution of the swinging problem. We also finalized our choice of cameras after some range tests off the bridge, and began designing our final housing.

For the drop tests, we first lowered the total added weight through rocks to exactly 1 pound (it was 600g before, whereas ~450g/1lb is the actual weight of our payload). We also charged the battery fully and ran additional thrust tests using the scale. We found that at 100% power using 16 AWG wure, we were getting almost 700g of thrust, a marked improvement over previous week’s results.

For the actual drops, as usual, I held up the device by the parachutes as Lahari initiated the device. We first ran with 50% power to the motors and noticed a very minute movement. We then decided to change this to 75% and dropped an additional two times, noticing noticeable levels of movement in the direction of the motors. I also took pictures from the bridge looking down to document where the device landed, so we could compare to the no motor drop.

To try alternative cameras, I ordered an 120 degree camera from the ECE parts inventory to see if the smaller FOV would make the circle easier to detect. I tried setting it up with the Pi only to find that the one we had was for the Nvidia Jetson, so the Firmware was not compatible. I then helped Lahari and Vikram modify existing scripts we had in order to support a USB webcam (as the exiting PiCamera module only supported the CSI Pi Camera module). This involved modifying the code to use the generic OpenCV camera access functions, rather than PiCamera specific methods. After this was done, we went out to test this new webcam against the old 160 degree FOV camera.


We came up with a quick testing strategy where I would hold the device over the bridge, say the testing parameters out loud (resolution + camera type + lateral distance from target), and Lahari would record the screen and mic feed using OBS. Using a tape measure, I measured up to 4 meters away from the location of the target, with increments of 1 meter such that we measure the performance of each camera at each resolution at each distance away from the target center. Lahari and I started with the 160 FOV fish eye camera and noticed that it was not even picking up the target on the ground, even after we doubled its diameter to 2 meters (the fish eye effect made everything in the center of the frame super small). We decided to call of the further tests with the fisheye, and moved onto the Webcam. With the Webcam, the target was large and clear in the frame, and we were able to proceed with the entire suite of tests. (640 x 480 vs 1280 x 720, at 1, 2, 3 and 4 meters away from the center of the target).

Lastly, we brainstormed ideas together for a final housing. As we ended up choosing acrylic, Lahari and I worked together to quickly create DXF files for the hexagonal base of the device, as well as the rectangular side walls using Solidworks. Our next phase is to fully construct this housing, and test the device with the full camera to propulsion piepline.

Daniel’s Status Report for 11/13/2021

This week, I helped the team prepare for and present our demo to the two groups on Monday and Wednesday, as well as work on modifications to the device to solve swinging problems in the air.

On Sunday before the demo, we worked together to fine tune the demo setup by testing what thrust to use, and how high up to tie the device to appropriately show off our system. Additionally, we ran various tests to see if the system performed as expected, as Vikram and I moved the circle under the device, and Lahari modified the detection code accordingly to improve detection. For the demo itself , I gave a brief overview of the main components of our project, then explained specifically my part (the vectorization algorithm), and how it fits into the overall data pipeline (my algorithm receives data from the CV subsystem, and passes it onto the arduino via UART).


After the demos, due to the low thrust we measured for each of our motors using the smaller props (~220g), I researched various thrust tests to see what other people were getting. I noticed people were getting easily over 800g using our motor with a 4S LiPo and 5 inch props (like us), so I suggested to the team that we should construct a proper motor thrust setup (motor attached to a raised platform, with a flipped propeller to generate thrust downwards onto a scale. Lahari and I found a wooden rod, and with the help of a friend, cut it to an appropriate size. I then measured the middle of a rectangular platform and glued the rod to it to give it a solid base. We then attached the motor to the top, and ran several tests. We found 330g of thrust with the 3S LiPo (the 4S was low battery), and 400g using thicker wires for the ESCs.

In order to solve the swinging problem we noticed in the air (which we attributed to lightness), we thought it would be beneficial to add weight to the device. I suggested we could use fish aquarium rocks (which I knew Lahari may have) as a stand in for the payload. We measured three 200g  bags of rocks, and Lahari and I cut a hexagonal insert such that we could put the bags at the bottom of the device, and cover them with this platform. After this, we tested  the device off the bridge (I dropped it, Lahari initiated it and Vikram caught it) and noticed the swinging no longer occurred, but minimal to no movement was observed. We then found that the 4S LiPo was low battery, causing the motors to slow down mid flight, so we charged it. Our next tests will have a fully charged battery, and 2 instead of 3 rock bags; we will then change thrust and weight accordingly to what we observed.


I also helped the team run detection tests for the circle off the Pausch bridge. I held the device off the side as Lahari viewed the camera feed on the laptop and made on the fly edits to the detection code. We found that the circle was too small to be detected in the thresh-holded image, so we printed a circle two times the size (2 meter diameter), and will be testing with that as soon as we construct it.


Lastly, I researched and ordered larger propellers (7 inches) in order to increase thrust further, incase we still cannot move after lowering the weight and using a fully charged battery.

Vikram’s Status Report for 11/13/2021

This week, after we demo’d I went to work on improvements to our physical drop system. First, I wanted to explore whether we were getting peak thrust from our motors by using a higher-gauge wire from the controllers to the motors. This change yielded a gain of 70g of thrust per motor, which was not insignificant. We will be moving all of our connectors to this 16AWG wire. Additionally we ordered 7 inch propellers that fit our shaft diameter, which are a 1 inch step up from our current props. I also measured our battery voltage and realized that it needed to be charged because on some of our pre-drop tests, the motors were struggling to keep thrust. I hope with topped off batteries, larger propellers, and thicker wires we will be able to push the most thrust possible out of our motors.
We also did a drop with the added weight to try and reduce swinging. This succeeded, but the added mass was a bit much, and we didn’t swing but we also didn’t move very much laterally. I think that using a solution in-between, where we decrease the added weight slightly, and use the aforementioned thrust changes will help balance the variables so that purposeful movement is possible.

Additionally, we went to the bridge again to test our perception system, where we found that the camera lens was working against us, and the fisheye made it hard to see the circle at the bottom with our processing added on. Using a standard camera or different lens should help to alleviate this, especially because the ultra wide fisheye is overkill when dropping from such height.