Posted on

Gaurav’s Status Report for 11/16/2024

This week, I was able to get object detection working on the Raspberry Pi 5 as well as transmission of the coordinates to an Arduino. I also worked with Bhavik to get object tracking working and to get radio transmission between an Arduino attached to the Raspberry Pi 5 and an Arduino attached to a computer to simulate our ground station environment. This week much of my time was spent modifying the run script to only transmit detection of humans and the human with the highest confidence. In the picture below, we switched it to a bottle just for that test to make sure that the drone controls were working with the position of the bottle. But as you can see behind the detection of the bottle, the Raspberry Pi is also deciding where the drone should move based on the position of the bottle and is transmitting that to the Arduino. We know this is working because the output that is being seen in the image is actually the Arduino transmitting what it is receiving from the Raspberry Pi.

When we (me and Bhavik) first started working this week, we were trying to compile the balloon detection model we had to work with the Hailo accelerator. However, we noticed that in order to compile the model for the Hailo accelerator, we had to compile the onnx file we had into an hef format. We spent a couple days trying to get that to work on an ARM machine by spawning AWS instances of X86 machines to compile the model. However, when we finally compiled it, we noticed that the model wasn’t working with the script they gave. We finally realized that we would have to write our own TAPPAS app, which is a framework provided by Hailo to optimize inference for the AI accelerator.  We then did some additional searching and found a pre-trained, pre-optimized model that would work for our purposes. All we had to do was modify the script to get only information about people, which in and of itself was quite an obstacle. However we were able to achieve 30 FPS on a 640×640 image.

I also worked with Ankit and Bhavik on training the PID controller in the drone cage in Scaife. Much of this work was headed by Ankit as we debugged different parts of the drone that were not working. First we did training for the P portion, and then we moved on to D. However, D exposed problems with the setup, the outdated hardware we were using, as well as parts shaking loose as we tested the functioning of the drone. This is the biggest bottleneck currently.

In order to verify my portion of the design we have run a variety of tests and will run more once it is mounted on the drone. For now, since we have set up the information pipeline from the camera feed straight to the radio, we are checking that the coordinates that the radio is receiving are representative of where the person is in the frame. We also have access to the information at each stage of the communication interface (stdout of the program in the RPi, serial monitor on the sending Arduino, and serial monitor in the receiving raspberry pi) and checking that the information is consistent across all those points. To run more tests, we will test with the object in various other positions and make sure that it is detecting in those scenarios too. We also will test by having multiple objects and no objects in the frame and checking that the arbiter is working properly as well.

object detection model detecting bottle
same model detecting a human
Posted on

Bhaviks Status Report for 11/09/2024

This week, I spent a lot of time getting the drone ready for PID tuning and setting up the various components required to relay coordinates back to the base. I began the week by attaching all the motors to the drone, testing them to ensure they all worked, and setting up the polarity so the drone could properly take off.

However, as discovered last week, our propellers don’t fit properly onto the motors. Thus, we spent some time talking with techspark experts to figure out a method to drill into the propeller. After many trials and various techniques, we couldn’t figure out a proper method to drill into them. Therefore, we decided to change our motors instead so that the properllers are compatible. Once we changed the motors, we again had to perform testing to make sure they all worked and set their polarity correctly. We found out that the required voltage for these motors are lower that the ones we were previously using, thus, we realized the need to order a different battery. But to continue testing while the batteries were being ordered, we set software constraints to ensure the motor operated safely.

On the other hand, I spent some time setting up the radio mechanism for the drone. This required me to set up the radios by installing the correct firmware, setting up its configurations, and interfacing with them to read data. Once these aspects were figured out, we were able to send data between the radios. The next step was to set up the radio such that one will be connected with the arduino board and one to my laptop (base station). I wrote code such that the arduino can understand the radio and read its values. Once this was set up, I tested the radio by sending messages from both sides of the radio to one another. We saw that the radio was working as expected.

The next steps are being able to interface with the GPS to transmit its values over by radio and connect the computer vision to the Arduino as well.

 

Posted on

Gaurav’s Status Report for 11/09/2024

This week, I was not able to accomplish as much as I would like to because we ran into many issues trying to recompile the model to work with the Hailo accelerator. I was not able to recompile it on my laptop, so we had to try to compile it on Ankit’s laptops but that came with its own set of errors. Currently we are stuck on the recompile step, but once that is ready everything else should work.

Our progress is on schedule. We have talked to the professors about what we will have for our interim demo, and we should definitely have that ready in time. Our drone is very much on track and having the vision component working soon is good for our overall progress.

By next week, I hope to have the model recompiled for the Hailo accelerator and a chassis designed to put the Raspberry Pi on the drone. I also hope to have an idea of how to convert the drone battery power into a current/voltage that the raspberry pi can accept.

Posted on

Ankit’s Status Report for 11/09/2024

This past week was very productive. We started off facing an issue where one of motors seemed to not be spinning. After some diagnosis we realized that this motor was probably dead from an earlier test where it had smoked up, causing us to have to replace the motor. Then, I changed the formulation of our Kalman Filter based state estimation as Kalman Filters require a prediction step that is very hard to encode for through quadcopter dynamics. Instead, I changed to a Complementary Filter, which is a measurement only approach that combines gyro and accelerometer readings essentially through a linear sum. After some quick tuning, this worked very well. Finally, i added in a safety mechanism where if the throttle stick is set to zero, then regardless of the PID outputs the motors stop spinning. This is necessary to ensure our safety during PID testing.

This week will finally all be PID tuning. We will setup a mechanism by which the drone is hanging from a string, fully unsupported, turn on the drone, and hopefully see if balance as we tune the PID gains. I am a little worried about the little time we have to tune both the pitch and roll gains, considering our interim demo is next week, but I do think it is possible.

I believe I am a bit behind schedule as I was hoping to get to PID tuning last week but because of the motor issue and the reformulation of the Complementary Filter, I wasn’t. That being said, I have set aside this entire week to work on PID tuning for the drone, so I think if I put my head down I can make it work.

Posted on

Team Status Report for 11/09/2024

The most significant risks that could impact the success of the project are not being able to tune the PID in time for the interim demo. Currently we have the drone motors running, and partially tuned. However we still have to test the drone in the air and on the harness.

No major changes were made to the existing block diagram. We are all working on our individual parts and collaborating when necessary.

There is no updated schedule, everything is on track to finish in time.

Posted on

Ankit’s Status Report for 11/02/2024

This past week was quite productive. I spent the week building the electronics of the frame and getting the motors tested. I started by writing some code to control the ESCs via Arduino PWM and then created a small test setup to verify the motor speed could be varied. Then, I got our transmitter to transmit to the arduino through a receiver, and while our drone is autonomous, this setup will be used to test our PID manually. Finally, me and Bhavik soldered the ESCs to the power distribution board built into teh frame and tested all four motors together. The entire setup works and we now have a drone with motors that spin.

This upcoming week, I will spend time actually tuning the PID and merging in the kalman filter state estimation. The goal is to tune the PIDs for roll and pitch to teh point where we can have stable hover flight, with throttle controlled by the transmitter. I think this will be an ambitious goal, but we should be able to accomplish it.

After the intense progress of this week, I believe I am on track.

Posted on

Bhaviks Status Report for 11/02/2024

This week was mostly spent on building the actual drone and connecting all the electronic components. We began the week by putting the entire drone frame together. Once we had the basic drone frame together, we had to connect all the ESCs to the motors. Once this step was complete, I worked with Ankit to solder all the wires to the power joints. This involved soldering the wires, running a continuity test, and finally doing a final test with the actual power on. After soldering and performing a continuity test on all the wires, we were able to verify that the wires were placed in the correct spots and that there was no short circuiting. Then we plugged in the battery could see the motors were powered. This signifies that we can properly power the drone and have all motors working.

We then began to run basic tests on the ESCs by plugging in the motors to the ESCs and connecting the ESCs to an arduino. We also had a drone receiver connected to the arduino. We programmed the arduino such that it would take the input of the drone controller and equally power the motors. Ex, if we are at full throttle, all the motors would be spinning at its fastest rate. We ran the program and confirmed that the drone motors were correctly responding to the inputs. The next step would be to test the pid control code by hanging the drone and checking if it can be stable. If not, we will need to fine tune the PID control system. This part of the project is at the highest risk right now since the rest of the project relies on this aspect working. If the drone is able to be controlled in a stable manner, the path planning should hypothetically be simple.

In regards to the computer vision aspect of the project, we were delayed due to the switch we made to the Rasberry Pi 5. The parts arrived at the end of the week and we are currently working on setting up the pi. This process hopefully shouldn’t take too long. I will be prioritizing this step next week and getting the computer vision algorithms up and running on the pi.

Posted on

Team Status Report for 11/02/2024

The most significant risks that could jeapordize the success of the project are getting the integration to work and the battery pack. These risks are being managed by finishing the other components earlier. We are about to start the integration step already, and we will be testing the efficacy of the battery pack soon once we get the drone motors installed.

No additional changes were made to the system design.

Current drone status wiring

Video of drone turning on

 

Posted on

Gaurav’s Status Report for 11/02/2024

This week, I ordered all the parts needed to start with the Raspberry Pi transition as well as flashed linux onto the Raspberry Pi. Much of my time was spent reading about how to setup the AI model onto the Raspberry Pi and any peripheral hardware we would need to access the GPIO on the Raspberry Pi to communicate with the Arduino.

Our progress is on schedule. Bhavik and Ankit have done a lot of work getting the drone running and setting up the model onto the Raspberry Pi should be extremely easy. Our next major step is integration and testing.

By next week, I hope to have the model running and figure out how to board the Rpi onto the drone.

Linux running on RPi
Posted on

Ankit’s Status Report for 10/27/2024

This past week I focused on reformulating the kalman filter. As I stated last week, our original formulation for the kalman filter was entirely wrong and so I spent significant time this week rederiving the dynamics equations for the quadcopter and understanding how to properly incorporate the gyro and accelerometer measurement updates into the state estimate. This led to a very productive week but I need to figure out how to properly tune the process and measurement noise variables in order to ensure that the state estimation does not lag the real variable.

This upcoming week I will focus heavily on tuning the kalman filter. Additionally, now that Bhavik and Gaurav were able to assemble the frame, I will be able to test out my code for controlling the drone motors. This is some boilerplate code I have lying around from a previous project. This should allow us to delve pretty heavily into tuning PID which is the next major step of our project and something that needs to happen quickly so we can get to actual flying testing.