Author: sthati

This week I gave the final presentation to the class and discusses the various steps we have taken so far to verify our design such that it aligns with our design requirements we have set forth.

This week was mostly spent planning and ordering the remaining components for the project (GPS, planning power distribution, and Rpi attachment). I also cleaned up the code from the demo and prepared it for the final demo. I have also spent some time making sure that the radio communication can work bi-directionally. Moving on to the next parts of testing, we want to be able to send signals from the base station to the drone, hence we need to make sure this step works. Next week we will continue to integrate our system and test our drone to start flying.

Additonally, a majority of the time was spent creating the final presentation slides. I created a full outline of the presentation. I will continue to work on the slides by rerunning some demo scripts to capture more media for the presentation. Since I will be the one presenting this time, I have also begun to practice and get the required information.

Throughout this project, there were a lot of new tools and knowledge I had to learn in order to accomplish my tasks. Working a lot of various different hardware systems, I had to learn a lot about efficient methods to communicate between systems, the various communication protocols, and how to interface with third-party tools. I also had to learn a lot about controls (PID specifically). Some learning strategies were watching tutorials on related topics, looking at other project documentation, or talking with friends who were knowledgeable about the specific topic. I also learned a lot by trial and error. For example, using the knowledge I gained by reading documents and watching videos, I was able to implement basic functionality, learn from the mistakes made there, and retry. This has helped me take the theoretical knowledge and apply it in a fast manner.

This week I focused my efforts on finalizing the CV and helping with drone controls.

At the beginning of the week I worked closely with Gaurav to get a model working on the rpi. We got the model working but the frames were really low as it wasn’t being accelerated. Thus, we spent time looking through the documentation to figure out how to compile the model. In our attempts we compiled the model multiple times in various configurations but have run in various PC related issues (ex. incompatiblae architecture or not enough hardware specs such as ram). We resolved this problem by configuring an AWS EC2 instance and setting up an additional memory. Once we did this we were able to successfully compile the model after lots of efforts.

Once we put it on the rpi, the script wasn’t able to utilize the model and detect anything. We believe this is a compiler issue with the hailo software as its very new and doesn’t fully support using custom models very easily. Therefore, we have tested this theory by compile a base yolov8n model and uploading it to the rpi. This seems to have worked. Thus, we have decided to move forward with objects trained on the base yolo. Another reason for the change is that we realized the drone propellers produce a lot of windforce and would easily cause the balloon to move around a lot, causing the detection and tracking algorihtims to fail.

On the drone pid controls aspect, I worked closely with Ankit to set up a drone testing rig. We have tried multiple rigs and finally decided to use a tripod attachment with the drone on top. This seems to have worked fairly well and we have been tuning with this. While tuning we found lots of hardware inconsistencies which seem to be like the major risk moving forward. The PID seems to be working well one day and the next day we seem to have gone backwards. We have spent multiple days trying to limit these hardware issues to fully ensure that the drone PID controls work well. There is still a lot of testing to do to make sure the drone is reliable.

In regards to testing, we will gather specific metrics required that related to our use case requirements. For example, for the computer vision model detection, we will be measuring that the computer vision model is able to accurately detect the object in the various frames. This will let us measure the accuracy and thus we will be able to analyze this metric to check if we meet the design requirements. For tracking we will check if the camera is able to provide directions on which way the drone should move. If the algorithm is able to make sure that  object is always centered we know we have met the design requriment.

This week, I spent a lot of time getting the drone ready for PID tuning and setting up the various components required to relay coordinates back to the base. I began the week by attaching all the motors to the drone, testing them to ensure they all worked, and setting up the polarity so the drone could properly take off.

However, as discovered last week, our propellers don’t fit properly onto the motors. Thus, we spent some time talking with techspark experts to figure out a method to drill into the propeller. After many trials and various techniques, we couldn’t figure out a proper method to drill into them. Therefore, we decided to change our motors instead so that the properllers are compatible. Once we changed the motors, we again had to perform testing to make sure they all worked and set their polarity correctly. We found out that the required voltage for these motors are lower that the ones we were previously using, thus, we realized the need to order a different battery. But to continue testing while the batteries were being ordered, we set software constraints to ensure the motor operated safely.

On the other hand, I spent some time setting up the radio mechanism for the drone. This required me to set up the radios by installing the correct firmware, setting up its configurations, and interfacing with them to read data. Once these aspects were figured out, we were able to send data between the radios. The next step was to set up the radio such that one will be connected with the arduino board and one to my laptop (base station). I wrote code such that the arduino can understand the radio and read its values. Once this was set up, I tested the radio by sending messages from both sides of the radio to one another. We saw that the radio was working as expected.

The next steps are being able to interface with the GPS to transmit its values over by radio and connect the computer vision to the Arduino as well.

 

The beginning of the week I tested the yolo balloon detection at a long range (20ft and beyond). For distances of 20 ft I ran over 50 trials and noticed that the ballon was detected 98% of the time. The confidence score was also very high at >80%. I noticed if I reduced the confidence threshold there were some false positives. I tested the model using other objects too try to break it and the model continued to perform well. I placed objects such as small balls, phones, hoodies and etc. however the model seems very robust and wasn’t affected by the distractions. Overall, I’m very confident in the models ability to work.

Additionally, this week I worked closely with Gaurav to get the kria set up for the vision model. We began by setting up the vitis AI tool, and attempted to load the yolo model on to it. In order to do that, we began by first quantizing the model. In order to do this, we began by exporting the model into a onnx format. Then, wrote a script to quantize the model (from float to int8).  Using the quantized model we tried to upload it to the kria following the “quick start” guide. However, we realized that we would have to write our own script that utilized the Kira’s xillinx’s library to analyze the model to ensure it’s compatible with the dpu. Once we wrote the script and ran it on the kria, we kept getting errors that the hardware isn’t compatible. Thus we tried multiple different approaches such as using their onnx library and PyTorch libraries to try to get vitas . After discussing with Varun (FPGA TA), we were recommended to look into using an AI accelerated raspberry pi 5.

We looked into the pros and cons of new hardware and found that is would be much more efficient and would simplify our issues.  Working towards fixing this issue in the next week and getting the model working on the new hardware.