Category: Kushaan’s Status Reports

 

This week I worked on fine tuning the model. I found a dataset that had fire from various point of views and perspectives. My main concerns during this process were: generalization of the perspective (can the model find our use case), quantity of sata (can this actually fine tune), and generalization to picam quality.

 

I fine tuned 11n, 11s, 11m. 11n was the smallest and took half an hour to fine tune 50 epochs. The results were promising. I also took some test images of fire in pans (our use case) on my phone and it generated a clean bounding box. 11s and 11m gave pretty bad results in comparison. Much noisier bounding boxes and under detection. We decided to use 11n going forward.

 

Once the picam arrived I did some testing using that image quality. However, it performed really well and detected 95% of our testing images. I wrote a script that took images every 2 seconds and then inferred on them.

 

One limitation of the current model is it’s night time performance. For the sake of MVP, this is not a pressing concern but is something I want to focus on after MVP. This will likely involve me making custom data.

 

I also helped out on the circuit testing. On that front, we found how to get a clear wave to drive the speaker with. We saw promising results on candle testing but need the collimater to get better results.

 

Next week I plan to work on the controls loop and interfacing since we didn’t have all the necessary hardware before break. Then I will integrate this with the vision code for a rudimentary loop.

This week, I finished setting up the PI. I was able to ssh into it and started setting up the python environment. I installed the necessary packages and troubleshooted some installations. I started running some benchmarks of the base yolo model we want to get an idea of any inference time issues. I also set up the github for the fine-tuning, proof-of-concept, and final code. I will be working on the fine tuning code on Sunday.  I set up sshing and internet on the Pi as well. Once I finish the benchmarking, I will try different serving frameworks to see if the inference time can be further reduced. While doing fine-tuning, I will be writing boilerplate code for the logging, so we can see how the ML model influences the pipeline. I also looked into manipulating GPIO on the Pi.

In addition, I worked on the circuit to test the amplifier. We wanted to make sure the amplifier worked at lower wattage, because we don’t believe we can power it at full spec. However, if it runs at lower spec, we can hit the wattage requirements of the speaker, which is the important part. I helped diagram the circuit, and use the signal generator, DC-power supply, and oscilliscope. Ultimately, we were able to boost a 60Hz wave from 100mV to 600mV. We used a very conservative amount of amps, 200mA instead of the ~2-3A we expect to be running at full power.  We need to perform further testing once we are more confident about this circuit, but it should be fine because our ESP outputs 3V, so we should hit our required ~80W into the speaker.

This week I looked into the Raspberry Pi more in-depth, and also into thermal camera imaging. I looked into benchmarks regarding different inference frameworks, and their benefits. I settled on using PyTorch to begin with, as it is the easiest to proof-of-concept with. Then, I will try using more advanced parallelism frameworks. I also looked into some depth-estimation models, since this will be an important part of our product. I found a few models that look promising, but want to do some benchmarking.

 

We got the RPI this week, so I will be running benchmarks over the weekend on these models, to verify the numbers as well. I want to generate intuition for the accuracy and limitations of these models.

 

I also worked on the design presentation, primarily focusing on my sections, and updating the theme and visuals, since I will be presenting

This week we were primarily focused on creating the presentation. We took a look at parts and determined which would be suitable for our use-case requirements. Particularly, I looked at the computer vision section and what hardware would be necessary. I looked at benchmarks on various compute platforms (RPI vs Jetson) of object detection algorithms. I found that an RPI can infer in ~500ms if we tweak the parameters correctly, which we found suitable for this use case. We also looked at camera options, and I believe that a regular pi camera will provide suitable resolution at a reasonable price. 

 

In our presentation, someone brought up concerns about the effect of smoke on image detection. I considered this problem, and while I don’t have a definitive answer, one workaround I thought about was sensor-fusing with a thermal camera. We could use the combination of inputs to more accurately detect fires. 

 

I am currently on track with the schedule. Next week, I will be looking into creating some notebooks and doing proof-of-concept testing to make sure my intuition is correct about my sensing components.