Posted on

Rayann’s Status Report for 4/25

I have been collecting more training data on the robot to add to the handheld training data in an attempt to increase the accuracy of the SVM model. I also had to replace the ultrasonic sensors on board the robot because they were no longer working. These sensors are very responsive to changes in surface and distance, so when they were no longer responding, I knew I had to change the sensors. I also collected some raw voltage data on my computer from the set-up on the robot, meaning that I connected to the Teensy MCU, so that I could monitor the voltage and see if there may be other features we could potentially process the data into that would increase the accuracy of the SVM model. I will continue collecting data with the robot and running the tests described below.

To validate the SVM model, we first train the model on the processed data we have collected. Then, we input new, unseen processed data into the SVM and validate its classification. I repeat this last step ten times so I can form an understanding about the reproducibility of the results. From this test, we have decided to increase the amount of training data and are looking into including more features. Occasionally, based on the data taken from the sensors, we replace them. When we first started collecting data with the robot, we realized we had to increase the sampling rate because it was too low to collect the full waveform.

 

My work for this week can be found here: https://docs.google.com/spreadsheets/d/1n9EZVZOw4e8DMoP9_O2lPJV9oe4U9V2cjh6RSExgdro/edit?gid=0#gid=0

 

Posted on

Mahati’s Status Report for 04/25

This week, I focused on improving our robot’s data collection and classification pipeline. I worked closely with Rayann to continue training the robot’s SVM model, refining its ability to distinguish between defective and non-defective regions. As part of this effort, we also re-evaluated our feature set to identify the most informative signals, aiming to enhance model performance and reduce noise in the data.

On the hardware side, we made significant progress by assembling the 3D-printed components of the robot, bringing us closer to a fully integrated system. We also added a second acoustic emission sensor, allowing us to collect data more efficiently and speed up the testing process. This improvement is especially valuable as we scale up data collection for training and validation.

Additionally, we revisited our data strategy and made an important decision regarding dataset composition. After analyzing the differences between handheld and robot-collected data, we observed that the sensing characteristics varied significantly. To maintain consistency and improve model reliability, we decided to exclude the handheld data and focus solely on robot-collected samples for training moving forward.

Posted on

Adithi’s Status Report for 4/25

This week, I continued preparing for our demo day. I have been setting up our demo system, working on the user report from the robot that is displayed, and have also helped with further unit testing of our overall system. I have also begun working on our final poster and continued with our final report. Overall, the project is going well. We are on schedule. Next week, I plan to finish up final testing, finish the poster, video, and report, and get ready for demo day. No schedule changes have been made and we are on track to finish as a group. Earlier in the week, I also gave the final presentation for our group in class.

Posted on

Team Status Report for 4/25

Overall Status Update

This week we have been working towards finishing our poster, final report, and gearing up towards demo day. We have 3D printed parts for the casing of our robot, and have done more system testing which we will explain in detail below.

Unit Tests

  • Precision Testing – How many times are defects accurately detected?
  • Recall – Number of defects identified
  • Dice Score
  • Position Error
  • Relative Error
  • Turn Accuracy
  • Turn Reliability
  • Signal Clarity
  • Accuracy in the dark
  • 2-6N of force for tapper

System Tests

  • HVAC Engineer interaction with robot and report UI
  • Full navigation of robot in duct with 2 90  degree bends
  • Power test to make sure robot is sufficiently powered

Our Findings

We found that overall, our system performs well apart from ome voltage readings which we are trying to sort out before our demo. The overall integrated system needs a few changes to our model parameters to adjust.

What are the most significant risks that could jeopardize the success of the project? How are these risks being managed? What contingency plans are ready?

No significant risks to report now we are happy with our progress and should be able to make it to the final demo. We are a little concerned about testing requirements which set us back a little since having a handheld scanner system is different from when it is moving on the robot. We will seek to address these challenges in the coming week, aiming to be finished with our capstone and ready for the final demo by Thursday morning.

Were any changes made to the existing design of the system (requirements, block diagram, system spec, etc)? Why was this change necessary, what costs does the change incur, and how will these costs be mitigated going forward?

No changes were made this week, we are proceeding with the design that was in place last week.

Provide an updated schedule if changes have occurred.

No changes have occured.

Posted on

Mahati’s Status Report for 04/18

Our teensy mcu broke, so I rewired the teensy mcu with the ADC, motors, and sensors. This time, while rewiring, I made sure to be very thorough and neat with the way that I was wiring so that it would not break again. We have been very careful storing the parts as well, so that there would be no issues for our final. 

I also worked alongside Rayann to test and train our robot’s SVM model, helping ensure that our classification pipeline was functioning correctly. We tested the model with the robot and added robot data to the training dataset. We collected 200 raw voltage data values, processed the features, added them to a robot dataset, and then let the robot move forward through the duct for 1 second. 

As you’ve designed, implemented, and debugged your project, what new tools or new knowledge did you find it necessary to learn to be able to accomplish these tasks? What learning strategies did you use to acquire this new knowledge?

I had to learn several new tools and technical skills. I gained experience booting and setting up an NVIDIA Jetson Orin Nano, writing Arduino code, and wiring a Teensy microcontroller with ADCs, motors, and sensors. As I am more on the software track, I sought out help and did the wiring alongside Adithi, and that was very helpful.

To build this knowledge, I used online tutorials and YouTube videos to quickly get hands-on familiarity with the hardware and software tools. Since I was working on soldering in AirLabs, I got to ask the researchers there about their inputs and was helped by Mark. This helped me effectively troubleshoot issues and integrate different components into a working system.

Posted on

Team Status Report for 4/18

Overall Status Update

This week, overall, we have conducted testing for our HVAC robot, designed the 3D CAD casing, which we want to go over our HVAC inspection robot to give it a cleaner look. Our overall demo HVAC duct has been set up and will be here for demo day too.

This week we also spent our time doing the onboard ML testing and verification. We ran into some problems with voltage levels not representing our handheld testing but we have reached an accuracy level of 90% and we are hoping to get it to 95%, but if not, our aim was 85% so the goal has been achieved.

We also worked on our final presentation and have started working on the final report together.

We are nearing the end of our capstone 🙂

What are the most significant risks that could jeopardize the success of the project? How are these risks being managed? What contingency plans are ready?

Right now, our risks do include hardware being fried at the last minute. We have taken all the necessary precautions and have back up parts, but for some reason, we could accidentally fry our teensy MCU like we did earlier in the week so we need to be prepared for this.

Another thing we are a little worried about is the recovery mechanism for our robot. We have developed a system where the robot can exit through its entry point should it fail halfway through its deployment. Right now, it is hitting the sides of the HVAC duct so we need to correct our LiDAR a little but this should be solved very soon.

Were any changes made to the existing design of the system (requirements, block diagram, system spec, etc)? Why was this change necessary, what costs does the change incur, and how will these costs be mitigated going forward?

No changes were made this week, we are proceeding with the design that was in place last week.

Provide an updated schedule if changes have occurred.

No changes have occured.

Posted on

Adithi’s Status Report for 4/18

Accomplishments

This week, I worked on the UI that is user-facing when the robot detects a defect in the system. I also continued working on our final demo HVAC duct as well as fine-tuning the issues with our LiDAR for the system. In addition to this, I helped Rayann and Mahati with the testing and the verification of our machine learning model and how it operates when on the robot inside the duct.

I have also sourced some additional corroded materials for our demo which I am excited about because while our machine learning model isn’t perfect, it is looking promising.

Something else I worked on this week was the CAD files to create an encasing for our HVAC robot. We want a case over the robot that protects the different components, especially the Nvidia Jetson Orin Nano. I just created a basic encasing and we sent it to techspark and are waiting for it to be printed.

Lastly, since I will be the presenter for our final presentation, I have been praticing and making the slides along with Rayann and Mahati. Our professor also emphasized how important it is to explain technical concepts in an easier manner. I have spent the second part of the week making sure I fully understand the signals side of our project and if I understand what I am saying, I believe my audience will as well because I have no signals expertise.

My work this week has mainly been on Github and working on the physical robot itself and can be found here: https://github.com/aphadke234/ece_capstone_C7

Schedule

I am on schedule now and currently on track. I am happy with my progress and think that I will be done with my part of the project and able to help my peers with their deliverables after the end of next week.

Next Week

Next week, we will make our product look nice for the demo. Technically speaking, all of our code, testing, and integration has been completed so I am very excited to present our project to everyone!

New Learnings

I have had to learn how to use an Nvidia Jetson Orin Nano, as well as understand how computer vision and LiDAR works. These were new to me and I had never used them before. The way I approached learning was using online tutorials and youtube videos. I also used different textbooks and research papers to guide my learning.

I have also had to understand more about Machine Learning than I ever will learn. The way I approached my learning was to sit in on the introduction to machine learning classes this semester, and talking to my teammates, especially Rayann, who is an expert in signals and systems.

Lastly, I have had to learn how HVAC engineers work, what HVAC technicians usually do to fix commercial HVACs, and what kind of problems they encounter, as well as what working with a robot might look like in their field. I have had many talks with HVAC experts from trade schools in Pittsburgh and they have also allowed me to observe them while giving me some reading and videos to go through to guide my learning.

Posted on

Rayann’s Status Report for 4/18

This week, I collected more training data from corroded samples. We now have 100+ samples of healthy and corroded samples combined. I also did an analysis of the data we collected and chose a couple new features to include in the processing. These features work together to showcase the differences between the healthy and corroded data. The graphs I created to analyze these features are available in the team Github. Nobody should stare at these graphs for too long because they are too confusing. I simply used these graphs, as well as other graphs of other features from the data I collected, to choose which features were the most different between the defect data and the non-defect data. They are histograms of the feature values from each data input (red is for defect data and blue is for non defect). The feature graphs that had the least overlap between red and blue bars were chosen to be included in the processing pipeline. I wrote a new Python file for extracting these features. This file named extract_features can be found in the team GitHub. Then, I trained an SVM using the data I collected and processed into these features. The file for training this SVM as well as the trained SVM file we downloaded onto the Jetson are in the team Github linked below. The SVM achieves .94 accuracy. Mahati and I worked on debugging the data collection on the robot. The sampling rate was too low, so we increased it. Then, we tested how the data collection system and the SVM classification model works on the robot.

Throughout the design, implementation, and debugging of this project, I developed both technical and communication skills by learning how to select and integrate components based on specific application requirements. For example, I learned how to choose appropriate ultrasonic sensors by considering use-case constraints, such as detecting the acoustic signature of galvanized steel. To support these decisions, I reviewed research papers and analyzed the reasoning behind prior designs, which helped me establish a more structured approach to my own design choices.

In addition to component selection, I had to learn how to effectively use these sensors and integrate them with other system elements. I relied on a variety of informal learning strategies, including watching instructional videos and reading technical documentation and articles from sources such as MATLAB and Adafruit. These resources allowed me to quickly gain practical knowledge and troubleshoot implementation challenges.

Beyond technical skills, I also worked on improving my ability to communicate complex concepts to individuals with much less technical experience. I incorporated feedback from our advising professor to refine how I present and explain my work, which has helped me become more effective in conveying technical ideas clearly and concisely.

My work for this week can be found here: https://github.com/aphadke234/ece_capstone_C7/blob/master/extract_features.py

 

Posted on

Mahati’s Status Report for 4/4

This week’s progress:

This week, I focused on refining the machine learning model, with the goal of improving both its accuracy and overall robustness. I explored ways to better structure the data and tune the model so that it can generalize more effectively to real-world scenarios. This refinement process is an important step toward ensuring that the model performs reliably when integrated into the full system.

In addition to the machine learning work, I made progress on the Jetson-side implementation. I restructured the codebase by separating the robot control logic from the ADC signal processing pipeline. This separation makes the system more modular and significantly improves the ability to visualize and debug sensor data independently from control behavior, which will be valuable as the system becomes more complicated.

Things to do for next week:

Looking ahead to next week, I plan to collect more training data to further strengthen the model. With a larger and more diverse dataset, I aim to improve the model’s performance and reduce potential edge-case errors. I will also continue testing the system end-to-end to ensure stable and consistent communication between all components, including the Jetson, Teensy, and sensors.

Additionally, I will be collaborating with Aditi to integrate the LiDAR system and begin working toward autonomous navigation of the robot through the HVAC environment. This will involve combining perception and control components into a more cohesive system. 

My work can be found here: https://github.com/aphadke234/ece_capstone_C7

Posted on

Team Status Report for 4/4

This week, our team focused on preparing our project for the interim demo. Adithi and Mahati worked together to debug code for the motors to make the robot chassis move forward, backward, and turn (both right and left). Adithi worked on the code to drive the LiDAR camera in such a way so that it can detect whether or not the robot is in the duct and the physical integration between the camera and the robot chassis. Adithi has also started building our final demo HVAC duct to show our project on. Mahati worked on rebooting the Jetson and integrating all the systems including the acoustic data collection system, LiDAR, motors, and ML model with the Jetson. Mahati physically integrated the ultrasonic sensors with the ADC and Teensy MCU on the robot chassis. Rayann collected healthy duct and corroded material data using the data collection circuit. She plans to collect more samples. She also debugged the MATLAB processing code to take in the data she collected, and processed this data so it is ready to be used to train the SVM. She also wrote another processing pipeline in MATLAB specifically meant to be downloaded onto the Jetson and take data from the Teensy MCU, process the data, and hand it over to the SVM. She is currently debugging this code.

 

We are currently on schedule, with a few things to accomplish in the coming weeks. We will finish collecting data, debug our on board processing pipeline, and train the ML model. We will finish building our demo structural HVAC duct so that we can have a good presentation to show viewers what our project is. We also want to make sure the robot does not have to be connected to a charger by the NVidia and that we can have our LiPo battery pack on the Jetson. We will also begin working on the final presentation, final report, and demo poster.

 

A potential risk is the ML model not being able to reliably classify defects vs healthy ducts. This is quantified using false/true positives and negatives. In this case, we plan to either change hyperparameters or change the processing pipeline after analyzing the data (adding/removing features).

 

All of our code can be found in the github: https://github.com/aphadke234/ece_capstone_C7