This week, I continued preparing for our demo day. I have been setting up our demo system, working on the user report from the robot that is displayed, and have also helped with further unit testing of our overall system. I have also begun working on our final poster and continued with our final report. Overall, the project is going well. We are on schedule. Next week, I plan to finish up final testing, finish the poster, video, and report, and get ready for demo day. No schedule changes have been made and we are on track to finish as a group. Earlier in the week, I also gave the final presentation for our group in class.
Team Status Report for 4/25
Overall Status Update
This week we have been working towards finishing our poster, final report, and gearing up towards demo day. We have 3D printed parts for the casing of our robot, and have done more system testing which we will explain in detail below.
Unit Tests
- Precision Testing – How many times are defects accurately detected?
- Recall – Number of defects identified
- Dice Score
- Position Error
- Relative Error
- Turn Accuracy
- Turn Reliability
- Signal Clarity
- Accuracy in the dark
- 2-6N of force for tapper
System Tests
- HVAC Engineer interaction with robot and report UI
- Full navigation of robot in duct with 2 90 degree bends
- Power test to make sure robot is sufficiently powered
Our Findings
We found that overall, our system performs well apart from ome voltage readings which we are trying to sort out before our demo. The overall integrated system needs a few changes to our model parameters to adjust.
What are the most significant risks that could jeopardize the success of the project? How are these risks being managed? What contingency plans are ready?
No significant risks to report now we are happy with our progress and should be able to make it to the final demo. We are a little concerned about testing requirements which set us back a little since having a handheld scanner system is different from when it is moving on the robot. We will seek to address these challenges in the coming week, aiming to be finished with our capstone and ready for the final demo by Thursday morning.
Were any changes made to the existing design of the system (requirements, block diagram, system spec, etc)? Why was this change necessary, what costs does the change incur, and how will these costs be mitigated going forward?
No changes were made this week, we are proceeding with the design that was in place last week.
Provide an updated schedule if changes have occurred.
No changes have occured.
Team Status Report for 4/18
Overall Status Update
This week, overall, we have conducted testing for our HVAC robot, designed the 3D CAD casing, which we want to go over our HVAC inspection robot to give it a cleaner look. Our overall demo HVAC duct has been set up and will be here for demo day too.
This week we also spent our time doing the onboard ML testing and verification. We ran into some problems with voltage levels not representing our handheld testing but we have reached an accuracy level of 90% and we are hoping to get it to 95%, but if not, our aim was 85% so the goal has been achieved.
We also worked on our final presentation and have started working on the final report together.
We are nearing the end of our capstone 🙂
What are the most significant risks that could jeopardize the success of the project? How are these risks being managed? What contingency plans are ready?
Right now, our risks do include hardware being fried at the last minute. We have taken all the necessary precautions and have back up parts, but for some reason, we could accidentally fry our teensy MCU like we did earlier in the week so we need to be prepared for this.
Another thing we are a little worried about is the recovery mechanism for our robot. We have developed a system where the robot can exit through its entry point should it fail halfway through its deployment. Right now, it is hitting the sides of the HVAC duct so we need to correct our LiDAR a little but this should be solved very soon.
Were any changes made to the existing design of the system (requirements, block diagram, system spec, etc)? Why was this change necessary, what costs does the change incur, and how will these costs be mitigated going forward?
No changes were made this week, we are proceeding with the design that was in place last week.
Provide an updated schedule if changes have occurred.
No changes have occured.
Adithi’s Status Report for 4/18
Accomplishments
This week, I worked on the UI that is user-facing when the robot detects a defect in the system. I also continued working on our final demo HVAC duct as well as fine-tuning the issues with our LiDAR for the system. In addition to this, I helped Rayann and Mahati with the testing and the verification of our machine learning model and how it operates when on the robot inside the duct.
I have also sourced some additional corroded materials for our demo which I am excited about because while our machine learning model isn’t perfect, it is looking promising.
Something else I worked on this week was the CAD files to create an encasing for our HVAC robot. We want a case over the robot that protects the different components, especially the Nvidia Jetson Orin Nano. I just created a basic encasing and we sent it to techspark and are waiting for it to be printed.
Lastly, since I will be the presenter for our final presentation, I have been praticing and making the slides along with Rayann and Mahati. Our professor also emphasized how important it is to explain technical concepts in an easier manner. I have spent the second part of the week making sure I fully understand the signals side of our project and if I understand what I am saying, I believe my audience will as well because I have no signals expertise.
My work this week has mainly been on Github and working on the physical robot itself and can be found here: https://github.com/aphadke234/ece_capstone_C7
Schedule
I am on schedule now and currently on track. I am happy with my progress and think that I will be done with my part of the project and able to help my peers with their deliverables after the end of next week.
Next Week
Next week, we will make our product look nice for the demo. Technically speaking, all of our code, testing, and integration has been completed so I am very excited to present our project to everyone!
New Learnings
I have had to learn how to use an Nvidia Jetson Orin Nano, as well as understand how computer vision and LiDAR works. These were new to me and I had never used them before. The way I approached learning was using online tutorials and youtube videos. I also used different textbooks and research papers to guide my learning.
I have also had to understand more about Machine Learning than I ever will learn. The way I approached my learning was to sit in on the introduction to machine learning classes this semester, and talking to my teammates, especially Rayann, who is an expert in signals and systems.
Lastly, I have had to learn how HVAC engineers work, what HVAC technicians usually do to fix commercial HVACs, and what kind of problems they encounter, as well as what working with a robot might look like in their field. I have had many talks with HVAC experts from trade schools in Pittsburgh and they have also allowed me to observe them while giving me some reading and videos to go through to guide my learning.
Adithi’s Status Report for 4/4
Accomplishments
This week, I worked on fine-tuning the issues with our interim demo, fixing some code. After our interim demo this week, we didn’t get a lot of feedback except that we seemed to be on track, which was good news.
After our interim demo, this week I have spent the time working on integrating our robot with the LiDAR in such a way that it can detect whether or not it is in the duct. It will take the sides of the duct and figure out whether it is in the middle of the duct and autocorrect. I have worked on making the robot move in a smoother manner.
I have also started building the final demo duct that we will use. I plan to use plexiglass so that people can see the robot moving inside the duct. I have also made sure there will be 2 bends, and naturally have been fine-tuning the 90 degree turns the robot will have to make. We worked on making the robot autonomous this week.
My work this week has mainly been on Github and working on the physical robot itself and can be found here: https://github.com/aphadke234/ece_capstone_C7
Schedule
I am on schedule now and currently on track. I am happy with my progress and think that I will be done with my part of the project and able to help my peers with their deliverables after the end of next week.
Next Week
Next week, I will finish building our interim demo structural HVAC duct so that we can have a good presentation to show viewers what our project is. I will also begin working on the final presentation and report and poster. I also want to make sure the robot does not have to be connected to a charger by the NVidia and that we can have our LiPo battery pack on the Jetson.
Mahati’s Status Report for 3/28
This week’s progress:
- Hardware Setup and Embedded System Integration
This week I focused on setting up and integrating key hardware and embedded system components required for our pipeline, working closely with Adithi. We successfully set up and flashed the Jetson Orin Nano, which serves as the main compute unit for running higher-level control and machine learning tasks.
We then connected the motor driver to the motors and the Teensy microcontroller and established communication between the Teensy MCU and the Jetson, ensuring that control signals can be transmitted reliably between the embedded controller and the main system. We also connected the piezo sensor to the Teensy MCU and verified its connection to the Jetson, which is important for enabling the HVAC inspection.
In addition, we integrated the LiDAR and IMU devices into the system and worked on configuring them for use. I also figured out how to SSH into the Jetson, which will make development and debugging easier especially for our interim demo. Finally, we implemented code on the Jetson to respond to signals from the Teensy MCU and execute control logic, enabling coordinated behavior between sensing and actuation components.
Things to do for next week:
Next week is our interim demo, so the focus will be to prepare for that. The main tasks include:
- Collect more training data
- Improve the machine learning model using the newly collected data
- Continue testing the system to ensure stable communication between all components
Since the ADC arrived late, we were not able to collect enough data earlier, so a key priority will be improving the machine learning model using additional data collected this week.
My work can be found here: https://github.com/aphadke234/ece_capstone_C7
Adithi’s Status Report for 3/28
Accomplishments
This week, I worked very closely with Mahati in the week leading up to our interim demo. We accomplished a series of things.
With Mahati, I worked on fine tuning the motor control of the robot that was already in place and we soldered our final circuit onto a permaboard so that it could be elegantly presented. Currently, the motors move at the set speed according to the specification we set when we initially came up with our design. Then, we moved onto helping Rayann with collecting data from our tranceiver and receiver modules, running it through an ADC, then feeding in the values that were read into the ML model that was built by Rayann and Mahati. We can now run the ML model on our Jetson, and the Jetson controls the Teensy MCU. Mahati and I also worked on integrating the LiDAR with our robot.
On my own, I have also been building the HVAC duct with one 90 degree turn that will be used for our demo. I also took a look at our remaining schedule and how we plan to utilize the time we have before the end of the semester and before our final demo presentations.
My work this week has mainly been on Github and working on the physical robot itself and can be found here: https://github.com/aphadke234/ece_capstone_C7
Schedule
I am on schedule now and currently on track. The interim demo is this upcoming Monday and as a team, we achieved everything discussed with our Professor and our TA, except for the solenoid tapper, which we have set up but have not had success integrating in time for the interim demo. This will have to be looked at in the coming week, and be addressed immediately.
Next Week
Next week, I plan to prioritize making our robot movements smoother so they don’t damage the duct while moving through them. I also plan to integrate the solenoid driver so that Mahati and Rayann can focus on the ML model. I am still meeting with the HVAC trade school contact we made to figure out how to build the system for our final demo. I have also been going to junkyards with him and will continue to do so once a week until 2 weeks from now, at which point, I think we will have all the training samples we need for our project. I will also aid Mahati with the IMU integration next week.
Team Status Report for 3/28
Overall Status Update
This week, overall, we have been working towards our interim demo. Since had our robot’s motors somewhat working and moving, we wanted to link the rest of our parts together. We finally started collecting data since our ADCs arrived, completed the initial training of our ML model, and we worked towards integrating the LiDAR with our robot. The goal we have set for ourselves by the interim demo is having a robot with a LiDAR feed that the user can view, a robot that can be controlled by the keyboard as we move towards making it autonomous, and a basic ML model that can identify cracks as they occur. After interim demo, we begin fine tuning and integrate the solenoid tapper to indicate where the crack or defect has occured in the HVAC duct, working towards a more robust and accurate ML model, and an autonomous robot. We want to also end with thorough testing, and throughout the entire process we will be collecting more data.
What are the most significant risks that could jeopardize the success of the project? How are these risks being managed? What contingency plans are ready?
We are worried about using LiDAR to enable our robot to move autonomously. Since this has been one of our pain points for the last 2 weeks, during the interim demo week we plan to work and read papers to familiarize ourselves with using LiDAR and enabling the robot to become autonomous.
Another thing that poses as a risk currently to our group is getting enough data to train our ML model. To mitigate this, we have found various surface panels with corrosions and cracks and will begin training on this while also finding other surfaces we can use for validation testing.
Were any changes made to the existing design of the system (requirements, block diagram, system spec, etc)? Why was this change necessary, what costs does the change incur, and how will these costs be mitigated going forward?
No changes were made this week, we are proceeding with the design that was in place last week.
Provide an updated schedule if changes have occurred.
The overall schedule of the project remains mostly consistent as of now, however after the initial feedback after the interim demo this may change.
Adithi’s Status Report for 3/21
Accomplishments
This week, I set up the motor controller code to not only function, but move our robot forward controlled by a user input on the laptop. I began to integrate the code for moving our robot chassis left, right, forward, and backward, and have begun figuring out how to ensure our robot can make 90-degree turns.
I looked into remote controls for our robot and have purchased the part so that instead of the current model, where the user controls how the robot moves by pressing a keyboard button or a button on screen, they will be able to use a remote control with a joystick to control the robot instead.
Since the robot is moving, I began looking into writing the code for our LiDAR camera. After discussion with the team, we have decided that we will move forward with using the camera to enable our HVAC engineer to see as much of the duct that they can. Since we have scoped our project down to get rid of mappings, this is much easier.
Some of the videos aiding me this week were:
youtube.com/watch?v=gJPIJ3yxME0&t=79
Mahati has also been working on the solenoid tapper and I helped her a little bit in the development with the code for the tapper.
My work this week has mainly been on Github and can be found here: https://github.com/aphadke234/ece_capstone_C7
Schedule
I am on schedule now and currently on track. The interim demo deadline is in 2 weeks or so and I am feeling confident about having a robot that can move through a duct, and classify defects at some level. I am a little behind on building the HVAC duct for our demo however.
Next Week
Next week, I plan to prioritize integrating the machine learning model on our Jetson Orin Nano with the current motion robot controls. I will also finish integrating the LiDAR camera with our robot so the user can have real time visual feedback. I will finish building our duct system for the interim demo using 5 to 6 ducts, and then I will offer help to Mahati with the solenoid tapper as well as data collection from the duct.
Rayann’s Status Report for 3/21
I discussed the ethics and risks of our project with my group this week. I also attended the ethics lecture and discussed the ethics and risks of other projects in the class. We investigated the risk of using ultrasonic sensors and found that if the transmitter is driven with a high enough frequency, people in the area could suffer from dizziness and nausea. The risk threshold is 20 kHz. If we drive the transmitter with low-level voltage, the frequency range for the ultrasonic waves emitted should never exceed 1 kHz.
I characterized the ultrasonic sensor by trying different inputs to the transmitter and observing how the receiver responded, going through the process step by step for each variation. I started the collection of acoustic data using the ultrasonic sensor on healthy ductwork. We have not received our ADC yet, so I only attempted to characterize the use of our sensor on our specific duct material. Basically, I drove the transmitter and collected the output voltage range from the receiver using a voltmeter. This is not a fully detailed waveform that I can input into the processing code I have. I used one piece of duct (out of the 25 we collected) and took in total six measurements every 5 cm (the duct is about 15 cm long). I drove the transmitter with 3V, 5V, and 9V. The voltage limit for testing was 9 volts, which is the same limit provided by our current hardware design. I took two measurements for each input voltage because this seemed enough; after two, the voltage values became very repetitive without much variation. The data collection is documented in the file linked below.
I also used an AD3 explorer to look at the waveform captured by the ultrasonic sensor. From using this software, I was able to obtain preliminary values such as frequency and peak-to-peak voltage. An example of the image is below.
This is my work for this week: https://docs.google.com/spreadsheets/d/17F5QZGwymYqmGGup4V3u2_v8MdibjNQDQthjUrzVp3I/edit?gid=0#gid=0
This is a picture of the testing set-up:
The transmitter is hooked up to the battery and the receiver is positioned next to it to collect the echo of the signal. The prongs for the sensors seem slightly too long for this breadboard so the signal is unstable. For a proper, reliable connection, I may have to redesign this system.