Smart Stethoscope

This week our team made good progress, however, we still have a decent amount to do in the upcoming weeks. We are aiming to finish the real time system tomorrow on Sunday in lab. We are also going to decide if we want to do AR/AS vs MR/MS depending on how much time we have. We are also going to try to increase our accuracy to above 85% consistently. We were able to reach 85% a few times, but are going to try to make it higher. Finally, with the remaining time we have, we will focus on complete testing of our finished system to make sure our device work. We have not a ton of time left, but I think we can finish what we need to get done to have a good final project.

This week was when we began our integration into a real time system. I personally accomplished a lot this week and am excited to integrate before our demo. The raspberry pi that we ordered arrived along with the touch screen that we will use to run the system. I flashed the pi and was able to configure the touch screen to work with it. I also started a python program that will be the basis for our entire project. This codebase will create a simple GUI which will live display the signal received from the stethoscope, and will have a button that will begin the analysis. This code will invoke our matlab code to classify a heart sound and will display the result. We aim to have this real time integration ready for our pre-final demo and will work a lot on it tomorrow to make sure it is ready. The main things we have left to do is to work on our testing after the integration is done. We also want to try to get a little more accuracy on our ML.

• What did you personally accomplish this week on the project? Give files or photos that demonstrate your progress. Prove to the reader that you put sufficient effort into the project over the course of the week (12+ hours).
  • We finished our double blind trials on our testing environment this week
  • Through training the larger dataset on the SVM and CNN we finally were able to reach an average accuracy of around 83% using the CNN, which is close to our goal.
  • We started integrating all of the necessary things for the demo on Monday, and tomorrow we will focus more on transforming our MATLAB code into C.
• Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?
  • Our progress is behind schedule since we wanted to be done with classifying AS/MR by the end of this week, but we realized we may just scrap that completely and just make this stethoscope work for classifying abnormal vs normal heart sound since we do not have a lot of time left.
• What deliverables do you hope to complete in the next week?
  • Finish integrating our code with our stethoscope and start working on using the raspberry pi to be able to display whether the user has a normal or abnormal heart sound.

This week I worked on training the CNN with extra preprocessing. First, the audio files were previously trimmed to be 5 seconds long, and also passed through Eri’s Shannon Expansion noise removal algorithm. I then took the spectograms of this new processed audio data and trained my CNN on it. Since the spectograms were now all of the same time frame of 5 seconds, we expected our accuracy to go up. However, the newly trained CNN produced an accuracy of around ~65%, so it was a decrease from the previously trained CNN’s with the same number of audio files. More testing has to be done on how to improve this algorithm.

I was travelling last week and this Sunday-Monday for interviews and Spring Carnival was this weekend, so I could not invest too much time into this project this past week. However, more time will be invested in future weeks to ensure that we reach the desired 85% accuracy by final demo. Next week I want to research LTSM to preprocess the data instead of trimming at a random 5 seconds length.

• What did you personally accomplish this week on the project? Give files or photos that demonstrate your progress. Prove to the reader that you put sufficient effort into the project over the course of the week (12+ hours).
  • Double blind trials on testing environment – so far people cannot tell the difference between the heart sound from our stethoscope and the dataset heart sound
  • Researched ways to find the similarity of the heart sound from stethoscope and the dataset so we can get a percentage to prove the testing environment is good enough
  • Trained the CNN on larger dataset from Physionet.
• Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?
  • Behind schedule – I did not work on this project enough this week due to carnival and other commitments.
  • I will put in more time next week and work with Ari and Ryan in lab more.
• What deliverables do you hope to complete in the next week?
  • We will be working on our CNN for AR/MS to reach an accuracy of 85%.
  • Start training on abnormal heart sounds for MR/AS.
  • Find a percentage accuracy of the similarity for our testing environment

Earlier this week I worked on sending data from Ari’s stethoscope to my ML algorithm. There were difficulties with this portion because we tried to send in real time data at first. There are no libraries or packages for this in Matlab so it was all experimental. The stethoscope was inputting weird audio data at first, so we switched our method to a more simpler method of just reading audio data from the stethoscope and saving it locally. I then trained my network on the training data and tested our stethoscope data on it to classify. Every time we tested it on our own heartbeat, it was classified as ‘normal’ which is a good sign, but we had no subject with an abnormal heart sound to also test on. Therefore, this is something we must test more thoroughly once Ari has finished making the testing setup with the speaker. I also plotted the input so that the heart sound could be visualized to the viewers during the demo. I was not able to make any improvements to the ML algorithm because I was busy travelling this week, but I will be working on that for this coming week.

For next week, I want to improve the ML portion by adding Eri’s denoising algorithm to the preprocessing. I also want to improve the communication between Matlab and the stethoscope to analyze data directly from the input instead of having to save it locally.

• What did you personally accomplish this week on the project? Give files or photos that demonstrate your progress. Prove to the reader that you put sufficient effort into the project over the course of the week (12+ hours).
  • This week I worked more on the hardware components and trying to make the signal look like the ones in our testing data. I also worked out a testing plan which involves a double blind between our microphone and our ML’s training data. The logic is that if the algo cannot tell the difference and a person cannot tell the difference (>80%), then the signal is about the same.
  • I also helped out with the integration for our demo, getting all of the separate matlab components to connect with each other and worked on modularizing our code.
  • Additionally, I looked into setting up the matlab code to work standalone with python so that way the stethoscope could work without being restricted to having a computer running matlab on it.
  • I also created a team to-do list with our highest action items based on what we learned during the demo, and from the feedback we were provided.
• Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?
  • Our progress seems to be on track, the next thing we need to do is get the testing working and sort out our process for doing so.
• What deliverables do you hope to complete in the next week?
  • For next week, I am to have a working testing set up, and want to validate our process. I also would like to connect Eri’s shannon energy denoising algo with Ryan’s ML and hopefully that will make the accuracy higher.
  • We also want to find approx 1000 more sound files so that we can train our ML more.