Nick Lee, Sojeong Lee, Max Lutwak, Jacob Hoffman
Other than working with the team on refocusing, I didn’t get much done this week — the RPi and IMU showed up in the mail from Nick on Friday.
This week I’ll be figuring out & writing a simple app and the RPi Python code to talk to it over Bluetooth. This will constitute the interface between the hardware subsystem and the app/ML.
Once I have that data connection working, I’ll start collecting data to send to Sojeong and Jacob.
Updated Gantt Chart
Updated Risk Management
Data Collection :
Our major concern with the project is collecting the data set that is large enough to train our machine learning algorithm. In order to do this, we ordered a dummy to easily collect a large data set without us having to fall all the time. The test dummy is an easy way to collect data, but there is a risk that it might deviate from the data collected from actual humans. In order to mitigate this risk, we are also collecting the data of us falling.
Machine Learning Based Prediction :
Another concern was the low accuracy of the machine learning algorithm. In order to reduce the accuracy risks, we compared the SVM and RNN approaches to fall detection. We will also try training our model with a variety of preprocessed features (tuple of raw acceleration, magnitude, and angle) and find the feature combination and hyperparameters that achieve the best accuracy.
Hardware Compatibility and Accessibility:
At the outset, we anticipated that data collection and the accuracy of our ML approach would be the greatest risks. Now that we’re separated, data collection through our device is an even bigger problem, and so we’ve decided to begin collecting and tagging accelerometer data from our phones to supplement it. By configuring the discrete IMU a bit differently (50Hz datarate), we can match the data formats and provide both to the ML.
Ordering of Components:
We ordered all of the required components to complete the project successfully and by now they have been shipped to the relevant team members.
This week, The test dummy arrived to my home. I commenced collecting fall data with the test dummy.
As well, I designed a system to adjust the parameters of the FFT and Wavelet frequency features which will be fed into the SVM, and also made a system to adjust the parameters of PCA compression that will happen during pre processing of features.
This week we made adjustments due to the COVID-19 Situation. The work we did this week can be found in the group posting.
For this week, I mainly focused on implementing the geolocation features for the application. As we are adjusting our project remotely, we decided to Max holds the RPi for the rest of the semester, so I mailed it to his address. The application’s major features like GPS and sending notification progress are continued, and we will work on the mobile application receiving data either from the RPi or a computer to adjust the current situation.
After team discussion, we decided to transition to React Native from Android App as React Native supports both iOS and Android App as we are working remotely. I began to set up an initial environment for the application and rebuild the major features like getting the current location and sending notifications to the first responders. Also, I worked with Max on connecting networks with RPi for integration. We are still on progress fixing the connection issues. For the upcoming week, we will continue working on the integration before the demo.
This week we discussed about our adjustments to the project as we all are required to work remotely because of the COVID-19 situation. Although the fall detection algorithm can be developed as planned, integration with the hardware component will be hard. To account for this situation, we decided to develop each component of the project separately. My goal is to develop a fall detection algorithm using SVM, but it will not be real time as planned before. Instead of using real time data from the RPi device as the input, I would have to use pre-collected data stream to simulate the real time inputs. A possible solution would be to set up a client-server connection using Python’s socket library, and make the client send data to the server at the same time interval as the collected data. The server can then run the classifier to detect falls. For demonstration, graphing the data stream and notifying when a fall is detected could show that the algorithm correctly detects falls.
My work is currently behind schedule because of the change of plans for the project. Next week, I will continue collecting data for the SVM using my phone. It would be best to collect data with our RPi device, but since the IMU sensors on RPi will provide more accurate data, I predict that the classifier will work as expected on the RPi if it works with the data collected from my phone. I will also have to come up with different ways to use phase data for classification as there was a concern about it being different depending on the orientation of the device. In addition to that, I will have to figure out how to use data stream as an input to the SVM, as the current version only supports importing data from csv files.
This week I mostly worked on figuring out how to proceed with the rest of our team.
It turns out that there isn’t an easy way to get the parts for another setup — Adafruit and similar companies are shut down and Amazon has shipping dates for late April at the soonest. Because of this, we’ll have to keep working with the one device.
I also spent some time this week troubleshooting some issues Nick was having with connecting to it over different networks.
For our SOW, we decided to split the major parts of the design into separate goals to avoid needing access to the device, but we didn’t fully specify the end products. This week I’ll decide on concrete goals for demonstrating the hardware side’s functionality (and probably get Nick to mail the rpi since the Bluetooth interface will be on the hardware end).
To adjust to the COVID-19 situation, our team decided to make some adjustments to our initial project. The MVP for our project originally intended to detect falls in realtime on the raspberry PI by the project deadline. The new plan is to prove the concept due to the difficulty in developing remotely or for multiple devices. Instead, we will record acceleration data on the raspberry pi and perform the machine learning analysis on an external machine.
Because the Raspberry Pi is not classifying the falls in real time as originally intended in the old plan, the new plan is to demonstrate the ability for the raspberry pi to make a bluetooth call to a smartphone to alert contacts. By demonstrating the communication framework we designed between the Pi, smartphone, and emergency contacts as well as the machine learning framework we designed for fall detection intended to eventually run on the Raspberry Pi in real time, we will successfully prove a proof of concept for our fall detection device. We are also going to order another Raspberry Pi and the sensors to set up the entire hardware component so that Max can use it for the hardware system development and Nick can use the other for testing the Bluetooth communication between the mobile application and the hardware.
Also, we decided to develop the mobile application with React Native instead of using the Android Studio. As we are working on our project remotely, Nick, who is responsible for the development, has only access to the iPhone. Since the React Native supports both Android and iOS platforms at the same time, once the application is released, another team member who has either an Android phone or iPhone can download it for the integration of the system later.
For data collection, our original plan was to collect data from people with different weights and heights to ensure that the data does not differ depending on people’s size. However, this became hard due to remote work, so we decided to have one person do the data collection. We also ordered a gym mat and a dummy for data collection before spring break, and we shipped it to Jacob so that he can use them for collecting data. The fall detection algorithm can be developed as planned, except that it will run on a computer instead of running on a Raspberry Pi. Our stretch goal would be to integrate the software component with the pi.
For this week, I received feedback from our teammates on the App UI and make sure the UI is intuitive to users, who are mostly elders. Max and I started working together on integrating the App and RPi. We will continue to improve the integration to make it automatic instead of manual pair between phone and RPi after the spring break. Our progress is on track, and we are looking forward to seeing much progress after the break as we approach the end of the semester.