Author: zrudnick

This week, I got the Bluetooth signal to work between the mat and the lock. I bought a new BLE module (an HM-10) to replace our old part because it wasn’t applicable to what we were doing. This one was the most recommended BLE module for Arduino Uno R3 according to multiple websites online and had comprehensive documentation online that made understanding its AT commands easy. I added client code to the mat XIAO ESP3S3 and made the HM-10 module act as a server. So, the app acts as a client for the mat server, and the mat acts as a client for the lock server. I ran the entire data pipeline from song choice on app to song received on lock, and I did not run into any issues with data transfer as long as the devices are all initialized correctly. While doing this work I also worked on updating the mat code and making it more readable for Brooke and Jada, because they will need to understand it while doing integration.

I am on time with my work, because I got Bluetooth to work for the whole system, which I needed to do by the end of this week. All I need to do this week is finish the mat code to update the dance logic (as explained in the Team update) and edit the code to be readable for Brooke and Jada. Since I am in New Hampshire this week for a competition, I will help them virtually with the system integration, communicating over calls and sending code updates if required. In addition, I will help with the final poster.

This week I made considerable progress on the project. First, I finished my app. I made the Bluetooth connection work from my app to the mat, so now when I click on a specific song on the app, the mat microcontroller receives the corresponding song number. It works reliably well when the correct procedure is followed. Tomorrow I am going to work on testing this connection, to 1) check that the correct procedure always works, and 2) to find and fix edge cases where the connection might not work. I also made the app better looking by adding a font and splash screen, and removing an unnecessary page menu. In addition, I continued working on the other mat code. The mat should be completed soon, so then I can debug this code on the mat. Lastly, I started working on the connection between the mat and the lock Arduino. I am going to complete these in the next few days, so that I have time to perform all of the testing that I want to by the end of the week.

I am on time with my work, because I can get everything done that I need to this week (I don’t have anything else to do, so I have a lot of time).

This week, I got Bluetooth working on our app! It was able to scan for devices and connect to my iPhone.  Below are photos showing 1) my iPhone being connected to the Android phone and b) connecting to my iPhone (named mtn dew) from the Beatlock app on my Android phone. 

This means our app is mostly done. To finish it, we need BLE implemented on the Seeed XIAO microcontroller in our mat. So, after the app was working, I went back to working on the mat code. Once BLE works on the mat, I will change the app code to not scan for devices but simply locate and connect to the mat. Getting BLE working in the mat and then making them work together is my goal for this week, since I have already finished most of the code for the mat. I just need to try out the code on the microcontroller and debug any problems that arise. I also have other code to test out (the code to deal with the dance input) that I will debug as soon as the mat is functional.

I am on schedule, because I have made the progress that I was hoping to make. There is still lots of work to be done, but since Booth and one of my classes will be done by this Wednesday, I will have even more time to work on it after then.

I spent a lot of time this week working on our app. First, I setup my Android phone to have the Expo app. It could run an Expo Go app fine. However, when I created a development build (image below) and downloaded it on my phone, it always crashed upon opening.

To debug the app, I used two tools. The link to the build contains information about the build, so it can tell you if an error was encountered with the package.json, dependencies, app config, expo doctor, and gradlew, and more. While debugging, I ran into issues in all of these areas, but the logs provided allowed me to fix the errors. However, sometimes there were no errors in any of these, but the app still crashed. To fix this, I downloaded Developer Tools on the phone (which I believe are now integrated into Android phones, but since the phone is so old, it was a separate app) and turned on Debug mode for my phone. Then, when the phone is plugged into my laptop, I was able to see a log of all errors coming from the phone. I saw that the same vague error was encountered every time I tried to open the app. After doing research online, and having no luck with any of the quick fixes suggested by other people encountering the same error, I decided to uninstall all the packages I had installed, and then reinstall the required ones. I did this in case 1) I was using incompatible versions of packages together or 2) one of the packages was causing a problem, and I would be able to isolate it. After reinstalling the necessary ones (not related to BLE), the app finally worked on my phone! Image shown below. This process took too long, but I have never made an app before, so it was cool. Now, I have already written the code for BLE, and I know which packages to install. I have not yet actually implemented it, but that is my next step. I anticipate running into more problems, but at least I know how to deal with them now. I also plan on polishing the app design. It is currently a prototype of the real design.

In addition to the app, I made sure my software was up-to-date with Jada and Brooke’s work.

My progress is on schedule. I wanted to have the app complete before the Interim Demo, and it is at least partially complete already. Tomorrow I will start integrating BLE functions again.

In the next week, I should have the app completely finished and communicating with the mat. This will require adding BLE connectivity to the mat too, which I have been working on but have not completed yet. While I am adding this connectivity, I will also add connectivity to the lock, since Jada has been getting BLE to work on the lock.

This week, we have been working to prepare for the interim demo. I worked with Jada to research how to set up audio and Bluetooth for the lock. We had not decided yet whether to use a Seeed microcontroller or an Arduino Uno R3 for the lock, so I looked into software options that would work for both architectures. The best way to store and play audio clips seems to be storing .wav files (generated online using a program like Audacity) to an SD card and then playing the songs using a built in module. Once we have the SD card ordered, we will be able to test the code we have for playing our songs. For Bluetooth, we had a similar problem in that how we implement it will depend on whether or not we use the Seeed microcontroller. If we do, then BLE is built into the Seeed microcontrollers and their website provides documentation for how to implement both client and server protocols. If we use an Arduino, we will have to buy and implement its own Bluetooth module to connect to the board. I spent a lot of time learning about BLE on the Seeed microcontroller, because regardless of whether we use it for the lock, we are using it for the mat, and it needs to act as a client and server in the mat.

Another thing I worked on this week was the correctness algorithm (how to determine if the input should be marked as correct). In my current algorithm, I model each correct dance move with a corresponding Gaussian curve, and performing the step before or after the required time provides ‘points’ according to the y-value of the curve. A specific percentage of total points must be achieved in order for the dance to be considered correct. In addition, if one move is outright incorrect or very badly timed (+ or – 3 seconds), the dance is considered incorrect.

Lastly, I have revisited my app with the hope of making the BLE connection work. To do this, I found an old Android phone (Galaxy Note 5) that I have and ordered a charging cable for it, because it was dead. Now that I have a working Android phone, I should be able to develop my app directly on the phone and avoid the problems that I had with iOS.

I might be slightly behind schedule, because I had to spend a lot of my week preparing for the Greek Sing musical I was a part of. However, that is done, so next week I should be able to finish a lot of the work that I started this week.

Since we have a week until the interim demo, I am going to focus on wrapping up everything to make the demo successful. This would ideally include 1) making BLE work for the app, mat, and lock, 2) getting audio set up, and 3) making sure we can get input from the foot pads, decide whether or not the dance is correct, and relay the information to the lock. At this point, we have started everything we need for the demo, we just need to make some decisions and wrap up some loose ends.

This week, I worked mostly on the code for the mat and the door lock. I spent some time researching Bluetooth for our app, and ultimately decided it may be easier to run our app on an Android phone instead of an iPhone. Since I have an old Android phone at my apartment, I plan to charge it and download the development build of the app onto the phone, if possible (the phone is pretty old). I have not yet tested this, but I intend to do it by Monday. After this research, I moved my focus once again to the mat and door lock software. Jada has been testing the door lock electronics using starter code online, and I have been brainstorming how to rewrite that code for our exact purposes. Further, I updated the mat code so it will be ready to test as soon as the physical components are connected to the mat board. This should be very soon, since Jada and Brooke have been working on making prototypes of the footpads to connect to the weight sensors. I am confident with the current control flow of the program, so I am just waiting on a mat prototype to be put together so that I can develop the feature integration.

I am slightly behind schedule, because I did not anticipate spending as much time trying to get Bluetooth to work and because I have not had a physical prototype to test with while developing my code. However, I have made consistent progress, and I think it will be easier to write the code once Jada and Brooke construct the first prototypes.

Next week, I intend to try running my app on my Android phone, or to find an alternative solution if that doesn’t work. In addition, I am going to test my code on the mat and lock prototypes and fix any problems that arise.

I have made significant progress with both the app and the mat software, though I’ve run into several issues. For the app, I learned that the development environment I was using (Expo Dev) would not support Bluetooth, which we need to connect the app to the mat. To solve this problem, I transitioned to using Expo Go and EAS Build, which will allow me to export the app binary and run it in an emulator app.  I’ve been having some issues using this new workflow, so I will need to ask someone on Monday. I have finished the actual code, except for any minor changes I may make, so I just need to figure out how to get the app to run. I don’t have a background in mobile/ web app development, so this is mostly new to me and has taken some time just to learn.

For the mat software, I switched from using the Arduino IDE to using PlatformIO in VSCode, because of the limited functionality of the Arduino IDE. I continued working on the code, and it should be ready to test as soon as the Seeed Xiao is connected to the weight sensor input. I have had some problems, such as not being able to rely on built-in features that I have used before. The Seeed XIAO has a Xtensa LX7 processor, which I have never used before.

I am currently back on schedule, due to the progress I have made. To stay on schedule, I will need to continue making progress at the same rate. I will need to figure out how to get the app to work and start testing the mat software.

This week, we received the components that we ordered and we started sensor testing. After seeing the components in person, and after considering the rough sketch of our PCB that I had made, we decided that we will make our PCB later. For now, we are going to wire everything directly. This is because the PCB is not strictly necessary, and we want to make everything work first.

Since I didn’t have to work on the PCB schematic, I spent my time working instead on the mat software and the React Native app. I made considerable progress with the app, which I show in photos below. Specifically, I made the page that will have the list of songs to choose from. I also made it so that when you click on a song’s image, it takes you to a new page. On this new page, I will make it so that a Bluetooth signal is sent out to the mat.

These photos show the Songs page, and what it looks like when the user hovers on the song choice. The next page is currently empty.

My progress is behind schedule, because I have not had much time to work on the mat software, and I will be traveling this week to attend a visitation day for a Ph.D. program, so I will not have a lot of time this week either. However, I plan to work on the software during Spring Break, because I have nothing else to do then. I hope to finish the app and make significant progress on the software during the break, so that I can start testing on the microcontroller as soon as we get back from break. That will get me back on schedule.

This week, I am going to research how to implement Bluetooth in the app, and I am going to continue to work on the mat software.

This week, we finalized our list of materials and ordered our first parts. The parts most relevant to my work are the microcontrollers we decided to use for the mat and the door lock. We decided to use the same microcontroller for both, so we ordered one, which we will use to test both. We bought the Seeed Studio XIAO ESP32S3 microcontroller, which can run code developed for Arduino and supports wireless communication via Wi-Fi. Now that we have compiled our materials list, I can start writing the schematic for our PCB, which we are going to use for our mat. I plan to use Fusion360 for PCB development. We have allocated one week for schematic design and one week for board design, so we should expect our PCB to be here after spring break at the latest. If we can expedite the development of our PCB, we might be able to get it before break.

In addition to doing research into required parts, I worked on our app development and pseudocode for the mat. For the app, I am using React Native and Expo, which allows me to write in JavaScript and create an iOS app from my laptop (which uses Windows). So far, I have created a user interface with song names and images. The next step is to make it so that if an image is clicked, then the corresponding song will be sent via Wi-Fi to the mat. For the code for the mat, I only wrote high-level pseudocode, because I was not sure which microcontroller we would choose to use (we were considering Arduino and Raspberry Pi). Since we have chosen to use microcontrollers using Arduino, I can now rewrite the pseudocode in the Arduino IDE.

My progress so far has been on schedule, but I will need to put in more hours during the next few weeks in order to stay on schedule. This is because the amount of work that we can do is now much greater, and the more work we do sooner, the more time will be left for debugging, ordering new parts, and testing. This week, after finishing our design presentation slides, I will focus on the PCB schematic. After that is complete, I will continue working on the app and the software for the mat. I will also help Jada and Brooke if needed with manufacturing.

My main focus for this past week was on presenting our proposal presentation well. Additionally, I researched my plan for the software for the two-factor authentication app. We discussed as a team what materials we want to use and what kind of user experience each decision leads to.

I feel that we are on schedule. This upcoming week, I plan to continue researching software for both the app and the dance mat. I plan to work with Jada and Brooke to ensure that their hardware plans will work with my software plans, and make suggestions as necessary.