Category: Mayur’s Status Report

Honestly, there isn’t much to say now. Akash sent me the code which bounded the songs to -10/+15 BPM in the song selection algorithm, and I integrated the new version into the app. Integrating the time warping algorithm with the app code remains difficult. All that’s left otherwise is the final report and the demo video. Overall, capstone has been very design heavy. I gave my feedback to the course instructors via the FCEs, and I honestly had a positive experience with the course.

Very short update for this week. We are paradoxically both wrapping up and trying to maximize the amount we finish. I continued attempting to integrate without much success. Namely, my Android phone was not excepting USB connections, which made it difficult to actually physically try out the app. The issue was eventually solved by removing the battery from my phone and putting it back in. Otherwise, I worked with Akash to get the App working on his phone. There were a few problems that we solved, which he will explain in his report. Finally, the presentation is next week, so I am working on the slides. I am still hopeful in getting the integration done, but it is pretty difficult to do.

While I believed that the Song Selection Algorithm had been fully implemented with the code, it was not the case. My code was building, but I had not actually attempted to test it on my phone (which was a mistake). As it turns out, I needed to adjust memory settings within the gradle files of the App so that I could increase the allotted memory to hold more songs at runtime. Afterwards, I started to work on integrating the time warping code. This is extremely complicated for several reasons. First, the code requires passing arguments via the JNI. So, code needs to be adjusted so that data is properly serialized/typecast when it is sent between portions of the code. Secondly, the audioread function from matlab needs to be rewritten for C/C++. Finally, several more files need to be “included” with the project, which requires working with CMake and its associated files.

At the moment, I have about 20 tabs open on my computer trying to figure out how to make this work. For now, I believe that the best approach is to send a string with the file name to the C++ code, warping it within C++, writing a new file with C++, and then returning the location of the new file to be played to the Java code. The number one tip for using the JNI according to the official docs in regards to speed is minimizing the amount of data that is sent across it. The JNI needs to marshall/unmarshall all data, which takes a long time. For this reason, doing it the way described would most likely be the fastest approach. Next week I am hoping that we will have finished completely integrating the app and possibly begun testing certain parts. The presentation is the week after!

This week, Akash rewrote his code in Java and sent it to me. On my end, I added the song selection algorithm he sent into the code. This week highlighted one source of time sink in the future; up until now, I have only been writing code with a “proof of concept” mindset. Unfortunately, this means that the other two parts of the project are much more difficult to integrate, as the code is not designed well for this to happen. In the coming week, I will be refactoring the code so that it will be [hopefully] as simple as dragging-and-dropping Arushi’s code into the app. It’s pretty obvious integrating will be more complicated, but I want to get the code to as close as that state as possible so that it will be less of a hassle in the future.

This week saw a bit of a pivot for me. Originally, my plan was to add music to the app and show that it was possible to run C code (call a function) from within the codebase.

First, I did the former. Indeed, I can play “Africa” by Toto with a WAV file when a user clicks the Play button. And, I can stop it when the user presses the Pause button. At this point, adding more music is fairly straightforward.

The pivot comes in from the second goal. Arushi was able to write a Vocoder in Matlab, and she found a link about how to run MatLab on Android. My network was really bad, and it took me 10 hours to download the Matlab software to my computer. I am hoping to investigate the possibility of using Matlab instead of C on Android using the suggestions at this link: https://www.mathworks.com/matlabcentral/answers/417602-can-matlab-code-is-used-in-android-studio-for-application. This way, I can explain the progress during the demo and maybe allow Arushi an easier time in developing the Wavelet code.

Next week is the Demo, for which my subsystem should be close to done. The idea is to have each part done to some degree (C/Matlab code, UI, music playability), so that all that is left is to polish the parts and integrate it with the other subsystems. Hence, that is my goal for the next week (to make sure the subparts have a preliminary implementation).

My goal for last week was to finish off the basic step counting functionality of the app so that Akash could run… and I finished! I downloaded the app to my phone before sending it to Akash. After pressing a “play” button, the app begins counting steps. Instead of using the step counter sensor, I am currently using the step detector. From our discussion with Professor Sullivan, we decided to gather data every second so that we could test the limits of the warping algorithm. As described in our research phase, the step counter takes multiple extra seconds to gather its measurements. This is in order for it to filter out false positives (which takes extra processing time). When I tried it out with the app, I found that I wasn’t getting any measurements for the first 6-9 seconds after starting up the app. Since the step counter took a variable number of multiple seconds to gather data that we wanted at the granularity of a second, it was not possible to use it in this case. If we switch back to the granularity of 60 or 90 seconds, we will switch back to it then.

Initially, I displayed the step counts on the screen in the form of an alert dialog. This did not work out for two reasons. The first was that I was unsure if there was a limit to the character count of a dialog box. The second was that I could not find a way to copy the text of the dialog into an email program. As a solution, I implemented code that brought up a ready-filled email to send out the step counts.

My goal for next week is to play music off the phone. As a stretch goal, I would want to call some function in C/C++ just as a proof-of-concept. Admittedly, the UI looks hideous at the moment. This is something else I can fix.

Pictures:

 <– The UI

 

  <– Automatically pulls up email with pre-filled body after user presses pause button

As I described in last week’s report, the outbreak of COVID-19 has forced all classes online. Our group wrote a Statement of Work that describes how we plan on dividing the project (see the team status report for this week). Essentially, we will be continuing with our initial division of labor, with small modifications to compensate for the difficulty of integrating our individual portions. During this next week, I will be creating the framework that allows Android to record step counts. I will be writing an app that will allow Akash to record his pace at 90 seconds intervals during a 30 minute run. Currently, my plan is to emit the recorded data as a list and send it in an email with a hard-coded address. This is subject to change, however. Aarushi will be using the data we gather to test out her time warping implementation.

In regards to the implementation of the project this week, I spent time attempting to work with the Native C/C++ App with Android Studio. Before, I was working with an Empty Activity instead. As it turns out, there were many more problems than I initially anticipated with setting up the new app. Right from the start, I received errors when Android Studio tried to build the project.  Specifically, I received errors because I had not accepted certain licenses. I attempted to follow the solutions in this link https://stackoverflow.com/questions/54273412/failed-to-install-the-following-android-sdk-packages-as-some-licences-have-not-b to solve the problem. But, I got an error along the way that told me the JAVA_HOME environment variable had not been set. I was confused at first, since I assumed that Android Studio would just set it to the JDK (actually JRE) that came embedded with the program (apparently not). To solve the issue, I downloaded the latest version of Java (jdk-14) and set my environment variables manually. Surprisingly, Android Studio does not support versions of Java above 8, and so I had to uninstall it. Then, I found out that there are GUI issues with using the default shell with Windows when trying to accept the licenses. As a response, I downloaded Git Bash. Eventually, this was my solution on how to fix the issues I had:

1. Download Git Bash
2. Navigate to  ~/AppData/Local/Android/Sdk/tools/bin
3. Run $ export JAVA_HOME=”C:\Program Files\Android\Android Studio\jre”
4. Run $ ./sdkmanager.bat –licenses
5. Reply yes to every prompt
6. Reload Android Studio and attempt to re-build the project
7. Accept every prompt to resolve issues related to getting/using a C/C++ compiler

These last two weeks have been pretty wild. As I stated as the goal of my last post, I wanted to finish up the design report on Monday as my goal for the week. This ended up taking both of Monday and Tuesday, as we had a sick member of our team. We got a PDF for feedback, with a couple points highlighted. I think it will be beneficial to discuss it with our mentor and TA(s), since the scan did not turn out that well. One thing I noted was that we increased the 60 seconds for music changing to 90 seconds, as we believe that long-distance runners will prefer less warping. Furthermore, from testing, we are aware that a minimum of 60 seconds is needed for accurate pace measurements, with more time preferred.

Our group is currently in discussion on how to proceed with our project in light of COVID-19. Our group members will no longer be on campus, so we will need to be more careful with how we coordinate pieces of the project. The virus has introduced a couple dangers to our project. For one, we will need to reevaluate our team member responsibilities. The previous division of tasks assumed that we could have face-to-face meetings, but now that is not possible. Another pain point will be the integration phase, as we cannot physically meet. Finally, testing will have to be reevaluated, since each member will need individual testing equipment and a complete repository of the project.

This week, we will be hashing out all of these ideas. Individually, I plan on creating the new UI for the main page of the app.

I spent most of my time for capstone this week working on the design presentation (finishing touches) and the design proposal. Our team wanted to ensure that we incorporated the feedback from the proposal that we received from both the mentors and our peers. We discuss this more in depth within our team status report. For the design proposal, we split up the paper into portions to work on individually. I am tasked with completing a piece of the system decision, part of the design specification, and part of the tradeoffs section.

In terms of deliverables, my top goal is to finish the design with the team. Finalizing each portion of the project is important so that we can begin the actual creation.

This week, I finished my exploration into using Python with Android Studio. I discovered several different methods. Each came with a host of problems that made them illogical to use.  Most of their problems were either that they didn’t have a way to access the Android Step Detector and Step Counter sensors, or that they didn’t provide ways to make callable functions in Python to interface with the app built in Java. We will be including a section in the design presentation and report with more details.

There was no specific reason we wanted to use Python other than for its familiarity, so we have decided to write the discrete wavelet transform with C/C++, which is natively supported by Android Studio. We also had the option of using Java, but feel that its efficiency for audio processing will not be as strong as C’s. I did a bit of searching for libraries that already implement the wavelet transform, and found a couple.

Finally, I worked on the design presentation slides to update them for our presentation next week. Specifically, the team discussed the data we had gathered to make conclusions on how we wanted to adjust our design goals and platform specifications.

I would say that the project is currently on track. We have taken the feedback from the proposal and used it to refine our project appropriately. Next week, the deliverables I want are the design doc (obviously) and having an app that uses a little C code as a proof-of-concept.