Very briefly, there are only three items left in the agenda. These are the integration of the warping code with the app (which we will attempt in the next two days), writing up the final report, and creating the demo video. From our discussion with Professor Sullivan, we know that it is already an accomplishment to have the individual components working and the project integrated 2/3 of the way. In a perfect world, the app and warping algorithm would be have a shared language and be easy to mix together. Our current ability to have inputs/outputs of each piece flow into each other is enough to show that with such a situation, we would have been able to add this final piece. The other two items have due dates in the next week. We are working on creating them now!
Mayur’s Status Report for 5/2
Honestly, there isn’t much to say now. Akash sent me the code which bounded the songs to -10/+15 BPM in the song selection algorithm, and I integrated the new version into the app. Integrating the time warping algorithm with the app code remains difficult. All that’s left otherwise is the final report and the demo video. Overall, capstone has been very design heavy. I gave my feedback to the course instructors via the FCEs, and I honestly had a positive experience with the course.
Team Status Report for 4/25
We have two weeks left before the end of the semester. At this point, most of our project is wrapping up, although we still have things to aim for. Each component of our project works individually, and 2/3 of the pieces are integrated. We are able to demonstrate that the inputs/outputs of each components match our expectations, and would flow as desired if integrated properly.
The final presentation is on Monday, and will be done by Akash. Accordingly, our group spent time this week running tests for our components, working on the slide deck, and generating graphs to include in the powerpoint. We finalized our decision to move to the phase vocoder as well in consideration of the fact that the wavelet transform experienced greater loss and took longer to process music.
Before “demo day”, we have the following goals:
- Finish testing, and possibly re-test components post-integration
- Attempt to integrate the final two portions
- Implement the -15/+10 BPM requirement for Song Selection
- Create the Video
- Final Report
Akash’s Status Report for 4/25
This week I worked on testing our app and collecting some running data. I ran into some issues with Android Studio, but with the help of Mayur, I was able to download the app onto my phone (the testing device). However, even after that, my phone was not able to collect data with an issue we have still not figured out. I then switched over to using an older Android phone (Galaxy S6) and it worked fine. We got good data that we can use to show the time-scale warp.
I also started working on the slides for our final presentation, that will be given this week. Our presentation is mostly focused on our actual implemented system and our metric validation.
Mayur’s Status Report 4/25
Very short update for this week. We are paradoxically both wrapping up and trying to maximize the amount we finish. I continued attempting to integrate without much success. Namely, my Android phone was not excepting USB connections, which made it difficult to actually physically try out the app. The issue was eventually solved by removing the battery from my phone and putting it back in. Otherwise, I worked with Akash to get the App working on his phone. There were a few problems that we solved, which he will explain in his report. Finally, the presentation is next week, so I am working on the slides. I am still hopeful in getting the integration done, but it is pretty difficult to do.
Team Status Report for 4/18
This week we focused on integrating our independent components.
The matlab code was ported to C++. Documentation for how to integrate the C++ files/functions were created. Upon integration, we ran into issues with (1) typing input variables from Java to C++, (2) translating some matlab functions to Java functionalities — namely, audioread. This is to be done by the Android OS. However, inputting a signal from Java to C++ causes a challenge in typing and storing variables.
To work around this, we are considering creating a C++ function manually to “audioread” wav files. This implementation, however, would then require our java functionality, that initiates music modification every 60 seconds, to be written and integrated in C++. In parallel, we are experimenting with performing “audioread” in Java through the Android OS and storing, typing, and transforming the signal matrices as necessary.
Details about the high-level approach for JNI usage can be found in Mayur’s status report, and details about the time-warping integration and audioread function can be found in Aarushi’s.
Mayur’s Status Report for 4/18
While I believed that the Song Selection Algorithm had been fully implemented with the code, it was not the case. My code was building, but I had not actually attempted to test it on my phone (which was a mistake). As it turns out, I needed to adjust memory settings within the gradle files of the App so that I could increase the allotted memory to hold more songs at runtime. Afterwards, I started to work on integrating the time warping code. This is extremely complicated for several reasons. First, the code requires passing arguments via the JNI. So, code needs to be adjusted so that data is properly serialized/typecast when it is sent between portions of the code. Secondly, the audioread function from matlab needs to be rewritten for C/C++. Finally, several more files need to be “included” with the project, which requires working with CMake and its associated files.
At the moment, I have about 20 tabs open on my computer trying to figure out how to make this work. For now, I believe that the best approach is to send a string with the file name to the C++ code, warping it within C++, writing a new file with C++, and then returning the location of the new file to be played to the Java code. The number one tip for using the JNI according to the official docs in regards to speed is minimizing the amount of data that is sent across it. The JNI needs to marshall/unmarshall all data, which takes a long time. For this reason, doing it the way described would most likely be the fastest approach. Next week I am hoping that we will have finished completely integrating the app and possibly begun testing certain parts. The presentation is the week after!
Akash’s Status Report for 4/18
This past week I worked on converting the Python implementation of the song selection algorithm to Java. This way we can easily integrate this feature into the app. I had a little difficulty with this as I haven’t coded in Java in a long time, but I was able to use the Python skeleton I wrote to help guide me. Now that it is all in Java, Mayur is going to work on integrating it into the app. The next feature we could add to this would be user preference of songs so that scores could be modified based on how much the user likes specific songs.
Team Status Report for 4/12
After our demo, our team decided that we would work on finishing up loose ends on our individual components. This included (1) porting the song selection algorithm from python to java, (2) porting the audio modification component to C++, and (3) writing code that would break up a song into 60 second chunks before warping. This would allow us to start integration the following week.
We made this decision based on our conversation with Professor Sullivan and Jens as they heavily reemphasized our goals for the end of the project. Additionally, we realized that our work progress was behind in relation to our Gantt chart. According to the plans, we would have completed integration this week and would move on to extended features in our following two weeks of slack time.
Thus, we will take what we have so far, integrate, and continue advancing on our individual parts in the remainder of our slack time, and then integrate again.
Mayur’s Status Report for 4/11
This week, Akash rewrote his code in Java and sent it to me. On my end, I added the song selection algorithm he sent into the code. This week highlighted one source of time sink in the future; up until now, I have only been writing code with a “proof of concept” mindset. Unfortunately, this means that the other two parts of the project are much more difficult to integrate, as the code is not designed well for this to happen. In the coming week, I will be refactoring the code so that it will be [hopefully] as simple as dragging-and-dropping Arushi’s code into the app. It’s pretty obvious integrating will be more complicated, but I want to get the code to as close as that state as possible so that it will be less of a hassle in the future.