Posted on

Luke’s Status Report for 04/20/24

As said last report, my main focus this week was to work on the integration of our backend managed queue with the actual Spotify queue. This consisted of writing a queue request scheduler that uses data about the length of the current songs playing to ensure the state of our queue is consistent with regards to the Spotify queue in terms of timing. I then used futures and the java concurrent safe scheduler class to implement this in an efficient way. With this in place, all of our veto and ordering of the queue functionalities work well with no problems. This was one of the lat big steps we needed to take so it is great we were able to get it done so efficiently.

Also, I worked more on various other tasks this week. I worked on some verification tests for my subsystems, primarily involving the accuracy of the semantic matching algorithm. In addition to this, Ice worked more on the authorization driver by replacing our current method of using a web driver with using refresh tokens instead, which makes our lives a lot easier snd limits the communications between the pis.

Further, I worked on the endless queue functionality. This basically just means, that when it is time for the scheduler to queue whichever song is at the top of the queue, if there are currently no songs in the queue, then the system will just queue a song that is a recommendation based on the previous song that played. However, this is slightly inefficient because it takes a couple of seconds to generate the recommendation and communicate between the pis, so we implemented this function as a prefetching process. Basically, the system will prefetch a recommendation if the queue is empty, and then when it is time to actually queue it, if the queue is still empty we use the rec and prefetch another. This was great progress and now makes our system fully functional as originally planned .

We are on schedule. This week was massively important in terms of all of the progress we made, so we just need to continue strong for these last two weeks. Next week I will focus mainly on small robustness improvements and verification and validation testing

What new tools or new knowledge did you find it necessary to learn to be able to accomplish these tasks? What learning strategies did you use to acquire this new knowledge?

I definitely learned a lot about new classes and packages that Java supports, such as the concurrent library, and some of the ML related classes that we use for the semantic matching. My core learning strategies involved reading lots of code documentation and examples – this helped me understand the use cases of these tools and then figure out how to incorporate them into our own project.

Posted on

Team Status Report for 4/20/24

  • One of the most significant risks that could jeopardize this project is not having access to the lighting fixtures and accompanying devices during these last few weeks for experimentation, testing, and demonstrations. This risk is being mitigated by ordering the same parts for ourselves so that we will have full control over a proper set of lighting equipment.
  • A secondary risk that could jeopardize this project is having trouble integrating all the modules of the system together on the Raspberry Pi’s. While these processors are indeed capable of multithreading, to operate the core queue manager or the recommendation service simultaneously with continuous lighting control signal generation, this may slow down the subsystems perceptively. This risk is being mitigated by having a third Raspberry Pi on reserve just in case it is necessary to dedicate an entire Raspberry Pi computer to each major subsystem. 
  • In order to mitigate the risk of not having enough time to run as many or as comprehensive of tests as necessary, we started building our testing systems early last week.
  • No changes have been made to the system diagram or the project schedule.
Posted on

Thomas’ Status Report for 4/20/24

Thomas Lee

  • I spent this week trying to understand the new lighting fixtures, and creating the different lighting schemes for these IDeATe Lab lights instead of our original Furious Five RG lights. Besides locating and procuring the lights, the ENTTEC DMX to USB converter, and the proper cables and DMX terminator plug, I ordered a set of all these components through the Purchase Request form to ensure that we will have an operating set these coming weeks. I also deciphered the cryptic channel & intensity mappings for the new SlimPAR PRO Q USB lighting fixtures, as the channel IDs sensitized by the Java port of DmxPy did not align exactly with the values provided in the User Manual for the lights. Using this mapping I was able to create the first two lighting schemes we would be using, and successfully tested them by operating the lights in real time with the Java processes administering the patterns. The video below shows one lighting test sample, and demonstrates that we are finally generating reliable DMX signals and controlling the lighting at a fine-grained resolution using a lighting microservice operating right out of the recommender module (which was previously demonstrated to work correctly on the second RPi):
    https://drive.google.com/file/d/1pNcK16PDUi-hqoaK9zqlAS_2xBKROgXs/view?usp=sharing
  • My progress is on schedule, however I would have liked to help start module testing a little earlier.
  • In the next week I hope to finish the rest of the lighting models & integrate the light system, collect more survey & testing data, and polish the whole system while checking complete end-to-end functionality.
    ==========================================================
  • As I have worked on this project, there are a few new tools and pieces of engineering knowledge that I have acquired as necessary for different parts of my project. For example, I learned how to use a Raspberry Pi, how to use web sockets under a Springboot framework for a live service web app, and how to control hardware lighting fixtures without on-board processors using a computer program by generating & transmitting DMX signals. In order to learn these skills, I had to use a range of different learning strategies, my primary methods being online research, following public tutorials, and reading documentation from hardware suppliers’ websites. I also consulted previous ECE Capstone groups for their expertise in utilizing certain technologies (such as controlling a DMX lighting fixture using a computer program), or asking my teammates (to learn more advanced version control tactics on Github) learning how to do different things directly from a real person. I also applied the programming strategies I learned from classes at Carnegie Mellon, such as Web Apps & Distributed Systems.
Posted on

Matt’s Status Report for 4/20/24

  • I started by making tests for heavy user load. I made it using chrome recordings which you can then download into a JavaScript file. I think I will run a few of those scripts at the same time to simulate concurrency. I plan on actually running the test next week. I also made much needed front end changes. I made it so that the app is responsive to different sized screens, does not have the weird zoom in that it used to have, made sure each song element is the same size, and also added a feature that scrolls through the text if it is overflowing (video included). I also ran into some issues with the user timeout, it was working fine on chrome and I knew safari would have some differences but I did not account for when users leave the page that the javaScript code would stop completely. Fortunately there are events that handle those cases so I added those to the code. The rest of my week was thinking about solutions to the problem of: if users refresh the page it will make them log back in as a new user. My biggest issue was differentiating a refresh and a page close on purpose, there is no 100% reliable way (to my knowledge/research) to do this since the browser treats them very similarly. I then just decided to have users stay logged in all the time. After having a few issues with reconnecting the sockets I finally got it to work, but then after some light testing, the app crashed, I think I was sending too many requests to the server but I will look into it more in the future. It could be a different thing implemented recently. Right now I think I have an acceptable solution. There is no reason for users to refresh the page since the WebSockets make it so the screen refreshes after every single change, but I realize people will do it anyways. When this happens on some browsers (like chrome) it will display a warning that they will have to re-login but on others (like safari) it will not display anything (I will also look into this a little more). If the user then clicks they want to refresh it will log them out and they will have to log back in and I will discard their old user votes as if the user was gone for an extended period of time. While thinking about this issue I also worked with Luke and Thomas to integrate our system together.

    Link to frontend:  https://share.icloud.com/photos/0182jqTvykRs-CBnIoQ5J29qQ
    I think the buttons are slow to load since I’m on a bad network right now. I did not notice this on other networks but I will double check when I get back to campus.

 

  • I am on schedule. I achieved what I wanted to do the past two weeks.
  • Next week I plan on helping integrate the lights to our system aswell as testing the app and thinking of that refresh problem more, maybe asking friends for their opinion.

 

  • New tools or knowledge:

There have been a lot of new technologies and knowledge I have learned to do this project. Websockets, everything Raspberry Pi related, Maven projects, Github commands, Testing methods, and probably more I can’t think of right now.

The first thing I always looked for if I was doing something I never did before was documentation / examples of other people doing it (a video is a plus). After struggling with getting websockets to work from java to JavaScript and debugging to no success I found a video of someone using springboot to do it along with their repo. So I was able to learn enough about the process by following along the video and then just messing around with the code by myself to see what happens when I change certain things.  I also was lucky enough to find a video for getting a program to run on raspberry pi from the start but then when I tried it myself it did not work. So then I went to online forums like stack overflow and tried a lot of things until my solution worked. It is safe to say that one of the most important things I needed for these new tasks was patience and persistence. Github commands are very easy to lookup, aswell as Maven commands to run certain files and to solve errors. It is also very helpful to talk with teammates when struggling with a problem as they can provide a different view / a solution you were not thinking of. Our team is good at helping each other come to solutions to their problems.

Posted on

Luke’s Status Report for 04/06/24

This week, I took a bit of a detour from my original plans and worked more on the authorization model. Basically, to access Spotify’s resources you need to get a proper access token, but this token expires after an hour. So if we want to be able to host larger events then we need to be able to automatically regenerate access tokens for use on both the core and recommender pis. This involved some more automation with selenium, but the main task was querying and sending the access tokens across pis. This involved some more intra-pi communication code and integration into the rest of the subsystems. Overall, this process went well and we were able to test it, which was successful. Now we can operate music mirror for an indefinite period of time without having to worry about losing authorization to Spotify’s resources.

With this in mind, I did start to work on the timing systems that we will use on the core pi by introducing timestamp timers. This infrastructure will be used with the Spotify queue integration which I will work on more this upcoming week.

Verification: I started to write a comprehensive test base for the semantic match. One of the big requirements for accuracy was being able to translate user input into accessible Spotify resources, and I’ve developed multiple solutions for this but we need to be able to test efficiently. So, I created a bunch of test cases for common situations we’ll run into, and began to compare the 1-gram frequency map with the embedding transformer solutions to see which is more accurate.

Validation: In terms of the overall system, we started testing more live concurrent users and also did some robustness checks, such as spamming session req requests to see how the system would handle it. To our satisfaction, it was actually quite robust to these failures but we’d still like to incorporate some sort of limit on the requests a single user can make to ensure nobody monopolizes the queue

Overall, we are still on schedule but need to keep working hard for these last few weeks. I plan to work on the queue integration further in the upcoming days before carnival.

Posted on

Team Status Report for 4/6/24

  • We borrowed the lights later in the week this week and are still trying to figure them out and will work on the timings this week. We did figure out the keep-alive functionality though.  So for the first two our risks are still the same as they were last week: The most significant risk that could jeopardize this project is still the lighting system. Since our lighting fixture displayed some behavior before starting to fail consistently, we need to test some other configurations, or consider obtaining a second set of lighting fixtures. We plan to mitigate this risk by borrowing equipment from the IDeATe lab: first, we will borrow an ENTTEC DMX to USB converter, which a previous ECE Capstone team (Group D4) told us they used. If this does not work we will borrow their DMX lights as well, to test whether it is an issue with the fixture. Our second concern is managing real-world song timings, as the Spotify Web API does not issue us the capability to wait for a callback when a song finishes playing, although it does return the exact duration time of any track. We intend to mitigate this risk by experimenting with different clocks and internal song tracking.
  • No changes have been made to the system diagram or the project schedule.
Posted on

Thomas’ Status Report for 4/6/24

Thomas Lee

  • I started this week by working on system integration and verifying that our whole project was operating smoothly end to end. This was in preparation for the interim demo that we had this past week. This involved some code refactoring and submodule implementation, as well as manual testing in the lab to make sure we could have a polished demo. I also went and acquired a lighting kit from the IDeATe lab, getting two new DMX-capable lights, more necessarily cables, and the ENTTEC DMX to USB converter, which a previous group had told us was required to transmit signals from our laptops/Raspberry Pi’s. I set up the new lights and compared their functionality to the previous lighting fixture we had by testing a few different DMX signal generation libraries. I also wrote a few different configurations of Java test scripts to see if they would better and more organically than the older lighting fixture. Below is a video of setting different colors at regular time intervals on the new lights, which are simpler but may have a lower barrier of entry:
    https://drive.google.com/file/d/1ZJ4hMDnmAVAk5n2S4THXyMkTPESUGbx9/view?usp=sharing
    Unfortunately, we still do not have that great of an understanding of how the different channels and intensities work for the DMX controlled lighting fixtures, especially since these new lights did not come with a user manual. This will definitely require some more systematic testing to reverse engineer the behavioral specifications of the lights next week.
  • Verification: I will use timestamped requests to determine the user input to internal system round-trip time. This will help us determine the effective latency of our web app, backend, and main RPi core system for our use-case requirement of a responsive & tactile system for users. We will also create a test script that simulates multiple users to check that the system can maintain websockets for and accept requests from 50-150 concurrently connected users. This will verify our system’s effective capacity and robustness. In terms of testing we have already done we have only ran basic functionality and behavioral correctness tests for the queue, recommendation  & Spotify API request systems, and the web app.
  • My progress on the web app and queue manipulation core modules is on schedule, but the lighting system is behind schedule. There is no way to catch up on progress in regards to the lighting system besides focusing on it and spending more time on the lighting modules, which I intend to do next week.
  • Next week I will be putting much more focus on the lighting system, as there are still some issues with controlling the lights finely. I will attempt to learn how to use them better and have some skeleton lighting schemes ready for integration with the other subsystems functioning on a basic level.
Posted on

Matt’s Status Report for 4/6/24

  • This week I started off by adding the functionality to send the updated queue after a song recommendation is sent from the recommender pi.  It first sends the updated queue with just a request added to the queue then when the queue is updated, the backend sends that to all clients. I added the keep-alive functionality for users to be active at least once for 15 minutes. If the timer runs out or if the user closes the page then all of their votes will not be counted towards the likes/dislikes of the songs. The user’s likes and dislikes are still saved so when they do make another action on the page then their votes will be restored to the songs still available. I was also playing around with what should log users out. The old version had it so if they refreshed then it would log them out. Now I have the app save the data so refreshing will not make a new user. I also made some UI fixes. There is now a designated dj color and all the songs recommended by the dj say it was queued by the dj. Also when song recommendations are pending there is no option to like or dislike. 
  • Testing:  I have only run very basic tests with Luke and Thomas so far where we all queue a bunch of songs and verify if it is what we want. To test the app/Pis further I want to make a script to simulate 100-150 users making requests at random times and monitor the app for correctness and to see how it handles the load. Also for the latency we can either input timing code for the simulated users, or I can just run a user of my own while the app is under a heavy load and see if there is a noticeable difference. 
  • I’m on schedule. I did all of the things I said I wanted to do last week. 
  • Next week I plan to make tests like the ones described above and also help out in other areas of the project. The most important thing is having the backend know what song is playing when so that is probably where I’ll go first.
Posted on

Luke’s Status Report for 3/30/24

As discussed last week, the primary goal for this week was to integrate all of our subsystems. With that being said, I focused on the intra-pi communication which bridges the gap between the core logic and recommendation code I’ve written over the past couple of weeks. This involved some more work with websockets, but after a little bit we were able to connect the two pis using a unified data transmission object (MessageRequest and MessageReponse classes).

This process involved a lot of small subtle debugging. For example, I caught a few different minor bugs with my recommendation code, particularly some edge cases with the session recs. We had some issues if there were fewer than 3 songs that have been queued, and also if the songs all had 0 likes for the session. that would cause the weighted centroid computation to be incorrect so I had to fix that.

After we fixed some minor bugs, we did some fun testing. I had a bunch of my teammates from soccer go to our website and concurrently queue songs and test around with the recommendation functionality. It was really cool to see everything working well. This also exposed some minor bugs that I worked on.

I would say that we are on schedule, but there is still a lot to be done. The integration process revealed many robustness challenges we are going to have to address, mainly with the management of the queue.

Next week I am going to work on the integration between our backend managed queue and the actual Spotify queue. this is going to be a little tough because we need to implement some timing of the songs by querying the duration of each song. This will be my main priority, and if I have any extra time I will work more on the semantic matching accuracy.

Posted on

Team Status Report for 3/30/24

  • The most significant risk that could jeopardize this project is still the lighting system. Since our lighting fixture displayed some behavior before starting to fail consistently, we need to test some other configurations, or consider obtaining a second set of lighting fixtures. We plan to mitigate this risk by borrowing equipment from the IDeATe lab: first, we will borrow an ENTTEC DMX to USB converter, which a previous ECE Capstone team (Group D4) told us they used. If this does not work we will borrow their DMX lights as well, to test whether it is an issue with the fixture. Our second concern is managing real-world song timings, as the Spotify Web API does not issue us the capability to wait for a callback when a song finishes playing, although it does return the exact duration time of any track. We intend to mitigate this risk by experimenting with different clocks and internal song tracking. The final potential risk is keeping track of which Users have recently engaged with the app, in order to determine the majority count of active users. We plan to tackle this issue with keep-alive signal communication with our web app clients to check if Users have engaged with our service within the last timeout epoch. 
  • No changes have been made to the system diagram or the project schedule.