Posted on

Thomas’ Status Report for 4/27/24

Thomas Lee

  • This week I started by writing and testing the different lighting schemes, and set up the lighting Maven program to accept the Acousticness, Danceability, Valence, Energy, and Tempo data from the Spotify Web API relayed through the main RPi core and modulate the behavior and rhythm of the lights accordingly. I integrated the new set of lighting fixtures and signal transmission hardware, and made sure they worked as intended and exactly like the set of lights we were using before. I also tested the functionality for daisy-chaining multiple lighting fixtures up in a row, all controlled by the same RPi program. Next I worked on making the communication work with the lighting program, such that on input on the web app the lights would rotate their colors, and developed a different easier to understand lighting script for this. I also helped work on the final presentation slides, and helped create content for Matt to discuss in the presentation.
  • Testing:
    Queue Latency: 102 ms (20 trials)
    Recommendation Generation Latency: 6.2 s (20 trials)
    Queue Capacity 100 Songs: passed (memory usage & latency acceptable)
    User Stress Test: 200 Users passed (memory usage latency acceptable)
    Concurrent User Stress Test: ran out of memory when trying 20+ users all adding songs. This may be due to inefficient usage of data structures and redundant memory copying. We plan on investigating this further in the lab.
    UPDATE:  RESOLVED we went into the lab and fixed the problem. The memory usage with for 5 simultaneous requests from different users each second over 30 iterations each consumes less than 2% of available memory.
  • We are approximately on schedule.
  • Next week I hope to continue working on integration, and help make the posters and testing the final demo.
Posted on

Luke’s Status Report for 04/27/24

This week, as discussed I focused mainly on tying up some loose ends in my parts of the system and also switched gears to focus on validation testing. Also, I spent a lot of time working on the final presentation that we had this week.

I primarily worked on robustness improvements, such as handling failure cases. This includes being able to smoothly handle the situation where a user input is not found in Spotify resources (ie. the semantic match fails), in which we now handle by removing the item from the queue and having the use try again. This makes the overall user experience better because while our semantic match handles legitimate input very accurately, there is always the chance the user tries to queue something that isn’t even a song or something along those lines. In addition to that, I helped with the rest of the system integration with Matt and Tommy.

In terms of unit and validation tests, I focused on a few things. For one, we needed an accurate metric for semantic matching accuracy, which I tested on Sunday prior to the presentation. This included the following results on 30 test cases, each carefully structured to test a common situation arising from the use or misuse of the system by users.

In addition to this, I worked on a survey to ask users how our system recommendations compare to Spotify’s traditional recommendations. I am still collecting the results but will finish that process tomorrow.

In terms of schedule, we are approaching the final few days of the project where we will continue to collect results from our validation tests and will put the finishing touches on the robustness of the system.

Posted on

Team Status Report for 4/27/24

  • The biggest risk of this project is not having enough memory to handle 150 concurrent users & 100+ songs on the queue. This is being managed by adding a database to the project. If sending the queue over the web socket is still too much then we will have the JavaScript call the database directly. If all else fails then we will host the web app on a cloud provider with a lot of memory or come up with a solution to compress our data. We have until Friday to make these changes and will all put our heads together starting tomorrow so we are not too worried about finding a solution.
  • We have not made the change yet, but as talked about above we plan on adding a database to combat our memory issue. This incurs a monetary cost for the database storage, makes our project a little more complex, and adds a possibility for attacks on the database.  This cost will be mitigated going forward by only getting enough storage that is necessary.
  • Updated schedule:
    https://docs.google.com/spreadsheets/d/1nq6JVOsJ9P9bE2MCSX8J-95vvIR2aPtEa-vl6Zx3WIE/edit#gid=0

    Sunday edit:
    We figured out our issue. Our queue can now hold a lot of users (150) with concurrent queuing of a song each which meets our requirement

Posted on

Matt’s Status report for 4/27/24

  • This week I first helped make our final presentation slides. I am the one who presented this week and wanted to give the best presentation possible, so I also spent a bunch of time studying the slides and practicing both by myself and in front of others.  For project work: I first helped Thomas set up and figure out how to use the new lights we ordered. Once we were confident in our ability to control them I set up the communication between the raspberry Pi and the lights. Since the lights need to operate on a while true, receiving communication from the first pi makes the light commands slower, so we split up the listening ports between the song recommendations and the lights so the lights will only get sent a message about once every three minutes. We will be running these programs concurrently on the second raspberry pi. I next wanted to fix the user keep alive. I am not really sure what I was thinking last week in my solution. Having a user that switches off the screen automatically deemed inactive is not very smart. I scrapped all of my old solutions and made this one: The server will set a time once it starts running and on every user connection they will receive this time. Also the backend will check every 30 minutes (could be changed) for all the users that are inactive. The way to tell the server that you are active is if you have had a screen interaction within the 30 minute server block. I do this by checking (on every user action) if the last action is earlier than the last inactive check by the server. If so then I send a message to the server saying that the user is inactive. Every 30 minutes the server sets everyone to inactive after removing the likes for all the new inactive users from the last 30 minutes. I then ran the concurrent user tests which I will talk about in the testing section
  • Testing:
    Queue and reccomendation latency. 102 ms (20 trials) for Queue and 6.2 seconds (20 trials) for reccomendation
    Queue capactity of 100 songs: Still had memory left over and latency was fine
    User stress test: I made a javaScript test to make 200 users (without songs) and it had memory over and latency was fine
    Concurrent user stress test: This one was an issue. This is combining a lot of users, song queuing and concurrency. I ran out of memory fairly quickly which was a shock (a little over 20 users).  I am still trying to understand exactly what is the big deal with the memory, I think its between the dataStructures kept locally and passing the whole queue into the socket to send to every single user. I plan on now storing the data in a database and retesting.

    Sunday edit: Concurrent user stress test works.

  • I did not anticipate this memory issue (for the Concurrent user stress test) and thought I would be done with this part of the project by now so I am behind schedule.  I plan to get back on schedule by finishing the remaining parts of the project as soon as possible.
  • Next week I plan on fixing this memory issue as well as any other issue that pops up in our system, do all of the final reports, and then have a great demo and be done with the semester [and cmu 🙂 ].
Posted on

Luke’s Status Report for 04/20/24

As said last report, my main focus this week was to work on the integration of our backend managed queue with the actual Spotify queue. This consisted of writing a queue request scheduler that uses data about the length of the current songs playing to ensure the state of our queue is consistent with regards to the Spotify queue in terms of timing. I then used futures and the java concurrent safe scheduler class to implement this in an efficient way. With this in place, all of our veto and ordering of the queue functionalities work well with no problems. This was one of the lat big steps we needed to take so it is great we were able to get it done so efficiently.

Also, I worked more on various other tasks this week. I worked on some verification tests for my subsystems, primarily involving the accuracy of the semantic matching algorithm. In addition to this, Ice worked more on the authorization driver by replacing our current method of using a web driver with using refresh tokens instead, which makes our lives a lot easier snd limits the communications between the pis.

Further, I worked on the endless queue functionality. This basically just means, that when it is time for the scheduler to queue whichever song is at the top of the queue, if there are currently no songs in the queue, then the system will just queue a song that is a recommendation based on the previous song that played. However, this is slightly inefficient because it takes a couple of seconds to generate the recommendation and communicate between the pis, so we implemented this function as a prefetching process. Basically, the system will prefetch a recommendation if the queue is empty, and then when it is time to actually queue it, if the queue is still empty we use the rec and prefetch another. This was great progress and now makes our system fully functional as originally planned .

We are on schedule. This week was massively important in terms of all of the progress we made, so we just need to continue strong for these last two weeks. Next week I will focus mainly on small robustness improvements and verification and validation testing

What new tools or new knowledge did you find it necessary to learn to be able to accomplish these tasks? What learning strategies did you use to acquire this new knowledge?

I definitely learned a lot about new classes and packages that Java supports, such as the concurrent library, and some of the ML related classes that we use for the semantic matching. My core learning strategies involved reading lots of code documentation and examples – this helped me understand the use cases of these tools and then figure out how to incorporate them into our own project.

Posted on

Team Status Report for 4/20/24

  • One of the most significant risks that could jeopardize this project is not having access to the lighting fixtures and accompanying devices during these last few weeks for experimentation, testing, and demonstrations. This risk is being mitigated by ordering the same parts for ourselves so that we will have full control over a proper set of lighting equipment.
  • A secondary risk that could jeopardize this project is having trouble integrating all the modules of the system together on the Raspberry Pi’s. While these processors are indeed capable of multithreading, to operate the core queue manager or the recommendation service simultaneously with continuous lighting control signal generation, this may slow down the subsystems perceptively. This risk is being mitigated by having a third Raspberry Pi on reserve just in case it is necessary to dedicate an entire Raspberry Pi computer to each major subsystem. 
  • In order to mitigate the risk of not having enough time to run as many or as comprehensive of tests as necessary, we started building our testing systems early last week.
  • No changes have been made to the system diagram or the project schedule.
Posted on

Thomas’ Status Report for 4/20/24

Thomas Lee

  • I spent this week trying to understand the new lighting fixtures, and creating the different lighting schemes for these IDeATe Lab lights instead of our original Furious Five RG lights. Besides locating and procuring the lights, the ENTTEC DMX to USB converter, and the proper cables and DMX terminator plug, I ordered a set of all these components through the Purchase Request form to ensure that we will have an operating set these coming weeks. I also deciphered the cryptic channel & intensity mappings for the new SlimPAR PRO Q USB lighting fixtures, as the channel IDs sensitized by the Java port of DmxPy did not align exactly with the values provided in the User Manual for the lights. Using this mapping I was able to create the first two lighting schemes we would be using, and successfully tested them by operating the lights in real time with the Java processes administering the patterns. The video below shows one lighting test sample, and demonstrates that we are finally generating reliable DMX signals and controlling the lighting at a fine-grained resolution using a lighting microservice operating right out of the recommender module (which was previously demonstrated to work correctly on the second RPi):
    https://drive.google.com/file/d/1pNcK16PDUi-hqoaK9zqlAS_2xBKROgXs/view?usp=sharing
  • My progress is on schedule, however I would have liked to help start module testing a little earlier.
  • In the next week I hope to finish the rest of the lighting models & integrate the light system, collect more survey & testing data, and polish the whole system while checking complete end-to-end functionality.
    ==========================================================
  • As I have worked on this project, there are a few new tools and pieces of engineering knowledge that I have acquired as necessary for different parts of my project. For example, I learned how to use a Raspberry Pi, how to use web sockets under a Springboot framework for a live service web app, and how to control hardware lighting fixtures without on-board processors using a computer program by generating & transmitting DMX signals. In order to learn these skills, I had to use a range of different learning strategies, my primary methods being online research, following public tutorials, and reading documentation from hardware suppliers’ websites. I also consulted previous ECE Capstone groups for their expertise in utilizing certain technologies (such as controlling a DMX lighting fixture using a computer program), or asking my teammates (to learn more advanced version control tactics on Github) learning how to do different things directly from a real person. I also applied the programming strategies I learned from classes at Carnegie Mellon, such as Web Apps & Distributed Systems.
Posted on

Matt’s Status Report for 4/20/24

  • I started by making tests for heavy user load. I made it using chrome recordings which you can then download into a JavaScript file. I think I will run a few of those scripts at the same time to simulate concurrency. I plan on actually running the test next week. I also made much needed front end changes. I made it so that the app is responsive to different sized screens, does not have the weird zoom in that it used to have, made sure each song element is the same size, and also added a feature that scrolls through the text if it is overflowing (video included). I also ran into some issues with the user timeout, it was working fine on chrome and I knew safari would have some differences but I did not account for when users leave the page that the javaScript code would stop completely. Fortunately there are events that handle those cases so I added those to the code. The rest of my week was thinking about solutions to the problem of: if users refresh the page it will make them log back in as a new user. My biggest issue was differentiating a refresh and a page close on purpose, there is no 100% reliable way (to my knowledge/research) to do this since the browser treats them very similarly. I then just decided to have users stay logged in all the time. After having a few issues with reconnecting the sockets I finally got it to work, but then after some light testing, the app crashed, I think I was sending too many requests to the server but I will look into it more in the future. It could be a different thing implemented recently. Right now I think I have an acceptable solution. There is no reason for users to refresh the page since the WebSockets make it so the screen refreshes after every single change, but I realize people will do it anyways. When this happens on some browsers (like chrome) it will display a warning that they will have to re-login but on others (like safari) it will not display anything (I will also look into this a little more). If the user then clicks they want to refresh it will log them out and they will have to log back in and I will discard their old user votes as if the user was gone for an extended period of time. While thinking about this issue I also worked with Luke and Thomas to integrate our system together.

    Link to frontend:  https://share.icloud.com/photos/0182jqTvykRs-CBnIoQ5J29qQ
    I think the buttons are slow to load since I’m on a bad network right now. I did not notice this on other networks but I will double check when I get back to campus.

 

  • I am on schedule. I achieved what I wanted to do the past two weeks.
  • Next week I plan on helping integrate the lights to our system aswell as testing the app and thinking of that refresh problem more, maybe asking friends for their opinion.

 

  • New tools or knowledge:

There have been a lot of new technologies and knowledge I have learned to do this project. Websockets, everything Raspberry Pi related, Maven projects, Github commands, Testing methods, and probably more I can’t think of right now.

The first thing I always looked for if I was doing something I never did before was documentation / examples of other people doing it (a video is a plus). After struggling with getting websockets to work from java to JavaScript and debugging to no success I found a video of someone using springboot to do it along with their repo. So I was able to learn enough about the process by following along the video and then just messing around with the code by myself to see what happens when I change certain things.  I also was lucky enough to find a video for getting a program to run on raspberry pi from the start but then when I tried it myself it did not work. So then I went to online forums like stack overflow and tried a lot of things until my solution worked. It is safe to say that one of the most important things I needed for these new tasks was patience and persistence. Github commands are very easy to lookup, aswell as Maven commands to run certain files and to solve errors. It is also very helpful to talk with teammates when struggling with a problem as they can provide a different view / a solution you were not thinking of. Our team is good at helping each other come to solutions to their problems.

Posted on

Luke’s Status Report for 04/06/24

This week, I took a bit of a detour from my original plans and worked more on the authorization model. Basically, to access Spotify’s resources you need to get a proper access token, but this token expires after an hour. So if we want to be able to host larger events then we need to be able to automatically regenerate access tokens for use on both the core and recommender pis. This involved some more automation with selenium, but the main task was querying and sending the access tokens across pis. This involved some more intra-pi communication code and integration into the rest of the subsystems. Overall, this process went well and we were able to test it, which was successful. Now we can operate music mirror for an indefinite period of time without having to worry about losing authorization to Spotify’s resources.

With this in mind, I did start to work on the timing systems that we will use on the core pi by introducing timestamp timers. This infrastructure will be used with the Spotify queue integration which I will work on more this upcoming week.

Verification: I started to write a comprehensive test base for the semantic match. One of the big requirements for accuracy was being able to translate user input into accessible Spotify resources, and I’ve developed multiple solutions for this but we need to be able to test efficiently. So, I created a bunch of test cases for common situations we’ll run into, and began to compare the 1-gram frequency map with the embedding transformer solutions to see which is more accurate.

Validation: In terms of the overall system, we started testing more live concurrent users and also did some robustness checks, such as spamming session req requests to see how the system would handle it. To our satisfaction, it was actually quite robust to these failures but we’d still like to incorporate some sort of limit on the requests a single user can make to ensure nobody monopolizes the queue

Overall, we are still on schedule but need to keep working hard for these last few weeks. I plan to work on the queue integration further in the upcoming days before carnival.

Posted on

Team Status Report for 4/6/24

  • We borrowed the lights later in the week this week and are still trying to figure them out and will work on the timings this week. We did figure out the keep-alive functionality though.  So for the first two our risks are still the same as they were last week: The most significant risk that could jeopardize this project is still the lighting system. Since our lighting fixture displayed some behavior before starting to fail consistently, we need to test some other configurations, or consider obtaining a second set of lighting fixtures. We plan to mitigate this risk by borrowing equipment from the IDeATe lab: first, we will borrow an ENTTEC DMX to USB converter, which a previous ECE Capstone team (Group D4) told us they used. If this does not work we will borrow their DMX lights as well, to test whether it is an issue with the fixture. Our second concern is managing real-world song timings, as the Spotify Web API does not issue us the capability to wait for a callback when a song finishes playing, although it does return the exact duration time of any track. We intend to mitigate this risk by experimenting with different clocks and internal song tracking.
  • No changes have been made to the system diagram or the project schedule.