Team C2: LearNet

This week, I primarily worked on upgrading our implementation of the video server to make it more suitable for our testing environment. In the previous week, when I changed the server so that it started retrieving the video from an S3 bucket instead of local files, it lost the functionality of being able to retrieve the video in chunks instead of retrieving the entire video. I attempted to solve this problem by querying the video size from the S3 bucket, but it seemed like the video server lacked the permissions to properly query that information. Therefore, since the scalability of being able to query the video size isn’t important to the functionality of the video server in the context of the testing environment, I decided to instead hardcode the video size of the video into the code.

Another way I upgraded the application was by adding a catalog so that users can select multiple videos to play instead of just one. This means that the load on the load balancer can now be greatly varied by requesting videos of different types. A 360p video that is mostly static would not need many requests to fully load, whereas a 1080p video that features an action scene would send many requests to the server. This will make the testing environment much more robust and realistic.

Next week, after the testing environment has been set up, I plan on helping with implementing our own custom load balancing algorithm as well as taking the data that is collected from our user scripts and compiling it in a coherent manner.

This week, we mainly continued to develop and finalize our testing environment for the custom load balancer. First, we made our application more robust and appropriate for load balancing testing by implementing the delivery of the video in chunks. Furthermore, we also added a catalog of videos for users to choose from instead of just one so that the load to the server would be more varied.

As for user testing, we developed user scripts that continuously send video requests to the load balancer and logs metrics such as user response time and bit rate. We plan on deploying these scripts to AWS instances when running tests for our server.

Finally, we managed to obtain the two metrics from the video servers we intend on using, namely CPU utilization and server response time. With the CPU utilization metric, we implemented the load balancing algorithm that forwards requests to the server with the lowest CPU utilization, and next week, we plan on using server response time for our own custom load balancing algorithms.

This week I spent my time working on user simulation scripts. The goal is to create python scripts that will mimic user behaviour on our video server so that we can test it’s performance as well as benchmark the performance of our load balancer.

I spent time learning about python requests library and created a script that can send requests to the RandomLB and RoundRobinLB instances that are deployed. After the request has been sent, I print the response object received from the server and isolate the HTML information.

Going forward I know need to calculate metrics such as response time as well as buffer length and bitrate. Even though these user end metrics are not going to be used for optimising our load balancer, they are crucial in testing the performance of both our video server and load balancer. Since our video server loads video files chunk by chunk, I will have to calculate the response time every second or so and then divide the total response time by the total size of the video to determine the average bit rate for that particular video file.

This week I worked on the processor utilization percent based traditional load balancer as well as response time calculation for our custom load balancer. We originally planned to get processor utilization metrics from AWS CloudWatch, an interactive monitoring tool for AWS EC2 instances. Unfortunately, I had issues trying to collect data in increments shorter than 5 minutes and such a delay would be excessive for load balancing decisions that happen in seconds. My current approach is to calculate processor utilization directly in our video server code and send it as an additional request header for the LB to parse. I may have to adjust how often the parsing happens based on how much load this creates.

On the response time end, we had to significantly change our plan for how to collect response time due to our new understanding that expecting this information from users, simulated or otherwise, is infeasible. Instead we plan to now get the response time from when the load balancer chooses a server to request to when said server sends its corresponding response. Specifically, the load balancer would send the start time as a request header, and the video server would include response time as a response header.

I also documented a more refined initial approach for our custom load balancer. Because volume of datapoints is not a good metric for choosing a server (old data is far less useful in load balancing than in traditional multi-arm reinforcement learning), I find that epsilon-greedy is a better starting point algorithm than UCB1. Furthermore, to ensure recent data is most valued and to reduce memory concerns, the algorithm will remove datapoints prior to a certain point (e.g. only most recent 20 response times are kept for decisions). I also plan to limit the algorithm from choosing one of the last k-chosen servers again. This accounts for delays in number adjustment while also promoting dynamism in server exploration and monitoring.

This week we met regularly in order to share personal updates and repository code for our working video stream with load balancing. Specifically, a new repository was made for streaming from the Amazon S3 database and the details were communicated for the necessary script additions for EC2 deployment. A similar process was done later in the week for the random and round robin proxy servers that load balanced between the streaming deployment group. We also logged several different repositories and their respective purposes; the purposes were categorized as experimental, locally working, or deployment ready.

We also managed to resolve a lengthy issue with the initial deployment of the video streaming servers. We intended to use AWS CodeDeploy to automate group deployment of many identical servers. However, this was an unfamiliar process for us and presented new problems. The issues were three-fold consisting of missing security permissions for our intended port usage, ending processes improperly managing residual files, and inaccurate file navigation in the virtual machine. We now know to keep an eye on these aspects for future deployment and benefited from a smooth deployment of the load balancing proxies. Solving major issues of individual assignments as a group worked well so we plan to continue that practice going forward.

Last week I was in progress of adjusting our video application to retrieve video from separate MongoDB database servers. However, we discussed and instead decided to replace that with a single auto-scaling Amazon S3 bucket to serve our database needs. This will let us more easily upload new videos and verify and database integration errors on AWS consoles. This past week I completed the app adjustment to retrieve video from S3. Unfortunately, the AWS node.js function suite did not fully follow the structure of our previous video chunk retrieval framework. I cut out that framework so that we could have working deployment but unfortunately the video currently fully preloads and does not support scrubbing. I will be fixing that issue over the following week.

I also did some research on node-http-proxy as a module to enable our simple and lightweight load balancers, specifically the random choice and round robin (set sequential choice). Both of these load balancers can utilize high-level proxy functions for a fully pass-through request-response architecture in which the contents of the stream do not need to be read. I completed locally working versions of these load balancers that could each proxy for 10 different AWS front-end servers described in the first paragraph. I’m currently working on extracting front-end response times for our first reinforcement learning load balancers. I plan to build a latancy-based framework around this response time to complete our first reinforement learning algorithm by next week.

This week, I primarily worked on deploying both the video server and load balancer on Amazon ec2 instances using AWS’s CodeDeploy module. The process was quite different from how I had previously deployed code on virtual machines, involving setting up configuration files and scripts in order to deploy code across multiple servers. However, after gaining some familiarity with the module with the first code deployment, I will now be able to use it to mass deploy code across multiple instances which will be helpful further on in the project as we update our implementation and also scale up operations to accommodate more load.

I also created an AWS S3 bucket to fetch our video files from which is already being used in our current implementation of the video server. Using the S3 bucket does currently remove scrubbing and returning the video in chunks from our application which could be a potential problem when load balancing incoming requests. Therefore, I am currently working on returning this functionality to the video servers while using S3.

This coming week, I will be working on obtaining the metrics that will be used for our custom load balancing algorithm: mainly CPU utilization and response time so we can also set up the monitor node. I may try to use Amazon’s own load balancers to gain insight into how these metrics can be obtained.

This week, we primarily continued our respective programming roles for the application. We successfully implemented a simple video streaming web application that works while locally deployed as well as began deployment of this application on AWS instances. While discussing our architecture, we decided to combine our backend and database server nodes due to the simplicity of our application. This will reduce some unnecessary complexity in our implementation as well as deployment costs.

Over the next week, we will begin implementing the backend/database nodes with MongoDB, supporting our front end with websockets, implementing our load balancer to fit into the server architecture, and simulating users using scripts running on additional instances

This week, with the basic front-end application implementation from Nakul, I was able to deploy the code and get it working on a few EC2 instances on AWS in our server architecture. I tested the functionality of the code that was deployed using my own browser, but in the coming weeks, I will work on implementing scripts in order to simulate users instead of having to manually test connections. Additionally, aside from the one video that the front-end video application currently uses, I also plan on finding additional videos from Youtube that our application will service in order to create a larger variety in load.

Another thing that was brought up to me by Mitul was condensing the back end and database server tiers into one comprehensive database access tier. After this suggestion, I looked into how to combine these instances in AWS and rework our architecture a little to accommodate this change.

In the coming week, I will continue to deploy any changes in code with our application as well as implement a monitor node to keep track of relevant metrics fro our project.