Category: Sid’s Status Reports

This week, I gathered and labeled more training data for our model and helped conduct further testing. Since our model’s classification varies in lighting, I helped gather more images in dim lighting to account for this difference. Right now, our test accuracy is just below 98% (but above 97%), so we are very satisfied with the accuracy results. One of our initial goals to account for lighting changes was to have a custom enclosure. I spent a few hours serving as an extra pair of hands for Ethan to build an enclosure for our card shoe, but our enclosure impaired our camera captures. As a result, we might not use the actual enclosure in our final product.

I spent some time working on our final poster and writing a script for my part of the final video. Finally, I also added an “edit” widget to allow users to make edits to the web application if a card was classified incorrectly. I am on schedule, but the next 5 days will be very hectic to get all our required deliverables done (poster, video, demo, and report). Tomorrow will be dedicated to finishing our poster and video. Then, the majority of next week will consist of writing our final report.

This week was dedicated to slack time and integration. Since our system has slightly changed due to the addition of LEDs, the images slightly changed in appearance. Hence, I spent most of this week gathering more training data with Jeremy and labeling the images to constitute training, validation, and testing data for our model. Since our previous model was able to achieve high validation accuracies with the previous images (without LEDs), not many changes needed to be applied to our machine learning code.  These were my main tasks for this week (gathering more data and training our ML model), so I am satisfied with my progress. Our model hits our overall latency and accuracy requirements. One of our user requirements is having the web app update within 2 seconds of a card being pulled from the trigger, and our web app updates (on average) 0.18 seconds after the trigger. Our accuracy requirement was 98%, and our final test accuracy is 98.1%. For the remainder of the weekend, I will help gather more metrics and performance results for our final presentation. I am definitely on schedule. Next week will be dedicated to further testing and making our final video, poster, and report. If we fail to satisfy our desired requirements during future testing, then I will have to help make modifications to our existing system. If time permits, I might also make the web app even more robust by allowing users to edit card classifications on the web app (for example, if a card was classified incorrectly). This is not a necessary feature, but it would add value to the overall functionality of our system.

 

Since the last status report on 4/10, I made some progress with the web app by adding the necessary logic to Blackjack in case there are multiple winners. In addition, I implemented error handling in case of empty and faulty user input from web app users. An example of faulty input is if a user inputs a negative number of players into the web app. Finally, I implemented authentication in our web app so that only verified users can modify the state of the game through the text forms and buttons. I was able to showcase these updates during the interim demo, but here is a snapshot below for reference.

 

I have also been able to make progress with the machine learning component of our project. Jeremy and I have spent the last few weeks performing the following tasks: collecting image data, labeling the data, writing and modifying Python code to train and test a convolutional neural network (through interfacing with the PyTorch library), and experimenting with various hyperparameters (number of layers, size of kernel, etc). Below are some of my results from my experimentation, but as we get more data, these results are subject to change. K refers to the size of the kernel.

 

1. Num convolution layers = 4 and K = 3

• Epoch 56: 93.45% – best validation accuracy
• Suit accuracy: 97.86%
• Rank accuracy: 93.57%

2. Num convolution layers = 5 and K = 3

• Epoch 58: 86.90% – best validation accuracy
• Suit accuracy: 98.66%
• Rank accuracy: 87.67%

3. Num convolution layers = 4 and K = 5

• Epoch 48: 95.83%- best validation accuracy
• Suit accuracy: 98.93%
• Rank accuracy: 95.71%

4. Num convolution layers = 4 and K = 7

• Epoch 56: 83.63% – best validation accuracy
• Suit accuracy: 96.78%
• Rank accuracy: 84.99%

 

Overall, I am on schedule. My main future tasks involve gathering more training data and experimenting with our hyperparameters to reach our accuracy requirement.

This week, I have been working with Jeremy on developing ML code and testing our models. Since I finished writing code to train and test a SVM model (with an RBF kernel), Jeremy was able to train and test this model on a limited dataset. We do not have enough training data, so we achieved very low validation accuracies. The fact that our training accuracy was high further proved that our training data is not comprehensive enough. Since my quarantine is ending soon and my symptoms have gotten better, I plan to go into the lab in person and obtain more card image data to use for training. This is one of my primary goals in the coming week, as we need to achieve higher classification accuracy for whichever model we choose (SVM vs neural network).

 

I have also started writing Python code on configuring a neural network. This has required interfacing with the PyTorch library. I plan to have the code finished by next week as per schedule. However, training the neural network will require even more training data than the SVM since neural networks don’t make modeling assumptions about the underlying data. As a result, finishing the neural network implementation is not as immediately pressing as obtaining training data.

 

I have also achieved considerable progress in making the web application more robust, intuitive, and accurate. To make it more accurate, I had to fix several logic bugs to ensure the card dealing order remained correct and the proper winner is determined at the end of Blackjack and War. I also added the following features: allowing the user to input player names, showcasing which player’s turn it is, highlighting the losers and winners of Blackjack and War, and implementing a sticky left column in our player table so that it remains constant when the user scrolls horizontally. This last feature is especially important in War, as there are up to 52 cards in a player’s hand, so users would need to scroll horizontally to view all the player’s cards. Lastly, I added some CSS styling to make the UI more elegant. In terms of UI styling, there is not much I need to do for the web app. With regard to next steps, I recently realized I need to implement the necessary logic for Blackjack in case there are multiple winners. In addition, I need to be able to handle empty and faulty user input from web app users (ex: inputting a negative number of players). This error handling is another one of my minor goals for the coming week. The feature I want to prioritize for next week, however, is implementing authentication in our web application to ensure only verified users can make changes to the web app. In a real-life professional poker setting, only casino officials and tournament officials should be able to input information into the web app (not audience members), so this level of security is necessary.

 

Overall, I am on schedule with my tasks. Below is an updated look at the web app (the password text field does not have any underlying logic yet, but I will work on that this week).

 

I was able to accomplish both of my desired goals this week. The first goal was writing the necessary code to visualize the card games War and Blackjack. This required writing some backend Python game logic and some JavaScript/HTML to convey the hands of each player. Blackjack does require user input to click on which player’s turn it is (as it isn’t predetermined how many cards a player will want to draw before stopping). This can be seen in the below picture, where there are buttons saying “This Player’s Turn”. Users can specify when they are done drawing cards by clicking on the appropriate button to indicate whose turn it is.

In addition, I wrote Python code to communicate with the web app via POST requests (this code will be stored on the Jetson Nano), and I wrote Python code to train and test a SVM model with an RBF kernel. This choice of model and hyperparameter should achieve our desired accuracy given online research. As our first iteration of preprocessed data becomes available in the coming days, I will input this data as training/validation data and analyze the model’s performance. In addition, I will start the process of writing code to train and test a fully connected neural network through PyTorch. These are my main goals in the coming week. If I have time, I will also try to make the web app’s UI more intuitive to create a better and more complete user experience (highlight which player’s turn it is, allow user to specify players’ names, add button to indicate when game is over, styling, etc). In addition, based on conversations with Professor Fedder last week, our web application could utilize some sort of security/authentication to ensure only verified users can submit requests on the website. This would preserve the integrity of information on the web app, and so this is another action item to possibly be completed. I’m adding these two tasks to my schedule, and so below is an updated look at my schedule. These action items for the web app do not bear much significance to the rest of the team, so the overall team schedule will not change as a result of these updates.

 

I am currently on schedule, but these next few weeks will be very tough. I recently contracted COVID and am experiencing mental/physical symptoms. Hence, my ability to focus and do work has deteriorated. I have been in contact with Professor Fedder, Ryan, and the rest of my team to ensure they are aware of my current health status. As of now, I still plan on completing all my work on schedule.

The past two weeks have been very productive. I was able to deploy my flask app on an AWS EC2 Ubuntu Server.  In addition to installing the necessary python packages on the server, I had to configure Remote Sync (Rsync) between the server and my laptop to transfer the necessary code files. This entailed enabling the Windows subsystem for Linux, starting the SSH Open Server and installing the OpenSSH client, and generating appropriate key pairs for authentication.

I was also able to test the web application’s latency by sending RESTful API requests from the Jetson Nano to the web app hosted on the AWS server. Unfortunately, I ran into a major problem, as the web app was taking six to eight seconds to respond to the POST requests. One of our user requirements is being able to update the web app within two seconds of a card being withdrawn from the card shoe. Hence, I spent much of this week optimizing the web app. The first modification I made was establishing a long-term connection to the MongoDB instance instead of making a new connection to the database every time an HTTP request was received. This significantly sped up the web app. However, there was another issue to be addressed: the web app operated by refreshing several times a second to fetch new data. This constant refreshing created an inconvenient user experience, so I migrated much of my Python logic to JavaScript to avoid refreshing. I wrote a JavaScript function to continually run and fetch new data without causing the whole browser to refresh. This further lowered latency by reducing the amount of data received from the server, and this also created a more seamless user experience. Now, the web app updates instantaneously, as seen below.

I’ve also started researching different models I plan to utilize for image classification training and evaluation. The first model I plan to experiment around with are SVMs with Gaussian kernels. Based on my research with similar image data, these models should achieve the desired classification accuracy of greater than 98%. Our team initially planned to start training next week, but due to delays with hardware shipping, training won’t be able to occur until the week after. That said, I still plan on writing Python code to work with existing ML packages, like sci-kit and PyTorch, and configure their respective models (SVMs and neural networks). Hence, I won’t be behind schedule, as the training/testing process will go by very quickly since the code will all be written. This is one of my main goals in the coming week. In addition, even though the web app has all necessary components (I recently added an input field to allow the user to specify the number of players), I will add logic to allow the web app to visualize multiple card games (instead of just poker). This is my other goal for the coming week.

I’ve spent the first two days of this week (Sunday and Monday) finishing up the design slides and rehearsing my presentation. After presenting on Monday, I spent the remainder of the week finishing making the web app dynamic/interactive to HTTP requests. Before, the web app was able to accept POST requests and update the MongoDB database. I’ve now cleaned up the code so whenever a user sends a POST request (containing the suit/rank of a player’s card), it accurately updates the database for the proper player (ex: player 1 vs player 2).  In addition, I connected the Python Flask backend to the HTML front-end code, so the backend software is able to pass the suit/rank information to the frontend software for rendering. Then, when a user visits the web page, it showcases this updated information. I’ve also finished implementing card images for the web app, so instead of displaying “2 Hearts”, the web app is able to showcase an actual card image corresponding to the suit/rank. Hence, the web app is now completely stateful and showcases consistent information for all visitors. This was one of my main goals this week, so my progress is on schedule. Today, I am meeting with Ethan and Jeremy to pick up our shipped hardware (Jetson Nano) and I am currently trying to set up the Nano. My goals for the next week are to finish setting up the Nano and to migrate my web app to AWS. I will also spend some more time researching machine learning algorithms that I plan to experiment around with for image classification.

This week, I created a MongoDB cluster to act as a centralized database for our card data. I realized that without a centralized database, there may be inconsistent information presented to users about the current state of the game. After experimenting with several databases, such as SQLite, I realized MongoDB would be our best option given its flexible, unstructured schema and reliability. After writing some Python code and working with the Pymongo package, I was able to connect my web app to our database and make queries. In addition, my web app now accepts POST requests. As a result, other machines (in our project, this will be our Jetson Nano) can send HTTP requests to my web app to update the data stored in our cluster, which also updates the visual display of the web app. I’ve also spent time updating our design presentation slides by refining our use case, metrics, risks/uncertainties, and software stack. I plan to spend tomorrow practicing my presentation for next week.

 

I am currently on schedule. Next week, I plan to finish making the web app dynamic/interactive so that I can start migrating it to AWS.

I spent the earlier part of this week viewing our classmates’ proposal presentations and learning from them. They all had unique approaches to combining software, hardware, and signals to solve a user problem. I look forward to learning more about their progress in the future. Towards the latter part of this week, I worked on designing and developing a basic web application. Right now, I have written Python and HTML code. The Python code utilizes the Flask framework as well as other libraries to interface with the front-end code. The web app is hosted on my local machine, and so I plan to spend the remaining week migrating this application to the cloud. In addition, there are still many logical elements that need to be added, so I plan to utilize JavaScript to accomplish that. These are my deliverables I hope to complete in the next week.

 

I’ve also been meeting with Ethan and Jeremy to stay in sync with our progress and start working on our Design Presentation. I plan to contribute to my slides in the coming week and continue meeting with them to ensure our components are compatible. So far, my progress is on schedule.

 

This week, I spent most of my time trying to refine our project’s scope to ensure I would be contributing a reasonable amount. After discussing internally as a team on 2/15, we initially revisited the idea of using RFIDs. Because I have taken 18330, I considered implementing cryptography between the RFID tags and readers. I spent most of 2/15 and 2/16 doing research on the feasibility and effectiveness of encrypting communication between tags and readers. After our meeting on 2/17, we decided as a group to focus more on image processing and using the camera for computer vision. Hence, I realized my best contribution would be in training our ML model, experimenting with various models and hyperparameters for the best results, and developing the web app for visual display. I created and updated our proposal presentation slide deck with our recent design changes, and I plan to spend the rest of today making more updates. So far, I am right on schedule. Next week, I hope to dive into further research over the best ML models to utilize and which resources/tools I’ll use for the web application. This will help in creating the design review presentation.