Author: sdomala

This week, I gathered and labeled more training data for our model and helped conduct further testing. Since our model’s classification varies in lighting, I helped gather more images in dim lighting to account for this difference. Right now, our test accuracy is just below 98% (but above 97%), so we are very satisfied with the accuracy results. One of our initial goals to account for lighting changes was to have a custom enclosure. I spent a few hours serving as an extra pair of hands for Ethan to build an enclosure for our card shoe, but our enclosure impaired our camera captures. As a result, we might not use the actual enclosure in our final product.

I spent some time working on our final poster and writing a script for my part of the final video. Finally, I also added an “edit” widget to allow users to make edits to the web application if a card was classified incorrectly. I am on schedule, but the next 5 days will be very hectic to get all our required deliverables done (poster, video, demo, and report). Tomorrow will be dedicated to finishing our poster and video. Then, the majority of next week will consist of writing our final report.

This week was dedicated to slack time and integration. Since our system has slightly changed due to the addition of LEDs, the images slightly changed in appearance. Hence, I spent most of this week gathering more training data with Jeremy and labeling the images to constitute training, validation, and testing data for our model. Since our previous model was able to achieve high validation accuracies with the previous images (without LEDs), not many changes needed to be applied to our machine learning code.  These were my main tasks for this week (gathering more data and training our ML model), so I am satisfied with my progress. Our model hits our overall latency and accuracy requirements. One of our user requirements is having the web app update within 2 seconds of a card being pulled from the trigger, and our web app updates (on average) 0.18 seconds after the trigger. Our accuracy requirement was 98%, and our final test accuracy is 98.1%. For the remainder of the weekend, I will help gather more metrics and performance results for our final presentation. I am definitely on schedule. Next week will be dedicated to further testing and making our final video, poster, and report. If we fail to satisfy our desired requirements during future testing, then I will have to help make modifications to our existing system. If time permits, I might also make the web app even more robust by allowing users to edit card classifications on the web app (for example, if a card was classified incorrectly). This is not a necessary feature, but it would add value to the overall functionality of our system.

 

Since the last status report on 4/10, I made some progress with the web app by adding the necessary logic to Blackjack in case there are multiple winners. In addition, I implemented error handling in case of empty and faulty user input from web app users. An example of faulty input is if a user inputs a negative number of players into the web app. Finally, I implemented authentication in our web app so that only verified users can modify the state of the game through the text forms and buttons. I was able to showcase these updates during the interim demo, but here is a snapshot below for reference.

 

I have also been able to make progress with the machine learning component of our project. Jeremy and I have spent the last few weeks performing the following tasks: collecting image data, labeling the data, writing and modifying Python code to train and test a convolutional neural network (through interfacing with the PyTorch library), and experimenting with various hyperparameters (number of layers, size of kernel, etc). Below are some of my results from my experimentation, but as we get more data, these results are subject to change. K refers to the size of the kernel.

 

1. Num convolution layers = 4 and K = 3

• Epoch 56: 93.45% – best validation accuracy
• Suit accuracy: 97.86%
• Rank accuracy: 93.57%

2. Num convolution layers = 5 and K = 3

• Epoch 58: 86.90% – best validation accuracy
• Suit accuracy: 98.66%
• Rank accuracy: 87.67%

3. Num convolution layers = 4 and K = 5

• Epoch 48: 95.83%- best validation accuracy
• Suit accuracy: 98.93%
• Rank accuracy: 95.71%

4. Num convolution layers = 4 and K = 7

• Epoch 56: 83.63% – best validation accuracy
• Suit accuracy: 96.78%
• Rank accuracy: 84.99%

 

Overall, I am on schedule. My main future tasks involve gathering more training data and experimenting with our hyperparameters to reach our accuracy requirement.

This week, I have been working with Jeremy on developing ML code and testing our models. Since I finished writing code to train and test a SVM model (with an RBF kernel), Jeremy was able to train and test this model on a limited dataset. We do not have enough training data, so we achieved very low validation accuracies. The fact that our training accuracy was high further proved that our training data is not comprehensive enough. Since my quarantine is ending soon and my symptoms have gotten better, I plan to go into the lab in person and obtain more card image data to use for training. This is one of my primary goals in the coming week, as we need to achieve higher classification accuracy for whichever model we choose (SVM vs neural network).

 

I have also started writing Python code on configuring a neural network. This has required interfacing with the PyTorch library. I plan to have the code finished by next week as per schedule. However, training the neural network will require even more training data than the SVM since neural networks don’t make modeling assumptions about the underlying data. As a result, finishing the neural network implementation is not as immediately pressing as obtaining training data.

 

I have also achieved considerable progress in making the web application more robust, intuitive, and accurate. To make it more accurate, I had to fix several logic bugs to ensure the card dealing order remained correct and the proper winner is determined at the end of Blackjack and War. I also added the following features: allowing the user to input player names, showcasing which player’s turn it is, highlighting the losers and winners of Blackjack and War, and implementing a sticky left column in our player table so that it remains constant when the user scrolls horizontally. This last feature is especially important in War, as there are up to 52 cards in a player’s hand, so users would need to scroll horizontally to view all the player’s cards. Lastly, I added some CSS styling to make the UI more elegant. In terms of UI styling, there is not much I need to do for the web app. With regard to next steps, I recently realized I need to implement the necessary logic for Blackjack in case there are multiple winners. In addition, I need to be able to handle empty and faulty user input from web app users (ex: inputting a negative number of players). This error handling is another one of my minor goals for the coming week. The feature I want to prioritize for next week, however, is implementing authentication in our web application to ensure only verified users can make changes to the web app. In a real-life professional poker setting, only casino officials and tournament officials should be able to input information into the web app (not audience members), so this level of security is necessary.

 

Overall, I am on schedule with my tasks. Below is an updated look at the web app (the password text field does not have any underlying logic yet, but I will work on that this week).