Author: rratnam

This week I shifted the format of YOLO to be the newest version 8 which they released recently which makes the whole process a lot more efficient with a lot less code. With that shift, the ML model was trained for 50 epochs and achieved a 0.995 accuracy. With this high accuracy, the model was finalized and I tested it thoroughly with various images I took for testing and validation and my accuracy was still very high. Because of this, I was able to create the detection file and reformat the output so that it was easily integrable with the hardware. We are a bit behind on schedule for the integration but we will be working all week to get the integration working and get the report, poster, and video done as well.

This week I worked on improving the accuracy of the multi card detection. I ran into a couple of problems. For the YOLOv7 algorithm, it needed a lot of data but the labelling software I had setup, could not handle that amount of data and was able to transfer the labels to the algorithm. Because of this, I had to reframe the network to be the YOLOv5 framework and set up a new labelling software that worked through a website rather than locally which allowed it to handle a lot more data. Because of this set back, a lot of data had to be relabelled which set me back on the progress. However, I have been working on that and the training is all set up so I hope to get the entire model trained by tomorrow so that integration with the camera can be completed with the ML model and we can get the software and hardware completely integrated before we put in the game logic.

This week I got the machine learning model to train with 99.4% accuracy in 5 epochs. With such a small amount of epochs and large accuracy, I want to see if I can get it even closer to 100% card detection accuracy with more epochs. Because training took a lot of time, I was not able to get to testing the model and measuring detection accuracy so I hope to get to that early this week. My progress is still on schedule according to our updated schedule because the ML is working locally but it needs to be tested and made more efficient before we integrate it with the camera. This week, I hope to get the testing of the ML completed and have the model finalized so that we can integrate it as soon as the camera is working properly as well. In order to properly test this, I will be sorting my data into test, train, and validation data and ensure that it runs properly on the testing data and meets the requirements that we set for the user and design requirements.

This week, the cards were all finalized and were printed. Therefore I was able to take photos of all the cards in different lightings, backgrounds, and angles. With at least 50 different photos of each card, I was able to start putting them into the labelling software that I set up last week. In this labelling software, I used bounding boxes to manually select each of the items and then assign them to a suit and rank. With these labels generated, I can now output the necessary values to feed into the YOLO algorithm and be able to train the model. My progress is a bit behind schedule but now that all the data is ready to go into the model, I will train the model tomorrow and spend the starting part of this week making the algorithm more efficient locally before integrating it with the system later on this week.

This week I set up the software necessary to label the input data for the YOLO algorithm. The ML model essentially takes in images as well as coordinates for where in the image it is detected associated to its label. In order to generate these coordinates accurately, it is necessary to use the YoloLabel software. With this created and the ML model set up, it is all ready to take in the cards to train the model effectively and then optimize it to be faster. We also got the cards figured out. With the trouble we were facing with the bitmapping, we got two backup plans so that if we can not figure out the bitmaps by Monday, we can use the backup designs of the cards to start printing them out and training our model.

I am a bit behind on schedule because we did not account for spring break but once we get the data printed out, I should be able to train the model quickly and be able to start working with the camera and transferring my model over to the system.

In the upcoming week, I hope to generate all the coordinates and labels for all of the input data pictures. I hope to then feed them into the model and start being able to perform object detection. I also hope to optimize the model by changing the logic to only look for cards that have been dealt to the user which can be determined by the game state.

Last week, Mason got the printer working. With the thermal printer set up, I was able to start designing the cards and printing out what I want them to look like. I worked with the bitmap encoding to attempt to recreate the different suits. I have also set up the YOLO algorithm so that it is ready to start training once I feed in the photos of the cards. My progress is currently on schedule. According to the gantt chart we had originally planned, I have until the end of this week to build the computer vision locally. I hope to complete designing the cards by the start of this week and have the pictures put into a dataset so that it is ready to be fed into the YOLO algorithm. This week the deliverables I hope to finish are having 52 cards printed out and ready to take photos of and the YOLO model completely trained so that it can start detecting cards locally.

This week I worked on the Design Presentation. As the presenter of this presentation,  I ensured all the numbers lined up and made sure we had a logical foundation behind all of our metrics. For the project, I was able to decide on the YOLO framework and started testing it with other datasets that are not ours. I am still on schedule for my progress because this week I will be designing the bitmaps for the printer so that I can print out the cards and using the camera take photos of the different cards to train the YOLO model accordingly. This week I hope to have the entire dataset created so that the model can train over spring break.

This week, my goal was to read documents about the potential ML models we could implement in order to perform the computer vision object detection as well as the image classification. From some research and local testing, I found that the best choice for us at this time is the YOLOv3 algorithm which performs real time object detection and image classification in an extremely efficient manner which can be made more efficient due to the fewer amount of images we will have classify and the lower frequency at which we will need to classify them at. My progress is on schedule at this point because I was able to read all the documents I wanted and narrow it down to the model and overarching parameters I want to use. Now, since we have ordered the thermal printer, cardstock, and clips, as soon as they arrive, I can create all 52 of the different cards in order to build our dataset and start building and training my model locally on my computer. This week, I hope to start building the model locally, and if all the parts come on time, hopefully also start building the training dataset. By the end of the week, I also hope to have designed what the card looks like and figure out how to encode it on the thermal printer so that I can start printing out the cards for the datasets.