Category: Rachel’s Status Reports

This week I shifted the format of YOLO to be the newest version 8 which they released recently which makes the whole process a lot more efficient with a lot less code. With that shift, the ML model was trained for 50 epochs and achieved a 0.995 accuracy. With this high accuracy, the model was finalized and I tested it thoroughly with various images I took for testing and validation and my accuracy was still very high. Because of this, I was able to create the detection file and reformat the output so that it was easily integrable with the hardware. We are a bit behind on schedule for the integration but we will be working all week to get the integration working and get the report, poster, and video done as well.

This week I worked on improving the accuracy of the multi card detection. I ran into a couple of problems. For the YOLOv7 algorithm, it needed a lot of data but the labelling software I had setup, could not handle that amount of data and was able to transfer the labels to the algorithm. Because of this, I had to reframe the network to be the YOLOv5 framework and set up a new labelling software that worked through a website rather than locally which allowed it to handle a lot more data. Because of this set back, a lot of data had to be relabelled which set me back on the progress. However, I have been working on that and the training is all set up so I hope to get the entire model trained by tomorrow so that integration with the camera can be completed with the ML model and we can get the software and hardware completely integrated before we put in the game logic.

This week I got the machine learning model to train with 99.4% accuracy in 5 epochs. With such a small amount of epochs and large accuracy, I want to see if I can get it even closer to 100% card detection accuracy with more epochs. Because training took a lot of time, I was not able to get to testing the model and measuring detection accuracy so I hope to get to that early this week. My progress is still on schedule according to our updated schedule because the ML is working locally but it needs to be tested and made more efficient before we integrate it with the camera. This week, I hope to get the testing of the ML completed and have the model finalized so that we can integrate it as soon as the camera is working properly as well. In order to properly test this, I will be sorting my data into test, train, and validation data and ensure that it runs properly on the testing data and meets the requirements that we set for the user and design requirements.

This week, the cards were all finalized and were printed. Therefore I was able to take photos of all the cards in different lightings, backgrounds, and angles. With at least 50 different photos of each card, I was able to start putting them into the labelling software that I set up last week. In this labelling software, I used bounding boxes to manually select each of the items and then assign them to a suit and rank. With these labels generated, I can now output the necessary values to feed into the YOLO algorithm and be able to train the model. My progress is a bit behind schedule but now that all the data is ready to go into the model, I will train the model tomorrow and spend the starting part of this week making the algorithm more efficient locally before integrating it with the system later on this week.

This week I set up the software necessary to label the input data for the YOLO algorithm. The ML model essentially takes in images as well as coordinates for where in the image it is detected associated to its label. In order to generate these coordinates accurately, it is necessary to use the YoloLabel software. With this created and the ML model set up, it is all ready to take in the cards to train the model effectively and then optimize it to be faster. We also got the cards figured out. With the trouble we were facing with the bitmapping, we got two backup plans so that if we can not figure out the bitmaps by Monday, we can use the backup designs of the cards to start printing them out and training our model.

I am a bit behind on schedule because we did not account for spring break but once we get the data printed out, I should be able to train the model quickly and be able to start working with the camera and transferring my model over to the system.

In the upcoming week, I hope to generate all the coordinates and labels for all of the input data pictures. I hope to then feed them into the model and start being able to perform object detection. I also hope to optimize the model by changing the logic to only look for cards that have been dealt to the user which can be determined by the game state.

Last week, Mason got the printer working. With the thermal printer set up, I was able to start designing the cards and printing out what I want them to look like. I worked with the bitmap encoding to attempt to recreate the different suits. I have also set up the YOLO algorithm so that it is ready to start training once I feed in the photos of the cards. My progress is currently on schedule. According to the gantt chart we had originally planned, I have until the end of this week to build the computer vision locally. I hope to complete designing the cards by the start of this week and have the pictures put into a dataset so that it is ready to be fed into the YOLO algorithm. This week the deliverables I hope to finish are having 52 cards printed out and ready to take photos of and the YOLO model completely trained so that it can start detecting cards locally.

This week I worked on the Design Presentation. As the presenter of this presentation,  I ensured all the numbers lined up and made sure we had a logical foundation behind all of our metrics. For the project, I was able to decide on the YOLO framework and started testing it with other datasets that are not ours. I am still on schedule for my progress because this week I will be designing the bitmaps for the printer so that I can print out the cards and using the camera take photos of the different cards to train the YOLO model accordingly. This week I hope to have the entire dataset created so that the model can train over spring break.

This week, my goal was to read documents about the potential ML models we could implement in order to perform the computer vision object detection as well as the image classification. From some research and local testing, I found that the best choice for us at this time is the YOLOv3 algorithm which performs real time object detection and image classification in an extremely efficient manner which can be made more efficient due to the fewer amount of images we will have classify and the lower frequency at which we will need to classify them at. My progress is on schedule at this point because I was able to read all the documents I wanted and narrow it down to the model and overarching parameters I want to use. Now, since we have ordered the thermal printer, cardstock, and clips, as soon as they arrive, I can create all 52 of the different cards in order to build our dataset and start building and training my model locally on my computer. This week, I hope to start building the model locally, and if all the parts come on time, hopefully also start building the training dataset. By the end of the week, I also hope to have designed what the card looks like and figure out how to encode it on the thermal printer so that I can start printing out the cards for the datasets.

This week, I worked on the proposal presentation slide deck for our team’s in class presentation. I worked on clarifying what the use case requirements and technical challenges were going to be for the project and researching different values for our quantitative requirements. Specifically, I looked into the frequency and latency of various machine learning models that worked with detecting objects and being able to classify them and found the specific frequency goals we wanted to achieve for our playing/vision area updates. I also looked more into what hardware and software we would be using to perform the classification and what camera we could use that would be easily integrable with the raspberry pi. This week was primarily focused on clarifying our goals and expectations for the project over the next couple of weeks and creating a reasonable timeline for each of the different tasks of the project. My progress is currently on schedule with the project schedule. As the team member on the ML track, according to the schedule, I should currently be starting to read documents and papers that perform similar tasks to what we are trying to accomplish with object detection of cards placed in the vision area and specific card classification. This week, I will be focusing on that specific task, by searching for different papers that implement various networks and ML models that pertain to the task at hand. I will then compile the models that I think will be best fit for our card classification and be ready to test and compare them next week according to the schedule. By the end of the week, I hope to have at least three different ML models researched and understood to ensure that I am ready to test them.