This week I shifted the format of YOLO to be the newest version 8 which they released recently which makes the whole process a lot more efficient with a lot less code. With that shift, the ML model was trained for 50 epochs and achieved a 0.995 accuracy. With this high accuracy, the model was finalized and I tested it thoroughly with various images I took for testing and validation and my accuracy was still very high. Because of this, I was able to create the detection file and reformat the output so that it was easily integrable with the hardware. We are a bit behind on schedule for the integration but we will be working all week to get the integration working and get the report, poster, and video done as well.
Author: rratnam
Rachel’s Status Report for 4/22
This week I worked on improving the accuracy of the multi card detection. I ran into a couple of problems. For the YOLOv7 algorithm, it needed a lot of data but the labelling software I had setup, could not handle that amount of data and was able to transfer the labels to the algorithm. Because of this, I had to reframe the network to be the YOLOv5 framework and set up a new labelling software that worked through a website rather than locally which allowed it to handle a lot more data. Because of this set back, a lot of data had to be relabelled which set me back on the progress. However, I have been working on that and the training is all set up so I hope to get the entire model trained by tomorrow so that integration with the camera can be completed with the ML model and we can get the software and hardware completely integrated before we put in the game logic.
Rachel’s Status Report for 4/8
This week I got the machine learning model to train with 99.4% accuracy in 5 epochs. With such a small amount of epochs and large accuracy, I want to see if I can get it even closer to 100% card detection accuracy with more epochs. Because training took a lot of time, I was not able to get to testing the model and measuring detection accuracy so I hope to get to that early this week. My progress is still on schedule according to our updated schedule because the ML is working locally but it needs to be tested and made more efficient before we integrate it with the camera. This week, I hope to get the testing of the ML completed and have the model finalized so that we can integrate it as soon as the camera is working properly as well. In order to properly test this, I will be sorting my data into test, train, and validation data and ensure that it runs properly on the testing data and meets the requirements that we set for the user and design requirements.
Rachel’s Status Report for 4/1
This week I worked on labeling the over 300 data points of pictures of cards I took this week. This involved creating bounding boxes for each of the cards in the pictures and then defining the labels for each of the bounding boxes. At this point, I got the outputs of each of the labeled bounding boxes, and fed them into the machine learning model for the YOLOv7 algorithm. These thousands of data points lead to a very high time to train the model. After training the model, I have realized that I have to make the algorithm more efficient and training should be a lot more efficient so this week I will work on training the network in different ways to optimize the time and ensure that it will work with our resources. To do this, I will try lower versions of YOLO which don’t require as much computing power. The progress is still on schedule and I hope to have the finalized ML model ready to integrate by the end of the week.
Rachel’s Status Report for 3/25
This week, the cards were all finalized and were printed. Therefore I was able to take photos of all the cards in different lightings, backgrounds, and angles. With at least 50 different photos of each card, I was able to start putting them into the labelling software that I set up last week. In this labelling software, I used bounding boxes to manually select each of the items and then assign them to a suit and rank. With these labels generated, I can now output the necessary values to feed into the YOLO algorithm and be able to train the model. My progress is a bit behind schedule but now that all the data is ready to go into the model, I will train the model tomorrow and spend the starting part of this week making the algorithm more efficient locally before integrating it with the system later on this week.
Rachel’s Status Report for 3/18
This week I set up the software necessary to label the input data for the YOLO algorithm. The ML model essentially takes in images as well as coordinates for where in the image it is detected associated to its label. In order to generate these coordinates accurately, it is necessary to use the YoloLabel software. With this created and the ML model set up, it is all ready to take in the cards to train the model effectively and then optimize it to be faster. We also got the cards figured out. With the trouble we were facing with the bitmapping, we got two backup plans so that if we can not figure out the bitmaps by Monday, we can use the backup designs of the cards to start printing them out and training our model.
I am a bit behind on schedule because we did not account for spring break but once we get the data printed out, I should be able to train the model quickly and be able to start working with the camera and transferring my model over to the system.
In the upcoming week, I hope to generate all the coordinates and labels for all of the input data pictures. I hope to then feed them into the model and start being able to perform object detection. I also hope to optimize the model by changing the logic to only look for cards that have been dealt to the user which can be determined by the game state.
Rachel’s Status Report for 3/11
Last week, Mason got the printer working. With the thermal printer set up, I was able to start designing the cards and printing out what I want them to look like. I worked with the bitmap encoding to attempt to recreate the different suits. I have also set up the YOLO algorithm so that it is ready to start training once I feed in the photos of the cards. My progress is currently on schedule. According to the gantt chart we had originally planned, I have until the end of this week to build the computer vision locally. I hope to complete designing the cards by the start of this week and have the pictures put into a dataset so that it is ready to be fed into the YOLO algorithm. This week the deliverables I hope to finish are having 52 cards printed out and ready to take photos of and the YOLO model completely trained so that it can start detecting cards locally.
Rachel’s Status Report for 02/25
This week I worked on the Design Presentation. As the presenter of this presentation, I ensured all the numbers lined up and made sure we had a logical foundation behind all of our metrics. For the project, I was able to decide on the YOLO framework and started testing it with other datasets that are not ours. I am still on schedule for my progress because this week I will be designing the bitmaps for the printer so that I can print out the cards and using the camera take photos of the different cards to train the YOLO model accordingly. This week I hope to have the entire dataset created so that the model can train over spring break.
Rachel’s Status Report for 2/18
This week, my goal was to read documents about the potential ML models we could implement in order to perform the computer vision object detection as well as the image classification. From some research and local testing, I found that the best choice for us at this time is the YOLOv3 algorithm which performs real time object detection and image classification in an extremely efficient manner which can be made more efficient due to the fewer amount of images we will have classify and the lower frequency at which we will need to classify them at. My progress is on schedule at this point because I was able to read all the documents I wanted and narrow it down to the model and overarching parameters I want to use. Now, since we have ordered the thermal printer, cardstock, and clips, as soon as they arrive, I can create all 52 of the different cards in order to build our dataset and start building and training my model locally on my computer. This week, I hope to start building the model locally, and if all the parts come on time, hopefully also start building the training dataset. By the end of the week, I also hope to have designed what the card looks like and figure out how to encode it on the thermal printer so that I can start printing out the cards for the datasets.
Team Status Report for 02/11
The most significant risks that could jeopardize the success of the project are being able to handle the multiple signals we will be receiving from the different games going on and the different players in the games, classify the cards placed on the table in a quick and efficient manner, identify the various unique devices and assign them to the specific corresponding player, and being able to interface between our keyboard device, LCD screen, dealing device, and our raspberry pi. In order to manage the risk of handling multiple signals from different devices and being able to properly assign them to the right users, we are utilizing a test driven development approach. We will be breaking it down into smaller pieces and attacking them from there. We will build “mock device” and “mock server” harnesses in order to be able to test receiving and sending signals along the way during production. In order to deal with and detect errors right away, we also plan on building a peripheral debug dashboard to be able to detect any bugs right away. For the card classification risk, we plan on locally building a classification network and getting that to the highest accuracy possible prior to integrating it with our camera and using it on the port. Finally, in order to interface with our input and output devices, we will be utilizing test driven development and we will be building test suites for all three of our games in order to continuously test them as we are developing. We will also perform unit testing on each of the different I/O devices to ensure they work separately as well as together. We have not made any changes to the existing design of the system from the original proposal. Our schedule has not changed yet and every member of the team is on track for their progress according to our originally planned schedule. Our project includes considerations for social and environmental issues. For the social side, we are providing a source of entertainment and a way to bring people together specifically for people who can not see each other or are separated by great distances. It also creates infrastructure that makes socially distanced living more bearable, if it is to happen again in the future. Due to feasibility and the timeline of the project, we decided to go with a receipt printer due to the implementation risk of a mechatronic device that can use a single card deck. However, we understand that this can lead to a lot of unnecessary paper waste and if time permits, we hope to transition from a thermal printer to a mechatronic device that can sort and deal from a single deck of cards.