This week our team did our Design Review presentation. The thermal printer finally arrived and we have tested it to make sure we can print basic text. Waiting for the thermal printer to arrive and getting it to work was our main prioritization (as we need our printer to begin training), so this set us a little bit behind schedule on creating the testing harnesses (software tasks). As a result, we’ve adjusted our schedule to get the playing card mockups done as soon as possible and have been looking into doing a full bitmapping for the thermal printer. This is so we can get a grasp on what the cards look like in order to start training the model to recognize the suits and values. The cardstock and clips we ordered last week came in this week as well, so we will use them for gameplay testing in the coming weeks. Aside from the schedule setback, there haven’t been many changes being made to our current system.
Rachel’s Status Report for 02/25
This week I worked on the Design Presentation. As the presenter of this presentation, I ensured all the numbers lined up and made sure we had a logical foundation behind all of our metrics. For the project, I was able to decide on the YOLO framework and started testing it with other datasets that are not ours. I am still on schedule for my progress because this week I will be designing the bitmaps for the printer so that I can print out the cards and using the camera take photos of the different cards to train the YOLO model accordingly. This week I hope to have the entire dataset created so that the model can train over spring break.
Team Status Report for 2/18
So far, our team has been following the schedule we defined from a few weeks ago. This week primarily focused on going over documentation and preparing for the design review next week. We received the Raspberry Pi 3 and camera module, and have been trying to configure it thus far. The thermal printer, LCD screen, and keyboard were also ordered this week, so we plan to start testing and integrating them as soon as they arrive.
As of right now, not much change has been made to the design of the system. There were some concerns over player experience being sacrificed since our physical cards would be very thin (receipt paper), so we ordered some cardstock and clips to give the playing cards a more authentic feel. Below is a diagram of the system we plan to build over the next few weeks.
Miya’s Status Report for 2/18
This week my goal was to read documentation/learn how to configure the Raspberry Pi Camera module. I had hoped to start working more this week, but since I was symptomatic with Covid, I was unable to work with the team or configure the devices in-person. I’ve since been reading through the Picamera2 library and Raspberry Pi Camera Guide in order to better prepare for next week when I finish isolating. Since we have the Design Review next week, I also helped prepare the slides/diagram for the presentation.
Next week, I plan to start testing the scanning properties of the RPi Camera Module and write code that can help troubleshoot the game logic (Go Fish, to start).
Rachel’s Status Report for 2/18
This week, my goal was to read documents about the potential ML models we could implement in order to perform the computer vision object detection as well as the image classification. From some research and local testing, I found that the best choice for us at this time is the YOLOv3 algorithm which performs real time object detection and image classification in an extremely efficient manner which can be made more efficient due to the fewer amount of images we will have classify and the lower frequency at which we will need to classify them at. My progress is on schedule at this point because I was able to read all the documents I wanted and narrow it down to the model and overarching parameters I want to use. Now, since we have ordered the thermal printer, cardstock, and clips, as soon as they arrive, I can create all 52 of the different cards in order to build our dataset and start building and training my model locally on my computer. This week, I hope to start building the model locally, and if all the parts come on time, hopefully also start building the training dataset. By the end of the week, I also hope to have designed what the card looks like and figure out how to encode it on the thermal printer so that I can start printing out the cards for the datasets.
Mason’s Status Report for 2/18
This week I did two main programming tasks. I set up the Raspberry Pi with the RaspbianOS image, and managed to ssh into the device. Unfortunately I was not able to get the device registered onto the CMU network, so I currently can only use it at home. Second, I wrote the protocol to communicate between the server and the device. The actually content of the protocol is fairly simple, more effort went into researching ways to serialize the messages and send them over the internet. Fortunately, Rust has a very good crate called serde that allows me to write enums. My work can be seen in this PR. I also did some administrative work ordering parts and working on the Design Presentation slides.
This week I was supposed to define the protocol, which I did. Next week I will begin working on the mock server and device which will be used to build the test suites.
Miya Higuchi’s Status Report for 2/11
This week I helped in preparing for the design proposal presentation. I also assisted in helping set up the website. As a team, we looked at documentation, and narrowed down the tasks that we needed to accomplish, what the project timeline would look like, and what the division of labor would be. Since I was out of town this week starting Tuesday (due competing at our university’s conference swim meet 7am – 10pm) until today (Saturday), I will be presenting for one of the later presentations and will be doing my best to catch up/making up work the next week.
Next week, I plan on looking more into the documentation (the software aspect of the project), and helping with deciding which devices we need to order/work with as our physical devices for the project.
Team Status Report for 02/11
The most significant risks that could jeopardize the success of the project are being able to handle the multiple signals we will be receiving from the different games going on and the different players in the games, classify the cards placed on the table in a quick and efficient manner, identify the various unique devices and assign them to the specific corresponding player, and being able to interface between our keyboard device, LCD screen, dealing device, and our raspberry pi. In order to manage the risk of handling multiple signals from different devices and being able to properly assign them to the right users, we are utilizing a test driven development approach. We will be breaking it down into smaller pieces and attacking them from there. We will build “mock device” and “mock server” harnesses in order to be able to test receiving and sending signals along the way during production. In order to deal with and detect errors right away, we also plan on building a peripheral debug dashboard to be able to detect any bugs right away. For the card classification risk, we plan on locally building a classification network and getting that to the highest accuracy possible prior to integrating it with our camera and using it on the port. Finally, in order to interface with our input and output devices, we will be utilizing test driven development and we will be building test suites for all three of our games in order to continuously test them as we are developing. We will also perform unit testing on each of the different I/O devices to ensure they work separately as well as together. We have not made any changes to the existing design of the system from the original proposal. Our schedule has not changed yet and every member of the team is on track for their progress according to our originally planned schedule. Our project includes considerations for social and environmental issues. For the social side, we are providing a source of entertainment and a way to bring people together specifically for people who can not see each other or are separated by great distances. It also creates infrastructure that makes socially distanced living more bearable, if it is to happen again in the future. Due to feasibility and the timeline of the project, we decided to go with a receipt printer due to the implementation risk of a mechatronic device that can use a single card deck. However, we understand that this can lead to a lot of unnecessary paper waste and if time permits, we hope to transition from a thermal printer to a mechatronic device that can sort and deal from a single deck of cards.
Mason Loyet’s Status Report for 02/11
This week I worked on creating the and presenting the slide deck for our proposal. One part that I spent a larger amount of time working on was the gantt chart to separate out our tasks. I also spent time trying to distill and predict the most difficult technical challenges.
After the presentation, I ordered and picked up our Raspberry Pi. I haven’t yet started writing code for the device, but I plan to start figuring out how to communicate over the internet with other devices in the coming week. I also spent time researching the ways that GPIO can be read and written from the board. I discovered several libraries that implement some types of interfaces that allow you to access the GPIO pins and other peripherals. There are APIs for both Rust and Python. I also dug a bit deeper and discovered that under the hood, it is really MMIO. There exist some headers as well as helper functions in C that make it possible to access this low level interface directly. For some devices, like the keyboard and printer, it is likely going to be better to use a library solution, because they use established protocols like SPI and USB. But, for the screen, it might be easier to build a solution from the ground up instead of trying to make an existing higher level interface work. More design planning will need to go into this.
Rachel Ratnam’s Status Report for 02/11
This week, I worked on the proposal presentation slide deck for our team’s in class presentation. I worked on clarifying what the use case requirements and technical challenges were going to be for the project and researching different values for our quantitative requirements. Specifically, I looked into the frequency and latency of various machine learning models that worked with detecting objects and being able to classify them and found the specific frequency goals we wanted to achieve for our playing/vision area updates. I also looked more into what hardware and software we would be using to perform the classification and what camera we could use that would be easily integrable with the raspberry pi. This week was primarily focused on clarifying our goals and expectations for the project over the next couple of weeks and creating a reasonable timeline for each of the different tasks of the project. My progress is currently on schedule with the project schedule. As the team member on the ML track, according to the schedule, I should currently be starting to read documents and papers that perform similar tasks to what we are trying to accomplish with object detection of cards placed in the vision area and specific card classification. This week, I will be focusing on that specific task, by searching for different papers that implement various networks and ML models that pertain to the task at hand. I will then compile the models that I think will be best fit for our card classification and be ready to test and compare them next week according to the schedule. By the end of the week, I hope to have at least three different ML models researched and understood to ensure that I am ready to test them.