I spent time this week diagraming a solid interface to connect the software UNO implementation to the embedded code that will manage rotating the device, dispensing cards, etc. I began implementing some of this functionality in code. I also spent some time playing around with other models for classification. The first is vision transformers. Although I first thought that it would be too slow on a Pi, after some testing, it might be feasible. Secondly, I would like to try tuning a pre-trained symbol-recognition model to see if I can achieve higher accuracy there. I started setting up the framework for training the vision transformer, although I haven’t quite tested it yet. With our current setup, we’re able to get around 99.5% accuracy. I also helped to 3D print a lot of the parts that are being used for the mechanical part of the project. I am on schedule, although some things are out of order. This coming week I hope to collect more data on real images of the cards, finalizing the interface, and improving the accuracy of classification on real data.
David’s Status Report for 2/24/2024
This week saw a big step forward in the mechanical side of the design. I wrote drivers for the servo and dc motors for testing. We got the rotating platform working along with the stepper motor. We are getting a smooth and consistent rotation. We also got the card dealer working as we were able to use the dc motor to push the cards out with enough speed. We were unable to test automated card extrusion as the gear for the servo motor had not been printed yet. However, we were able to test just using our hands to turn the roller and we can test the servo motor tomorrow as the gear is now printed. We have attached videos in the team status report showing the subsystems working. I also tested the white LED’s we bought for lighting inside the chassis for card classification and they work fine. Lastly, I started working on the event loop that the raspberry pi will run during actual operation. I planned out the functions we will need to add as well as the control flow of deciding what to do on each turn.
I am on schedule. Next week I will work on finalizing the camera drivers and start automating data collection for our ML model. I will also work on fleshing out the event loop and modifying the UNO software implementation.
Team Status Report for 2/24/2024
The risk of being able to move the chassis as well as dispensing one card consistently has become less of a concern. We haven’t yet given the full weight on top of the rotation platform, as well as we still have the jamming issue on the dispenser that happens here and there, but with the amount of time we have, we will be able to figure a solution out. For the dispenser at least, it seems to work well when we apply some amount of pressure on the cards, which will be replaced by some type of weight on top of the cards later down the road.
We are still unsure about the classification latency/accuracy which we will start testing next week. One of the potential risks is lighting inside the chassis, but we have purchased some diffused LEDs we could use to brighten the inside or make a mesh-like cutout on the chassis so there’s enough lighting coming into the chassis for better visibility.
No changes to the system and schedule have been made. We do aim to focus on getting the design report done next week though.
Here are some videos of getting the chassis and dispenser working:
https://drive.google.com/file/d/1KGgJ0_tdnAvi6B1oQFJU8SHMi2bihiQO/view?usp=sharing
https://drive.google.com/file/d/1dAv9jNmSohXnl55oyNhz-Z-KTMuDrxDC/view?usp=sharing
https://drive.google.com/file/d/1LHxRdXYtc_W_PJOMiwY9bpuBw-nyfv2b/view?usp=sharing
https://drive.google.com/file/d/1nWOSIvm0QjXJGd85pnTdlnt-_bM_Pzob/view?usp=sharing
Thomas’s Status Report for 2/24/2024
I had a very productive week. We have received all parts and gears printed for the dispenser as well as the bottom chassis. We were able to test both of their functionalities. I assembled them while David wrote the drivers for each part. We were able to test and verify that the bottom chassis can rotate 90 degrees at a time, as well as the dispenser is able to dispense one card at a time fairly consistently.
For the bottom chassis, I was making holes on the board using a laser cutter, but I had to go through multiple trials due to the machine’s calibration being slightly off. When we print further prototypes or the final one, we will make the holes while printing other things as well, so this is not too big of a problem.
For the dispenser, we still have the gears for the servo motor that is in progress, but we manually rolled out a card and used the DC motor to shoot the card out further. It does get stuck here and there, but with more weight or force on top of the cards, we might be able to achieve our requirement pretty soon.
This puts me very ahead of the schedule. The only thing I would have to do is to make a couple more prototypes, verify that those work, and get some extensive testing on the dispensing and rotation of the chassis.
My goal for next week is to start re-cutting a chassis that is closer to what we would use for the final design since we have made some changes in the design to cut weights and get rid of unnecessary parts of the chassis. For the dispenser, we will likely just use the current design and only print a new one if necessary. With these, if we have a barebone event loop, I would like to test the dispensing on top of the chassis as a stretch goal.
Jason’s Status Report for 2/17/2024
I spent my time on two key areas of the project. The first of this was data creation. This week, we didn’t have our pi cameras, so I had to be more creative about gathering preliminary data. I found an online labeled dataset with images of UNO cards that are randomly rotated and saturated and placed on a random background. An example of this can be seen below.
Then, I cut out the top-left corner of the cards, and created my own large dataset. Example images from this dataset can be seen below.
This dataset consists of ~65k images and is split into train, test, and validation sets. The second key area I worked on was the classification model / algorithm. Using PyTorch, I set up a parameterizable convolutional neural network for classifying the cards, as well as a script that uses SGD to train the model. Using this, I was able to achieve accuracy up to 99.5%. On top of the training script, I also wrote a script that runs a single forward pass on a preloaded model.
My progress is slightly behind schedule, since we didn’t have the cameras for getting data that is more accurate to our setup. We have just received a camera, and are planning on creating the dataset on 2/18/24. Although, I’m not truly behind, since I’ve already made significant progress on the classification algorithm. In this coming week, I hope to create an extensive dataset for training, provide a better script for running inference on the Pi, and finish the independent color classification algorithm.
Team Status Report for 2/17/2024
We have noticed that it will be very difficult to make the card dispenser dispense one card at a time, both due to inaccuracies of the 3D printer and having to calibrate the rollers and the pressure on the cards very accurately. We will try to identify issues of each trial and make improvements to each prototype to carefully calibrate for the next couple weeks. We are not pressed for time on this at the moment as the software needed for the dispenser’s full functionality is still being implemented.
Furthermore, similar to last week, since we just got our relevant parts so we weren’t able to fully test rotating the platform. These are scheduled to be handled next week, and we plan to make changes to the motors and the motor shield if needed as soon as possible.
There were no significant changes in the system design. We had some schedule changes that occurred due to delay in getting the parts, but it’s mostly changing the orders of 2 tasks, so everyone is on schedule.
Part A written by Jason Stentz, Part B written by Thomas Kang, Part C written by David Peng.
Part A: Although our project is not directly geared towards drastically improving public health, safety, and welfare, there are a few key benefits. Since we are removing the burden of managing game flow, players will be more social during the game instead of worrying about managing the game. Of course, increased social interaction is almost always a health benefit. Also, UNO is a cognitively stimulating game. It may not seem like it on the surface, but there is actually quite a bit of strategy involved in picking when to play, draw, or call a bluff. Creating an automated UNO device will attract others to the game, promoting a healthy game to stimulate the mind. Lastly, an automatic UNO device will lower the barrier to entry for new players, since the rules will be enforced for them! This will mean they will have less anxiety when playing for the first time. Overall, an automatic UNO controller will allow for more social interaction during the game, reduce anxiety for new players, and help spread a cognitively stimulating game.
Part B: While UNO is one of the most popular card games around the world, there are a good amount of people who are not as familiar with it, especially in Asian countries. Also, even among people who know how to play the game, depending on where and who they played the game with, they can have their own sets of rules. This potentially leads to people getting into arguments or getting too focused on the game itself all the time that they miss out on the opportunity of using UNO as an opportunity to bond with other people and socialize. Our machine will be able to not only educate and assist people who have never played the game, but also eliminate arguments over rules as well as facilitate people’s social interaction by automating the flow of the game. This way, regardless of people’s background in the game, everyone will be able to enjoy the game to the fullest.
Part C: With UNO being one of the most played card games in the world, there is a large group of people, both casual, and competitive, who can benefit from this product. For example, in the UNO world championships, the need for dealers would be reduced as our product can manage the games being played with high accuracy and efficiency. It can also allow for other tournaments with lower budgets to use this machine to control the games instead of having to hire a large number of dealers. The automatic scoring will also augment the home experience and introduce more people to the game and make it more engaging.
David’s Status Report for 2/17/2024
This week all of the parts arrived which allowed me to start testing the drivers for the motors and cameras as well as testing the batteries. I soldered the motor shield pins and got a proof of concept for all motors working. I also was able to get images from the Pi camera and output them to a jpeg file.
I also got UART communication between the Pi and Arduino working which will allow us to send commands to control the motors from the Pi.
All code I worked on this week is included in the UNOmatic repo that Jason shared last week.
Lastly, I tested the batteries and the voltage converter. We proved that we were able to power the motors from the 12v battery and get 5v for powering the raspberry Pi from the 12v battery as well. We also powered on the Arduino using the 9v rechargeable batteries.
I am currently on schedule. Next week, I want to get started in assisting Jason with collecting image data for the card classifier. We will have to decide on an image size and format. The picamera library provides support for outputting images as numpy arrays so that will be our first try. I also will work with Thomas to tune the motor parameters to get smooth rotation and card dealing.
Thomas’s Status Report for 2/17/2024
A lot of the parts came in this week, so David and I tested the parts out together. We were able to check the battery output, buck converter output, Pi camera connections, as well as motor functionalities. In terms of personal tasks, I was able to create a rudimentary card dispenser model, and also 3D print it with the rollers so we could start calibrating the amount of pressure we might need on top of the cards, as well as making sure we could dispense one card at a time. We figured out that with smaller designs, the 3D printer was struggling to get precision, meaning I should insert more tolerance into the design starting from the next design. Also, I have started preparing for the design presentation, as I will be presenting this time.
With the modified schedule, I am still on schedule. Instead of doing the lower body integration this week, I have been designing the upper body/card dispenser, scheduled to take about a week, which is what I have finished. This coming Tuesday, we will be ordering screws and some other missing parts. Once all of these arrive and have the gears 3D printed, we can start motor testing on the lower body while still prototyping with the dispenser which takes about half a day to print. I would like to get the lower body to turn by the stepper motor by the end of February according to the schedule.
Team Status Report for 2/10/2024
In the mechanical realm, we have to make sure the stepper/DC/Servo motors don’t draw too much current through the Arduino Motor shield, so we made sure to choose options with the least amount of current draw. We could also swap out the motors with something else down the road if we find it necessary. Also, we do have a concern of RPI 5 not being able to run on full throttle due to requiring 5V 5A, but even on the low power peripheral mode, we expect the 2 cameras and the serial interface to stay within the max current output. Finally, the stepper motor options we could use were heavily restricted by the 1.2A current draw, making us choose a relatively weaker one. It is possible that it will struggle to move the entire chassis, but we do plan to use possibly lighter material for the casing, make the batteries smaller in case we need to reduce weight, and change the gearing design to require less torque on the motor.
In terms of software, the biggest risk is a failed classification model. In our proposal, we promised at least a 95% accuracy in card classification, but we want to achieve something higher than this. Initially, we will be attempting a standard CNN architecture without much preprocessing. If this does not achieve a high enough accuracy, we will preprocess the image (extracting the symbol) to classify the symbol and color separately. This will decrease the dimensionality of both the input and the output, hopefully simplifying the problem for the model. If all of this fails, our backup solution is to tag the cards (say with a QR code) to reliably identify them.
On a systems level, nothing changed. We are calibrating sizing of the case and gear ratios, and minor physical features, but our overarching design is still the same.
We chose a smaller battery pack for the motors as we realized that we could save power by cutting power to the motors when not in use. This allows us to save weight and space in the enclosure. This also saves us money.
We might have to swap what we are doing next week based on the parts delivery. If we do not get the parts, we will just start developing the card dispenser CAD model and 3d print as well as begin work on the classification model itself.
Jasons’s Status Report for 2/10/2024
In the first part of this week, I spent time preparing for the proposal presentation, which I gave on Monday. On the hardware side, I was a part of the discussion surrounding the main body of the project. I also 3D printed a prototype gear + rail system for rotating the main body. I looked into cameras and camera drivers that would work in conjunction with the Raspberry Pi 5. I spent most of my time this week on a fully-functional software implementation of UNO. Here is a link to the repository I’ve been developing in. I designed the OOP structure to be intuitive and easily extendable. The current implementation is around 450 lines of code. I am currently on track with our original Gantt chart, as the UNO implementation was scheduled for this first week. This next week, I’m hoping to begin work on the CNN card classifier and start creating a labeled dataset composed of pictures from a Pi camera. If that is delayed, I found a general-purpose UNO dataset online to begin testing the model.