Author: lctummal

As a team, we are attempting to work through two main tasks. The first is the integration of the web app and the ML model. As mentioned in individual reports, the web app can now receive data from a local source, so now we are trying to test this process in a real-time system. There are some small issues to consider, such as duplicate cards (which can happen with multiple decks), not double counting cards, etc. We hope to finalize this process by next week, but for now, we can start the second main task, which is testing the metrics. For the presentation, we hope to get most of these metrics done, but for some metrics, such as the user survey, we won’t have results immediately. We have finalized our methodology, and very soon, we will conduct these tests.

I was able to address the changes mentioned in the last status report. I laser cut two long planks and glued them to the bottom of the box, with each of them protruding out in the same direction, so that if the camera were to stick out in that direction, the box would no longer tip over. Some of the planks fell apart, so I plan to hot glue them soon. Also, to further stabilize the camera mount, I laser cut a plank with a hole in it. The camera mounting hole almost perfectly sticks through it, so once I hot glue that to the top, the camera mount should exhibit very limited movement when the user moves the camera. On top of this, I have been working on the slides for the final presentation.

We were able to run the ML model on the Jetson Nano at last. We initially thought that this would entail reflashing the microSD card, but it was actually possible with just a few Python installations to get to 3.11. Now, this leaves the final major integration step, which is the transmission of card detections from the ML model to the processing algorithm, which we are working on. We also would like to broadcast the camera feed to the web app so that the user can calibrate the camera. Once we finish these integration steps, we then will start testing the key parameters of our product to assess viability.

Since the last status report and demo, a lot of work has been done on improving the apparatus that houses the Nano and the camera mount. This improved design is made of plywood and stuck together with hot glue. After piecing the structure together, I tested the integrity of the structure, and it seems to be good, except for the case where the camera significantly tilts over the edge, at which point the structure falls. The solution here, which I will implement shortly, is to glue the box to long flat planks that jut out forward. In this case, even if the camera leans forward, the box won’t fall forward.

This week, I have received both the camera mount and the camera. Later today, I will test the functionalities of both and see if I can integrate it smoothly with the Jetson. Much of our integration steps for the interim demo will take place tomorrow. Also, I have laser cut the parts for the box that will house our camera mount and jetson. I don’t have a picture right now since I’m on campus, but I can update this post at some point with a picture. This is a basic prototype model that was made out of cardboard since scrap wood was not available, though for our final version I will use wood. The dimensions are perfect to fit our Jetson, and the camera mount wire is actually very long, so I imagine I can wrap it inside the box and then attach the camera. Once these pieces go together, we will have a very basic working implementation.

This week, I have finished most of the model for the product that will house the camera and the jetson device. Pictures are attached. Each face has a bit of a jigsaw pattern so that a preliminary assembly can be done for prototyping without actually using a permanent solution like hot glue or screws. I intend to use wood to laser cut the material, so once I learn to how access wood, I will do so. The jetson will fit in the bottom compartment (almost perfectly), and the camera mount will stick out of the top. There will be a small platform inside the box that will act as the mount for the camera mount to attach to, so once I pick up the camera mount on Monday, I can configure how that will look. The holes in the box back side are for ventilation. I’m not sure if that would even be a concern, but it’s good to make sure, in case we happen to close the top for whatever reason. Wires will run from the bottom opening (where the jetson would be inserted), and this consolidation will make the product usage much easier. I may design a quick way to keep the jetson in place during operation.

This week, I have been getting back on track on my tasks after spring break week. Finally, we have settled on the gooseneck camera mount, which is the most effective way to stick the camera and then adjust it. The previous idea of a lid is very error prone and involves a lot more iteration, since it relies heavily on the materials used for the hinge as well as the design. The gooseneck mount should work better. This means that we need a platform inside the box to attach the mount to. Also, to attach the box, I plan to use a jigsaw structure, as this will support the structure best, and gives a degree of portability in case the user wants to disassemble and carry it to places. By either next week or the one after, I should have a printed prototype of the box.

After items were picked up, I wanted to set up the camera and Jetson together to make sure they work, since initializing devices can often have compatibility problems. The first step was to flash the SD card with the image for the Linux operating system. Then, I inserted the microSD into the Jetson and connected the Jetson to a monitor. There were several issues with setup on the first attempt, related to running “sudo apt-get update.” This required me to flash the microSD, and to not update. I downloaded a demo program with “DepthAI” (which we likely will not use on the first attempt of writing code), and connected the camera. The result was a (somewhat) functional object detection. Currently, I am on track, since I anticipated setup to take more debugging efforts. Now, we can begin to run some smaller sample computer vision programs to determine the accuracy of card detection. I can also start profiling different methods of card detection, such as machine learning (which can usually guarantee a higher accuracy) and computer vision (which can be simpler but with no guarantee of accuracy).

As a team, we made final decisions for the camera and Jetson products, then placed orders for them. We are ready to start creating a program for card detection, as well as working on the math required for card counting. We are trying to find an easy way to develop without moving the hardware around too much. For example, it would be nice if we could develop significant parts of a computer vision program without needing to actually connect the camera. We’ll work on how to make this process easy. We do not have any significant schedule changes or block diagram changes. The most significant risks would be a hardware device breaking, which could delay progress as we may need to order a replacement. We still have a contingency plan to develop a CUDA kernel for additional performance if accuracy and performance suffer with computer vision or a python ML model.

 

Nicholas wrote part A, Lohith wrote part B, and Joe wrote part C.

(A) The product does not strictly meet a specified need to public health, safety, or welfare, but rather is designed to ensure these factors are not infringed upon. This product is meant to teach people how to count cards when playing Blackjack at a Casino, as well as providing a fun user experience with friends to all learn as a group. However, we could see that if a product like this gained wide spread adoption, without proper boundaries, arguably public welfare and health might decline. We note that gambling addictions are real problems that need to be properly addressed and mitigated when making a product such as our own, so we have designed it to have constant reminders of safety and proper risk management when playing Blackjack. Luckily, since we are teaching players how to optimally play Blackjack, we are giving them a lower chance of losing money and causing themselves financial harm, but we still need to be aware of the real risk and issues associated with a gambling addiction. Through occasional warnings during gameplay and a warning when starting the system, we hope to mitigate possible issues related to public health, safety and welfare.

(B) This product meets needs with consideration to social factors. Blackjack can be and is often a social activity. Many people going to the casino will go with friends or family, and playing blackjack is often a bonding experience, since a game involving both luck and skill will usually prompt discussion between members of the table, with feelings of jubilation (winning a hand), disappointment (a correct play leading to a loss), or surprise (a misplay or a lucky card). Our product, which aims to help a beginner player make the right decisions when playing, will also help the player take part in these discussions and therefore increase their social interaction with friends and family. Even a person who goes alone can start interacting with other players (including the dealer as is often the case), and maybe make some friends along the way. 

People also play blackjack with their friends at home, and it is possible that a beginner who isn’t confident enough in their skills to bet money will feel alienated. This product can help the beginner start making the right moves in blackjack and increase their confidence, enabling them to start betting and also having fun with their friends.

(C) This product does not really have that many economic considerations since it is an educational application that helps with making accurate probabilistic decisions. From an indirect perspective, the application does also help teach individuals how to manage their finances and make prudent financial decisions. However, from a strictly macro-economic perspective there are not that many relevant considerations for this application and thus the design of the product can more or less ignore this area.

Over the past week, I have been exploring options for camera models. The camera is an integral piece to our project, and since we have a goal of 90% accuracy for card identification, the choice of camera is very important. Out of the many options, one option I found was the Intel RealSense camera series. These cameras have sub-millimeter accuracy, which is sufficient for us, since this camera will be on a short tripod close to the hands of cards. Out of the many options, I settled upon the D435 model, a camera that records at 90 frames per second, with RGB capabilities, and a USB-C connector. Also, I have been researching videos on how to interface the Jetson Nano with the camera. This is a common practice that people do, so there is support for if we run into issues. I am currently on schedule. Early next week, we will place the order for the camera (unless the ECE department has one, in which case we will borrow one), and then we can start testing data transfer with the Jetson Nano with a small sample program. Then, we can start developing the ML model for identifying cards.