Team B0: Blackjack Buddy

We were able to run the ML model on the Jetson Nano at last. We initially thought that this would entail reflashing the microSD card, but it was actually possible with just a few Python installations to get to 3.11. Now, this leaves the final major integration step, which is the transmission of card detections from the ML model to the processing algorithm, which we are working on. We also would like to broadcast the camera feed to the web app so that the user can calibrate the camera. Once we finish these integration steps, we then will start testing the key parameters of our product to assess viability.

Since the last status report and demo, a lot of work has been done on improving the apparatus that houses the Nano and the camera mount. This improved design is made of plywood and stuck together with hot glue. After piecing the structure together, I tested the integrity of the structure, and it seems to be good, except for the case where the camera significantly tilts over the edge, at which point the structure falls. The solution here, which I will implement shortly, is to glue the box to long flat planks that jut out forward. In this case, even if the camera leans forward, the box won’t fall forward.

The demo went well, but we ran into some issues with fixing the camera angle, so I have retrained the model on less epochs and started working on some testing, both with the current model and looking for possible alternate datasets, to try and improve the vision algorithms improvement in more difficult scenarios. Lohith also re-flashed the Jetson and now we can load a version of the model on it, so I will continue testing and working with the ML model and then I will move to Jetson tests throughout the week.

I successfully trained the model. However, the data fidelity was not what we wanted, so we are exploring various data augmentation options, and possible alternative datasets, to get the model working the way we intend it to.  This issue has caused the expected output of the ML Model to be different than what we anticipated, so for the Interim demo’s, we will not be integrating with the Processing Algorithm. I am also currently working with Lohith to get a version of the model on the Jetson so we can start working on using video feed instead of just images.

This week, I have received both the camera mount and the camera. Later today, I will test the functionalities of both and see if I can integrate it smoothly with the Jetson. Much of our integration steps for the interim demo will take place tomorrow. Also, I have laser cut the parts for the box that will house our camera mount and jetson. I don’t have a picture right now since I’m on campus, but I can update this post at some point with a picture. This is a basic prototype model that was made out of cardboard since scrap wood was not available, though for our final version I will use wood. The dimensions are perfect to fit our Jetson, and the camera mount wire is actually very long, so I imagine I can wrap it inside the box and then attach the camera. Once these pieces go together, we will have a very basic working implementation.