Author: mloyet

This week I learned how to build an enclosure using the laser cutter. The enclosure is not quite finished, the attachments of the walls and the mounting points need to be fixed/finalized. Rachel and I also finalized the interface between the ML model and the device manager. It runs in a child thread and communicates through pipes. I also got another raspberry pi to be the server and wrote the base software.

We were supposed to start doing the ML integration, but that is going to happen Sunday due to delays in training.

Now we are working on the poster, report, video, and integrations done.

 

This week I finally got the new camera that works with our Raspberry Pi. This week I worked out the interface with the camera and wrote the software to capture the images and send them over to Rachel’s image detection software. This included figuring out how to instantiate and execute Rachel’s script and capture all of the I/O.

This week I think I did all of the objectives I was planning on doing. Miya has been working on the all of the backend game logic things.

In this next week, particularly tomorrow, we will be integrating the game logic, device drivers, and the detection software.

 

This week I finished up the interface for the printer. There were a few issues with the formatting for the cards which made the cards look slightly different from the card we used to train the dataset. I had to modify the bitmaps slightly in order to make sure that the alignments were consistent. Also, there were a few font and sizing discrepancies which needed to be fixed by adding some commands to the rust driver that communicates with the printer. In terms of new things, I have been researching writing a new driver for the LCD screen so that we can take advantage of all 4 rows of the screen instead of just the top two.

I previously was planning on writing the camera driver, but the new lens arrived and it was the wrong one. In order to get things back on track, we have bought the newest camera which has much better support than the camera which we were using before (this camera we have is from inventory was the oldest version. On top of that it might have been broken).

This week I want to finish writing the camera driver as soon as the new camera arrives. Also, after out mid-semester review and feedback, we have realized that we have a lot of server design and integration to do in order to meet our design goals.

 

 

This week I wrote the driver for the printer. Previously, we had been generating the cards using an Arduino because we were having issues with the interface to the receipt printer over the raspberry pi. There are couple of reasons I discovered as to why the receipt printer might not have been working before. First, the buad rate default for the interface we were using to talk to the printer was too slow. By fixing the baud rate, the printer and raspberry pi would be communicating at the same rate. Second, there is a limited buffer for the printer which needs to not be overfilled. By reading the Arduino library for interfacing with the printer, I discovered that there need to be delays inserted in order to prevent buffer overflows.

This week I was supposed to write the interface for the camera, but I didn’t get a chance to do that because we did not get the camera lens yet. As soon as we get the filter I will be able to productively write this interface.

Next week I am going to focus on writing the camera interface, even if we don’t have the lens yet. I’ll look at some open source examples to figure out the standard way for interfacing with raspberry pi camera modules.

This week I was working on the device driver for the camera. The plan is to interface with the camera through v4l2 (video for linux 2) which comes preinstalled on the raspberry pi and is supported by the raspberry pi HQ camera. We managed to capture some raw images from the camera but realized that without a lens, they don’t look very good. So I ordered a camera lens. I also worked on improving the LCD screen device driver to make it print out special characters for the suits.

Since I wasn’t able to finish the camera driver, I am a bit behind schedule, but once the lens arrives, I expect finishing the driver to be fairly straightforward.

Next week, I’ll finish up the camera and work on writing the receipt printer device driver. Hopefully in the process of formalizing the driver I will be able to figure out exactly what the confusion with respect to the bitmap printing commands was.

 

This week I was working on the device drivers for the keyboard and the LCD screen. The keyboard was a holdover from the last week which I wasn’t able to finish on time. This week I was able to catch up and make an MVP version of both drivers. Implementing the drivers also included rearchitecting the device code in order to support the multithreaded model outlined in the design document. The keyboard implementation (https://github.com/mloyet/joat/pull/3) was fairly straightforward but required reading about the some linux documentation and source code to figure out how the key codes are formatted. The LCD screen (https://github.com/mloyet/joat/pull/4) was slightly more complicated. It needed a new device driver to be installed, and since the protocol is not on a standard bus, it had to be registered as a platform device in the device tree. Fortunately the device is pretty well supported by linux, so the actual device driver almost works for our use case. I say almost because I discovered that our device is actually interfaced as two lcd screens that share a bus and have different enable lines. Currently our lcd peripheral only makes use of the top half of the screen. I may make some modifications to the linux device driver in the future in order to utilize both the top and bottom halves.

I am pretty much complete with my tasks for this week, but there is some code that prints out suits to the LCD screen that I haven’t had the chance to test yet.

Next week I plan on getting the camera operational, and also spend some time looking into the receipt printer. Rachel and Miya have been working with the interface to figure out how to make cards, but the bitmap logic hasn’t been fully working. Perhaps a fresh set of eyes can get it working.

This week I was supposed to finish writing the testing framework, which was held over from last week. I was able to write the framework how I wanted it before working on the design document, which was a major team effort. I helped Rachel understand how to use the receipt printer, and explained how it was configured. This week she began constructing the card designs in order to begin her ML training.

 

Over spring break I was scheduled to figure out how to make the keyboard work. I didn’t do this as of today, but I expect to be able to figure it out tomorrow. Since keyboard support is very common, I don’t expect it to be a real holdup.

 

Next week I will figure out the keyboard and begin working on getting the LCD screen operational. The LCD screen is the most obscure peripheral, but it implements a very common LCD screen protocol.

This week I was supposed to write testing frameworks for the device and server, which I started work on. However, I did not completely finish them because I started working ahead on the receipt printer. It came to my attention that in order for Rachel to do her ML training, we would need to construct a set of training images of the cards we will be using. In order to do that, we need to be able to print cards, so I prioritized being able to print receipts. There were a couple of hang ups I had to address along the way.

1. I needed to be able to connect the raspberry pi the the school internet, which I was able to do with Ankita’s guidance. This required me to mess with some of the network configuration files and learn how the wpa_supplicant daemon works and how network cards are represented on Unix systems.
2. Then, I needed to figure out how to connect the receipt printer in a productive way. I discovered that serial ports are represented as tty ports. There was some more configuration files I needed to mess with to make sure the actual pins were connected to the file though. Then I was able to write to the file and send data to the printer for the first time.
3. Finally, did some research into making it possible for my partners to ssh into the pi instead of just me. Next time we meet, I will register their public keys as authorized keys so they can do work.

This week I got behind on the testing frameworks. I am confident that I can finish them before the end of next week along with my other tasks so that we can get back on schedule.

Next week I am going to make the interface I started with the printer concrete. This task is high priority so we can start printing cards and Rachel can be unblocked.

This week I did two main programming tasks. I set up the Raspberry Pi with the RaspbianOS image, and managed to ssh into the device. Unfortunately I was not able to get the device registered onto the CMU network, so I currently can only use it at home. Second, I wrote the protocol to communicate between the server and the device. The actually content of the protocol is fairly simple, more effort went into researching ways to serialize the messages and send them over the internet. Fortunately, Rust has a very good crate called serde that allows me to write enums. My work can be seen in this PR. I also did some administrative work ordering parts and working on the Design Presentation slides.

This week I was supposed to define the protocol, which I did. Next week I will begin working on the mock server and device which will be used to build the test suites.

This week I worked on creating the and presenting the slide deck for our proposal. One part that I spent a larger amount of time working on was the gantt chart to separate out our tasks. I also spent time trying to distill and predict the most difficult technical challenges.

After the presentation, I ordered and picked up our Raspberry Pi. I haven’t yet started writing code for the device, but I plan to start figuring out how to communicate over the internet with other devices in the coming week. I also spent time researching the ways that GPIO can be read and written from the board. I discovered several libraries that implement some types of interfaces that allow you to access the GPIO pins and other peripherals. There are APIs for both Rust and Python. I also dug a bit deeper and discovered that under the hood, it is really MMIO. There exist some headers as well as helper functions in C that make it possible to access this low level interface directly. For some devices, like the keyboard and printer, it is likely going to be better to use a library solution, because they use established protocols like SPI and USB. But, for the screen, it might be easier to build a solution from the ground up instead of trying to make an existing higher level interface work. More design planning will need to go into this.