Author: tfakhri

What did you personally accomplish this week on the project? Give files or photos that demonstrate your progress. Prove to the reader that you put sufficient effort into the project over the course of the week (12+ hours).

This week, I finished writing the code for the final components of my subsystem I had left, the LED array and the keypad. Now that all of the components of the Embedded Controller are there, I have written an Arduino sketch that plays out a whole game with everything except gaze tracking input, using serial input instead. Code here.

I built a 4×4 LED array to test my code and it worked. See video. Expanding this circuit to 8×8 will require a lot of wiring, but the logic is no different, so it should work in theory.

I also designed the final three chess pieces in Fusion to 3D print them at IDeATe. With this, we can now print an entire set of chess pieces. We did run into an issue where some of the pieces were too heavy for the electromagnet to move, so we are attempting to print them in “vase mode”, where the pieces are virtually hollow, and therefore a lot lighter.

Finally, I started by validating certain components of my subsystem, such as the magnet, motor calibration, limit switches, keypad, and LED array (4×4, will validate 8×8 once built..

Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?

I am right on schedule.

What deliverables do you hope to complete in the next week?

Aside from expanding the LED circuit from 4×4 to 8×8, all I have left to do is thoroughly test and validate my subsystem and the project, and write a short UART parser to integrate with the  gaze-tracking.

As you’ve designed, implemented and debugged your project, what new tools or new knowledge did you find it necessary to learn to be able to accomplish these tasks? What learning strategies did you use to acquire this new knowledge?

What did you personally accomplish this week on the project? Give files or photos that demonstrate your progress. Prove to the reader that you put sufficient effort into the project over the course of the week (12+ hours).

I did a great deal of things this week. I expanded the Embedded Controller subsystem with a few more libraries, all available on our organization’s GitHub repo.

I added the Magnet, Gantry, LimitSwitch and ChessGantry libraries, and I am almost done with the GameState library. The Magnet library controls the electromagnet, the Gantry library builds on the StepperMotor library by allowing positioning of the gantry in two directions. The LimitSwitch library provides control to detect when a limit switch is being pressed (this is for calibration). The ChessGantry builds on and abstracts the Gantry class by providing an interface for chess squares as opposed to position in inches, it can also execute a move using the following code:

// Move to the origin square center and pick up piece

go_to_square(origin_row, origin_col);

go_to_square_center();

_magnet.on();

// Return to square bottom-right corner

go_to_square(origin_row, origin_col);

go_to_square_center();

// Move to the destination square center and drop off piece

go_to_square(dest_row, dest_col);

go_to_square_center();

_magnet.off();

delay(500); // Allow time for magnet to turn off

// Return to gantry origin

go_to_origin();

I added some Arduino sketches to take in user inputs for functions over serial to test out these libraries.

I also wrote a Python script that takes in user voice and detects chess moves e.g. “A1 to H8”. This is the worst-case backup in case we are unable to get gaze detection working for the final demo.

I also 3D printed some braces for the gantry to ensure it is perfectly squared. In addition, I designed and 3D printed some pawns, a queen, and a rook with a hole for the magnet at the bottom (see attached picture). These were used to test out the electromagnet.

Finally, today I helped Trey assemble the enclosure (everything except for the lid).

Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?

I am right on schedule.

What deliverables do you hope to complete in the next week?

The main tasks I have left to do is write the short library to parse serial input from the Jetson (or the speech-to-text Python script as a backup), and wire up the LEDs and write code for that. After that, it will all be testing.

How will you analyze the anticipated measured results to verify your contribution to the project meets the engineering design requirements or the use case requirements?

So far, I have tested and calibrated stepper motor and gantry movement, to verify the number of inches it is intended to move is how much it actually moves. When calibrated, any given gantry move is within ±1% of the input position in inches. I have also written Arduino sketches to verify some of the smaller parts of my subsystem, such as the limit switches and magnet, and these work as intended. As soon as I am done writing the ChessState library, I will write an Arduino sketch and play 10 chess games on it, which I will use to validate the game state after each move function call is what it should be. Once we finalize the board this week, I will also be able to validate the ChessGantry library’s movement by using it to move a small magnet across the top of the board to a number of specified positions and measuring the error. Finally, I will integrate the Arduino with the Jetson/speech-to-text script, and validate that UART connection by measuring the difference in message sent and message received timestamps.

What did you personally accomplish this week on the project? Give files or photos that demonstrate your progress. Prove to the reader that you put sufficient effort into the project over the course of the week (12+ hours).

This week I made a great deal of progress. On Monday I wrote a basic Arduino sketch to test the limit switches we received and ensure they work as expected. This was successful. We’ll be using these for calibration.

I also spent some time designing the laser cut file to engrave our table. We’ll be purchasing a large piece of 1/8 in plywood, sawing it down to the right size, and engraving it using the IDeATe laser cutters.

I have begun by structuring all my code into libraries. I now have a stepper motor library. I tested this by writing an Arduino sketch to step both motors synchronously forward or backward by a number of steps entered by the user over serial input. This test was successful. In this test, we discovered that we will need one power supply for each motor driver.

Finally, once the second belt arrived Trey and I tested movement of the gantry using the stepper motor library. I wrote an Arduino sketch that let the user move the gantry by a positive or negative number of steps in the x and y direction sequentially. This test was also successful.

Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?

I am right on schedule.

What deliverables do you hope to complete in the next week?

I need to finalize the gantry library with better calibration for the number of steps per inch, so that we can have more accurate and precise gantry movement. I’ll also be helping Trey with some physical assembly.

What did you personally accomplish this week on the project? Give files or photos that demonstrate your progress. Prove to the reader that you put sufficient effort into the project over the course of the week (12+ hours).

I started off the week by working with the IDeATe staff to get our parts printed, and to be able to print parts quickly in the future. I 3D printed the pieces for the gantry and the camera stand and delivered them to the group. Given that I still couldn’t test my Arduino motor code until the gantry was fully set up, I started working on later tasks, such as the chess logic, overall main routine, LEDs, and additional peripherals.

I chose and ordered some shift registers, 74HC595s in particular, to control the chessboard LEDs. These are a better choice than the previously chosen MAX7219 LED driver because they can more accurately and responsively light a single LED at a time, as the MAX7219 uses row and column scanning, while the shift registers can simply set a row and column high. For a system where users are constantly looking around and high responsiveness is necessary, this is a better choice and worth the extra wiring. I also designed the circuit for this.

I also created a GitHub repository (I wasn’t able to do this while in Europe because 2FA wouldn’t let me log in), and uploaded commits of my previous Arduino code versions. I added a main routine that I will be using to guide the design of the other embedded controller libraries (motor control, magnet control, LED control, keypad, and chess logic) .

Finally, I assisted Trey with assembling the gantry, and continued working on the chess logic file.

Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?

Still bottlenecked by not being able to test the Arduino motor code, so I am behind on the gantry motion control. However, I spent time getting ahead on parts I was not meant to start later, so I am not too far behind overall.

What deliverables do you hope to complete in the next week?

Now that the gantry is finally assembled, I will be testing the Arduino motor code over the next two days, and then refining the motion of the gantry throughout the board (e.g. moving pieces between other pieces and recalibrating gantry after every move).

What did you personally accomplish this week on the project? Give files or photos that demonstrate your progress. Prove to the reader that you put sufficient effort into the project over the course of the week (12+ hours).

As mentioned in the last update, I had to undergo an unforeseen medical procedure at home which prevented me from returning to Pittsburgh for a few more days. I am now back in Pittsburgh, ready to continue working. While I was home, I designed two stands for the depth camera. One is a basic design for Liam to be able to use quickly so he can proceed with testing, while the other is a more aesthetic design shaped like a pawn meant to go on the table for the final product. I sent these, as well as the gantry pieces Trey designed, to be 3D printed at the IDeATe 3D printers, which I have access to for free. The problem is these have still not been approved, so they have not been printed, which has bottlenecked the whole team for the week.

I also spent a few hours working on the ethics assignment, and designing the board in Adobe Illustrator so I can laser cut it next week. Finally, I spent some time starting to program the chess logic code that the Arduino will use to keep track of the state of the game and know when a piece is taking another piece.

Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?

Being at home and being bottlenecked by the delay in 3D printing has set me back about a week. I plan to catch up each in the next two weeks so I am back on schedule by the interim demo.

What deliverables do you hope to complete in the next week?

I am working on making sure we can get the pieces 3D printed as soon as possible, ideally within the next 48 hours. Once that is done, I will finally be able to test the 2D gantry control code I wrote some time ago, and add the electromagnet control code.

What did you personally accomplish this week on the project? Give files or photos that demonstrate your progress. Prove to the reader that you put sufficient effort into the project over the course of the week (12+ hours).

Spring break and the design report meant that the last two weeks were less productive than the previous one. I spent the bulk of the first week writing the design report, this took more than 12 hours individually. I wrote the introduction, drew the block diagram, and wrote the parts relevant to my subsystem.

As for the actual project, I modified the Arduino embedded controller code to work for two motors, controlling the gantry in 2 dimensions instead of one. See code.

Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?

My progress is on schedule. My goal for this week was to finish the design report and add the second dimension to the motor controller code.

What deliverables do you hope to complete in the next week?

I had a medical procedure over spring break which is preventing me from getting to Pittsburgh until Thursday. I’m working on requesting a short-term “Full Absence from Class Accommodation” recommendation from ODR. I will be working on the Ethics Assignment until then. Once I am back in Pittsburgh, my goal is to upload this code to GitHub (2FA won’t let me log in because I am out of the country), and test this code if Trey has assembled the gantry. This will likely take some time. If I manage to complete this with some time to spare, I hope to add more sophisticated logic to move the gantry only along the edges of chessboard squares.

What did you personally accomplish this week on the project? Give files or photos that demonstrate your progress. Prove to the reader that you put sufficient effort into the project over the course of the week (12+ hours).

This week was key for my subsystem of the project: the embedded controller. I received the Arduino Mega 2560 and installed the required Arduino IDE to be able to develop on the board. I also spent some time reading up on the TB6600 stepper motor driver, and how to use it in combination with the Arduino to drive the stepper motors that will move the gantry. After some work figuring out how to properly wire the motor and motor driver, Trey and I were able to make the motor spin using a basic Arduino sketch. See video.

The rest of my time was spent adding calculations to the Arduino program such that given a microstep resolution, belt pitch, and pulley teeth count, we can drive the gantry to a specific position along the belt (in one dimension). See code.

Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?

My progress is on schedule. My goal for this week was to figure out how to control the motors using the Arduino and I accomplished that.

What deliverables do you hope to complete in the next week?

I will add additional logic to control the gantry in two dimensions. Once Trey assembles the gantry this week, we will thoroughly test this. That being said, we expect to be able to accomplish less work on the project in the next week as the design report will take up the bulk of our time.

What did you personally accomplish this week on the project? Give files or photos that demonstrate your progress. Prove to the reader that you put sufficient effort into the project over the course of the week (12+ hours).

I was sick for the majority this week and had to visit the hospital which hindered the amount of work I was able to do. That being said, I was able to complete the majority of my tasks.

I researched how to implement the through-board LED system. I thought of using a MAX7219 multiplexing LED chip, but after our faculty meeting and our pivot to magnetically controlled chess pieces, I realized this may actually be more complex than anticipated, as the electromagnet may destroy components of this circuit when moving pieces. As this is not a quality-of-life feature, and not a critical element of our design, I set this aside and focused on other tasks this week.

I ordered the Arduino Mega 2560 as well as cables to connect devices for the team. I also designed the communication protocol between the Arduino and Jetson. The Jetson will communicate over UART with the Arduino sending out messages as fast as the model can estimate gaze in the following format:

'''{row}{col}\n'''

The Arduino will only parse a line after seeing a newline character so that we can detect where messages start. We avoid fragmentation by discarding messages that don’t end in a newline character. The Arduino will select a move origin (or destination) as soon as it receives a complete coordinate message AND the user is using the lock-in mechanism (button/pedal).

I also researched libraries and implementation methods in case gaze estimation at our required accuracy is not possible and we are forced to pivot to having the user look at a screen. If this were the case, we would implement the eye-tracking and UI in Python. The eye-tracking would most likely be powered by the EyeGestures library. This would take in live webcam input from the user’s desktop machine and use it to output a coordinate of where on the screen the user is looking at. I would write a simple Python applet that displays a live-feed of a bird’s eye view of the board using the OpenCV Python library (this is just a commonly used live-feed library, OpenCV would not be actually using Computer Vision in this version of our project), then overlay the EyeGestures coordinate over the screen using tkinter to display where the user is looking. EyeGestures can also capture blinks, which the user would use to lock into a move.

Finally, I worked on designing and delivering the design presentation, which I will be giving unless my Monday morning doctor’s appointment takes too long, in which case Trey will give the presentation.

Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?

My progress is mostly on schedule. My illness and the change of plans to using magnets to move the pieces meant that I was not able to put enough time into finalizing the design of the through-board LED UI subsystem. I will park this aside for the time being and focus on the mechanical aspects of our system until the design report is due. Once that is working, I can go back to working on quality-of-life improvements.

What deliverables do you hope to complete in the next week?

Next week we will present our design presentation and receive the components we ordered. My main goal for next week is setting up the Arduino and figuring out how to use it to control the stepper motors.