Posted on

Team Status Report for 10/10

One of the significant risks we identified in our previous progress reports was a delay in ordering the PCB, and this risk has now occurred. When we looked at the price of the PCB, we realized we could not afford to order two boards (in case the first one did not work). This created an additional risk, since it left us with no buffer. We had to weigh the tradeoff between two options, sending the PCB out on schedule and risking a mistake that we could not recover from, or sending it out a bit late after checking it thoroughly. We ultimately chose to send the PCB out one week later than initially planned. Although this may set our integration back slightly, we decided it was the safer choice for the project.

No changes were made to our design. This week we finalized the design spec of digital hall effect sensors, which is represented in our PCB design. This decision will require a working CV element to the system to detect specific types of pieces. However, this is still in line with our current design. 

Schedule Update:

We moved the integration back by two weeks in consideration of the PCB delay. 

Photos from Sensor Testing:

For both Analog and Digital Sensors we ran some distance testing (as cross-talking was our main concern). We ran each sensor with the magnet on top and in a 1/2cm distance.

Directly on top of sensor (7cm = 0cm):

1/2cm away from sensor (7.5cm =0.5cm)

Analog Results respectively:

Digital Results respectively:

 

A was written by Justin, B was written by Kevin and C was written by Sabah.

Part A: The product solution we are designing will meet a specific need with consideration of global factors. 

Our project, while offering a much more accessible way to play chess for the visually impaired, is still limited in the global scope. Mainly because the braille and audio feedback features will be in English. This is a tricky barrier to cross in a single system because all major languages use a different writing system for the visually impaired. We suppose translated versions of the project may be created with little difficulty, although we aim to only help the English speaking visually impaired play chess more effectively. However, we do not require our users to be very tech savvy. The user interface on the web will be accessible to everyone on the local network, and on the device, we will have automated scripts that connect to known internet hosts upon startup. There will be equal accessibility to nearly everyone in the US and English speakers with minimal tech experience and access. With consideration to all global factors, language is a clear barrier that our project currently will not seek to expand on. Although with our target audience in developed, English speaking countries, we aim to provide a much more accessible way to play chess. 

Part B: The product solution we are designing will meet a specific need with consideration of cultural factors.

Our project is aimed to make chess more accessible to visually impaired individuals while also considering different cultural backgrounds and experiences of our different potential users. Chess is a pretty famous game played all around the world. While the rules are standardized, players may have different levels of familiarity with assistive technology and different preferences to how they would interact with the game. Additionally, players might even have different experiences and expectations for how chess is played. This may include how the chess community values fairness, independence, and equal opportunities. But in our current state of the world, because technology has progressed so much, online chess has become a popular mode of play. However, virtual play can often be limited to visually impaired players since they cannot see the screen or have any physical contact with the game. With the values of fairness, independence, and equal opportunities in mind, we are creating an accommodation for visually impaired players by incorporating tactile features so that they can participate in the same game as sighted players. Since chess is often played in a social environment, reducing these barriers between visually impaired and sighted players can help with play between friends, family, or even competition. By doing so, we are maintaining the social aspect of chess and encouraging inclusion within the chess community. 

Part C: The product solution we are designing will meet a specific need with consideration of environmental factors. 

In consideration of environmental factors we choose materials and designs to limit waste and conserve natural resources. The board surface is acrylic, which is both durable and scratch resistant, hence the board can last for years and be recycled at the end of its life. We are also considering 3D-priniting our chess pieces, in this case we will use a PLA. PLA is extracted from plant-based renewable resources and can be composted in industrial facilities. Our overall system is built with a modular design which allows users to replace the PCB, camera, speaker, or LEDs (neo pixels) individually instead of discarding the whole product. This design choice saves the raw materials used to manufacture these electronic pieces and additionally keeps the electronic waste to a minimum. Finally, the system uses little energy as the Jetson can run in a low-power mode, and the neo pixels only light up when giving feedback.  Lower electricity use means fewer fossil fuels burned to power the board, which helps conserve natural resources. 

Posted on

Sabah’s Status Report for 10/10

This past week, I finished the schematic and layout for the 8×8 PCB matrix. The layout took a lot of time because this was my first project in Fusion in about two years, so I spent time relearning the tools and wiring rules.

Here are the final schematic and the 3D layout:

8x8Digital SensorSchem

Our progress is slightly behind schedule. I decided not to send out the PCB before fall break, which technically set us back a full week. The PCB total came to about $364.50 (not including shipping), and we also need $25 for a stencil. That leaves roughly $30 in the budget for the NeoPixels, board materials, and chess pieces. While reviewing the order, we noticed a large $228.60 charge that came purely from the board’s size. We therefore decided to reduce the board from 2.5 x 2.5 inches per square to 1.1 x 1.1 inches per square. This lowers our cost and also helps our users, as a smaller board means less space for a visually impaired player to cover with their hands.

Over break, I have been shrinking the board and redoing the entire layout. To catch up, I rearranged some tasks. Instead of integrating the PCB next week, I will work on the front end of the website and on ordering and wiring the NeoPixels. By Monday, I hope to have the orders for the PCB and NeoPixels confirmed. For the rest of the week, I will figure out how the NeoPixels will work with the Jetson and begin work on the website.

Posted on

Justin’s Status Report for 10/10

This week I helped out with soldering and testing both the digital and analog hall sensors. We had solid results from both, but we noticed that the analog sensor output values could easily be misread if the spacing of each piece and sensor was inadequately alligned. And the digital sensor was very accurate in detecting the presence of the manget within a 1 inch radius. We also tested the sensors with a piece of aryclic on top. Readings were not affected by any noticeable effect. Additionally, me and my team heavilty discussed and mapped out how the entire system will behave for the design report. This was a good way for everyone to be on the same page about the system design and how implementation will be done. We also recieved the mircoSD card for persistent storage on the Jetson, this was key for finishing setting up the board. I intend on doing basic testing of all the in/output pins and USB ports for which our PCB and speakers will connect from. With our PCB officially using digital sensors, the interface between the PCB and the Jetson will be simipler and less error prone, altough basic verifcation that the Jetson can get the results of the sensor inputs is necessary.

Posted on

Kevin’s Status Report for 10/10

For this week’s progress report, I continued to develop the digital board-state tracking system responsible for maintaining the current configuration of chess pieces and updating the board after each move.  I also worked on integrating the python-chess library with the board-state tracking logic to check for legal moves and invalid board-states transitions. I was also focusing on structuring the software so that the CV outputs can be translated into board-state changes which then can be processed by the chess logic.

One challenge that I faced was making sure that the digital board state can accurately see the changes detected by the camera. This can come from when it tries to distinguish valid moves from incomplete / incorrect piece detections. Since the camera and Hall-effect sensor system are still being developed, the testing relies on the simulated board-state inputs than from the actual integrated system.

For next week, I will continue with refining the integration between board state tracking and python-chess to make sure that moves are being correctly checked and reflected on the digital board. I will also start implementing the Stockfish integration to provide move suggestions and board analysis. Another main thing would be to get the computer vision to begin detecting piece positions and board changes which also will be integrated with the game logic.

Posted on

Team Status Report for 10/3

Based on some of the feedback from the design presentation, it is going to be a priority for us to make sure that there is some benefit for the computer vision aspect of the project. As of now, it seems like it will be the most effective way to recover from corrupted game states, i.e. a hall effect isn’t triggered and the virtual state of the board becomes out of sync with the physical game. For the Jetson microcontroller, in the process of installing the operating system, we realized we need some form of persistent storage which needs to be ordered; the most affordable option seems to be a 128 gig microsd card. Once that arrives, we can boot the Jetson without the ISO image. This may set our software development back about a week, although we are confident that using a performant high level language such as golang will allow us to seamlessly manage concurrent processes safely and fast enough for our target requirements. Since go is also built with a nice networking interface, it should be also relatively easy to do the web backend with it; later down the line we need to decide if an on-device database or a cloud based solution will be best. Additionally, for the computer vision aspect, we are considering having the computer vision aspect to act as a secondary verification / recovery mechanism rather than having it as a primary means of detecting every move. We want to keep the hall sensors as the primary source for detecting board changes if piece classification is difficult to achieve under varying lighting or camera conditions.

One big feature we are going to add to our project is a website for logging the history of moves  which can be used by either a user who is able to navigate the web or a caretaker/ coach who could go through and analyze past games. This will be a simple add on to our software stack and requires no additional hardware or modifications to the overall software architecture. 

No changes have been made to our overall design since the design presentation, although some of the questions we received during our presentation have got us thinking about how the overall design may need to be refined. One of these possible changes is switching to a hardware-only product, by using analog sensors and magnets with variable strengths. We do want to create a smart chess board with as close to 100% accuracy as possible. So along with testing digital vs analog sensors this week, we will also try analog sensors with variable magnet strengths. If an analog hall sensing for determining piece types is very successful, we would prioritize development for this; it would also require ordering more magnets with more precise strengths and using some sort of ADC which would be integrated over I2C. 

As we just received the sensors and magnets we moved testing to this upcoming week. We have completed the full (8×8) matrix schematics for both sensors, so as soon as a sensor is selected we will move on to layout. 

Other updates to the schedule are to integrate ethernet much sooner, this would allow us to access our code base and download supporting software such as Jetpack 6.x for openCV. This will be completed with setting up the Jetson’s OS. Even though sensor testing and Jetson setup have shifted slightly, not all of the software tasks depend directly on the physical hardware so we are able to continue along the set schedule. During this time, we can also opt to use simulated board states to test board-state tracking, move detection, move validation, and move logging. Once the actual hardware is available, we can replace the simulated inputs with real sensor and camera data. 

This is our updated price sheet & ordered products:

Posted on

Sabah’s Status Report for 10/3

This week, I spent a lot of time practicing for the design presentation. In addition, I completed the matrix schematics for both the analog and digital sensor matrices. Here are the specs/connections of the sensor matrix: 

Analog Sensor Matrix: 

There are 8 sensors per row. Each sensor connects to a “mini” mux, and the 8 mini muxes feed into the master mux. The mini muxes share the same S0-S2 lines, while the master mux gets its own control lines.

For each mini mux:

A0-A1 Sensor0-Sensor7
S0-S3 Jetson GPIO inputs (control lines) (shared)
A Master Mux (first row mini mux A corresponds to A0 of master mux)

For Master Mux: 

A0-A1 MiniMuxRow0 – MiniMuxRow7
S’0-S’3 Jetson GPIO inputs (control lines) 
A ADC Input 
ADC Output  Jetson I2C Ports 

The Nvidia Jetson will have 6 GPIO ports for the designated control lines. The first three (S0, S1, S2) are connected to the S0, S1, and S2 ports of all the mini muxes. The next three (S’0, S’1, S’2) are connected to the S0, S1, and S2 ports of the master mux. The S0-S2 lines choose the column from each row, and then these 8 column sensor values, one from each row, go into the master mux. The master mux will then choose each row one by one and send each value into the ADC, then into the I2C ports of the Nvidia Jetson.

Here is some pseudo code for how the matrix will work: 

For c in range(0,7): # choose column for each row [c0, c1, c2, c3, c4, c5, c6, c7]^T

     For r in range(0,7): # choses the row from the set of columns   

          Return matrix[i,j]

Digital Sensor Matrix: 

In this matrix, there are 8 shift registers, one for each row. The shift register is a PISO, so it will take in all 8 sensor results in parallel and pass them out serially through one data output line (QH). This matrix works by daisy-chaining the shift registers together in order (SR0’s QH goes into SR1’s SER). The final output will be a serial line (from SR7’s QH) of 64 digital values, either 1 or 0. The first digit of the 64 will be for Sensor 0 in Row 0, and the last digit will be for Sensor 7 in Row 7.

In terms of the schedule, which has been changed for the design presentation, I am on track with completing my task for building the Hall sensor schematics (8×8), as shown above. However, our team task of testing the sensors and physical board has been pushed to next week because we just received the sensors on Wednesday and are not able to test them until Monday. I think as long as we complete the testing on Monday, I will be able to complete next week’s task of laying out the Hall sensor matrix after we have chosen which sensor to move forward with.

For next week, I plan to lay out the chosen sensor matrix and implement the LED matrix. Then, over the weekend, I should be able to order the full PCB.

Monday:

  • Run sensor tests
  • Discuss board dimensions

Tuesday-Thursday:

  • Layout board based on dimensions (LED & SENSOR)
  • Do final connection checks (power lines/register or mux connections/capacitors where needed)
  • Add to cart and calculate cost

Wednesday/Thursday:

  • Discuss with TA and send out the order if possible

I want to try my best to get the order out by Wednesday since the following week will be fall break and we don’t want to delay our pcb order. 

Posted on

Kevin’s Status Report for 10/3

For this week’s progress, I continued with developing the board-state software. I mainly worked on representing the physical chessboard as a digital 8×8 state and I also worked on developing the logic behind comparing a previous board state with an updated state. Since the camera is currently available, I have been working with simulated board inputs to test the board-state tracking logic and made sure the software can correctly identify piece movement. I also made some progress organizing the software so that the board-state tracking can connect directly with the camera/OpenCV pipeline and python-chess move-validation system.

As of now, I feel like the project is currently on schedule with the software development side. Right now, board state tracking is most likely the most important task to complete. The main risk that has come up is just not having the hardware at the moment so I am not able to test using real images. While I used simulated boards, it could turn into a potential problem if the simulated board and images do not work the same.

For next week, I want to plan on integrating the board-state tracker with python-chess so that detected board changes can be translated into chess moves and then we can check for move legality. I will also continue using simulated move sequences to test normal moves and invalid board changes. If the camera is available, I can begin connecting the software framework to the actual camera capture and calibration pipeline.

Posted on

Justin’s Status Report for 10/3

This week I made some progress on getting the Nvidia Jetson set up for development. I created a bootable USB with the Jetson-speficic ISO image. I am able to boot from it, but we will need some persistant storage to really get the environment set up.  After some more research into using OpenCV on embedded processing, I found that we should heavily consider adding the Jetpack 6.x software development kit to utilize CUDA processing for the CV.  Once the OS is fully installed, we can fully start software development. And over time as the PCB and the camera arrive, I can start integrating these onto  the Jetson and begin testing of each detection system’s accuracy and testing for improvements on the overall detection routine we have planned.

Overall I am slightly behind on schedule. I hope to have the Jetson set up this upcoming week, and once that is finished, I will meet with Kevin to make sure that we can download pyhton chess and stockfish to the board and build the system software. I have never really done much devlopment with using multiple languages in a single code base so this might cause some slight unexpected delays.

Over the next week I plan to split my time to spend on the design document and getting the entire system software stack ready for use after break. I plan on staying in Pittsburgh over Fall Break, so I can make sure that this will be completed before the second half of the semester.

Posted on

Justin’s Status Report 9/26

This week we were in the phase of ordering componets for the system. Orders were placed for magets, hall effect sensors, and a Nvida Jetson. We were approved for all, and I picked up the Nvidia Jetson Orin Nano board. I am currently in the process of drawing out the block diagrams for both the system and software level. I have the first iterations of these ready, and the fully formed diagram will be ready before our slides submission. I have also been in the process of finding an affordable camera that is compatible with Jetson MIPI Camera Connectors. I will need to do some testing with the board to see wheter the 40 pin expansion header or USB interfaces are best suitable for interfacing the sensors/PCB. My current plan is to use the 40 pin header for the actual PCB and USB for the speaker. I also have downloaded and prepared the Jetson specific ISO USB for starting to work with the board. I want to have all of this completed sometime next week. Also below are current iterations of the block diagrams (first system level, then software flow diagram).

 

18500 Block Diagram

Our weekly meeting with Professor Yu was also productive. We got insight of how we should formulate our design slides, as well as discussing the instructor feedback from the proposal slides. The most significant feedback was to nail down the use case and technical requirements because some of the requirments we listed in the proposal were too vauge.

Posted on

Kevin’s Status Report for 9/26

For this week, I continued focusing on preparing the software side of the project for digital board-state tracking and the chess logic since we have not yet gotten a camera for our setup. Since I couldn’t work on the camera capture and calibration pipeline, I researched how the computer vision output will eventually feed into the digital chessboard representation, move validation, and move logging. One key component to this was looking into the python library, python-chess, which would serve as the foundation for legal moves, updating board after valid move, and generating a PGN move history.

I also began to look into how the flow of moves should be represented internally, which includes the starting square, ending square, move number, and timestamp, so that the move detection and logging can be handled more consistently later on. I did some basic research on how to have the software components run on the NVIDIA Jetson Orin Nano and how the modules should be separated so that the board state logic can potentially be tested before the vision hardware integration.

As of now, my progress has been slightly shifted since we don’t have the camera for the configuration quite yet, but I am using that time to work more on the software components. Next week, I plan to continue developing the digital board-state representation and begin testing move detection and validation using either manually generated or simulated board states. This will help me to check move validation thru the library python-chess. Another thing that I hope to accomplish is an initial move logging structure so that valid moves can be stored in sequence.

Prototyping of board-state tracking + move logging: