Category: Team Status Reports

As I’m sure is common among many groups, this week our team was basically exclusively focused on getting our project ready for demo. On that front:

Engine: The engine is fully operational with high accuracy value and policy networks. In the demo version, a simulation depth of 25 is used, meaning the engine is looking up to 25 moves into the future (though the average would tend to be between 8-10, with a minimum of 4). The engine has been integrated fully into the web app, and both real-time and historical analysis work as intended.

Web-App: The web-app is fully operation on its own. Integration with the engine has been completed, as mentioned earlier, however integration with the physical board is not quite complete. That being said, the analysis function works as intended, and communication with the physical board is almost complete, pending a full debug of said board.

Physical Board: The physical board is almost fully complete. The LEDs have been soldered, meaning the only remaining issue is debugging 2 of the muxes that are having slight issues. This should be done easily before demo, as it is a known issue that has been fixed on 2 of the other muxes already. Full integration with the web-app has been done, but the overall product has not been fully tested, as it is not possible without working muxes. That is, with the outputs coming from the muxes currently, the web-app and engine work exactly as expected, so there is a 99.9% chance no more debugging will be needed once the final two muxes are done.

Once the muxes are debugged we are ready for demo, and the next week will be spent on the video and on the final report.

ABET:

Unit Tests:

Engine:
Basic board positions to insure all suggestions are legal moves. Found error with how passes were signified on a 9×9 board, pass was changed to be signified by an index of 81 and not 361.
Complex board positions to insure engine maximized or minimized depending on whose turn it is (minimize for black, max for white). Found error where exploration wasn’t inverted, so for black’s moves the optimization wasn’t working correctly. Fixed.

Physical Board:
Sensors tested manually when disconnected from board (i.e. only connected to Arduino) to make sure they would fit our purposes. These ended up having thresholding issues, which caused our change from 1 megaOhm to 10 megaOhm resistors in series with the photoresistors.
Tested multitude of board positions to ensure the correct board state is sent to arduino. Found issue with muxes, which is currently being debugged.
(Each intersection was tested individually)
Tested each individual button to make sure each signalled the web-app in the expected fashion.

Web-app:
Tested input states to make sure they were rendered as intended.
Tested responses to RPi signals (from Arduino) to ensure intended behavior.
Pytest suite for endpoints and web-socket.

Integration Tests:
Web-App -> Engine:
Tested interactions to make sure a prompt suggestion is delivered while also making sure all suggested moves are legal (implying the sent state is processed correctly). This identified a secondary issue with how passes were being conveyed which was again fixed.
Timing analysis was performed to be sure that the engine and communication components complied with out use case requirements.

Board->Web-App:
Extensive unit tests comprising of setting up the board and making sure the exact state on the board is the exact state on the web app. This caught issues with the muxes, which are currently being debugged, so I can’t give a super detailed writeup on the solution yet.

A hardware risk that ended up occurring was completing the 19×19 board in time for the demo. As such, we decided to change our design by downscaling our board from a 19×19 grid into a 9×9, making the board around 4x smaller. This change also affected the software component and the engine component for the project. On the software side, the data structures holding the board states had to be changed to accommodate for the smaller board size on both the client side code and the code on the raspberry pi server and the backend server. 

This created some risk for the Engine, as all the planning had been for a 19×19 version, including the training for the value network and initial training for the policy network. It was decided after deliberation that we would retain the 19×19 network architectures, but adapt their usage to fit a 9×9 board. This involved tweaking the engine so it converts 9×9 inputs into 19×19 via padding, allowing the networks to run as normal. Additionally it required a refactoring of the MCTS code to run 9×9 simulations. These are necessary, as switching the network to optimizing for 9×9 hurts its relative strength, so extra data points are needed to correct this.

As a quick note, the server and backend needed to be adjusted to fit the 9×9 board, but this was done very quickly and easily.

Our new schedule is shown below, and most of the changes are around adapting our system to the new 9×9 board:

Hardware:
There have been quite a few design changes made with regards to the hardware component. The main differences from the previous design iteration are that wires will replace the vector boards, and the instead of being located on said vector boards, the light sensors will be located on the wooden planks themselves. With regards to its development, we are still wiring sensors which is taking longer than expected due to design changes. This is also affecting the planned interfacing with the Raspberry Pi. This is the main risk for our project at the moment, as to mitigate it, Hang has started to work on the hardware component along with Israel, in order to reach the interfacing stage faster.

Software (backend):
We have a slight change in design for the software backend. After consultation with Prof. Tamal, we have decided to implement the communication between the Arduino connected to the physical board and the web server via a Raspberry Pi, instead of using the user’s computer. This will be done by hosting a Flask server on the Raspberry Pi itself. However, before this can be implemented Hang will continue his work helping Israel with the hardware component, as that has proved to be more complex than expected and thus requires more manpower in order to be finished in time.

Software (Engine):
The engine has come together nicely, and is in full working order, and is currently running MCTS in order to tune both the value and the policy networks. However, as mentioned in Nathan’s weekly report there is a bit of a risk in the execution speed of the MCTS. This does not threaten the existence of the engine, the engine already works as is, and can be used. However, the faster and more efficient the MCTS implementation is, the more scenarios can be examined in a given amount of time, and thus the more accurate the engine can play (assuming the evaluation skills of the network are not harmed in the speedup). Thus Nathan will continue working on improving execution to allow for deeper simulations. At some point, he may also need to evaluate the tradeoffs in reducing the network complexity to improve MCTS timing, as the increase in allowable depth could outweigh the network accuracy loss.

Schedule:
Other than Hang, whose changes are detailed above (helping with the hardware) there are no schedule changes. Because Hang is ahead of schedule this added workload will not cut into his slack, and this added help will allow Israel to stay on schedule.

ABET:
We’ve definitely gotten a lot better at working together as a team as the semester has proceeded. At first, we did all of our work together, in person. This had quite a few limitations, including timing and interest levels. We began expanding the ways in which we work together. Sometimes when working on collaborative efforts (ex. team reports, design documents, etc.) we work remotely but synchronously connected over a voice call. This allows us to communicate effectively while still being in the comfort of wherever we choose to be (usually at home). We’ve also gotten better at working asynchronously. Our project sections (hardware, software, and engine) are fairly disjoint, so we spend a lot of time working individually, however, there are obviously times when we need to consult each other on how parts interact. We have gotten good at doing as much as possible while leaving a generalized interface, prevent blockages while still making interaction efficient and easy. Finally, due to the recent overlap in jobs (Hang helping on hardware) all of us have had to learn a bit more about how the hardware works, and Israel has done a great job teaching us so we can help him out.

A hardware risk that we are currently facing is the delay in parts and our delay of ordering them. With our delay of these hardware components, it’s put us behind in building and testing our code. This would push our completion date as well as possibly affect our LED development in the later half of the semester. As of right now at least, we plan to have a pre-made circuit design layout on the vector boards to speed the build process when our components arrive as well as start soldering resistors as required. This would at least not waste too much of our time and allow us to gain back some later.

One of the risks of site development is that people may not find specific parts of it not very intuitive to use. Some issues can seem a little nitpicky (one of the 2 tabs should be highlighted so that the user can easily tell which page they are on) but having an easy-to-use site makes the product more appealing. To mitigate this issue, we’ll find some sample users and have them test out the site and record their feedback on specific issues with usage on the site.

No changes were made to the existing designs of the 3 different components of the project (hardware, software, go engine).

However, there were some changes made to the schedule:

One of the risks that we are handling in the hardware development of the project is starting mid-size scale development of the circuitry without having tested a subset of the circuitry design. Some concerns regarding this is that our Arduino component, even though theoretically able to provide enough Current for our required sensor subset read time, the Current the Arduino has may not be substantial enough to provide for all the sensors at once. If this Arduino can not provide Current required, we will need to add more into our circuit than expected to limit current usage of each sensor.

 

For the secondary risk mentioned in last week’s team status report, we were able to find an open-source database with expert Go games; this fully mitigates this risk from last week. The games are stored in a file format called SGF, or smart game format, where game moves are stored in a tree which allows for variations from the main line of gameplay. This database has over 60000 go games, and the go games are split into different categories.

For hardware, we have changed our usage of photodiode sensors to photo resistors with an additional static 1MOhm resistor in series. This decision was required due to unexpected value results not being a high enough range. This along with the component requiring an additional resistor to restrict Supply Current made our usage of this sensor impractical and even more expensive than other components. The new photo resistors are very simplistic in characteristic, cheaper, and will theoretically be able to provide a larger range of light values with our 1MOhm resistor we have chosen in series. In addition, we have added more parts to the physical board design, as we will require an additional platform in the physical board to support the multiple mini vectors to be held near the top of the board’s holes.

 

Besides the hardware design change, there are no design changes to the reinforcement learning side and the software side. The development of the reinforcement learning model is ahead of schedule. The development of the software side is a little behind schedule, but the work can easily be caught up in time.

Updated schedule:

Physical board assembled and subcircuit testing breadboarded.

General vector board placement internal view

First of all, to address the major risk from our last report, we have made significant progress on the physical board but we are not out of the woods yet. We have built the autocad design and prepped for laser cutting for our new 100cmx100cm wooden plank. In addition we have started circuitry assembly for testing purposes before making a final order for all our electrical components. We still have not finished these tasks and have yet to start testing our software on this circuitry but we plan to do so in the coming week. The risk here has evolved a little as well, as we realized we need fire and short-circuit protection, so have developed a plan to do this with a combination of insulation, and strategic spacing of components, but we still need to actually add that to the board.

 

A secondary risk from the reinforcement learning side is the inability to find an open-source database with expert go games. We felt confident that such a database existed, but should it end up not existing (or the data is unclean, or it is in an unusable state, etc.), the value network for MCTS would not be able to be trained to non-random weights before training commences. Fortunately, the mitigation for that comes in the form of MCTS itself, as the search generates more value network training data, that is, every board state plus whatever result the game it came from reached.However, this would harm the efficacy of training at first (the network would improve more slowly) and while we have enough slack for that possible time setback, it would be ideal to avoid.

Other than the aforementioned insulation and spacing, there are no changes to the design, and everything is running roughly on schedule. The development of the website is on-track with the feature for saving game history finished, and we have started on displaying the game history. Our hardware development is on track as well and the reinforcement learning model is almost ready for training.

The major risk that we are taking as of the moment is spending more time on the implementation of the board than expected. With less support from Techspark than initially expected, it has led us to focus on building a board of our own. To ensure this does not put use behind schedule, we plan to dedicate more time next week in catching up. If the Board implementation does require more time to build, which is not planned nor expected, we will be sure to look for external resources that may help us build the physical board if not look for possible substitutes to making a custom board.

We currently have no changes in our design other than our shift of projects from the earlier week.

Schedule update:

One principle of engineering that was used is modularity. One specific example in the site design would be the code for visualizing the go boards. Go boards are typically 19×19, but the code is made modular such that it can visualize up to any NxN board size. We attempted to make our code as modular as possible because modularity typically reduces complexity and makes parts more reusable.

In addition, we have used skills from Electrical Engineering and electro-magnetic physic principles when deciding on components to use as many components have limitations. Such principles applied would be Ohm’s law and KCL rules for our simple circuitry design as well as knowledge on components characteristics.

With the custom board development, one of the concerns we dealt with as an engineer was dimension requirements and constraints of our box that would hold our electronics. This board development required skills in product engineering to make sure the board was easy for users to use as well as safe for users.

One of our biggest risks currently is the possibility that the planned minimax strategy will not prove effective as an initial opponent for the self-play RL model (be it too strong or too weak). If it is too weak, the platform that we are building it on can be extended to look more than 2-ply into the future. While this will increase training time (as the calculations will take longer to compute) it will provide a stronger opponent. On the other hand, if it proves too strong, we have other, even more basic strategies waiting in standby, such as 1-ply maximization (just maximize the amount of stones captured in one move, ignoring the possible responses) or even a random agent.

With regard to changes, a possible problem pointed out during the presentation was the idea that some variants of Mancala were solved. While we had always planned on this, we had not made clear that the version we were building for our website was an unsolved variant (the seven stone variant of Kalah Mancala). Some players use other ways to get around the solved aspect of the game such as switching positions after the first move, but those add unnecessary complication to the game, raising the barrier for entry, especially for younger players. This will not cause any increase in price, or changes to the system itself, but does specify requirements a bit better. Other than that there have been no changes to the system or structure of the project.

For right now, everyone is on schedule, so no changes are necessary there.

The effect our project will have on public safety, the economy, or the environment are relatively minimal. Of course, we are using a small amount of computational power on training the RL model and maintaining our servers, but in the grand scheme of things it is next to nothing. That being said, our project certainly has a non-trivial effect socially, and could possibly improve mental health for some users as well. Multiplayer games are inherently social, and an online platform for them provides an outlet for users to connect with other like-minded individuals. The fact that there is no major website dedicated to Mancala makes it all the more important. Beyond even meeting new people and possible friends, our project would also allow for friends to play each other directly, for friendships where it is difficult for the participants to see each other (long-distance, etc.) this can help strengthen them. Finally, this may only be relevant a tiny percentage of the time, but the small amount of social interaction from online gaming can make a significant difference in mental health. It is all too easy to shut yourself away and not interact with anyone, and as this effect compounds it becomes harder and harder to break out of it. Online social interaction can be a small step in the right direction, and our platform could provide that.