Posted on

William’s Status Report for 3/26

This week, I was able to resolve the issues that I had last week regarding weird behaviors with the IMU readings and the resulting roll, pitch, and yaw. I realized one issue was that in my driver, I had initialized a register to the wrong value, which meant my readings were coming in at the wrong magnitude. In addition, I realized that the inaccuracies in the roll, pitch, and yaw were a result of the raw sensor readings drifting at the start, so I added a calibration function at the start to calibrate all the readings, which has helped a lot with the sensor accuracy. The sensor is now fairly accurate with the roll, pitch, and yaw, and when I tested it by manually moving the IMU around and plotting the sensor output, I’m getting that the orientation is generally within 10 degrees of the expected values.

I also tried to add some functionality so that I could leverage accelerometer data in conjunction with the roll, pitch, and yaw to provide some position data that can cross-validate our CV data. To do so, I would have to take the integral of the acceleration and then velocity to get the displacement, and find a way to translate the readings from relative to the IMU axes to relative to Earth’s absolute orientation. This is still a work in progress, but so far, I have done some research and found a way to filter out the noise in our sensor readings using a lowpass butterworth filter. I will have to now figure out how to leverage the smoother filtered data to generate some sort of positional data.

Moving forward, in addition to getting position data, when our CV subsystem is functional, I can begin integrating the subsystems together with my team.

Posted on

Henry’s Status Report for 03/26

This week I was able to reach my goals of implementing the multiplayer capabilities and adding Bluetooth communication code to give the VR headset the ability to interact with the paddle’s rPi and the camera’s Jetson.  The only thing that I am waiting on right now is the wifi card for the Jetson which I am going to order this weekend.  The wifi card is necessary for bluetooth connection between the Jetson and the Google Cardboard.

The multiplayer capabilities took a lot longer for me to implement than I expected.  Initially, I used an open source library called Riptide Networking to program this, but the problem was that it was a lot more low-level than I wanted it to be, which means more unnecessary work.  An example would be that I would have to explicitly code what information I wanted to send from a server to a client rather than the library taking care of that for me.  I spent a lot of time researching and starting the code using the Riptide library when I decided to change to a multiplayer library called Mirror, which does a lot more work for me than Riptide.  Mirror does a lot of client-server communication, so instead of me explicitly telling the server what to send to the client, Mirror automatically send the server’s game state to the client by itself.  The biggest thing I had to do with Mirror was program the spawn parameters for when players entered a game.  This includes limiting the game to have only two players,  making sure two players spawned on different ends of the table, and starting the game when two players connected to the game.  Another cool thing with Mirror is that the server and client run the same game, all I need to do is assign a computer as a server and the other two computers as clients.  This saved me a lot of time because with Riptide, I would’ve had to create a client version of the game where it read information from the server and reflected it in the graphics.

Going off our Gantt chart my progress is on time, I have about a week to get Logan’s CV information incorporated into my game, which shouldn’t take much time at all.  The biggest risk right now is how fast the wifi card can come in and how quickly we can get it working with the Jetson.  Any free time I have can be spent working on incorporating better graphics and finalizing the UI of the game.

Posted on

Logan’s Status Report For 3/19

This week I continued with the CV portion of our project and had success on detecting position in most cases, but we ran into considerations about how the cameras would recognize the paddle in different orientations. We decided to change our strategy from contours to color recognition which would allow more flexibility in recognition if the paddle was closer to parallel with the floor. This strategy would also open up the option of coloring the edge of the paddle so that could be recognized as well.

This week I will go forward with this path and hopefully finish it up so I can turn my attention to more nuanced issues with the CV or assist on other subsystems.

 

Posted on

Team Status Report for 03/19

By far the biggest risk that came up this past week was the issue of accuracy of the position and orientation of the paddle.  The IMU was very inaccurate at the beginning of the week as the orientation would off by a lot, sometimes as bad as 45 degrees.  However, William was able to fine tune the device so that the orientation data was realistic from eyeballing the orientation of the IMU.  We are still going to work through some smaller problems with the IMU that William mentioned in his report as well as do more extensive testing to get quantitative data as to how accurate the IMU is.  Right now, if any issues with the IMU come up, our contingency plan is to either supplement the open source code we downloaded with our own code or to find a new source that has the features we need.   There are also accuracy issues with the camera as well.  Right now, Logan is working on an OpenCV algorithm that creates a bounding box around the paddle and we just take the center of that bounding box as the position the paddle needs to be in.  There’s definitely concern about how accurate this model is, but we believe that this algorithm is accurate enough that it is not noticeable by the player.  As with the IMU, once this algorithm is completed, we need to get quantitative data of how well the camera can track static and moving objects.

One change in our final design is that the game will be run on the server.  Something that came up during our design of the multiplayer capabilities of the game was that our old implementation had two players running the same game on two different devices.  We realized that synchronizing two different devices to show the same game would be too complicated and too risky, so we decided to centralize the game on a server to take away the synchronization headache.  We don’t see any cost in this new solution as the servers we were planning on using for our MVP and final implementation should be able to handle this extra workload.

As of right now, everything is on schedule and we may be looking at completing our MVP as early as the end of next week, assuming no unforeseen problem arise.  The shift of having our game be run on a centralized server is a minimal effort change for us that should not take more than an hour to fix.

Posted on

William’s Status Report for 3/19

This week, I got the IMU up and running with the Raspberry Pi. This included physically soldering the connections between the IMU and the Raspberry Pi, as well as writing a device driver for the IMU on the Raspberry Pi. The device driver allows me to get readings from the sensor. I also performed testing of the device driver by getting the sensor readings of the accelerometer and gyroscope while moving the IMU around and displaying/plotting the output to see the accuracy of the IMU. Furthermore, I tested out various open-source implementations of the Madgwick Algorithm, which helps convert accelerometer and gyroscope data into roll, pitch, and yaw.  I performed testing of the Madgwick Algorithm using plotting as well. There are still some weird behaviors with the algorithm that I have to figure out how to fix, such as the roll/pitch/yaw numbers changing despite no movement.

In addition, I finalized my strategy for powering the raspberry pi wirelessly, and came upon the solution of using a power bank. The reason for this is some other products I looked at would be assembled on top of the current pi, which would make the construction a little too tall and thus unfeasible for a ping pong paddle. With a power bank, it can be laid out flat with the pi, thus keeping the paddle at a reasonable thickness.

For the next week, I can work on getting the IMU data more accurate, as well as beginning to potentially integrate the IMU data with other components to get some idea of how our system will look as a whole. This will involve using bluetooth to send data to the VR headset, as well as seeing how the network transmissions may look.

Posted on

Henry’s Status Report for 03/19

This past week I was working on creating the graphics and game design for our MVP as well as beginning to work on my deliverables for next week.  What I need to do for the MVP is to have the graphics, game interactions, rules, and multiplayer/bluetooth capabilities implemented.  This past week, I was able to get the graphics, game interactions, and rules implemented on Unity, so right now the game looks like a 3D version of pong where two players can control the paddles using the left/right and a/d keys.   The game can also be imported onto an Android and iOS device to be played on a Google Cardboard.  I also began work on creating the multiplayer aspect of the game so that two different devices can play a game against each other rather than having two players play on one device.  I found the necessary libraries and tutorials to integrate that functionality so all I have to do now is implement it and test it.  For next week, the goal is to finish implementing the multiplayer functionality as well as adding Bluetooth capabilities to get data from our cameras and paddles.  By the end of next week, the game should be completely implemented and all that would need to be done to finish the MVP would be plugging in the data from Will and Logan, which should be really easy.  Everything so far seems to be going according to schedule.  I envision there being some debugging and testing time with the bluetooth functionality, but my goals are definitely realistic for next week.

Posted on

Team Status Report for 02/26

This week all three of us were getting our microcontrollers and hardware set-up for software development.  We had to set up the OS for the raspberry pi and Jetson as well as get the Google Cardboard setup with the Unity game engine.  Since everything this week went smoothly, there aren’t any new issues to deal with or changes necessary to make.  As of right now, all three of us are a little bit behind schedule because we had a hectic week, but the plan is to catch up next week or during spring break.

Posted on

William’s Status Report for 2/26

This week, I spent quite a few hours trying to get my raspberry pi up and running. I had a lot of technical difficulties figuring out how to SSH into the device without Ethernet cables, but eventually, I was able to figure out how to SSH using the Pi’s Wifi. Because those technical difficulties took me so long, I haven’t really done anything else with that yet. Thus, I am behind my goal of last week, but I should have more time next week and over spring break to get some good work done.

Posted on

Henry’s Status Report for 02/26

I’ve been really busy with my other classes this week, so I haven’t been able to spend as much time as I would like to on our project this week.  I spent about 4-5 hours this week getting the VR headset set up and getting Unity ready on my computer.  I set up a demo game to see how the headset works, and I should be able to start working on the graphics and network connection next week.  Right now, I am definitely a little behind schedule as I need to set up the network connectivity and MVP graphics before the end of next week, which may be too much to ask for in one week.  The plan  is to get the MVP graphics created by next Saturday and catch up on the network communication functionality during spring break.

Posted on

Logan’s Status Report for 2/26

I did a very small amount of work on CV this week, as I was severely ill from the middle of the day Monday until Thursday evening. I am posting early as I am unavailable for the next 2 days due to other commitments.

Clearly, I was not nearly able to meet schedule for this week but I plan to make up for this some time next week and over break.