This week all three of us were getting our microcontrollers and hardware set-up for software development. We had to set up the OS for the raspberry pi and Jetson as well as get the Google Cardboard setup with the Unity game engine. Since everything this week went smoothly, there aren’t any new issues to deal with or changes necessary to make. As of right now, all three of us are a little bit behind schedule because we had a hectic week, but the plan is to catch up next week or during spring break.
William’s Status Report for 2/26
This week, I spent quite a few hours trying to get my raspberry pi up and running. I had a lot of technical difficulties figuring out how to SSH into the device without Ethernet cables, but eventually, I was able to figure out how to SSH using the Pi’s Wifi. Because those technical difficulties took me so long, I haven’t really done anything else with that yet. Thus, I am behind my goal of last week, but I should have more time next week and over spring break to get some good work done.
Henry’s Status Report for 02/26
I’ve been really busy with my other classes this week, so I haven’t been able to spend as much time as I would like to on our project this week. I spent about 4-5 hours this week getting the VR headset set up and getting Unity ready on my computer. I set up a demo game to see how the headset works, and I should be able to start working on the graphics and network connection next week. Right now, I am definitely a little behind schedule as I need to set up the network connectivity and MVP graphics before the end of next week, which may be too much to ask for in one week. The plan is to get the MVP graphics created by next Saturday and catch up on the network communication functionality during spring break.
Logan’s Status Report for 2/26
I did a very small amount of work on CV this week, as I was severely ill from the middle of the day Monday until Thursday evening. I am posting early as I am unavailable for the next 2 days due to other commitments.
Clearly, I was not nearly able to meet schedule for this week but I plan to make up for this some time next week and over break.
Logan’s Status Report for 2/19
This week I continued to look into CV while awaiting materials. As I further delve into the CV content, I have increasing doubts that we will be able to meet latency requirements. After discussions with the team we have developed fallback plans to adjust the speeds of the ball to buy more computation time, either by scaling all speeds globally, or locally slowing down opponents’ shots while keeping the player’s shots in real-time. We have also considered scaling up the size of the court to make the game more similar to tennis. I also began preparing for my assigned design presentation.
This week’s progress was decent, clearing up details on our design and creating mitigation planning for the foreseeable latency issues we may have. Now that our materials have arrived we can really make progress on our subsystems .
William’s Status Report for 2/19
For this week, I helped our team further our design process. I worked on creating our design diagrams that would depict how our subsystems would work individually as well as how they would interface with each other. I also tried to research ways to power up our paddle wirelessly. After doing some research about manually building voltage converters, I came across some products that are specifically built to convert to 5 volts from various different voltages, so I think eventually, we can try to acquire some of those products. Our sensors arrived this week, but we were missing SD cards, so I had to wait for that order to come through, and I was able to pick up the SD card towards the end of the week. With that, I should be able to begin playing around with the sensors and the Raspberry Pi next week.
I didn’t quite meet last week’s goal because our materials haven’t fully arrived. However, I think that we should still be on track as a team as my goal last week could have been a little ambitious, seeing that we still had to hash out some design issues. That said, I think now that our materials have arrived, I can begin tinkering with them so that our team will keep pace.
Team Status Report for 2/19
One issue that concerned us this week was how to communicate the position and orientation of an opponent’s paddle to a player. We were worried that the latencies from an opponent’s paddle to a player’s headset would be too much and would cause delayed graphics and drastically affect gameplay. An intermediate fix we have for the MVP is to use bluetooth communication. Since for our MVP, the distance between players isn’t large, we can have an opponent’s paddle connect to a player’s headset and the latency between the two devices is pretty much negligible. For the final product, we discussed adding delays to game to send paddle data, not showing the paddle to an opponent at all, or simplifying the game so that players are within range to have bluetooth connections.
We also made some adjustments to our server design. Before, we were planning on using AWS Gamelift as a gaming server to compute the ping pong ball path. However, because the server’s rate of sending and receiving data is low we decided to switch the server to either one of our computer or AWS EC2. The biggest factors in this switch comes down to pricing, lower latency, and ease of implementing. For example, if our server was one of our computers, we could really quickly set up a bluetooth network for quick communication and our computers should be powerful enough to calculate the ball projection. If the bluetooth connection between computer and paddle are too slow, we can switch to a different communication protocol or use EC2 which can guarantee single digit millisecond latency.
Our schedule looking forward seems to be on schedule. What we are going to be working on this week is data transfer and collecting data from sensors. We should also start implementation of basic graphics for our MVP.
Henry’s Status Report for 2/19
This week I have finalized our team’s network communication implementation for our MVP. Looking into bluetooth communications and Unity plugins for that, I decided that the latencies for paddle-headset and paddle-server communications are pretty unnoticeable. I did some research into some helpful libraries and found that the Arduino Bluetooth Plugin from the Unity Asset Store is going to help make data transfers easier between headset and paddle and server to paddle since it provides fast bluetooth communications between rPis and Androids/iOS/Mac devices. For camera-headset communications, there was a hitch because the camera sends paddle position data through a Jetson Nano, which doesn’t have built in bluetooth capabilities. My solution to this was to buy a wifi card and antennae that will give the Jetson bluetooth capabilities. I don’t think the plugin is compatible with the Jetson, so I will probably need to program the network communication from camera-headset by myself.
I would say that I am a little bit behind schedule because I need to still set up my Google Cardboard with the Unity Game Engine before I can implement the code for the network communication. However, I think that I should be able to get caught up and back on schedule within the next two weeks since I project creating the MVP graphics shouldn’t take too long. Next week, I want to be able to send data from an rPi to an Android and a Jetson to an Android. If I have time, it would be great if I could get a quick start on the graphics.
Logan’s Status Report for 2/12
This week I worked on tinkering with OpenCV to get oriented with the computer vision aspect of our project. Specifically, tutorials on the core functionality, image processing, camera calibration, and 3D reconstruction along with some other documentation.
Next week I plan to get some initial metrics on the accuracy and latency of the software. Ideally, we will have our cameras for this but I should be able to do some cursory testing regardless. This will also give us an early chance to develop workaround or solutions for high latency which is a major concern at the moment.
William’s Status Report for 2/12
This week, I completed researching the possible technologies that our team could use for tracking our paddle using Inertial Measurement Units. My conclusion was that we would use the 6-degree-of-freedom MPU6050 IMU connected to a Raspberry Pi Zero Wireless through I2C. I also delved into researching ways to bring up this sensing system, and found several useful sites about how to interface the sensor with the Raspberry Pi, including how to set up the I2C connection, how to write a driver for the IMU on the Raspberry Pi, and also how we can write and run our own software on the Raspberry Pi to process the data from the sensor. I also researched and found existing open-source libraries that run the Madgewick Algorithm, which is a sensor fusion algorithm that will us interpret the sensor data relative to Earth’s axes. Lastly, I also researched potential servers we could use as a game server, and found that we could potentially leverage Amazon Web Service’s Gamelift product, which provides a low-latency game server SDK for Unity that we can integrate into our game. I discussed a plan with our team on how we can attempt to optimize our communications between our sensing system, VR device, and server to minimize latency.
As of now, I think I am perfectly on schedule, as we have ordered our parts and I have good preparation on how I intend to bring up the system. Moving forward into next week, my goal is to begin integrating the IMU with the Raspberry Pi. I hope to figure out how to physically connect the IMU to the Raspberry Pi, and subsequently write a driver on the RPi for the IMU so that we can read the raw data from the IMU into the Raspberry Pi in the next week, so that from there, we can begin figuring out how to process that raw data. This is contingent on our IMU sensor arriving, so hopefully, it will be here by next week. If it doesn’t arrive, I should still be able to get my hands on the Raspberry Pi as we have requested it from CMU inventory, and thus, at the very least, I can try to set up an I2C driver on that.