Posted on

Steven’s Status Report for 4/24

This week I worked on testing and validating our system in preparation for our final deliverables. I collected data on the hologram’s RPM, FPS, and latency and compared these values with our proposal goals to identify if we met the targets we set for ourself/if we needed to. I did this by recording videos of our hologram at 30 fps, I analyzed each frame and calculated the amount of time it took for user input to be registered on the screen. Once I had collected the necessary data, I began compiling and analyzing the data for our final presentation. I created the final presentation with andrew and kendric, and practiced the presentation extensively before our final presentation on Monday.

I dont believe we have any significant roadblocks in our way between now and our final demo. We are now just working on polishing our project. The biggest task we have left is to complete our final deliverables like our poster, video, and final paper before our demo.

We haven’t made any changes to our schedule and are on track to finish on time.

 

 

Posted on

Andrew’s Status Report for 4/24

After some bug fixes last week, our project is generally in a state that I think is ready to demo. At this point, I think we’ve kind of hit MVP and any further changes are just for additional content to demo. On the software side, I’m doing some further debugging to isolate latency. I think the only place we can feasibly get any decent gains is from optimizing the computer vision pipeline, as network latency is mostly due to our hardware and seems unfixable.

To fix the vision pipeline I implemented some instrumentation to print out latency of specific operations. Once I identify which operations take longer than expected I will try to further improve them. I also made a hardware fix by borrowing a friend’s webcam. The one we used before is a cheap one that seems to have worse drivers which made it repeatedly disconnect under high load. The new webcam seems to not have this problem.

Over the next week I will probably implement minor changes that might further optimize performance. I might also try to find other interesting applications to display.

Posted on

Team Status Report for 4/24

At this point, our project is pretty demoable and there aren’t any significant risks unless we break something. We are trying to prioritize safety for the demo by limiting speed, which should also mitigate risks of anything breaking. There haven’t been any changes to any designs, and we are still on track for demo day.

Tests:

  • Motor RPM: We tested a safe motor speed by incrementally increasing our target duty cycle and carefully observing any changes. At around maybe 24% duty cycle we observed some slight slippage, which might be attributed to PLA warping. So, we will try to reprint in PETG before demo day. If that isn’t achievable we can simply limit speed to 20% duty cycle, which seems to work fine for extended periods.
  • LED Display Time: This metric was measured because this is the hard physical limit for how fast we can update the display. We measured a mean of 184 microseconds and a maximum of 479 microseconds. This informed our decision to increase the angular resolution from 120 to 240 slices per revolution.
  • End to End Latency: We decided the best way to measure the end to end latency was taking a video and seeing when gestures would be processed. In our video we measured around 500 ms of latency, which is a bit higher than we expected. This finding motivated us to do a bit more instrumentation to identify if there are any places we can optimize the computer vision pipeline.
  • UDP gaps: We measured the difference between successive instances of receiving network information. We got around 280 ms, which informs us a lot about where our latency is likely coming from. Unfortunately, this number seems difficult to change because our setup needs to stay wireless and our hardware can’t really be changed at this stage.
Posted on

Team Status Report for 4/18

No significant risks jeopardize the completion of our project. The only big risk would be if one of our parts decided to spontaneously combust during testing, which is why we’ve decided to use our remaining budget on buying backup parts in case anything was to happen.

We haven’t made any significant design changes, and our project is proceeding smoothly. We are on track to present at demo day.

Posted on

Steven’s Status Report 4/18

This week I worked primarily on getting the final project ready for final documentation. In the next couple of days, we have many deadlines coming up, so I’ve been preparing the final deliverables. I’ve begun brainstorming video ideas, started trying out different poster layouts, and started working on the final presentation slides.

We don’t have any significant setbacks this week. Since our interim demo, things have been going smoothly for the most part. We’ve had some difficulties here and there getting the belt to tension, so we reprinted some parts and readjusted the clamp which has so far solved the slipping issue.

We are currently on schedule and are on pace to complete the project by our final demo.

One of the skills I learned while working on this project is how to read schematics and pcb boards. While working on the project, a couple weeks back, we had an issue with reading from our hall-effect sensor on the adafruit hub adapter. In order to find the root cause, I needed to review the schematics for the adafruit hub adapter and look at the data sheets for each of the components before finally diagnosing that the hub adapter only allowed for one-way communication of data from the pi to the panels, and not from the panels back to the pi. Without learning to read the schematics and datasheets of various components, I’m not sure we would have figured out this bug.

Posted on

Andrew’s Status Report for 4/17

This week I mainly worked on just improving the software and trying to get the display to look better. We’ve slightly increased speed to about 200 rpm, and though this is much under our goal of 900, I think at this speed the display is decent enough to be able to visually distinguish if there are any software bugs, which I worked on fixing.

There were a bunch of bugs and improvements I made all over the place to try and make it look better. Two that likely made the biggest difference were increasing angular resolution and fixing a display slicing bug.

For the angular resolution, we previously displayed 120 different images for every full rotation the displays made. However, I noticed that when we tried to display straight lines near the edge of the display, they would appear warped (since the distance between 2 successive images becomes farther as you move further from the center). I doubled this number so we now display 240 images for every full rotation while also testing to ensure both the Jetson and Pi were fast enough to meet this.

The major display slicing bug I fixed had to do with how we sampled frames to display. The way the logic works is that the Jetson stores the entire 3D volume of data in memory, and picks out a 128×64 2D plane to send over to the Pi to display. Previously, this plane we sampled crossed the center axis (like a diameter). However, this is inaccurate for our real physical display because the LED panels have width, so they are actually on planes that are slightly offset from the center (more like chords). Fixing this hopefully will help when things look like they are displayed completely wrong or warped.

Tools, Knowledge, Learning:

I didn’t really find the need for tools I hadn’t used before, but there were a bunch of knowledge gaps throughout the design and implementation of this project. For the overall idea itself, I used previous work to understand the fundamentals behind design decisions and architecture. For example, I often looked back on a video of another person who built a similar device to look at how he designed specific parts to fit together. For lower level details where there was no previous work to build off of, I used AI tools to help fill in the gaps so I could get instant feedback on if my understanding of concepts was wrong. I mainly focused on asking questions in order to develop a robust design so that implementation was easier.

Posted on

Kendric’s Status Report for 4/4

This week I worked on the new display mount for the new LED panels. Since the last displays broke, we had to buy new ones, but their design was a bit different. Based on the demo feedback, I am planning on printing the new mounts in black instead of white so it blends better with the display and improves the hologram effect.
For verification, I tested the mechanical behavior of the system while it spins. We’ve seen that the whole system has some wobble while it spins and we observed it even before we tested full speed. So we are considering adding weight to the base. I also saw that the pulleys under operation are holding up well and not breaking, especially the one connected to the motor, which was one of the initial concerns. However, after running the system for a while, the belt has become looser, most likely due to stretching. So, we either need to buy replacement belts which is an easy fix, or reposition the motor to increase tension. The first option is most likely the best since the belt would most likely stretch more if we reposition the motor.
Next week, the mount needs to be reprinted and I will address the stability issues more as we push toward full integration and pushing the system to run at higher speeds

Posted on

Andrew’s Status Report for 4/4

After the interim demo, the biggest area for improvement is the refresh rate of our display. We need every piece to be faster for this to happen, so this week I worked on improving the laser cut chassis and some of the software. The chassis we used for the demo had one hole drilled in manually since I forgot to laser cut it, and at higher speeds, the imperfections in the hole caused our tube to rub against wood creating friction and bad sounding noises. I redesigned the laser cut pieces to actually have this hole so we can hopefully reach higher speeds. I also added some holes to hold it better for transportation and some minor improvements to allow for easier assembly.

On the software side, I noticed during the demo at times the display could seem a bit laggy, which motivated me to do more testing. This is likely either due to networking logic or display driving, so I investigated display logic at first (since we also got our full-sized displays which means more data to be displayed).

I created some tests to measure peak performance in terms of how fast we can update both displays, and I originally measured a maximum of 95 fps, which is too low when we are spinning at high speeds. I found that the library we used, rpi-rgb-led-matrix, used a somewhat slow implementation for using pwm to drive the display. To fix this, I replaced the library with custom direct memory access (DMA) so there’s less cpu time spent on driving LED outputs. This vastly improved performance to over 7000 fps, and next week I will work on fully testing to make sure this doesn’t affect any other part of the software.

Other verification will eventually include doing network testing to see if our throughput is enough. If not, we might have to look at improving our wifi hardware or wrapping everything in aluminum foil or something.

Posted on

Team Status Report 4/4

We did not face any significant challenges this week. We successfully pulled off the interim demo and are now preparing for our final demo.

No significant schedule or design changes to our project at this time. Our LED panels have finally arrived and we’re now ironing out how to program the displays.

For full system validation, we’ve been testing while integrating to make sure everything would work well. For example, for power draw we have measured components individually and ensured that they pull around the amount we estimate. Once we have a more final product, we will also take the time to measure framerate and latency to ensure they are within reasonable bounds. We will also make some qualitative observations to see which aspects we will focus on improving.

Posted on

Steven’s Status Report 4/4

This week marked our interim demo, where we completed our first full system integration. Overall, the demo was quite successful, and we received largely positive feedback from both our instructors and TAs. While the LED display has not yet achieved 3D image rendering and we are not operating at our target RPM, we were still able to present a strong proof of concept. I was honestly surprised by how much we were able to accomplish in time. Andrew and Kendric both put in significant effort to make the interim demo a success.

One major challenge I overcame prior to the demo was getting the hall-effect sensor to work with the Adafruit HUB adapter. Because the adapter occupies all of the Raspberry Pi’s GPIO pins, we needed to route the hall-effect sensor through the HUB itself. Although the adapter supports up to three LED panels and we are only using two, leaving some pins physically available, we initially overlooked an important detail: data direction. LED panels are designed to receive data, while hall-effect sensors transmit data. As a result, when we first connected the sensor to the available GPIO pins, we were unable to read any signal.

After further debugging, we discovered that the Adafruit HUB adapter only supports unidirectional data flow on most pins, with the exception of the SDA and SCL lines. We then rerouted the hall-effect sensor to these pins, allowing proper data transmission back to the Raspberry Pi. Fortunately, this resolved the issue, and we were able to demonstrate a fully functioning hall-effect sensor during the interim demo.

After having a working project for the interim demo, I’m much more confident about where we are in our progress. I feel that we are now back on schedule and in good shape for the final demo.

Next week, I’m planning on working on the Raspberry Pi software to drive images to display as well as preparing final demo logistics.