Category: William Shaw’s Status Reports

This week, the rest of the critical parts arrived. As such, I was able to move into the testing phase for the components. Since I am still in the earlier stages of testing and integrating the components, my focus has been primarily on the setup process and ensuring basic connectivity. First, for the camera (IMX219), it is connected via CSI-2, so the system detects it under “/dev/video*; v4l2-ctl –list-devices”, instead of with “lsusb”. I then made sure that v4l-utils and gstreamer were installed and updated to interact with the camera. More testing needs to be done to actually access the video feed, but the device is being detected. Second, for the IMU (BNO055), it communicates over I2C. As such, I use the command “i2cdetect” to check that the module is detected on the I2C bus of the Jetson Orin Nano. Next, I will use the smbus python library to read the raw sensor data. 

I also worked on configuring the Jetson Orin Nano for headless operation, ensuring that we can all access and interact with the system without needing an external monitor, keyboard, and mouse. Headless will be the operation mode for the project’s final phase, as we will not be able to have a monitor attached to the user. This is done through SSH’ing on my laptop. I also set up VNC (Virtual Network Computing) to get a visual remote desktop. Initially, I was experimenting with using Vino, but that has varying performance depending on the exact Jetson device being used. As such, I ended up swapping to x11vnc. This can be connected to using the built-in VNC client on a MacBook (“Screen Sharing” app). Separately, there were some initial issues with getting the wifi to work properly (due to some user privilege issues), but they have been resolved. 

Regarding the schedule, I am about a week behind the Gantt chart. This is because I did not consider that Spring Break was my “Week 5”, so I misjudged the actual dates. I plan to finish testing of each component by this week to get back on schedule. I also want to double check that the Jetson Orin Nano works on CMU-Secure/Device, as I have just been testing on my home network.

This week, I ordered the other essential parts. This included the GPS module and a USB-C to DC barrel jack adaptor. Neither of these are technically system-critical as of right now, but ordering them together saved on shipping costs from Adafruit. I opted to wait on ordering the USB Sound Card, as the driver used for audio output should not change, and we may opt to use wireless bone conductive earphones instead of the on-ear headphones. This could be a safer alternative, as they block the least ambient noise.

Regarding the Jetson Orin Nano, I spent this week setting it up in preparation for future tasks. This included updating the board’s firmware, loading a new boot image with JetPack SDK, and setting up Ubuntu. After completing these preliminary steps, I moved on to installing the dependencies we would need for future tasks. Many of these were included in the JetPack SDK, so it took less effort than expected. I also began trying to run a few demos like Ollama on the Jetson. Ideally, this makes me more familiar with the platform, which should make later work smoother.

In terms of schedule, I am right on track. The parts should arrive in a few days, which is on schedule for me to begin testing. Next week, I plan to complete much of the testing for interfacing the hardware to the Jetson. I will focus on the camera and the IMU first, as these are our most system-critical components. I also want to begin drafting our overall mounting mechanism.

This week, I primarily focused on ensuring that all our parts were in order. This was a much deeper dive into each hardware component than before, and I checked for things like compatibility, interfacing, and outputs. This caused me to revise a few of the prior hardware choices and realize that more needed to be added. In particular, the GPS module we planned to use did not give accurate heading data for stationary subjects. As such, I added a new IMU module to act as a compass, the Adafruit 9-DOF IMU Fusion Breakout BNO055. The module allows us to get accurate user heading without movement up to ±2°, while automatically compensating for head-tilt and sensor noise. 

Another update regarding parts is the audio output and power supply. I had previously thought a Type C Power delivery power bank could power the Jetson board. However, for the Jetson Orin Nano Development Kit, the Type C port is data only and does not power the board. As such, I am looking into alternative power supplies/options for when we make the system portable. Additionally, the board does not come with a 3.5mm audio jack. While there is audio over the HDMI port, that is not a viable solution since we will not connect the board to a display. As such, I need to find a compatible USB sound card for the board. 

So far, I have the Jetson Orin Nano on hand. The Arducam IMX219 (camera) and BNO055 IMU (Compass) are being shipped. These are necessary for us to begin testing the navigation system of our project, so we should be able to start testing actual inputs when they arrive (assuming that interfacing goes smoothly). There are a few remaining components to order (speakers, soundcard, portable power supply), but they are not system-critical for the work that needs to be done so far. I plan to order these components by the following weekly report. I am on schedule so far. By next week, I hope to have placed orders for all the components. I also aim to successfully interface the IMU and camera to the Jetson Orin Nano board. 

This week, I spent most of my time looking into hardware components for our project. More specifically, I looked into the camera, GPS/Compass, and battery components. I also began drafting plans for mounting these components cleanly and comfortably. Currently, I plan to mount the camera to the helmet with a GoPro mount. This would let us adjust the camera angle based on testing results. Furthermore, I would like to create a case for the Jetson board and its components using laser cutting or 3D printing. If it is light enough, the board and battery can be mounted to the helmet, but otherwise, we may need to use a hip-mounted pack for comfort.

I also placed the order form for the Nvidia Jetson Orin Nano. I noticed a 4GB variant (non-Orin) was also available in the existing ready-to-use stock, but I am unsure if it is sufficient to run our models.

I am on track for our Gantt Chart Schedule. Although I have not placed all the orders for the other parts yet, testing of the code can begin once we receive our Jetson board. I would like to get this before placing orders for some of the parts (like the camera), so we can test the interfacing available. I also want to check how well it runs the model, as the top contender for the camera (Arducam IMX219) outputs a 3280 x 2464 pixel image. Although the FOV seems promising, the resolution might be too high to run our model at a suitable refresh rate.

For next week, I hope to finalize all of the parts and place the orders! However, I will emphasize that I want to ensure the components work first, which may require testing on the Jetson board beforehand.