Posted on

Status Report – Week 1 (Rohit)

This week, I focused on researching and planning the web application architecture and database design.

Frontend Research and Planning:
I primarily evaluated Django’s server-rendered HTML/CSS approach for the web interface, researched responsive design patterns to ensure 30-second page load time, started planning the calendar-based interface for daily entries, and began sketching some initial UI wire frames for the daily summary view.

Database Architecture:
Regarding the database, I investigated a schema design for storing the user diary entries, associating images with timestamps, location metadata from Google Maps API, and LLM-generated summaries. I also researched heavily on efficient storage mechanisms and retrieval patterns for image data using AWS S3.

LLM Integration Planning:
I explored AWS Rekognition API capabilities for image processing and started to work on an initial design for the data pipeline for processing daily images and generating summaries. I also looked into approaches for combining images analysis with location data for better context in the summaries.

The main challenge that I am anticipating from my research this week is optimizing the database queries to meet our performance requirements, especially when fetching and displaying multiple images and summaries for a single day’s view. I anticipate we will need to implement some kind of efficient caching system to address.

Posted on

Status Report – Week 1 (Dhruv)

This week, I did a lot of work on the slides and presentation. I spent many hours researching the design of our product to figure out what devices to use. In particular, I spent time researching how to have constant connectivity to the cloud to have continuous image transmission. I also spent time refining the requirements for our hardware and software.

My role on our team is on the software side. Therefore, I also spent time researching how the location would work and designing the system for getting images processed. In particular, I decided that we would like to use something like AWS S3 to store compressed images daily. When the user wants their summary, a function will fetch all files under the S3 folder for the current day and feed it into a multimodal LLM to handle summarization. We will also tag each image with a timestamp and google maps location to get an accurate summarization of the day!

Other than working on the slides, I began to brainstorm on how exactly to attack this problem. We also began thinking of what orders to put in for our device.

Posted on

Team Status Report for 2/8/2025

We believe that the most significant risk that could jeopardize the success of the project would be integration of the software and hardware end. We foresee potential compatibility issue with RPI Pico W’s communication with the cloud using blues notecard. The risk here is being managed by starting the hardware implementation side as soon as possible to test whether cloud connectivity is possible. Blues notecard is very well-documented with how to connect with RPI Pico W so William will deep dive into the documentation in Week 5. If the blues notecard does not work, we may alternatively change the hardware of the RPI Pico W into one that has built-in wifi capability. Blues notecard seemed more reasonable as our first choice due to its strong documentation with wifi connectivity and location data. Thus, using Blues notecard reduces the complexity on the hardware end.

Another important risk mentioned in William’s status report is the power consumption of the components on the glasses. We are hopeful that the selected 2000 mAH battery will be sufficient to continuously power all components on the glasses based off of William’s research of the consumption of the Blues Notecard and Arducam. However, complications may arise, and we may need to purchase a higher mAH battery to power the glasses.

A third significant risk may be the 2 MP arducam may not be good enough quality. William chose this camera due to its lower power consumption and strong resolution at 1600 x 1200. A contingency plan of this risk may be to upgrade our arducam to 5 MP with an image resolution of 2592 × 1944. However, due to the tradeoff of lower processing speed and higher power consumption, we are leaning towards 2 MP being “good enough” for our LLM analysis on the software end.

There has been no changes to the existing design of the system. We have been in the research stage and now we are focusing on the implementation stage next week. Currently, we don’t have any photos to display our progress, but next week William will start the RPI Pico W startup process while Dhruv and Rohit will work on the software end and have a UI ready via Figma or hand-drawing. Dhruv will additionally assist William in the connectivity/software aspect of the Pico’s connection to the cloud. We hope to implement and test this aspect ASAP because this is a key feature our design relies upon.

Posted on

William Wang’s Status Report for 2/8/2025

This week, I was the presenter of the proposal and heavily focused on flushing out the details and requirements of our system design. In addition to rehearsing and preparing the proposal presentation, I explored various options of network connectivity of RPI Pico W to the clouds as well as flushing out the component details:

Based off of initial research of product documentations, I drew out this graph to specify how each component will communicate with each other. The Blues Notecard will be communicating with the Raspberry Pi Pico over UART protocol and the Arducam will communicate with the Raspberry Pi Pico over SPI protocol. Since I am responsible on the hardware/embedded side of this project, I finalized the decision on the hardware end to use a Raspberry Pi Pico W rather than a Jetson Nano due to the ease of development with MicroPython library as well as the significantly smaller size and weight of the hardware. The size of the hardware is an important concern since we need to have all the components fit on the glasses and be wearable to the user.

We were initially debating whether to use a NVIDIA Jetson Nano or a Raspberry Pi Pico. However, after some research of online and previous working projects, I noticed that the Raspberry Pi Pico has a well-documented library MicroPython that could well-integrate with Arducam and the Blues Notecard. Both these specified components are specifically compatible to Raspberry Pi Pico. Thus, these hardware design choices mainly lied upon system compatibility and ease of software development with MicroPython.

In addition to the system compatibility, I also spent a lot of energy researching the component’s power usage. After conducting some calculations and research, I selected the 2000 mAh LiPo battery power the RPI Pico W, Blues Notecard, and Arducam for at least 8 hours in the day. Implementation complications with power consumption may arise; thus, I plan on starting the hardware implementation aspect of our project ASAP to validate my calculations as we are passing the research+design phase of the project.

My progress is currently on schedule. By next week, I will start learning how to set up the Raspberry Pi Pico. I currently don’t have any RPI Pico development experience. However, I have 18-349 background implementing SPI as well as UART communication. Thus, I plan on going back on my previous work/lecture notes to get a starting understanding of the implementation of these components.

By next week, I will focus on two deliverables

1: starting up Raspberry Pi Pico W

2: Orders list for the components of our system designs

In addition to working on these deliverables, I will be focusing on reading into the documentation of Arducam and Blues Notecard to make technical progress in that space next week.