Posted on

Team Status Report – 3/29/25

General Accomplishments From Team

This week on the hardware side we implemented MQTT-based communication between Raspberry Pi Pico and server. Along with this, we also automated image capture system working with Arducam. Additionally, the Blues Notecard can now send location data via HTTP, but with accuracy issues.

On the software side, we completed the user auth system with login/registration. We also were able to integrate Google Maps API to convert coordinates to place names which greatly enhanced the diary view entries. We are in progress of connecting this with the image pipeline from capture to diary entry generation.

Most Significant Risks That Could Jeopardize Success of the Project

Location Accuracy: Blues Notecard providing unreliable location data and shutting down after two communications. Solution: Ordering replacement GPS module with UART interface.

Integration Challenges: Minimizing with standardized MQTT communication protocol between components.

Testing: Four weeks until final demo so we are prioritizing integration testing early.

Changes made to the existing design of the system

We are planning to replace the Blues Notecard with dedicated GPS module for better accuracy.

We are also switching from serial communication to MQTT broker architecture for improved reliability in location change to capture pipeline.

Posted on

Dhruvs status report 3/29

Accomplishments this week:
This week I made a lot of progress. I worked on the raspberry pi and got the camera working and automated picture taking. We ran into many issues with trying to communicate with the pico and connecting the blues to the pico. Therefore, I decided to set up a Hive broker and use MQTT to send images and receive commands. I also connected this to my existing application and now whenever i am stationary in a new location for 10 seconds, it publishes a capture command to Hive. The pico is subscribed to the command topic and initiates a picture capture. After the image is taken, it publishes it to an image topic. My laptop is subscribed to this topic and downloads the image to a local directory. This is then connected to the backend pipeline to summarize and create a diary. Will and I also got the blue’s notecard set up and can now send us location data based off a command.

Reflection on Schedule:
I am ahead of schedule and have mainly finished all my parts.

Plans for next week:
Since I have done a lot of the early work on my part, I will continue working on the hardware. We will focus on automating location next and then full integration.

Posted on

William’s Status Report for 3/29/2025

Accomplishments This Week:

This week, I continued working on the micro-controller development of the hardware parts. We ran into a few issues with the automation aspect of capturing image to the server receiving the image serially through the Python library PySerial. Thus, we switched into MQTT transmission between the PICO captured image and the local server running on our laptop. The pipeline between the captured image and our local server host now works.

In regards to the location module of the system with the Blues Notecard, it is able to communicate the coordinates between the module and the local server via HTTP. However, the automation has a little bit of issues since it can only communicate twice then the device shuts down for some reason.  In addition, the location seem a little bit far from our expectations. Thus, I requested to replace the module with GPS tracker here.

According to some research, it seems a lot more accurate than the Blues Notecard and can provide real-time configuration of the location. This should be fairly easy to configure via UART, so we plan on trying to switch this component for the location aspect of the project. It should be a pretty easy transition here.

Reflection on Schedule:

I think that considering four weeks left till the final demo, our group is still on track to finish by the final demo. The location module seems intuitive to complete, so we will focus on integration and design for the remaining weeks.

Plans for Next Week:

We will be spending some time on the interim demo. In addition, we will also order another GPS device for our location and integration + location will be our main focus for the week.

Posted on

Rohit’s Status Report – 3/29/2025

This week I made more progress on the frontend components. I successfully implemented the user auth system, which now allow users to register new accounts and login securely. This includes all the necessary validation, password hashing, and session management. The login and registration pages are now fully functional.

I also integrated the Google Maps API with our vision model. This allows us to extract the latitude and longitude metadata from the captured images and uses the Google Maps Places API to identify nearby points of interest. The system can now automatically generate captions that include recognizable location names instead of just coordinates or addresses to help make the diary entries much more readable for users. As a result, I refined the daily summary view to incorporate these location names into the timeline visualization. When a user views their daily log, they can now see their chronological list of visits with the actual place names.

I am currently on schedule with the web app development. The auth system was completed earlier than expected and the Google Maps API integration was a planned feature that I was able to implement without major obstacles.

Next week, I plan on completed the settings interface that allows users to configure the capture prefs and continue enhancing/adding features to the front end.

As Will and Dhurv make progress with the Blues Notecard and GPS functionality. I’ll prepare to integrate the location data coming from the actual Arducam images.

Posted on

Rohit’s Status Report – 3/22/2025

Following the team’s decision to shift away from AWS, I refactored the Django backend to use SQLite for local database storage instead of the previously planned AWS RDS. I also modified the image storage system to use the local filesystem rather than S3 buckets, implementing a directory structure that organizes images by user ID and capture date.

Additionally, I worked on implementing the server-side components that will receive and process the image data from the Raspberry Pi Pico. Since Dhruv successfully established serial communication between the laptop and the Pico, I created the necessary endpoints to receive this data and integrated it with our local storage solution. I built a simple webhook listener that can accept the camera captures and trigger the necessary processing.

I also made significant progress on the user interface components. I completed the implementation of the calendar view with proper highlighting for days that contain captures, and refined the daily summary view to display images in chronological order. I added location visualization using a local map implementation that doesn’t rely on cloud services.

For the summary generation component, I researched and implemented a solution using a locally deployable LLM instead of the OpenAI API. I tested several options and found that a quantized version of Llama-3 can run on our development machines with acceptable performance for generating the diary summaries based on image metadata and location information.

I am currently on schedule with the web application development, despite the pivot away from AWS. The local development approach actually simplified some aspects of our implementation, removing dependencies on cloud configuration and allowing for faster iteration. The core functionality of the web interface is now working in our local environment, which aligns with our timeline.

Next week, I will focus on integrating with the hardware components as they become available. I plan to complete the communication layer that will receive images from the Pico once Dhruv transitions the serial connection to MQTT. Additionally, I’ll try to implement the location data processing system that will work with the GPS coordinates from the Blues Notecard that William is setting up.

Overall, the shift to local development hasn’t impacted our schedule significantly, and I remain confident that we can deliver a functional proof of concept on time.

Posted on

status report march 22nd – dhruv

Accomplishments this week:
This week I did a lot of work on the hardware software integration of our project. First, I worked on being able to send a signal to the raspberry pi pico from my laptop. I was able to do this through a serial connection. In the future, I want to be able to do this through MQTT. Then, William and I were able to work on the arducam. Will was able to connect the arducam to the raspberry pi pico. Then we worked together and read through the docs and were able to capture an image through the arducam. Overall, good progress was made!

Reflection on Schedule:
I am ahead of schedule now and have finished a large portion of the early developments of my part for the software back-end and begun working on the hardware software integration.

Plans for next week:
Since I have done a lot of the early work on my part, I will continue working on the hardware. I will be trying to set up MQTT and also sending GPS coordinates.

Posted on

William’s Status Report for 3/22/2025

Accomplishments This Week:

This week, I continued focusing on the microcontroller development. I shifted focus working on configuring the Arducam with GPIO pins into the Raspberry Pi PICO as well as flashing the microcontroller. After Dhruv finished up his work on the signalling between RPI Pico and laptop, he and I worked through the documentation of Arducam and was able to finally successfully capture an image. This is good progress on the hardware side as now the Arducam works on top of the  RPI being able to connect to the wifi successfully.

Reflection on Schedule:

I am currently on schedule now that the Arducam is working. With shifted focus onto the Blues Notecard next week, I am confident in the completion of the schedule.

Plans for Next Week:

I will continue setting up the Blues Notecard for GPS handling of the device as well as refining the picture quality and distance of the captured image on the Arducam. Currently the image seems a little bit zoomed in, so I will work on trying to allow the image to zoom out a bit. Hopefully, the Blues Notecard is completed by the end of next week without any complications.

Posted on

Team Status Report 3/22/2025

General Accomplishments From Team

The general accomplishment from the team this week includes both the software and hardware end.

Dhruv successfully sent the communication from laptop into PICO through serial connection. He will transition this signal through MQTT which should be smooth since the Pico is successfully connected to the internet on the microcontroller end.

On the microcontroller development, Dhruv and I (Will) were able to successfully capture the image.

I will be spending more time on optimizing the image and we will be focusing on the Blues Notecard this week.

Rohit will be continuing to work on the web development side of things.

Most Significant Risks That Could Jeopardize Success of the Project

The most significant risks that remains include the Blues Notecard connectivity. We will be continuing in focusing on the Blues Notecard development his week.

The integration between hardware and software seems resolved as Dhruv was able to send signal from software to hardware this week.

Changes made to the existing design of the system

Since we did not get approved for reimbursement on AWS, we will be shifting towards completing this project locally as a proof of concept.

Our hardware will still be a wireless connection to the laptop, however, it needs to be under the same wifi network/mobile hotspot we will be using.

We do not see foresee any changes in the schedule of completing this project by shifting this project locally as we can develop our LLMs locally.

Anyhow, good progress is made this week and we still remain confident in a successful completion of the project.

 

 

 

 

Posted on

Team Status Report for 3/15/2025

Will successfully resolved the Wi-Fi connection reliability issues with the Raspberry Pi Pico W, which was one of our major technical obstacles from last week. This achievement establishes a stable foundation for our hardware communication. While the Blues Notecard still has some connection challenges, progress is being made and a resolution is expected soon.

Dhruv made progress on the software backend, completing the location change detection program and implementing a simulation environment to demonstrate the system’s functionality. His work includes a map interface with user movement simulation that triggers image capture when the simulated user becomes stationary. He’s also integrated a vision model for image analysis and connected it to the OpenAI API for summary generation.

Rohit completed the implementation of the calendar view interface in Django and the API endpoints needed for handling image and location data from the hardware components. He’s also developed a working prototype of the daily summary generator using the OpenAI API, which aligns with Dhruv’s simulation work.

The most significant risks currently facing the project are:

  1. Blues Notecard Connectivity: While the Raspberry Pi Pico W connection has been stabilized, the Blues Notecard is still experiencing some connectivity issues with our hotspot. This risk is being managed by Will’s troubleshooting.
  2. Integration Between Hardware and Software Components: As we move towards integrating the hardware (Raspberry Pi Pico W, Blues Notecard, Arducam) with the software (location detection, image processing, web interface), there’s a risk of compatibility issues. We’re mitigating this by establishing clear API contracts between components and developing modular code that can be adapted if requirements change. Dhruv’s simulation environment also helps us test the software side independently of hardware completion.There have been no significant changes to the overall system design this week. The core architecture, consisting of wearable hardware for image capture, cloud infrastructure for storage and processing, and a web application for user interaction remains the same.

For the coming week, our focus will shift to:

  1. Will will continue working on the Blues Notecard configuration and then transition to setting up the Arducam.
  2. Dhruv, having made significant progress on the software backend, will assist William with the Arducam setup to accelerate hardware development.
  3. Rohit will focus on finalizing the user authentication system, implementing the settings interface, and preparing for integration testing with the hardware components.
Posted on

William’s Status Report for 3/15/2025

This week, I continued working on the script/microcontroller code with the Raspberry Pi Pico W. Last week, there were still some reliability issues with maintaining the connection. However, after adding some additional configuration code to the microcontroller, it appears that the connection is now stable, which is the main improvement from last week.

After fixing the wifi connection with the RPI Pico, I am still in progress with the configuration process of the Blues Notecard as it is also having some connecting issues with our hotspot, but I am close to getting the configuration of Blues Notecard completed.

I am still slightly behind schedule. However,  the Blues Notecard is close to being done, and I will almost next focus on the Arducam. Since Dhruv is close to being done on the software end, he will jump in to help me with the Arducam next week as well to speed up the technical development progress.