Category: Weekly Status Reports

This week I made more progress on the frontend components. I successfully implemented the user auth system, which now allow users to register new accounts and login securely. This includes all the necessary validation, password hashing, and session management. The login and registration pages are now fully functional.

I also integrated the Google Maps API with our vision model. This allows us to extract the latitude and longitude metadata from the captured images and uses the Google Maps Places API to identify nearby points of interest. The system can now automatically generate captions that include recognizable location names instead of just coordinates or addresses to help make the diary entries much more readable for users. As a result, I refined the daily summary view to incorporate these location names into the timeline visualization. When a user views their daily log, they can now see their chronological list of visits with the actual place names.

I am currently on schedule with the web app development. The auth system was completed earlier than expected and the Google Maps API integration was a planned feature that I was able to implement without major obstacles.

Next week, I plan on completed the settings interface that allows users to configure the capture prefs and continue enhancing/adding features to the front end.

As Will and Dhurv make progress with the Blues Notecard and GPS functionality. I’ll prepare to integrate the location data coming from the actual Arducam images.

Following the team’s decision to shift away from AWS, I refactored the Django backend to use SQLite for local database storage instead of the previously planned AWS RDS. I also modified the image storage system to use the local filesystem rather than S3 buckets, implementing a directory structure that organizes images by user ID and capture date.

Additionally, I worked on implementing the server-side components that will receive and process the image data from the Raspberry Pi Pico. Since Dhruv successfully established serial communication between the laptop and the Pico, I created the necessary endpoints to receive this data and integrated it with our local storage solution. I built a simple webhook listener that can accept the camera captures and trigger the necessary processing.

I also made significant progress on the user interface components. I completed the implementation of the calendar view with proper highlighting for days that contain captures, and refined the daily summary view to display images in chronological order. I added location visualization using a local map implementation that doesn’t rely on cloud services.

For the summary generation component, I researched and implemented a solution using a locally deployable LLM instead of the OpenAI API. I tested several options and found that a quantized version of Llama-3 can run on our development machines with acceptable performance for generating the diary summaries based on image metadata and location information.

I am currently on schedule with the web application development, despite the pivot away from AWS. The local development approach actually simplified some aspects of our implementation, removing dependencies on cloud configuration and allowing for faster iteration. The core functionality of the web interface is now working in our local environment, which aligns with our timeline.

Next week, I will focus on integrating with the hardware components as they become available. I plan to complete the communication layer that will receive images from the Pico once Dhruv transitions the serial connection to MQTT. Additionally, I’ll try to implement the location data processing system that will work with the GPS coordinates from the Blues Notecard that William is setting up.

Overall, the shift to local development hasn’t impacted our schedule significantly, and I remain confident that we can deliver a functional proof of concept on time.