Author: janetli

This week, I completed writing and implementing the Schedule Learning feature for our web application. The schedule learning feature makes use of two new models: a TimeItemListPair and an ItemCounter.

Multiple TimeItemListPairs essentially function as a dictionary, with each pair tied to a user and a timestamp with a list of items currently inside the backpack at that specified timestamp. An ItemCounter exists for each item tied to a specific event (e.g. there could be multiple ItemCounters for the same item, “Folder,” if Folder is assigned to more than one event) and keeps track of how many times this item, for this event, has been found in a TimeItemListPair.

For the assignment logic, our system runs through all the events for a user that are marked as “Schedule Learning” events. If a TimeItemListPair exists such that its timestamp falls within the window of an event (i.e. between the start time and end time of an event), the counter for each item in that Pair will increment by one. Finally, assignment continuously happens, assigning items with counts higher than 85% of all counts to the corresponding event.

Although this feature is not completely fleshed out, schedule learning is not included as part of our MVP, and is simply meant to help decrease the user burden of manually assigning items to each event.

Additionally, I completed an extra feature to add a “Register” button to unnamed items inside a user’s backpack for streamlined registration. This feature can be viewed here.

This past week, I also worked with my teammates on the Final Presentation slides.

Our progress is currently on schedule.

In the next week, our remaining focus will be on completing deliverables for the end of this project course, including the final video, poster, and report.

This week, I completed the features for missing item notifications (i.e., an event is coming up but at least one of the required items for the event is not currently in the backpack) as well as lost item notifications (i.e., the item list at the beginning of an event is not consistent with the item list at the end of an event). A demo of this feature can be found here [with no missing items] and [with missing items]. Two new functions were written for these new features, which can be viewed below:

Additionally, since our team members are all working to finish the remainder of our project, I picked up the schedule learning feature task for our web application. Currently, I’ve collected sample data for what our web application may need for this feature, which consists of large dictionaries with timestamps as keys and an item list snapshot of what items are inside the user’s backpack at that timestamp. This data would then be continually compared against events stored in the database and items would be automatically assigned to the user’s events based on this data. However, this feature is not a requirement of our MVP, and a fully fleshed-out version of this feature would require persistence (thin native client + webview) to be completed first. I’ve also started working with my teammates on our final presentation.

Currently, our schedule is a little bit behind. The components left involve:

1. Persistence (implementation of thin native client in Android + WebView)
2. Schedule Learning (not part of our MVP, but dependent on persistence)
3. Integration with image recognition component (currently depending on Joon to set up an API endpoint for the model)

Save for the integration with the image recognition component, our MVP is already completed. However, implementing features like persistence and schedule learning would greatly increase the cases in which our product can be most useful as well.

To catch up to the project schedule, we’ll be using this final week of development (week of 5/2) to complete the remaining components. Aaron and I will work on Persistence, I will work on Schedule Learning, and Joon will work on setting up an API endpoint for the model. These are the deliverables we hope to complete in the next week.

This week, I completed the removal functionality for items and events as well as web push notifications that get sent once the notification deadline for an event has arrived. I used the Django webpush package to send web push notifications to the user and had to register a service worker to enable this functionality. Additionally, I had to add another attribute for the Event model, “notified,” so that our checkNotifs function doesn’t repeatedly send notifications for events more than once, and created a Notification model to use for debugging and logging. This feature can be viewed here, and the logic for the checkNotifs function can be seen below:

I also worked with Aaron to test the delay between an action (adding or removing an item to/from the backpack) and the RPi receiving the item as well as the delay between the RPi and the web app interface updating. We found that the time fluctuates quite a bit between the action and the RPi, but once an RPi receives an item, the interface update is typically under a second.

We have had to push back on integrating Joon’s image recognition module with the web app until Joon finished migrating the model to AWS, so this aspect of our progress is behind schedule. We are also a little behind schedule on the schedule learning feature as well as Bluetooth persistence, though we’ve planned the bulk of these tasks for the upcoming week rather than this previous week. To catch up to our project schedule, we’ll likely focus solely on these two tasks for the upcoming week. Thankfully, we’ve also planned for a little slack time in our final planned week of work (week of 4/25), so we can also use that time to wrap up any leftover tasks. At this point, all my individual tasks for our project have been completed, and the remaining tasks all involve some form of integration or collaboration to complete.

In the next week, I hope to complete usability testing of our web app, the schedule learning algorithm, and Bluetooth persistence with Aaron. I also need to integrate Joon’s image recognition module into our web app’s registration process.

This week, I worked with Aaron to complete the integration with the RPi Zero with our web app and also continued to work on the onboarding and item addition on our web app. Since Joon’s timeline on the image recognition module will likely not allow for integration before the interim demo, I have decided to move ahead with the manual naming for items in our item registration process. I also spent time researching Bluetooth persistence for our web app, and currently, we may need to implement our mitigation plan of a bare native Android application with our web application included as a WebView. This is troublesome for some features we may want to implement, such as tracking lost items (e.g. “You left your notebook at Study Session”) or simply add too much to the user burden (if the user has to consistently open the app), so I’ve allotted some time to begin working on this during the coming weeks.

A bare-bones demo of our item registration process currently looks as so, and a demo of the interface update with our BLE tags can be viewed here.

Our progress is currently on schedule.

In the next week, deliverables I hope to complete include integration with Joon’s image recognition module (after the interim demo). I also plan to begin working on the mobile web push notifications for reminders/missing items as well as deletion of items on the web app, and Aaron and I will work together on automatic registration of tag MAC addresses.

This week, I completed the scheduling feature for our web application. Events can now be manually created via a Django Form, stored as an Event model, and displayed in a calendar view. Events can also be edited retrospectively. Additionally, the home page contains a view of upcoming events from these models. Much guidance was received from this blog post. A short demo of the scheduling feature on our app can be viewed here (video is too long to be directly included in post). Additionally, the login and registration functionality is complete (barring the integration with our item recognition module).

Additionally, I began development on the onboarding feature of the web app (i.e. when a new user first registers, they must tag all their items and register these tags). However, since this step cannot be completed until Joon completes the item recognition module, dummy items and information are currently being used. I also began development on the integration of the RPi Zero with the web app using the Web Bluetooth API.

Progress is currently exactly on schedule, as written in our Gantt Chart.

In the next week, per our Gantt Chart, deliverables I hope to complete the final integration between the RPi Zero and our web app as well as the integration between the item recognition and the onboarding on our web app. Additionally, we will all be preparing for the interim demo on the week of 4/11.

This week, I worked on setting up our web application in Django, building layouts for the pages, creating models in the database for our user Profiles, Items, and Events, and setting up the infrastructure for connection to the RPi Zero. All of routing and navigation is completed, and the two main areas of focus left on the web app are the scheduling and the connection with the RPi Zero via the Web Bluetooth API. Our new repository is now set up here.

We are currently using native Django models for our web application to meet our MVP goal for the interim demo, but plan to use databases on AWS before the final demo for scalability.

Since the last status update, our team has agreed on developing a web app rather than a native Android app, for the advantages stated before:

1. Ease of development between the three team members
2. Product no longer excludes users without Android phones
3. Accessibility via other platforms, such as laptops and tablets

Since our current focus is preparing the MVP of our project for the interim demo (week of 4/11), we wanted a route that would help optimize our development time as quickly as possible. Additionally, any concerns of loss of functionality in a web app vs. a native app can be easily mitigated. Our current plan is to create the web app with full functionality, however, if we run into issues with persistency or Bluetooth tasks, one mitigation we’ve planned is to create a bare native Android app (in Android Studio with Kotlin, as previously planned) to handle native functionality, but using the interface of our web app through a WebView, which allows you to deliver web applications as part of a client application.

While developing, I also ran into the question of how our project will be used in actual use cases, specifically with the RPi Zero. Currently, multiple users can create accounts on the web application, but this requires our web application to assume that each user has their own backpack device (aka the RPi Zero).  However, our team is currently only testing with one RPi Zero (and this will be reflected in the demos as well). I decided to include the UUID field of an RPi Zero as a required attribute for each user Profile on our web application, as this would be required if Backpack Buddy were a commercial product.

My progress is currently on schedule.

In the next week, I hope to have the interface completed for the schedule creation and the onboarding (registration) process implemented, in preparation for integration with Joon’s component (item recognition). I’ll also be working together with Aaron on integrating the RPi Zero with the web app’s interface.

This week, I gave the Design Review presentation and also worked with my team on our Design Report. I also created an entity-relationship diagram for the database of our phone app.

In the diagram, we have an intersection table, EventItems, that keeps track of two foreign keys: EventIDs, and ItemIDs. This is because each event has a subset of all the tagged items in our database, and intersection tables are useful when modeling many-to-many relationships, as in this case. I also created the basic layouts and navigation logic for our phone app in Kotlin this week. However, I’ve been running into constant issues with Git integration for Android Studio, as well as common build errors with Android Studio, and these issues may present a significant risk to the project (as discussed further in the team status report). I’ll be preparing to create a mobile-accessible web app version of our phone app, should we run into large integration issues in the upcoming weeks. Three reasons why switching to a web app may be advantageous to us include:

1. Ease of development between the three team members
2. Product no longer excludes users without Android phones
3. Accessibility via other platforms, such as laptops and tablets

Additionally, we could use the Web Bluetooth API to communicate with other Bluetooth devices over JavaScript. I did some research and found that this API would perfectly fit our needs, as it allows you to request and connect to nearby BLE devices, read/write Bluetooth characteristics & descriptors, and receive GATT notifications and when devices get disconnected.

Currently, our progress is on schedule, but I’ll be running some basic tests to see how accessible on phones a web app would be for our project. I’ll also be working with my team members mostly on the design report that’s due this upcoming week.

In the next week, I hope to have completed the design report with my team members and have done some research on using a web app instead of a phone app for our needs. Luckily, there’s quite a bit of slack time built into our schedule for the phone app before further development is needed (after integration with the BLE tags + RPi), so I’ll be able to use this time to set up risk mitigations.