Keaton’s Status Report 4/16

This week, I mostly worked on doing integration, specifically handling a fair number of edge cases with regards to unregistered items. This consisted mostly of work in the Django code, and a few minor updates to the CV code. Both Harry and Jay helped me quite a bit as I didn’t have much experience with Django prior.

The CV code previously just gave it’s best guess out of the approved iconic image classes, or none if it wasn’t sure. The CV code was changed to accept a set of items in the current inventory. This is used in the event that an item was removed, so we don’t have to check every possibility, which should help with accuracy. It also deals with the edge case where the cv code says that an item was removed, that it didn’t say was in the cabinet in the first place. The CV code was also modified to accept arbitrary sets of additional Iconic images, to support user item registration. Both of these changes were relatively minor, and mostly just involved filtering/adding to the set of iconic image checked against.

The majority of the code work was done in Django, dealing with user registered items. Essentially, we needed to be able to be able to add unidentified items to the cabinets, and later change their category when the user identified them. We also needed to be able to handle them, in the case that the user decided to place the items and removed them, all without manually identifying them. Long story short this was resolved by creating a Iconic Image model item, with an optional associated item field, that was assigned a UNKNOWN category be default. Essentially, we then treat it as an a regular iconic image, until the user manually identifies it, at which point, we can propagate the identified category to the associated item object. We can also easily generate the prompts the user by filtering iconic models by category/created user. This handles most of the edge cases, and doesn’t have a huge drawback unless the user has huge amounts of unregistered items in their cabinets concurrently, which is unlikely. We still need to do some UI/CSS work here (the alert/request to identify items needs to be placed in a few different places and made much more apparent), but we have the needed functionality. There was also some minor technical debt that I had to fix regards to categories (IE we were making every item have a unique category. instead of re-using them).

Overall, for not knowing Django very well before this week, I’d say I did a pretty good job. Again, I obviously had a large amount of help from Harry and Jay, but I think I learned fairly quickly and got a good amount done. For the next week, I plan to almost exclusively work on the CV component, and try and get it within our needed criteria, and get the testing done so we have it for the slides.

Team Status Report 4/16

This week has mostly been integration week, which has been overall successful. Highlights include:

  • Full integration of web app with hardware device
    • Fully working communication between RPI and web app
    • Support for users identifying images/adding new Iconic classes, based on cropped images sent from RPI && support for arbitrary categories
  • Slight improvements on the CV side
    • Can now take a list of expected items from the web app, and filter looking for only the expected items to improve accuracy (when item is removed)
    • Now accepts arbitrary additional iconic images to be compatible with user registration on Web app side
  • Web app/CV now supports custom registered items (fully integrated with previous debugging-stage custom items): users can now identify their custom items using the viewfinder on the webapp.
  • Several UI/CSS enhancements

Overall, I think we’re in a pretty good position. We have almost everything essential for the project finished, and we have about a week left to perform the needed testing and create the slideshow. The main concern is finalizing the AJAX for the current recipe listings, and possibly needing to eke out some extra performance on the CV side to meet our criteria. To that end, we’ve decided to shuffle around some responsibilities so that Jay can focus on AJAX (see Jay’s report). There are also some remaining less essential issues, including RPI online/offline indicators, CSS, and website UI enhancements.

Harry’s Status Report for Apr 16

This week I have been focusing on webapp deployment. I was able to launch an EC2 instance and setup Django and Apache to serve the webapp in the cloud. I also purchased a web domain for the webapp and configured an SSL certificate for it to enable HTTPS and get better security (the webapp can be found at https://b6foodtracker.com). I updated the settings in the GCP console and FB Developer console to make sure OAuth works on the server. We ran into some issues about file permissions and python imports when deploying but ultimately we were able to get them working.

I looked into the Django messages framework and incorporated it into our webapp. Now we are able to pass messages to the user and display them on the webpage. Current four types of messages are supported: info, success, error, and warning.

I also worked on some CSS styling and HTML code restructuring. I worked on multiple pages and was able to complete the sign in page.

I will keep working on styling and help Jay to get the AJAX component working. I will also continuously deploy the latest version of the webapp to the server and monitor if there’s any issues. Next week, when the majority of the tasks are done, we will start testing our project.

Team Status Report 4/10

Summary: last Tuesday was the demo, and the past few days was Carnival.

Demo summary

  • Good: working CV, functional webapp, Keaton’s CSS contributions
  • Bad: mis-ID’ed a can of tomatoes as beans (or vice-versa), webapp needs lots of fine-tuning

Harry has begun implementing the API from the RPi to the webapp. Jay continues to make progress with the webapp. Keaton continues to beautify the website, and is investigating the cause of the mis-ID.

Jay’s Status Report for 4/10

This week was a blur, from the demo to Carnival and back at it with the project. We got through the first demo alright, accomplishing most of the goals outlined for us. I implemented a functional MVC, though some fine-tuning still needs to be done.

One notable miss was the ability to notify the user of a list. That’ll be accomplished by this week.

Harry’s Status Report for Apr 10

This week I have been looking into implementing the API endpoint for communicating with the RPi. I also investigated potential security risks and how to mitigate them. I was able to implement the update_inventory views method with precautions against malicious actors. The current communication protocol should be able to handle the following situations:

  1. Someone tries to update another person’s inventory if they manage to get their serial number: we are including a client secret in the POST request for server to validate
  2. In case of a data breach where all of the serial number and secret pairs are leaked: we are storing the SHA256 hashed string of the client secret in the database
  3. Someone intercepts the POST request and tries to get the client secret: we will use HTTPS to send the requests so that all of the data is encrypted
  4. Someone intercepts the POST request and sends it repeatedly to the server (replay attack): we will send a timestamp along with the data and record the latest in the database. Since the request cannot be modified by the attacker, only the first request will be accepted
  5. Someone brute force the serial numbers on the registration page: we are planning to add CAPTCHA to the page

Team Status Report 4/2

This week has mostly been cleaning up stuff and adding some polish in preparation for the demo. Some highlights include:

  • Adding recipes/grocery list functionality
  • Finalizing backend user model registration (see Jay’s post)
  • CSS additions/improvements
  • Dealing with hardware crisis (see Keaton’s post)
  • Updated Gantt chart

Overall, we have almost everything we need for the demo, but we’ll probably need to put in some work on Sunday/Monday to ensure everything is polished and in a presentable state. On the updated Gantt chart, we’ve fairly heavily eaten into our flex time, but we may be able to catch up during Carnival weekend. We’re about where we should be in terms of progress, but the lack of flex time going into the last two integration weeks is definitely concerning.

Jay’s Status Update 4/2

After much procrastination, I finally got around to completing the model-view-controller of the website, moving on from dummy HTML to actually interactive data. I’ll still continue to work on it before our demo this week, but I’m optimistic that we’ll have a functional backend by the time we present.

Major goal by next week is getting a working grocery list for the demo. I hope to make substantial progress on Sunday and Monday.

Besides that, there were a handful of lesser bugs in our current version, namely…

  • empty items don’t return an error message, but rather crash the website. Cause still unknown.
  • remaining logged in for too long causes a NoneType error. Harry is chasing down the issue. I don’t think we’ll be able to get this working by the end of the weekend, but certainly by the final demo.
  • the same serial number of a tracking device can be registered multiple times. Very much intentional whilst debugging the website, but I wanted to write it down to remind myself to un-comment out those lines before the final presentation.

This week, after the first demo, I’ll continue to refine the site, ensuring that the bugs above don’t impede the user experience.

Harry’s Status Report for Apr 2

This week I have been continuing working on the user registration system. I discovered a bug where users are able to sign in with another account when they are already signed in, which would consistently break the webapp. I figured out a solution by inserting custom functions into the django social auth sign in pipeline. In order to streamline the sign in process for the users and avoid needing to repeatedly call a method for all view methods to retrieve user information, I scrapped the UserInfoFetcher I wrote previously. After investigating through the python-social-auth source code, I was able to create custom functions and insert them into the oauth pipeline. By also creating a custom User model, I am able to save users’ profile pictures and emails (which sometimes are not provided by default) into request.user without needing to manually fetch them. I also worked on some html pages to create a usable webapp and started working on the recipe page where people can get auto-generated grocery lists. In addition, I was able to work with Jay to debug some code and help Keaton setup his Django environment so that he can work on some css/bootstrap.

For next week, I will continue working on the recipe page and try my best to finish a working version before the demo day. I will also start working on the API endpoint for updating inventories. I have come up with a detailed design for the transmission encryption and I will discuss it next week.

Keaton’s status report 4/2

On Friday, I discovered that my hardware wasn’t working. Taken from my post to general chat general chat:

I unplugged the ribbon from the camera module, just to see if I could, for purposes of potentially using a 15 pin CSI to 21 pin CSI-2 ribbon. When doing this, I accidentally reinserted the ribbon upside down. After realizing my mistake, I fixed it, but it no longer worked. I assumed at this point that the camera module was bricked, and used our spare, which also didn’t work. At this point, I assumed that I had bricked the CSI port, so we got another RPI from Quin (ty so much Quin). However, the new RPI still didn’t work with either camera module.

There was a little bit of floundering, but (as of 3PM on Apr 2nd) I’ve since gotten a new RPI/camera combination that does work, and will be using this for the demo. I don’t think the slightly different fisheye lens will have a significant impact, but I do have some concerns regarding the pixel diff. As I’m unable to securely attach the camera to a single unmoving position, we do get some fairly large areas of change just due to noise:

Overall, I’m pretty sure everything will be alright for the demo, but I definitely need to invest some time today/Monday to be sure . While I was waiting on backup hardware, I did some minor stuff, such as updating the gantt chart, and I did some bootstrap CSS on a few of the Django HTML templates. Overall, this was a pretty crap week in terms of progress, which comes on the heels of another pretty crap week. I definitely expect to be putting in some time during carnival weekend to fix these issues, and catch up on my work.