Harry’s Status Report for Apr 30

This week I polished the style for some pages and caught minor issues on the webapp. Since most of my classes are done this week, I will spend more time on the project next week. I will finalize the page styles before we start working on the video and test all of the functionalities of the webapp and fix any issues I find.

May 3 Update:

I was able to finalize page styles and clean up the codebase. I also added a few functionalities.

  • display custom recipes on the dashboard page
  • added “recipe” page to show individual recipes
  • allow users to edit their recipes
  • allow users to delete recipes
  • allow users to publish their recipes
  • shows if an ingredient is in stock on the “recipe” page
  • added a “discover recipes” page for users to see published recipes
  • added a warning dialog when user tries to delete a device (as shown below)

Similar to device registration, I added reCAPTCHA to the “email grocery list” dialog to prevent spamming.

Team Status report 4/30

This week, very little was done, due to it being finals week, and our already being complete with the majority of the work needed for the live demo. As such, we mostly just worked on the presentation poster, and the video. We expect the next week to be much the same, minus some very minor HTML tweaking/ possible CV optimizing.

Keaton’s status report 4/30

As stated in the team status report, nothing much was done this week outside of the poster and the video. For me specifically, I just haven’t had the time to do any additional optimizations on the CV code. All of my work/finals except for this capstone should be complete as of May 3rd, so I expect that I will resume working on the capstone then, so we can have some better test results for the final paper.

Jay’s Status Report 4/23

This week I finally completed AJAX implementation to a satisfactory degree. Other than that, I spent the week cleaning up the TODO’s and other dead code in our project. Alas, much of our time has been spent on working on the final presentation, though I’m feeling better about where we are in the project.

Keaton’s status report 4/23

So, this week, I spent the majority of my time finalizing the last for secondary checks for the CV code, and actually did the testing. I started the actual testing itself far too late. I had hoped to finish by today, but even with Jay’s assistance, we were unable to get all of the taken/identified before the end of today.

 

There was a lot of just manually iterating, testing and retesting suitable HSV color bounds, which took more time than I expected. Also, I found that the dimensions of our image post localization generally were not very reliable. The SSIM method I was using for localization tends to pretty regularly overestimate the bounds, so it’s difficult to . The overinclusion of the background also had a cascading effect on the color tests, since it was harder to tell how much of an item was a particular color when we don’t know how much of the image is the background. As such, a lot of the secondary checks ended up being a lot less effective than I had hoped. I think we might possibly be able to mitigate this with a second localization pass using SIFT, but I’m not sure if it’s feasible at this point.

 

There’s not much to say about the testing, I just vastly underestimated the amount of time it would take. It’s not difficult to do, it’s just that our testing plan expects us to take >200 images manually, which takes a large amount of time. Preliminary results based on what we have done are mixed. We are well within the amount of time needed, and we get enough correct positives. However, we also misidentify objects (ID object A as object B) far more commonly then we fail to identify objects (ID object A as an unknown object). This is unfortunately exactly the opposite of what we wanted to occur.

 

Next week, and probably for all of the following weeks until the final live demo/paper, I will be working on trying to get the results such that we can meet our use case. I expect this to be fully possible given where we’re at, it’s literally the only thing left, and I can think of several possible secondary checks that I didn’t get to implement on the first p

Harry’s Status Report for Apr 23

I have been super busy with homework and labs this week so I wasn’t as productive as I hoped. However, I did manage to get some stuff done:

  • I kept working on designing and styling pages with Bootstrap. I was able to make most of the pages look decent and I worked on small features like phone number formatter to improve user experience. CSS and styling is in general hard to deal with to achieve desired results, so this task ended up taking most of my time.
  • I upgraded the EC2 instance to have more memory and CPU cores. Further testing is needed to determine the best configuration for us to meet our goals.
  • I also helped teammates debug some code

Next week:

  • Finish styling and make some final touches to web app functionalities
  • Integration testing
  • Find the best config for the EC2 server

Team Status Report for Apr 23

This week has mostly been preparing for our presentation, and finalizing our integration stuff/deployment. Highlights include:

 

  • Launching the website(see Harry’s report)
    • Some issues with instance size due to RAM limits and modules setup
  • Ajax is working (see Jay’s report)
    • Still some hard to isolate bugs
  • Testing is almost done (see Keaton’s report)
    • Initial results look pretty good, but our misidentification rate is FAR higher than what we need for our use case requirements
    • All other requirements seem like they’ll be fine (right now)
  • Various CSS/HTML fixes/improvements

 

Overall, we’re mostly happy with where we are, for the presentation. We’re fully integrated and deployed, minus a few minor bugs that crop up here and there. Honestly, I would say that we’re pretty ready to perform the live demo, if it weren’t for the issue of getting our results up to scratch to meet our use case. While this is obviously an issue, I don’t think it will be insurmountable. We can probably implement a few additional specific secondary checks, and hopefully just barely squeak by.

Team Status Report for 4/16

This week has mostly been integration week, which has been overall successful. Highlights include:

  • Deployed the webapp to an EC2 instance
    • Purchased a domain name for $1
    • Configured SSL certificate to get HTTPS working
  • Full integration of web app with hardware device
    • Fully working communication between RPI and web app (locally)
    • Support for users identifying images/adding new Iconic classes, based on cropped images sent from RPI && support for arbitrary categories
  • Slight improvements on the CV side
    • Can now take a list of expected items from the web app, and filter looking for only the expected items to improve accuracy (when item is removed)
    • Now accepts arbitrary additional iconic images to be compatible with user registration on Web app side
  • Web app/CV now supports custom registered items (fully integrated with previous debugging-stage custom items): users can now identify their custom items using the viewfinder on the webapp.
  • Several UI/CSS enhancements

Overall, I think we’re in a pretty good position. We have almost everything essential for the project finished, and we have about a week left to perform the needed testing and create the slideshow. The main concern is finalizing the AJAX for the current recipe listings, and possibly needing to eke out some extra performance on the CV side to meet our criteria. To that end, we’ve decided to shuffle around some responsibilities so that Jay can focus on AJAX (see Jay’s report). There are also some remaining less essential issues, including RPI online/offline indicators, CSS, and website UI enhancements, which we will implement during the next week.

Jay’s Status Report for 4/16

Earlier in the week I did some work completing the email functionality, as well as some light bug fixes.

In the latter half of this week I’ve been working exclusively on converting our existing code into AJAX. This is proving to be much more challenging than I had expected, and I’m concerned about the integration phase of this down the week. I’ve gotten as far as having responsive updates on multiple windows, but I’m still having trouble handling edge cases and displaying error cases.

I’ve asked Keaton and Harry to take over on some of the polish for the website, namely going back on our old code and ensuring that permissions policies on our items/recipes visibilities are user-specific.

Keaton’s Status Report 4/16

This week, I mostly worked on doing integration, specifically handling a fair number of edge cases with regards to unregistered items. This consisted mostly of work in the Django code, and a few minor updates to the CV code. Both Harry and Jay helped me quite a bit as I didn’t have much experience with Django prior.

The CV code previously just gave it’s best guess out of the approved iconic image classes, or none if it wasn’t sure. The CV code was changed to accept a set of items in the current inventory. This is used in the event that an item was removed, so we don’t have to check every possibility, which should help with accuracy. It also deals with the edge case where the cv code says that an item was removed, that it didn’t say was in the cabinet in the first place. The CV code was also modified to accept arbitrary sets of additional Iconic images, to support user item registration. Both of these changes were relatively minor, and mostly just involved filtering/adding to the set of iconic image checked against.

The majority of the code work was done in Django, dealing with user registered items. Essentially, we needed to be able to be able to add unidentified items to the cabinets, and later change their category when the user identified them. We also needed to be able to handle them, in the case that the user decided to place the items and removed them, all without manually identifying them. Long story short this was resolved by creating a Iconic Image model item, with an optional associated item field, that was assigned a UNKNOWN category be default. Essentially, we then treat it as an a regular iconic image, until the user manually identifies it, at which point, we can propagate the identified category to the associated item object. We can also easily generate the prompts the user by filtering iconic models by category/created user. This handles most of the edge cases, and doesn’t have a huge drawback unless the user has huge amounts of unregistered items in their cabinets concurrently, which is unlikely. We still need to do some UI/CSS work here (the alert/request to identify items needs to be placed in a few different places and made much more apparent), but we have the needed functionality. There was also some minor technical debt that I had to fix regards to categories (IE we were making every item have a unique category. instead of re-using them).

Overall, for not knowing Django very well before this week, I’d say I did a pretty good job. Again, I obviously had a large amount of help from Harry and Jay, but I think I learned fairly quickly and got a good amount done. For the next week, I plan to almost exclusively work on the CV component, and try and get it within our needed criteria, and get the testing done so we have it for the slides.