Kelly’s Status Report for 3/30/24

Work Accomplished:

• Made the video button direct to a page that will stream the video feed from the scanners.
  • Added a back button that goes back to the inventory page.
  • For now UI displays “Camera is not active” but once the server is set up, I’ll put the video stream when the camera is active.
• Made the filter button open up a pop up with options for dietary restrictions and cuisines.
  • This widget is reused between the saved tab and the generate tab.
  • Made the UI look better by having the buttons colored in when selected, as opposed to the default multiple select check boxes.
  • When clicking “Apply Changes” it will return a list of the selected options that we can use to sort saved recipes or ask the LLM for specific recipes.
• Displayed a warning label before clicking the “Generate Recipe” button.
  • This will warn users to check if their ingredients have gone bad before using them since our app does not have a way to know exactly if ingredients have gone bad besides getting an estimated number of days the produce usually lasts.
  • The animation starts playing after the button is pressed.
• Implemented a Swipe-to-delete feature for the inventory items and recipes.
  • This will be used if the user accidentally forgets to scan out items or would like to delete saved recipes
  • Might need to implement it to allow users to delete some quantity and not the entire ingredient (not as urgent since main use will be through scanning out)
  • This action will need to update the database.

Here is a video demo for the new UI changes in action

• LLM prompt engineering: made a prompting flow for recipe generation

• Tested Meta’s Llama2 7b and 13b and Google’s Gemma using this prompting flow
  • Llama2: the only recipes that it knows are different varieties of stir fry, but it includes questionable ingredients in it, like bananas. When I ask for a specific cuisine, it still gives me stir fry and claim it is Italian or Korean when they are the exact same recipe.

• • Gemma: the only recipe that it knows is tomato and garlic soup. It uses ingredients that are not in the inventory. It does not give any other recipe even when I ask for a different one or change the cuisine.

• • Deduced that these LLM Models are very limited for our use case and we do not have enough time to fine-tune/train the models (learning how to and actually doing it)

• Tested the prompts on ChatGPT
  • Worked a lot better
    • Gave a good variety of answers
    • Would change the recipes according to dietary restrictions and cuisine
    • Fixes itself if it uses an incorrect ingredient and I ask it to try again
  • Given the success rate of ChatGPT in creating recipes, our intended use, it significantly outweighs the pros of local LLM models, which are that they are more secure and can be faster.
  • Decided to use ChatGPT for our recipe generation although it is a cloud-based LLM and we originally said we would use a local LLM.

• Set up OpenAI API on Flutter app
  • Tried making a sample app and it should work but we do not have credits so it just gives me “429” rate limit errors.
  • Waiting to purchase credits using our budget
    • I asked on Wednesday. They told me that on Monday someone in ECE purchasing should reach out to me.
    • My plan was to finish it for the interim demo, but now I cannot.
    • Might need to pay out of pocket if this takes too long.

Progress:

• On track with the updated gantt chart.

Things to complete next week:

• Get OpenAI credits
  • Implement prompting flow in the app to generate recipes
  • Display recipes on new UI elements (pop up with recipe name and recipe page with instructions)
• Create custom item & custom recipe input pop up when clicking + button
• Cookable toggle on saved recipes page