With the upcoming deadline for demo coming, I shifted gears towards integrating all the ML components into the website. A lot of my focus was in integrating the ChatGPT API into the website for simple functionality. There were several issues with synchronization and syntax bugs that I spent a while debugging. The goal was to allow for label reading of an uploaded image, so there was less of a stress on database access; it was primarily just ML functionality. We came into an error involving the inability to upload images to the API without understanding complex transformations. As a result, we ultimately decided to scratch this idea after trying it out.
This led to me experimenting more with the Keras library and tensorflow. The code involved a lot of particular configurations which took a lot of time. This included python compatibility and virtual environment issues and various hardware protections that we needed to overcome. We ultimately were able to overcome this issue and have a working classification algorithm on the website that classifies between the various fruits. The algorithm worked super well in classifying bananas and oranges, and it also provides other classification results. To determine if it is canned food, we will classify as canned food if it is not a fruit. We still need to look more into error edge cases to notify the user.
Lastly, the focus shifted towards OCR and label reading now that ChatGPT API is not an option. The discussion involved research between doing it on the rapsberry pi or the website. I along with Surya experimented with both this week. We were able to use Tesseract to read nutrition labels after uploading image. However, we are currently in the process of testing it out on the hardware instead.
Overall, I made good progress towards practical implementation and the demo. I need to record everything I tested that ultimately did not work when it came to integrating with our website. However, I was satisfied with our ability as a team to adjust on the fly and have many backup options that allow us to make significant progress despite some complications.