Posted on

March 9th 2024 Status Report — Surya Chandramouleeswaran

Leading into the Spring Break, my main goals were to continue work on our web application while continuing to develop ideas for implementation of the scale integration in our project.

Regarding the web application, we thought it would be appropriate to have a working demonstration of some of its basic features ready by our next weekly meeting with TAs and instructors – planned for Monday, March 11th. Because hosting it on the Amazon Elastic Compute (EC) instance is not a must for this meeting, we will perform this demonstration on a localhost and will focus on site navigation, frontend development, user-user interaction, and view routing in the presentation of our progress to the course staff. This has been a shared effort amongst the entire group; I would now like to share some of my contributions to the web application:

Google Oauth Feature that registers and authenticates users with their Gmail addresses:

Weight-tracking visuals implemented with chart.js (a JavaScript framework for data management and plotting):

 

This, along with general routing of the pages, constituted some of the things I worked on the past 2 weeks on the web application front. We hope to flush out some of the main features under the context of user-user interaction, such as posting progress, friend networks, following/unfollowing, and chatting with Websockets, in time for the demonstration on Monday.

 

With the scale integration component of the project arriving soon, I took some time to compare and contrast the numerous market options. I had been hesitant to start ordering parts due to my unfamiliarity with the market options; however, in preparation for the design report, I performed a thorough evaluation of the possible microcontrollers that we could use for this stage of the project:

The market evaluation and design trade studies sections of the report required a thorough evaluation of factors I hadn’t previously considered, such as power consumption, pin layouts, and opportunities to streamline the entire design pipeline through custom SoC options. Favoring simplicity for the MVP, we will proceed with ESP-related options for scale communication, while researching more robust and impressive options in parallel. The benefit of these SoC designs is the potential to integrate camera and scale communication all in a single chip. This would greatly reduce the complexity of the project and alleviate concerns with separate stages of the project working with one another. Still, we consider this a potential stretch goal in the context of a simple yet effective MVP.

Posted on

Steven Zeng’s Status Report for 03/02/24 and 03/09/24

I wrote about section A for the team report; below is my answer.

With consideration of global factors, NutrientMatch can be used by just about anyone across the globe, whether or not their wellness goal consist of consuming more calories or to better keep track of the types of foods they are consuming daily. Regarding younger adults, as fitness is becoming a bigger trend with the emergence of its technology and the expansion of gyms, more of these people are aware of their nutrient intake. Our product is heavily targeted towards this group since they tend to live with roommates as opposed to other family members. With this in mind, it is easier to get foods mixed up when groceries are purchased separately. NutrientMatch is able to track each user’s food inventory as soon as they are logged to avoid these confusions. On the other hand, family members may also want to use this product to track their own intake. Our product can also be used by those who are not as tech savvy as others. The usage process is easy: after each food item is either scanned or classified and weighed, it is automatically forwarded to the cloud database that will make backend calculations to be displayed on the webpage for users to see. Hence, while NutrientMatch is not a necessity in peoples’ lives, the increasing trend of physical wellness makes it a more desirable product overall.

Regarding my progress, I primarily did work before Spring break, and used the designated slack time to relax and clear my head. I focused most of my effort on the design report including analysis of design trade-offs for the ML algorithms. Recently, I have tested soft-margin SVMs to distinguish between fruits and canned foods. I have computed optimal parameters and hyper-parameters using 5-fold cross validation due to a limited data set. It correctly classifies objects only around 74 percent of the time. However, I expect this number to go up as I tune the algorithm to only address the three types of fruits (banana, orange, and apple) and canned foods.

Furthermore, I did more research and experiments on ChatGPT4 and its API to read labels. The results were quite promising, and our focus will be on making sure the image is clear enough for ChatGPT to process. Likewise, I am experimenting with database storage and narrowing down what exactly our group wants to store in the database logs. The current plan is to take a picture of the image then pass it into the classification algorithm. If it is a fruit, our product will directly store the name of the fruit using the GoogLeNet classification algorithm along with caloric information calculated using the weight and online data. If it is canned food, our algorithm will read the label from the front and store the name into the database alongside an image of the back with the nutrition label. This will reduce the latency of storing an object into our database and offload the extra computation to database retrieval in which the image will need to be processed. The other alternative is to do all the computation and calculations when the object is being scanned. This will lead to better memory utilization but worse latency which will be an issue in terms of user experience. We plan to test both these approaches out and determine the differences in latency and memory utilization.

Lastly, I read up on a lot of the work Surya has done with the Arduino as well as cloud deployment. The ML portion is highly dependent on retrieving information from the Arduino (weight and image retrieval) and the ability of an Amazon EC2 server to efficiently run all the computations to populate the database. This involved reading a lot of documentation online to prepare myself for future stages. A lot of the detail and info came from https://aws.amazon.com/blogs/machine-learning/category/compute/amazon-ec2/ which I highly recommend. It gives a lot of fascinating stories and helped me understand the space better.

Overall, I had a relatively productive two weeks and got a good break to clear my head.

Posted on

Grace Liu’s Status Report for February 24th, 2024

This week, my group members and I carefully reflected upon previous feedback from peers and faculty to curate a cleaner and informational presentation. Because we have made some significant progress in meeting our target goals every week and have made a few design changes, it was a bit more difficult for Surya to include everything in under 12 minutes for the design presentation. Nonetheless, Surya did a great job of addressing previous concerns from peers (i.e. clear definition of MVP and quantitative design requirements from the user perspective) and reporting design progress. A lot of our progress and discoveries made during the process helped us visualize the bigger problem and additional factors that would need to be addressed in our design report.

In terms of our web application, after viewing other group’s presentations and use-case requirements from the perspective of their users, I focused more efforts on designing frontend that would be easy to use as well as attractive to our typical user. I started creating better mockups for the pages I have in views.py and HTML pages (user login, user profile, and user home page). Tabs/buttons that should be included on every page for user accessibility are the home page, a user guide that will have details on how to maximize the use of our product, perhaps (to attract more users) have a leaderboard of daily calorie intake between users in the same household, and a logout button.

I am a little behind on schedule as the goal was to have most of the frontend without backend connections from Steven and Surya to be completed soon. Next week, I plan on reflecting upon changes we may want to add to the user interface with my group members and complete the rendering functionalities between different HTML pages. With the design report being due as well, I plan on creating specific system implementation diagrams with changes derived from more recent testing operations.

Posted on

Team E2 Status Report: Feb 24

This week, our team focused on preparing our design presentation and report. We collaborated on designing slides, creating speaker notes, and beginning the writing process for the Introduction and Machine Learning documentation in our design report.

In terms of progress on our project’s design component, we made significant strides. We experimented with Support Vector Machines (SVMs), GoogLeNet, and ChatGPT’s API to optimize our classification results. We conducted tests on SVM formulations, continued building our dataset, and explored using GoogLeNet for image classification. Additionally, we tested ChatGPT4’s ability to read labels, which proved to be highly accurate.

Throughout the week, we didn’t encounter any major complications. However, I acknowledge that I could have contributed more to designing the physical product and creating the box. Moving forward, I aim to gain more hands-on experience with the physical design process while finalizing the Machine Learning algorithms (Steven).

Regarding the design presentation, we addressed key considerations such as project implementation, load cell amplifiers, food item differentiation, validation metrics, and backend complexity. We’re now focusing on completing the design report, which will require a deeper understanding of our project’s development procedures. For instance, we will use our block diagram and implementation plan to devise a more descriptive architecture plan.  We will also put more efforts towards connecting initial public health/safety/welfare, social, and economic factors we want our product to solve to the formulation of this solution. Of course, many tradeoffs emerge with designing an engineering solution, so we will dive into these considerations through our testing experience so far and feedback we have received from peers during our proposal and design presentations.

In terms of hardware-related risk mitigation strategies, we’re addressing concerns such as buffer overflow, data loss, recovery, and mismatched baud rates. We plan to implement control mechanisms for buffer overflow, backup data on nonvolatile memory for data loss recovery, and ensure proper synchronization of baud rates.

Overall, our progress is on track. We’re aiming to complete the imaging component of the project by Spring break, with each team member making significant contributions to their respective tasks.

Posted on

Feb 24th 2024 Status Report — Surya Chandramouleeswaran

This week was centered around building a convincing design presentation. With midterms and other commitments, there were challenging aspects to preparing a 10-12 minute talk discussing the primary features of our product, but overall, such an experience went well and was invaluable for our group in sorting out some of the finer details present in our design plan.

 

Some of the key takeaways from the design presentation include the following:

  • How much of the project do we want to implement by ourselves, as opposed to offloading the work to an API call?
  • Re-evaluating the need for load cell amplifiers to be placed between the scale reading to the ADC port of our microcontroller
  • Proper differentiation between labeled food items, versus standalone produce
  • Validation metrics and subsequent justification
  • Balancing how much complexity we want to offload to the backend. We are now working on the design report, which requires another level of understanding of our procedures in the development of our product.On my end (hardware), some of the risk mitigation strategies that I have thought about in the context of the microcontrollers include the following:
  • Buffer Overflow: This can occur when incoming data from the scale exceeds the capacity of the micro-controller \texttt{receive} buffer, potentially causing data corruption or subsequent instability. To address this issue, our team plans to implement control mechanisms in the controller scripting, following XON/XOFF software flow control protocols.
  • Data Loss and Recovery: In the case of failure, the data should be backed up on nonvolatile memory. This is present in each of our options for microcontrollers: Arduino holds the well-established EEPROM as an example of this, and SPIFFS is the equivalent option in the ESP8266 WiFi microchip. These are hardware backups that the database can read from if the system crashes. 
  • Mismatched Baud Rates: Arduino IDE handles synchronization, but because there is logic in the sampling of the scale values, this needs to be matched properly.

An additional consideration (pictured below) is a conversion from the RS232 serial protocol that the scale communicates data with to TTL logic employed by microcontrollers. This is something I realized in the building process. A converter, pictured and wired in the schematic below, will be necessary.

Progress-wise, we are doing well. We plan to finish the imaging component of the project by Spring break, so this upcoming week will be critical for Steven and me, while Grace is making good progress on the web application.

Posted on

Steven Zeng’s Weekly Status Report 02/24/24

This week I concentrated my efforts into the design presentation and report. I helped my team in designing slides and creating speaker notes. Likewise, I began writing the Introduction and ML documentation in our design report.

 

Regarding the progress in forwarding the design component of our project, I experimented with SVMs, GoogLeNet, and ChatGPT’s API. I was able to conduct tests on the weights and constants of the SVM formulation to best optimize the classification results. Likewise, I am still adding to a data set for training, testing, and validation data distinguishing canned foods and fruit. I plan to use 5-fold cross-validation to properly tune the hyper parameters (i.e. C and gamma). I also began looking at the GoogLeNet library and watching videos on how to use it. I seek to use function calls to distinguish between apples, bananas, and oranges. Finally, I tested ChatGPT4s ability to read labels which was surprisingly high and accurate. I plan to learn how to use the API to accept input and direct prompt responses in a formatted way to forward to the database.

 

There were no major complications or challenges this week in my progress. I wish I could have helped my teammates more in terms of designing the physical product and creating the box. I hope to get a more hands-on experience next week with the physical design process as I finalize the ML algorithms. Likewise, I hope to shift a lot of focus and effort into completing the design document for next week.

Posted on

Grace Liu’s Status Report for February 17th, 2024

This week, I focused more on building functions in views.py for incoming HTTP requests and responses and different HTML pages corresponding to user login, user profile, and user home page. While the rendering functionality has not been completed yet between pages on the frontend side, it is essential to build the views.py functions as it directly connects user web requests to a lot of backend logic in Django. In addition to this, since we were able to receive a lot of insightful feedback from the faculty and our peers on the proposal presentation, we worked together as a group to reflect on potential changes for the design proposal. I realized there were several things I overlooked while preparing for the previous presentation, so I wanted to go through them with Surya to ensure we meet the new requirements and fully demonstrate understanding of our project design.

Within views.py, action functions check if a user profile exists and whether or not it has been completed (so far, fields to be checked include name, gender, weight, and profile picture). I also included corresponding functions for the rendering of user login, user profile, and user home pages in which the latter two can only be displayed once a user is logged in. A convenient way to handle user login and registration is the forms.py module. This can authenticate the user’s username and passwords, otherwise, throw an error on the frontend side. We want to render these webpages if a GET request is detected.

More testing will have to be done with the HTML pages to ensure they meet our end goal aesthetic in CSS and JavaScript. Each page is built upon a base.html file that includes our product name display, user profile button, logout button, home page button, and daily intake button. This allows ease of navigation between pages for usability and helps us ensure consistency during testing phases.

Moving on regarding previous feedback, a big thing I failed to pay more detail to was defining MVP since these questions arose during the Q&A session. In next week’s presentation, we will clearly define the differentiation between image classification/label reading and scale reading foods in the testing and verification requirement. Although there was a mockup CAD to help explain the entire process of how users would utilize our product, we could have included more use-case requirements from the user perspective in addition to the product design perspective. We made the addition of including latency minimization to ensure data is forwarded to mySQL DB at a reasonable rate. There is also more specific quantitative data to demonstrate the viability of our idea. Surya will included changes in the design presentation to ensure questions from the feedback regarding our implementation will be addressed.

While we did include an overall flowchart in the proposal presentation, we noticed a lot of other groups included a block diagram that either showed an overview of the interaction/connection between interfaces or separated their components into hardware and software parts. Since our implementation has a clear separation between the two components, I thought this conceptual block diagram would be the best approach. Our first block diagram separates our hardware and which software integration fits under them with a clear overall flow. As seen below, the implementation plan includes more details including our software design/development, off-the-shelf parts, and where the user will be interacting with our product:

The CAD design has been updated to include our main design changes:

  • Replacing the Arduino camera with an Arduino driver/receiver that directly forwards scale measurements to the cloud as opposed to adding light illumination inside the box as well.
  • Using a ring light to ensure full illumination of classified object.
  • A hinge mechanism on the door for better user accessibility and help audience understand product use.

We look forward to presentations next week that while not only will address most people’s confusions but also give us more guidance on our ambitious goals for designing a useful product.

Posted on

Steven Zeng Status Report for February 17, 2024

 

This week, I focused primarily on image classification rather than label reading. I looked through the Tesseract library and watched several videos on using it to classify images. Likewise, I explored using support vector machines to hand-make our own classification algorithm. This included a compilation of training data from Kaggle’s 2022 Food Recognition set (https://www.kaggle.com/datasets/sainikhileshreddy/food-recognition-2022). This training data can help for whatever method we chose to apply. Finally, I tested ChatGPT and its API more to potentially use it for classification and label reading. As a result, we have these three potential design paths that we could take. Each of these paths have their own tradeoffs that I have explored and written down below, and I hope to make a decision next week after doing more research. 

1.  Online Label Reading and Classification Library: The benefits are that all the hard work and algorithms are created. Likewise, the method has been tested and published online. All I need to do is import the libraries and write functions that call these libraries. This will reduce the design process tremendously. On the cons side, it will take a lot of preliminary research and understanding of other people’s code. Likewise, some of the code may be outdated and lack the accuracy we need for our project.

2. Incorporate Tesseract and Create Our Own Algorithm: The primary benefit of this is better top-down understanding of everything. Likewise, I can fine-tune it to fit our product specifically. Maintenance and product updates would be quicker since I will have full understanding of a majority of the code. The con is that this will take a lot of time upfront with learning everything and small design choices along the way. For example, I have looked into the tradeoffs between using Support Vector Machines vs Naive Bayes vs Trees. All of them have their unique choices, but I was able to study up on the math and principals behind each of these algorithms.

3. Chat GPT API: The benefit of this is reliability, precision, and speed. I queried various inputs and formatted the replies of Chat GPT to be compatible with key-word extraction to store values in the database. The accuracy was quite remarkable, and it could classify images as well as read labels. The main con is understanding the API and the potential to have a high cost. However, we are definitely interested in somehow incorporating Chat GPT into our product.

In addition to that, I helped with creating the design presentation. I focused primarily on the implementation plan, testing, and finalizing our schedule via the Gantt chart. I worked with my team members to create the slides and write the script for the presentation. I went through feedback from our first presentation to better design our next presentation and our product in general. Thanks again to all the students, TAs, and professors who provided commentary on it.

Regarding the special prompt we are supposed to answer for this report, I have decided to write part A. With respect to considerations of public health, safety and welfare, our solution is designed to improve physical well-being for users by monitoring caloric intake. Likewise, our product aims to reduce the psychological burden of tracking what and how much you eat as our product does all the tracking for you. Furthermore, keeping track and inventory of all the food you buy at the store can be a burden, and people are quite forgetful sometimes. Our product reduces the stress in remembering what is in your fridge or pantry by keeping count of every item. We have ambitious goals for our product to reduce over and under eating which aims to better the physical health and well-being of the public. 

With respect to safety, our product will have limited hazards as it will consist of a sturdy physical structure with a scale, camera, and lights. The components will be as minimalistic as possible, and we will conduct rigorous testing to make sure there are no malfunctions. The lights will require as little power as possible and will be properly embedded into the walls. Likewise, the camera will be off whenever the scale reading is 0 as a result the product will only turn on when an item is put on the scale.



Posted on

Team E2 Status Report – Feb 17th

Ensuring that our product addresses a desired need in the market is critical to its reception and success amongst the public. This week, the team would like to explore this issue through 3 lenses: Steven will explore part A, Surya will explore Part B, and Grace will explore part C.

Steven:  With respect to considerations of public health, safety and welfare, our solution is designed to improve physical well-being for users by monitoring caloric intake. Likewise, our product aims to reduce the psychological burden of tracking what and how much you eat as our product does all the tracking for you. Furthermore, keeping track and inventory of all the food you buy at the store can be a burden, and people are quite forgetful sometimes. Our product reduces the stress in remembering what is in your fridge or pantry by keeping count of every item. We have ambitious goals for our product to reduce over and under eating which aims to better the physical health and well-being of the public.  With respect to safety, our product will have limited hazards as it will consist of a sturdy physical structure with a scale, camera, and lights. The components will be as minimalistic as possible, and we will conduct rigorous testing to make sure there are no malfunctions. The lights will require as little power as possible and will be properly embedded into the walls. Likewise, the camera will be off whenever the scale reading is 0 as a result the product will only turn on when an item is put on the scale.

Surya: Consideration of social factors, which relate to extended social groups with distinct sociocultural, political, and/or economic affiliations, is crucial in our increasingly interconnected environment. With this in mind, there are several key themes that we seek to account for in our design. First is the notion of accommodating varying dietary preferences: whether this be due to medical reasons, religious restrictions, or observance of regional cuisine, our algorithm looks to maintain as unbiased a performance as possible in its classification approach.  We look to achieve this accessibility in several ways. This involves details in the image recognition training process and enhanced user-user interaction on the application side that allows for features such as transparent sourcing of nutritional information and customizable dietary profiles (vegan, gluten-free, keto, etc.). Embracing diversity through building a prototype that lacks systematic bias is key to our consideration of social factors we look to meet in our design.

Grace:  With consideration of economic factors related to the system of production, distribution, and consumption of goods and services, a big concern that arose during our research is the amount of food waste generated by people on a daily basis in the United States. Not only does this pose sustainability challenges in the long run from an economic point of view, but also contributes to a substantial amount of landfill waste. Increased amounts of waste also require more management and environmental cleanup, leading to costly endeavors and potentially higher taxes for residents. Our solution, rather than focusing solely on mitigating the system of production and distribution of food goods, aims to assist users minimize waste during their food purchasing and consumption processes. While it does not directly address issues that may arise during production and distribution, helping users automate their daily calorie tracking can at least help them make more informed purchases and reduce waste as a whole. By providing accurate nutritional information, from label reading and food classification, along with better control over food inventory, our solution enables users to make more efficient use of the food at hand and consequently decrease food spoilage rates in general. We anticipate our product to not only  peoples’ wellness habits, but also make a positive economic impact on our society as a whole.

For our team’s status, we identified various risks that might prove to be an issue for our project. Surya was able to write down various risks regarding scale integration and using the Arduino as a middle component in our product. Likewise, Steven discovered risks with the image classification design and the various trade offs. As a result, we need to analyze our design decisions more next week and finalize it. Our contingency plans involve the various backup options we have which include the other design choices (i.e. buying multiple scales and Arduinos).

Likewise, we specified our schedule more as we got a better understanding of the necessary work and steps of our project. Here is the new Gantt chart that will be added to our design presentation:

 

Next week, we will make extensive efforts to produce results in terms of website design, image classification design, and physical product creation and integration.