Posted on

Grace Liu’s Status Report for February 24th, 2024

This week, my group members and I carefully reflected upon previous feedback from peers and faculty to curate a cleaner and informational presentation. Because we have made some significant progress in meeting our target goals every week and have made a few design changes, it was a bit more difficult for Surya to include everything in under 12 minutes for the design presentation. Nonetheless, Surya did a great job of addressing previous concerns from peers (i.e. clear definition of MVP and quantitative design requirements from the user perspective) and reporting design progress. A lot of our progress and discoveries made during the process helped us visualize the bigger problem and additional factors that would need to be addressed in our design report.

In terms of our web application, after viewing other group’s presentations and use-case requirements from the perspective of their users, I focused more efforts on designing frontend that would be easy to use as well as attractive to our typical user. I started creating better mockups for the pages I have in views.py and HTML pages (user login, user profile, and user home page). Tabs/buttons that should be included on every page for user accessibility are the home page, a user guide that will have details on how to maximize the use of our product, perhaps (to attract more users) have a leaderboard of daily calorie intake between users in the same household, and a logout button.

I am a little behind on schedule as the goal was to have most of the frontend without backend connections from Steven and Surya to be completed soon. Next week, I plan on reflecting upon changes we may want to add to the user interface with my group members and complete the rendering functionalities between different HTML pages. With the design report being due as well, I plan on creating specific system implementation diagrams with changes derived from more recent testing operations.

Posted on

Team E2 Status Report: Feb 24

This week, our team focused on preparing our design presentation and report. We collaborated on designing slides, creating speaker notes, and beginning the writing process for the Introduction and Machine Learning documentation in our design report.

In terms of progress on our project’s design component, we made significant strides. We experimented with Support Vector Machines (SVMs), GoogLeNet, and ChatGPT’s API to optimize our classification results. We conducted tests on SVM formulations, continued building our dataset, and explored using GoogLeNet for image classification. Additionally, we tested ChatGPT4’s ability to read labels, which proved to be highly accurate.

Throughout the week, we didn’t encounter any major complications. However, I acknowledge that I could have contributed more to designing the physical product and creating the box. Moving forward, I aim to gain more hands-on experience with the physical design process while finalizing the Machine Learning algorithms (Steven).

Regarding the design presentation, we addressed key considerations such as project implementation, load cell amplifiers, food item differentiation, validation metrics, and backend complexity. We’re now focusing on completing the design report, which will require a deeper understanding of our project’s development procedures. For instance, we will use our block diagram and implementation plan to devise a more descriptive architecture plan.  We will also put more efforts towards connecting initial public health/safety/welfare, social, and economic factors we want our product to solve to the formulation of this solution. Of course, many tradeoffs emerge with designing an engineering solution, so we will dive into these considerations through our testing experience so far and feedback we have received from peers during our proposal and design presentations.

In terms of hardware-related risk mitigation strategies, we’re addressing concerns such as buffer overflow, data loss, recovery, and mismatched baud rates. We plan to implement control mechanisms for buffer overflow, backup data on nonvolatile memory for data loss recovery, and ensure proper synchronization of baud rates.

Overall, our progress is on track. We’re aiming to complete the imaging component of the project by Spring break, with each team member making significant contributions to their respective tasks.

Posted on

Feb 24th 2024 Status Report — Surya Chandramouleeswaran

This week was centered around building a convincing design presentation. With midterms and other commitments, there were challenging aspects to preparing a 10-12 minute talk discussing the primary features of our product, but overall, such an experience went well and was invaluable for our group in sorting out some of the finer details present in our design plan.

 

Some of the key takeaways from the design presentation include the following:

  • How much of the project do we want to implement by ourselves, as opposed to offloading the work to an API call?
  • Re-evaluating the need for load cell amplifiers to be placed between the scale reading to the ADC port of our microcontroller
  • Proper differentiation between labeled food items, versus standalone produce
  • Validation metrics and subsequent justification
  • Balancing how much complexity we want to offload to the backend. We are now working on the design report, which requires another level of understanding of our procedures in the development of our product.On my end (hardware), some of the risk mitigation strategies that I have thought about in the context of the microcontrollers include the following:
  • Buffer Overflow: This can occur when incoming data from the scale exceeds the capacity of the micro-controller \texttt{receive} buffer, potentially causing data corruption or subsequent instability. To address this issue, our team plans to implement control mechanisms in the controller scripting, following XON/XOFF software flow control protocols.
  • Data Loss and Recovery: In the case of failure, the data should be backed up on nonvolatile memory. This is present in each of our options for microcontrollers: Arduino holds the well-established EEPROM as an example of this, and SPIFFS is the equivalent option in the ESP8266 WiFi microchip. These are hardware backups that the database can read from if the system crashes. 
  • Mismatched Baud Rates: Arduino IDE handles synchronization, but because there is logic in the sampling of the scale values, this needs to be matched properly.

An additional consideration (pictured below) is a conversion from the RS232 serial protocol that the scale communicates data with to TTL logic employed by microcontrollers. This is something I realized in the building process. A converter, pictured and wired in the schematic below, will be necessary.

Progress-wise, we are doing well. We plan to finish the imaging component of the project by Spring break, so this upcoming week will be critical for Steven and me, while Grace is making good progress on the web application.

Posted on

Steven Zeng’s Weekly Status Report 02/24/24

This week I concentrated my efforts into the design presentation and report. I helped my team in designing slides and creating speaker notes. Likewise, I began writing the Introduction and ML documentation in our design report.

 

Regarding the progress in forwarding the design component of our project, I experimented with SVMs, GoogLeNet, and ChatGPT’s API. I was able to conduct tests on the weights and constants of the SVM formulation to best optimize the classification results. Likewise, I am still adding to a data set for training, testing, and validation data distinguishing canned foods and fruit. I plan to use 5-fold cross-validation to properly tune the hyper parameters (i.e. C and gamma). I also began looking at the GoogLeNet library and watching videos on how to use it. I seek to use function calls to distinguish between apples, bananas, and oranges. Finally, I tested ChatGPT4s ability to read labels which was surprisingly high and accurate. I plan to learn how to use the API to accept input and direct prompt responses in a formatted way to forward to the database.

 

There were no major complications or challenges this week in my progress. I wish I could have helped my teammates more in terms of designing the physical product and creating the box. I hope to get a more hands-on experience next week with the physical design process as I finalize the ML algorithms. Likewise, I hope to shift a lot of focus and effort into completing the design document for next week.

Posted on

Grace Liu’s Status Report for February 17th, 2024

This week, I focused more on building functions in views.py for incoming HTTP requests and responses and different HTML pages corresponding to user login, user profile, and user home page. While the rendering functionality has not been completed yet between pages on the frontend side, it is essential to build the views.py functions as it directly connects user web requests to a lot of backend logic in Django. In addition to this, since we were able to receive a lot of insightful feedback from the faculty and our peers on the proposal presentation, we worked together as a group to reflect on potential changes for the design proposal. I realized there were several things I overlooked while preparing for the previous presentation, so I wanted to go through them with Surya to ensure we meet the new requirements and fully demonstrate understanding of our project design.

Within views.py, action functions check if a user profile exists and whether or not it has been completed (so far, fields to be checked include name, gender, weight, and profile picture). I also included corresponding functions for the rendering of user login, user profile, and user home pages in which the latter two can only be displayed once a user is logged in. A convenient way to handle user login and registration is the forms.py module. This can authenticate the user’s username and passwords, otherwise, throw an error on the frontend side. We want to render these webpages if a GET request is detected.

More testing will have to be done with the HTML pages to ensure they meet our end goal aesthetic in CSS and JavaScript. Each page is built upon a base.html file that includes our product name display, user profile button, logout button, home page button, and daily intake button. This allows ease of navigation between pages for usability and helps us ensure consistency during testing phases.

Moving on regarding previous feedback, a big thing I failed to pay more detail to was defining MVP since these questions arose during the Q&A session. In next week’s presentation, we will clearly define the differentiation between image classification/label reading and scale reading foods in the testing and verification requirement. Although there was a mockup CAD to help explain the entire process of how users would utilize our product, we could have included more use-case requirements from the user perspective in addition to the product design perspective. We made the addition of including latency minimization to ensure data is forwarded to mySQL DB at a reasonable rate. There is also more specific quantitative data to demonstrate the viability of our idea. Surya will included changes in the design presentation to ensure questions from the feedback regarding our implementation will be addressed.

While we did include an overall flowchart in the proposal presentation, we noticed a lot of other groups included a block diagram that either showed an overview of the interaction/connection between interfaces or separated their components into hardware and software parts. Since our implementation has a clear separation between the two components, I thought this conceptual block diagram would be the best approach. Our first block diagram separates our hardware and which software integration fits under them with a clear overall flow. As seen below, the implementation plan includes more details including our software design/development, off-the-shelf parts, and where the user will be interacting with our product:

The CAD design has been updated to include our main design changes:

  • Replacing the Arduino camera with an Arduino driver/receiver that directly forwards scale measurements to the cloud as opposed to adding light illumination inside the box as well.
  • Using a ring light to ensure full illumination of classified object.
  • A hinge mechanism on the door for better user accessibility and help audience understand product use.

We look forward to presentations next week that while not only will address most people’s confusions but also give us more guidance on our ambitious goals for designing a useful product.

Posted on

Steven Zeng Status Report for February 17, 2024

 

This week, I focused primarily on image classification rather than label reading. I looked through the Tesseract library and watched several videos on using it to classify images. Likewise, I explored using support vector machines to hand-make our own classification algorithm. This included a compilation of training data from Kaggle’s 2022 Food Recognition set (https://www.kaggle.com/datasets/sainikhileshreddy/food-recognition-2022). This training data can help for whatever method we chose to apply. Finally, I tested ChatGPT and its API more to potentially use it for classification and label reading. As a result, we have these three potential design paths that we could take. Each of these paths have their own tradeoffs that I have explored and written down below, and I hope to make a decision next week after doing more research. 

1.  Online Label Reading and Classification Library: The benefits are that all the hard work and algorithms are created. Likewise, the method has been tested and published online. All I need to do is import the libraries and write functions that call these libraries. This will reduce the design process tremendously. On the cons side, it will take a lot of preliminary research and understanding of other people’s code. Likewise, some of the code may be outdated and lack the accuracy we need for our project.

2. Incorporate Tesseract and Create Our Own Algorithm: The primary benefit of this is better top-down understanding of everything. Likewise, I can fine-tune it to fit our product specifically. Maintenance and product updates would be quicker since I will have full understanding of a majority of the code. The con is that this will take a lot of time upfront with learning everything and small design choices along the way. For example, I have looked into the tradeoffs between using Support Vector Machines vs Naive Bayes vs Trees. All of them have their unique choices, but I was able to study up on the math and principals behind each of these algorithms.

3. Chat GPT API: The benefit of this is reliability, precision, and speed. I queried various inputs and formatted the replies of Chat GPT to be compatible with key-word extraction to store values in the database. The accuracy was quite remarkable, and it could classify images as well as read labels. The main con is understanding the API and the potential to have a high cost. However, we are definitely interested in somehow incorporating Chat GPT into our product.

In addition to that, I helped with creating the design presentation. I focused primarily on the implementation plan, testing, and finalizing our schedule via the Gantt chart. I worked with my team members to create the slides and write the script for the presentation. I went through feedback from our first presentation to better design our next presentation and our product in general. Thanks again to all the students, TAs, and professors who provided commentary on it.

Regarding the special prompt we are supposed to answer for this report, I have decided to write part A. With respect to considerations of public health, safety and welfare, our solution is designed to improve physical well-being for users by monitoring caloric intake. Likewise, our product aims to reduce the psychological burden of tracking what and how much you eat as our product does all the tracking for you. Furthermore, keeping track and inventory of all the food you buy at the store can be a burden, and people are quite forgetful sometimes. Our product reduces the stress in remembering what is in your fridge or pantry by keeping count of every item. We have ambitious goals for our product to reduce over and under eating which aims to better the physical health and well-being of the public. 

With respect to safety, our product will have limited hazards as it will consist of a sturdy physical structure with a scale, camera, and lights. The components will be as minimalistic as possible, and we will conduct rigorous testing to make sure there are no malfunctions. The lights will require as little power as possible and will be properly embedded into the walls. Likewise, the camera will be off whenever the scale reading is 0 as a result the product will only turn on when an item is put on the scale.



Posted on

Team E2 Status Report – Feb 17th

Ensuring that our product addresses a desired need in the market is critical to its reception and success amongst the public. This week, the team would like to explore this issue through 3 lenses: Steven will explore part A, Surya will explore Part B, and Grace will explore part C.

Steven:  With respect to considerations of public health, safety and welfare, our solution is designed to improve physical well-being for users by monitoring caloric intake. Likewise, our product aims to reduce the psychological burden of tracking what and how much you eat as our product does all the tracking for you. Furthermore, keeping track and inventory of all the food you buy at the store can be a burden, and people are quite forgetful sometimes. Our product reduces the stress in remembering what is in your fridge or pantry by keeping count of every item. We have ambitious goals for our product to reduce over and under eating which aims to better the physical health and well-being of the public.  With respect to safety, our product will have limited hazards as it will consist of a sturdy physical structure with a scale, camera, and lights. The components will be as minimalistic as possible, and we will conduct rigorous testing to make sure there are no malfunctions. The lights will require as little power as possible and will be properly embedded into the walls. Likewise, the camera will be off whenever the scale reading is 0 as a result the product will only turn on when an item is put on the scale.

Surya: Consideration of social factors, which relate to extended social groups with distinct sociocultural, political, and/or economic affiliations, is crucial in our increasingly interconnected environment. With this in mind, there are several key themes that we seek to account for in our design. First is the notion of accommodating varying dietary preferences: whether this be due to medical reasons, religious restrictions, or observance of regional cuisine, our algorithm looks to maintain as unbiased a performance as possible in its classification approach.  We look to achieve this accessibility in several ways. This involves details in the image recognition training process and enhanced user-user interaction on the application side that allows for features such as transparent sourcing of nutritional information and customizable dietary profiles (vegan, gluten-free, keto, etc.). Embracing diversity through building a prototype that lacks systematic bias is key to our consideration of social factors we look to meet in our design.

Grace:  With consideration of economic factors related to the system of production, distribution, and consumption of goods and services, a big concern that arose during our research is the amount of food waste generated by people on a daily basis in the United States. Not only does this pose sustainability challenges in the long run from an economic point of view, but also contributes to a substantial amount of landfill waste. Increased amounts of waste also require more management and environmental cleanup, leading to costly endeavors and potentially higher taxes for residents. Our solution, rather than focusing solely on mitigating the system of production and distribution of food goods, aims to assist users minimize waste during their food purchasing and consumption processes. While it does not directly address issues that may arise during production and distribution, helping users automate their daily calorie tracking can at least help them make more informed purchases and reduce waste as a whole. By providing accurate nutritional information, from label reading and food classification, along with better control over food inventory, our solution enables users to make more efficient use of the food at hand and consequently decrease food spoilage rates in general. We anticipate our product to not only  peoples’ wellness habits, but also make a positive economic impact on our society as a whole.

For our team’s status, we identified various risks that might prove to be an issue for our project. Surya was able to write down various risks regarding scale integration and using the Arduino as a middle component in our product. Likewise, Steven discovered risks with the image classification design and the various trade offs. As a result, we need to analyze our design decisions more next week and finalize it. Our contingency plans involve the various backup options we have which include the other design choices (i.e. buying multiple scales and Arduinos).

Likewise, we specified our schedule more as we got a better understanding of the necessary work and steps of our project. Here is the new Gantt chart that will be added to our design presentation:

 

Next week, we will make extensive efforts to produce results in terms of website design, image classification design, and physical product creation and integration.

Posted on

Feb 17th 2024 Status Report — Surya Chandramouleeswaran

My roles this week involved a dive into the design process in preparation for the upcoming design presentations. In particular, I decided to take the opportunity to present our team’s design plans next week. Preparing for this presentation requires strongly re-evaluating current project parameters and scope. Additionally, we found the feedback from our peers and faculty quite valuable from the proposal presentation, and we are eager to reflect on their suggestions in our design process. Building these slides, in concert with feedback and an improved understanding of our project, forced our group to critically examine every component that we plan to develop, in the hopes of convincing our peers of the value of our product and the merits of our implementation approach.

For my specific role in the project, I started looking ahead into scale integration. Learning the requisite communication methods was a focus of mine this week, especially in the context of seamless integration with mySQL. I came up with 3 proposals, weighted carefully for their relative merits and drawbacks:

  1. Arduino RS-232: There is a lot of support for this chip, which serially communicates the weight reading to a computer through USB. This approach would involve a fair amount of scripting to implement a “polling feature” that Prof. Savvides and our group discussed this week in our meetings. In particular, the code would need to sample the weight reading at a high enough frequency (possibly every second) to detect significant weight changes, to then write to the database. Although this is a good starting point for the project and is largely scale-choice invariant, I am not too fond of this approach given that RPIs are a little easier to work with in this context than Arduinos, and the fact that a USB connection may be rather unwieldy.
  2. ESP-8266: This option was one that I had to do a bit of searching for, but this chip works quite seamlessly with mySQL. Along with the scale and the chip, the final design would presumably involve a load cell amplifier, 5-volt power supply, and jumper cables to hook everything together. The chip would have to be controlled through PHP code, which is something that I would need to familiarize myself with, and the documentation/support is not nearly as good as it is for a product like Arduino or RPI. Much like the previous example, it functions as a good proof-of-concept approach.
  3. Xiaomi Smart Scale with Bluetooth RPI communication: Also works seamlessly with mySQL, and has the benefit of being a Bluetooth communication in terms of elegance, but requires most implementation and background knowledge to successfully build. Some of the challenges include proper recognition of the MAC address of the Bluetooth dongle (for the program to recognize the RPI), and some additional code for the busy-polling feature we would like to implement.

 

My preference is to leverage one of the first 2 options towards a working idea, and then, maybe as a stretch goal, work with my teammates to getting a Bluetooth communication setup working. It would greatly improve the aesthetic of the project and would automate several of the features we are looking to incorporate.

 

As a group, we started laying the groundwork for our project’s essentials, and I helped across the scope for such collaborative ventures: exploring image recognition and classification architectures with Steven, to working with Grace on backend deployment. Grace’s expertise in local deployment (learned from her web applications course) gave us a proof-of-concept idea of deploying a scalable application through DB.sqlite3. We are debugging some of the view actions as part of the MVC architecture, and remain committed to our goal of transitioning to a flexible mySQL database. One of the main things we learned about is the importance of the database in achieving our latency goals. Updating and reading from this database are operations that should fit within our overall latency requirements of 30 seconds or less. Steven has expertise in efficient database manipulation and traversal, and I look forward to working with him to ensure the complexity of database traversals can be done in logarithmic time as opposed to linear. We plan on leveraging the fact that mySQL organizes workbench data as an object hierarchy (GRT Tree), which should go a long way toward efficient traversal and manipulation.

 

We look forward to design presentations this week as the semester begins to unfold.

Posted on

Grace Liu’s Status Report for February 10th, 2024

During this week, I presented for my group’s proposal presentation on Monday. My group and I initially worked together to expand on some of the ideas from our abstract and accurately reflected them onto the required themes of the presentation. I was then able to elaborate on the slide requirements to help the audience better gauge our product use. Since we had already established our problem and solution requirements, reasonable quantitative metrics were defined that state facts about our system.

Using the hardware components my team members scoped out, I was able to sketch a mockup of our envisioned product in CAD with the corresponding dimensions to be included in our solution approach. I believe this was an important step to visually demonstrate to our audience how users would utilize our food tracking product in realtime. Afterwards, the steps to the project were split into three different components that could be worked on asynchronously: image classification, storing scanned information to a cloud database, and the website interface for users to interact on. 

Moving onto technical challenges and testing verification, communication between each component would be hard to achieve in an efficient manner since image classification, scale weighing, and forwarding backend calculations to the database would all need to be processed and displayed in a relatively short amount of time. There also exists the decision of choosing between a wireless implementation or connecting our hardware to be offloaded to a computer for debugging purposes. We will decide between the two upon further testing. In parallel to the steps of our solution approach, verification and metrics were defined to guide our initialization process.

In terms of progress, I am on schedule with what I wanted to accomplish this week. However, upon receiving feedback during our Q&A session and overall notes from the faculty, there are many additional factors that need to be considered. I will need to review more on the technicalities of the OCR algorithms we will be testing with. Additionally, we have established that cooked meals will not be part of our testing plan and our product serves more as an inventory tracker for food.

A mockup of the website framework has been created that allows users to login through either a new account or Google OAuth. At the moment upon logging in, there is dummy data that will eventually be the food items users scan in. I plan to add functionality that lets users delete food items as they desire to be added to the total macronutrients count. Also, my group and I may discuss more on how interactive we would like to make this user interface and ask Professor Savvides for additional feedback.