Posted on

Team E2 Status Report: Feb 24

This week, our team focused on preparing our design presentation and report. We collaborated on designing slides, creating speaker notes, and beginning the writing process for the Introduction and Machine Learning documentation in our design report.

In terms of progress on our project’s design component, we made significant strides. We experimented with Support Vector Machines (SVMs), GoogLeNet, and ChatGPT’s API to optimize our classification results. We conducted tests on SVM formulations, continued building our dataset, and explored using GoogLeNet for image classification. Additionally, we tested ChatGPT4’s ability to read labels, which proved to be highly accurate.

Throughout the week, we didn’t encounter any major complications. However, I acknowledge that I could have contributed more to designing the physical product and creating the box. Moving forward, I aim to gain more hands-on experience with the physical design process while finalizing the Machine Learning algorithms (Steven).

Regarding the design presentation, we addressed key considerations such as project implementation, load cell amplifiers, food item differentiation, validation metrics, and backend complexity. We’re now focusing on completing the design report, which will require a deeper understanding of our project’s development procedures. For instance, we will use our block diagram and implementation plan to devise a more descriptive architecture plan.  We will also put more efforts towards connecting initial public health/safety/welfare, social, and economic factors we want our product to solve to the formulation of this solution. Of course, many tradeoffs emerge with designing an engineering solution, so we will dive into these considerations through our testing experience so far and feedback we have received from peers during our proposal and design presentations.

In terms of hardware-related risk mitigation strategies, we’re addressing concerns such as buffer overflow, data loss, recovery, and mismatched baud rates. We plan to implement control mechanisms for buffer overflow, backup data on nonvolatile memory for data loss recovery, and ensure proper synchronization of baud rates.

Overall, our progress is on track. We’re aiming to complete the imaging component of the project by Spring break, with each team member making significant contributions to their respective tasks.

Posted on

Feb 24th 2024 Status Report — Surya Chandramouleeswaran

This week was centered around building a convincing design presentation. With midterms and other commitments, there were challenging aspects to preparing a 10-12 minute talk discussing the primary features of our product, but overall, such an experience went well and was invaluable for our group in sorting out some of the finer details present in our design plan.

 

Some of the key takeaways from the design presentation include the following:

  • How much of the project do we want to implement by ourselves, as opposed to offloading the work to an API call?
  • Re-evaluating the need for load cell amplifiers to be placed between the scale reading to the ADC port of our microcontroller
  • Proper differentiation between labeled food items, versus standalone produce
  • Validation metrics and subsequent justification
  • Balancing how much complexity we want to offload to the backend. We are now working on the design report, which requires another level of understanding of our procedures in the development of our product.On my end (hardware), some of the risk mitigation strategies that I have thought about in the context of the microcontrollers include the following:
  • Buffer Overflow: This can occur when incoming data from the scale exceeds the capacity of the micro-controller \texttt{receive} buffer, potentially causing data corruption or subsequent instability. To address this issue, our team plans to implement control mechanisms in the controller scripting, following XON/XOFF software flow control protocols.
  • Data Loss and Recovery: In the case of failure, the data should be backed up on nonvolatile memory. This is present in each of our options for microcontrollers: Arduino holds the well-established EEPROM as an example of this, and SPIFFS is the equivalent option in the ESP8266 WiFi microchip. These are hardware backups that the database can read from if the system crashes. 
  • Mismatched Baud Rates: Arduino IDE handles synchronization, but because there is logic in the sampling of the scale values, this needs to be matched properly.

An additional consideration (pictured below) is a conversion from the RS232 serial protocol that the scale communicates data with to TTL logic employed by microcontrollers. This is something I realized in the building process. A converter, pictured and wired in the schematic below, will be necessary.

Progress-wise, we are doing well. We plan to finish the imaging component of the project by Spring break, so this upcoming week will be critical for Steven and me, while Grace is making good progress on the web application.

Posted on

Team E2 Status Report – Feb 17th

Ensuring that our product addresses a desired need in the market is critical to its reception and success amongst the public. This week, the team would like to explore this issue through 3 lenses: Steven will explore part A, Surya will explore Part B, and Grace will explore part C.

Steven:  With respect to considerations of public health, safety and welfare, our solution is designed to improve physical well-being for users by monitoring caloric intake. Likewise, our product aims to reduce the psychological burden of tracking what and how much you eat as our product does all the tracking for you. Furthermore, keeping track and inventory of all the food you buy at the store can be a burden, and people are quite forgetful sometimes. Our product reduces the stress in remembering what is in your fridge or pantry by keeping count of every item. We have ambitious goals for our product to reduce over and under eating which aims to better the physical health and well-being of the public.  With respect to safety, our product will have limited hazards as it will consist of a sturdy physical structure with a scale, camera, and lights. The components will be as minimalistic as possible, and we will conduct rigorous testing to make sure there are no malfunctions. The lights will require as little power as possible and will be properly embedded into the walls. Likewise, the camera will be off whenever the scale reading is 0 as a result the product will only turn on when an item is put on the scale.

Surya: Consideration of social factors, which relate to extended social groups with distinct sociocultural, political, and/or economic affiliations, is crucial in our increasingly interconnected environment. With this in mind, there are several key themes that we seek to account for in our design. First is the notion of accommodating varying dietary preferences: whether this be due to medical reasons, religious restrictions, or observance of regional cuisine, our algorithm looks to maintain as unbiased a performance as possible in its classification approach.  We look to achieve this accessibility in several ways. This involves details in the image recognition training process and enhanced user-user interaction on the application side that allows for features such as transparent sourcing of nutritional information and customizable dietary profiles (vegan, gluten-free, keto, etc.). Embracing diversity through building a prototype that lacks systematic bias is key to our consideration of social factors we look to meet in our design.

Grace:  With consideration of economic factors related to the system of production, distribution, and consumption of goods and services, a big concern that arose during our research is the amount of food waste generated by people on a daily basis in the United States. Not only does this pose sustainability challenges in the long run from an economic point of view, but also contributes to a substantial amount of landfill waste. Increased amounts of waste also require more management and environmental cleanup, leading to costly endeavors and potentially higher taxes for residents. Our solution, rather than focusing solely on mitigating the system of production and distribution of food goods, aims to assist users minimize waste during their food purchasing and consumption processes. While it does not directly address issues that may arise during production and distribution, helping users automate their daily calorie tracking can at least help them make more informed purchases and reduce waste as a whole. By providing accurate nutritional information, from label reading and food classification, along with better control over food inventory, our solution enables users to make more efficient use of the food at hand and consequently decrease food spoilage rates in general. We anticipate our product to not only  peoples’ wellness habits, but also make a positive economic impact on our society as a whole.

For our team’s status, we identified various risks that might prove to be an issue for our project. Surya was able to write down various risks regarding scale integration and using the Arduino as a middle component in our product. Likewise, Steven discovered risks with the image classification design and the various trade offs. As a result, we need to analyze our design decisions more next week and finalize it. Our contingency plans involve the various backup options we have which include the other design choices (i.e. buying multiple scales and Arduinos).

Likewise, we specified our schedule more as we got a better understanding of the necessary work and steps of our project. Here is the new Gantt chart that will be added to our design presentation:

 

Next week, we will make extensive efforts to produce results in terms of website design, image classification design, and physical product creation and integration.

Posted on

Feb 17th 2024 Status Report — Surya Chandramouleeswaran

My roles this week involved a dive into the design process in preparation for the upcoming design presentations. In particular, I decided to take the opportunity to present our team’s design plans next week. Preparing for this presentation requires strongly re-evaluating current project parameters and scope. Additionally, we found the feedback from our peers and faculty quite valuable from the proposal presentation, and we are eager to reflect on their suggestions in our design process. Building these slides, in concert with feedback and an improved understanding of our project, forced our group to critically examine every component that we plan to develop, in the hopes of convincing our peers of the value of our product and the merits of our implementation approach.

For my specific role in the project, I started looking ahead into scale integration. Learning the requisite communication methods was a focus of mine this week, especially in the context of seamless integration with mySQL. I came up with 3 proposals, weighted carefully for their relative merits and drawbacks:

  1. Arduino RS-232: There is a lot of support for this chip, which serially communicates the weight reading to a computer through USB. This approach would involve a fair amount of scripting to implement a “polling feature” that Prof. Savvides and our group discussed this week in our meetings. In particular, the code would need to sample the weight reading at a high enough frequency (possibly every second) to detect significant weight changes, to then write to the database. Although this is a good starting point for the project and is largely scale-choice invariant, I am not too fond of this approach given that RPIs are a little easier to work with in this context than Arduinos, and the fact that a USB connection may be rather unwieldy.
  2. ESP-8266: This option was one that I had to do a bit of searching for, but this chip works quite seamlessly with mySQL. Along with the scale and the chip, the final design would presumably involve a load cell amplifier, 5-volt power supply, and jumper cables to hook everything together. The chip would have to be controlled through PHP code, which is something that I would need to familiarize myself with, and the documentation/support is not nearly as good as it is for a product like Arduino or RPI. Much like the previous example, it functions as a good proof-of-concept approach.
  3. Xiaomi Smart Scale with Bluetooth RPI communication: Also works seamlessly with mySQL, and has the benefit of being a Bluetooth communication in terms of elegance, but requires most implementation and background knowledge to successfully build. Some of the challenges include proper recognition of the MAC address of the Bluetooth dongle (for the program to recognize the RPI), and some additional code for the busy-polling feature we would like to implement.

 

My preference is to leverage one of the first 2 options towards a working idea, and then, maybe as a stretch goal, work with my teammates to getting a Bluetooth communication setup working. It would greatly improve the aesthetic of the project and would automate several of the features we are looking to incorporate.

 

As a group, we started laying the groundwork for our project’s essentials, and I helped across the scope for such collaborative ventures: exploring image recognition and classification architectures with Steven, to working with Grace on backend deployment. Grace’s expertise in local deployment (learned from her web applications course) gave us a proof-of-concept idea of deploying a scalable application through DB.sqlite3. We are debugging some of the view actions as part of the MVC architecture, and remain committed to our goal of transitioning to a flexible mySQL database. One of the main things we learned about is the importance of the database in achieving our latency goals. Updating and reading from this database are operations that should fit within our overall latency requirements of 30 seconds or less. Steven has expertise in efficient database manipulation and traversal, and I look forward to working with him to ensure the complexity of database traversals can be done in logarithmic time as opposed to linear. We plan on leveraging the fact that mySQL organizes workbench data as an object hierarchy (GRT Tree), which should go a long way toward efficient traversal and manipulation.

 

We look forward to design presentations this week as the semester begins to unfold.

Posted on

Feb 10th 2024 Status Report — Surya Chandramouleeswaran

This week, I helped scope out some of the hardware tools needed to interface with our imaging system, which is the first component of interest for our product. I helped develop a plan for our team to have this component of our product finished and fully tested in about 6 weeks.

The first aspect of this involved researching existing cameras that met the frame rate and pixel resolution benchmarks that we needed for our OCR technology. Last week, during our weekly meetings, Prof. Savvides was kind enough to let me borrow a camera for the image recognition pipeline, suggesting that simple desk cameras would suffice for our applications, given the product would be held in a stationary manner. Additionally, Prof. Savvides offered to extend his expertise for making the detection algorithm as efficient as possible; I look to work with him and my teammates to build this processing pipeline.

Additionally, the layout of our apparatus, as well as details of the interconnect, were finalized; as the primary person in charge of hardware in this group, I took the initiative on this front. For computing reasons, and after prolonged discussion with the software-inclined members in our team, I decided that running the detection algorithms on an Arduino would not be feasible. While we certainly will order Ardunios (RS232 is rather cheap given the budget constraints of the class), I envision that offloading the computational processes to an external computer may be the best course of action in the beginning. After speaking with Prof. Savvides and our TA Neha last week, we agreed that it is best to test the performance of such tools before dismissing them.

 

We are on schedule progress-wise. We need to start building the recognition system and integrating that with the camera. I look forward to working with Steven on building some helper functions for our algorithm through Tesseract. Right now, we would like to test classification on our MacBook Laptop Camera, then integrate with the web camera that Professor Savvides provided us.

 

One of the design questions that has arisen is if we should implement an auto-capture feature on the camera (where the photo is captured when the camera deems it in focus). For training purposes, it is best if the image set is rather standardized in nature. This weekend I will brainstorm approaches to ensuring that the image classification algorithm gets quality images with ambient lighting and clarity in reading.

 

In the next week, I look to work with Steven in scaffolding the image recognition system and understanding what is expected from the camera which the system should speak to. I plan to begin testing on our built-in computer cameras, as I expect to work on some form of an autofocus-autocapture feature on the camera to ensure the training process of our classification neural network goes smoothly.

 

Attached please find a copy of our proposed design sketch:

Design Sketch