Author: yicheng3

This is the last status report for the project! As we step in finals week, I crafted and presented the final presentation, and the final presentation seems to have turned out well!

The CV system is now also capable of making quantity guesses. I updated the API specification on the backend to support the new capability in order to allow quantity information to be sent from the CV system to the backend, and then to the frontend for display in the confirmation dialog. This required a mini-rewrite of the schema as well as the relevant API endpoints, but it was not too troublesome and I was able to deliver it in just 2 days.

In preparation for the final presentation as well as poster, I updated the overall architecture diagram, and consolidated test results and compared them against our use case requirements. All our explicitly defined use case requirements have been achieved! (see presentation) This is a testament to our good work both individually and as a team.

Coming up, there are some stretch goals and “nice-to-haves” that we can try to implement. Of higher priority is the final report and demo, and with that we will wrap up our project, on time and on schedule!

It turned out that the fix applied last week on the backend relating to items not being properly decremented upon removal was incomplete, and there were still edge cases. Thankfully, this was uncovered during further testing and integration this week, way in time before the final demo. This bug has now been fixed (and verified through the same rigorous testing process) through a complete rewrite of the item quantity computation logic. In short, instead of returning a single result after one pass through the transactions, it now makes 2 passes in order to properly account for item returns. In terms of time complexity, both approaches have a complexity of O(n), so there should be no significant difference in backend latency as a result of the change. In exchange, not only is the bug fixed, but the logic and code is also much more clearer and readable, significantly helping in any future efforts to expand the backend.

I have also made the backend more robust against incorrect quantity data, e.g. “removing” more apples than there are in the fridge. In essence, the backend now computes as if there is a floor of 0, instead of going into negative numbers, which is not logical. We passed over this aspect of the project initially as we prioritized producing new features that are suitable for the demo, but the time is now right for this change to be implemented as the product matures.

My main priority this weekend is to produce and complete the final presentation for our project. I recorded several video demos that I intend to incorporate into the slides, and collected a series of performance and benchmarking data to compare against our use case requirements (good news: we’re meeting them!)

Coming up, the largest priority is to complete the wrap up of the project as the semester comes to a close, in the form of the final presentation, poster, and final demo. We also intend to expand user friendliness and productiveness even further by allowing the user to select multiple fruits in one confirmation, based off our discovery that the CV is capable to predicting all 3 different types of fruits when all 3 of them are placed on the platform.

We’re comfortably ahead of schedule! The project including the hardware is now all in place, integration tests are nearly complete, and the semester is coming to a close. This will mark a great end to our CMU careers!

This week I have been working on testing and ironing the bugs that have been identified. One bug that cropped up during the demo presentation itself was a bug in the backend logic that was resulting in items not being properly decremented after removal. This stemmed from our transaction-oriented database design, where instead of a running total, a list of transaction deltas are stored in the database instead. A bug in the logic in collapsing these transactions (i.e. one loop was reading from the front while an inner loop was reading from the back) resulted in the unexpected behavior. Fixing this bug was relatively nasty as it meant having to store more information for computation – the original quantities of each addition. This bit of logic was especially complicated, due to the simultaneous “first-in-first-out with replacement” nature of our application.

Additionally, I have also began work to expand the database to support presenting nutritional information, a new feature we are planning to introduce. I will be expanding the schema by adding new tables, and creating new API endpoints for the new feature. I expect this will take around 1-2 weeks.

Overall, we are on track and I am personally confident in our ability to finish this project satisfactorily.

This week was an incredibly productive week for me because the primary blockers, such as the development environment and library issues have been tackled in the preceding weeks, leaving me free to implement the meat of the back-end swiftly. I delivered multiple API endpoints and worked closely with Alex and Samuel to integrate the API with their front-end and CV system respectively, producing a cohesive product very rapidly.

One thing I also achieved this week is that I actually refactored the database schema in order to better work with Samuel’s idea of sending a list of items the CV system has predicted, in descending order of probability, and presenting it to the user. The previous schema did consider such a use case but was not well designed for it, and did not store the CV’s predictions in the database at all. By swiftly refactoring (which was made much easier by the Prisma ORM and TypeScript!) I managed to deliver the results we wanted for demo week and beyond.

Going by my personal Gantt Chart, we are now exactly 1 day ahead of schedule. It’s a fantastic sign, and I believe that with this pace, we can work even faster. In the following weeks, I have identified several improvements my API could perform, such as coalescing multiple transactions within a single day, as well as fixing some security vulnerabilities.

Indeed, as I’ve talked about in the team status report, we are now entering the phase of the project where the fun truly starts, and the product begins to take shape. This week, I kicked off the integration process between the front-end and back-end, ironing out kinks such as CORS header mismatches and API design issues, paving the road for full integration ahead. Right now, the front-end is now able to display real expiration dates fetched from the API.The back-end API had quite a bit of work done to it as well. Logic has been added for when items are replaced into the fridge, ensuring that the item count and expiry dates remain correct even when the same item is removed and placed back into the fridge. This is done by storing the state of the fridge – new items are only added to the tally when it is in “add” mode, i.e. when the user is loading new groceries from the store. All other additions in the “track” mode are considered replacements of previous removals.

We’re still healthy in terms of schedule, but with just a bit over a month left, time can get a little tight if we are not careful. Going forward, we will be integrating the back-end with the Jetson and CV system, and I will also continue working with Alex to bring other aspects of the front-end into fruition, such as editing previous transactions.

I did not make as much progress this week as I had planned for due to an illness from Wednesday (Mar 16) till Saturday (Mar 19). However, despite that, I did manage to get some work done and achieve the progress we needed this week to keep us from falling behind.

Specifically, I brought the back-end API online on a server connected to the Internet, allowing for integration process with the front-end and the Jetson to begin. For example, calls to the API can now be made as follows:

$ curl -u "<secret key redacted>":"" -d "" https://fresheyes.oliverli.dev/item/2/
{"id":2,"name":"Apple","shelfLife":7,"unit":"piece","itemExpiry":[{"expiry":1648050152638,"quantity":5},{"expiry":1648351218448,"quantity":2}]}

This fetches all details for items of ID 2, which in our example here consist of apples. This data, including the expiration dates, are all returned in JSON format which can be easily parsed by a JavaScript front-end and presented to the user visually.

I also implemented enough remaining API endpoints to enable us to begin the integration process meaningfully. Specifically, there are now API endpoints analogous to CRUD – create, read, update, and delete, the 4 essential operations on the various resources. Although the back-end is still incomplete, this means that my focus from now on is the key application logic instead of lengthy boilerplate, which should progress substantially faster. I will also be working on the front-end integration together with Alex, so as to speedily unblock his work and enable him to continue with the front-end.

Besides the code, I also worked on the ethics assignment this week. Thinking about the ethical implications of the project was a new and interesting take, since I did not initially think this project could raise any ethical concerns at all. However, thinking critically, there were indeed aspects of personal data collection that can become problematic, making the ethics assignment an enlightening endeavor all things considered.

This week, I have brought our back-end Express code-base to strict, 100%, type-safety. This type-safety is also enforced using automated code linters, specifically eslint and prettier, which are automatically run after each commit. This enforces a standardized and readable code style, ensures that strict type-safety is maintained, and even catches common bugs such as misuse of “null-able” values before the code is even merged into master.

In a large enterprise context, such type-checking and automated linting is a vital part of the development process, as it ensures a consistent code style and enforces a common standard across all developers. Even within this project, with a team of 3, each of us may write code in very different ways. Using an automated linter therefore automatically and safely transforms the written code to conform to our standard, ensuring maximum developer productivity across all 3 of us.

A stretch goal would be to extend the linting process to require automated tests, and to execute these automated process as part of a Continuous Integration/Continuous Deployment (CI/CD) process with each commit and each merge. This is another standard operating practice in fast moving and agile enterprise environments. However, considering the scale and timeframe of our project, I do not see the need to enforce this level of rigor at this point in time.

I am currently on schedule and am well poised to deliver on the back end APIs. Next week, I will focus on delivering a key set of API endpoints that Alex can integrate into the front end.

This week, I defined the database schema, together with data relations, in a Prisma file. Defining the schema as code in this way also allows for easy tracking in a VCS e.g. Git of any changes, allows for safer database migrations as the schema changes, and also allows other group members to speedily spin up a development database for their own use.

In addition, I have decided upon an Express.js stack, using TypeScript with Prisma as the ORM. Using Prisma and an ORM abstracts away the complexity of crafting SQL queries, and also protects the application from SQL injection attacks by design. It even provides type safety, which together with TypeScript provides for full type-safety across the backend. I have spun up a starting Express application with Prisma and the schema, allowing me to continue building out the application. I chose TypeScript for its type-safety guarantee and the extensive standard library provided by JavaScript, and I chose the Express framework because it has great support for API endpoint development, which will form the vast majority of the backend work.

In the coming week, I will implement and bring online the first few API endpoints, allowing for some rudimentary functionality and testing after integration.

This week, I focused on back-end architecture and API design, and also came up with our testing strategy, task distribution, as well as task schedule (Gantt Chart) to guide us in our journey ahead.

Before beginning any back-end development, it is of vital importance to plan out the details such as the technology stack used, database system, and the specifications for API endpoints. The entire system will consist of up to 4 discrete components including the back-end. Hence, coming up with an exhaustive list of API endpoints, their functionality, and their input and outputs is of vital importance not just in guiding the design of the API endpoints and the back-end architecture, but also guiding the technical direction of all the discrete components, serving as a sort of technical “glue”. It is much easier to come up with a definitive list of use cases and develop around that structure, than it is to add arbitrary use cases as they arise during development. The architecture I have developed is linked here. Together with the API, I have also been exploring technology stacks such as Node.js with Express and SQLite, and selecting the most appropriate technology stack that will serve our use case best, including factors such as scalability and ease of integration with our other system components.

I have also worked on the task schedule, including the task distribution. The Gantt Chart is a visual, “waterfall”, breakdown of our tasks as well as phases of development. It enables us to keep track of whether we are on schedule and any upcoming tasks, in every stage of our project. Currently, we are on schedule and I see no significant roadblocks in the immediate future – let’s hope this keeps up!