Posted on

Team Status Report – 26 Feb 2022

This week, we completed our design review presentation, which we think went quite well.  Our main focus for next week will be the design review report, due Wednesday next week. After Wednesday, we will continue working individually on our various responsibilities to implement the CV, UI and back-end systems  for Samuel, Alex and Oliver respectively.

Currently, we have made good progress on the implementation side, and are slightly ahead of schedule in this sense (see our individual reports for more information):

  • Samuel: Completed C++ testing for CV. Discovered that the CNN network found from Medium does not work well and has major flaws with the customized architecture. Will begin work next week training and testing a new ResNet18 or AlexNet model.
  • Oliver: Enforced a rigorous common standard in the back-end code-base by integrating automatic linting, type-checking, and even bug catching tools. Brought the code-base to strict, 100%, type-safe standards, setting up the back-end for seamless and co-operative development regardless of each team member’s individual style, and ensuring that code pushed meets high levels of rigor. Will deliver a core set of APIs next week built upon this level of rigor for integration with front-end
  • Alex: Completed most of basic UI. Will start collaborating with Oliver once API side is complete. Helped Samuel with the training of the classification algorithm
Posted on

Alex’s Status Report – 26 Feb 22

This week, I helped with the design review presentation and began work on the design review report. I also assisted Samuel in the training and fixing of the neural network for classification.

I was not able to start the API integration, as the endpoints aren’t ready yet, so I fell slightly behind schedule. However, the API is being built in a much more sustainable way, so making these early sacrifices will save us lots of debugging headaches down the line.

Next week, I will hopefully begin work on integrating the code with the API endpoints, as soon as those are ready.

Posted on

Oliver’s Status Report – 26 Feb 2022

This week, I have brought our back-end Express code-base to strict, 100%, type-safety. This type-safety is also enforced using automated code linters, specifically eslint and prettier, which are automatically run after each commit. This enforces a standardized and readable code style, ensures that strict type-safety is maintained, and even catches common bugs such as misuse of “null-able” values before the code is even merged into master.

In a large enterprise context, such type-checking and automated linting is a vital part of the development process, as it ensures a consistent code style and enforces a common standard across all developers. Even within this project, with a team of 3, each of us may write code in very different ways. Using an automated linter therefore automatically and safely transforms the written code to conform to our standard, ensuring maximum developer productivity across all 3 of us.

A stretch goal would be to extend the linting process to require automated tests, and to execute these automated process as part of a Continuous Integration/Continuous Deployment (CI/CD) process with each commit and each merge. This is another standard operating practice in fast moving and agile enterprise environments. However, considering the scale and timeframe of our project, I do not see the need to enforce this level of rigor at this point in time.

I am currently on schedule and am well poised to deliver on the back end APIs. Next week, I will focus on delivering a key set of API endpoints that Alex can integrate into the front end.

Posted on

Samuel’s Status Report – 26 Feb 2022

The highlight of this week for me was delivering the Design Review presentation. I think I did quite well for the presentation, with my teammates and Professor commenting that the presentation was polished, with good graphics and content. We are currently working on the design report, and will use the content of the design presentation to write it.

On the technical side, I was able to make significant progress on the PyTorch port for the CV application, which was coded in C++ so that it can run optimized code on the Jetson, for maximum speed. The C++ application is now able to run the trained PyTorch model and spit out a set of predictions for the top 5 highest probability classes.

There were several challenges with the porting process, including the lack of good PyTorch documentation, which resulted in it being very difficult to figure out how to properly convert between C++ formats (cv2 to torch Tensor for example) and also important considerations in ensuring that the model can be serialized properly for running in C++ (in particular, no nested functions etc.). This is a lesson learnt on the importance of good documentation, and the pain/need of having to pour through various forums and articles as a result.

However, after training and testing the network, I began to realize big problems with the trained model. Most notably, the model failed to produce correct predictions. After consulting with Prof Mario’s PhD student, we realized that we were using a highly customized model that was not designed properly, and was not even a proper “ResNet” (lacking fully residual layers). To this end, he advised us to use other preexisting models like ResNet18 or AlexNet. This is a lesson learnt as to not blindly copy code over intern

Next week, I will focus on trying to train our data on either ResNet18 or AlexNet, as well as test it in the new C++ classifier. (There is also a Python one for quick testing, in case the C++ one still has bugs). Hopefully I will be able to train up a network that will achieve our desired accuracy of 85% (the network itself should reach about 95% validation accuracy).

Fortunately, despite this setback, we are currently still on schedule, because we were previously ahead of schedule with the successful porting of the C++ application.

Posted on

Alex’s Status Report – 19 Feb 22

I worked on a significant portion of the front end, and was able to get a live demo hosted at https://capstone.astrasser.com:2096/

This week I finished the typescript for the site design. I used typescript since in the past it  has caught lots of bugs without needing to run and debug for me. Although it took a while to set up, it will pay off in the long run.

Additionally, setting up the domain forwarding was difficult for the demo, but will also speed up development a lot, as it allows me to quickly test on the tablet.

Next week, I will work on integrating the code with the API endpoints. I’m slightly ahead of schedule, but need to work hard since the rest of the semester will get busier.

Posted on

Oliver’s Status Report – 19 Feb 22

This week, I defined the database schema, together with data relations, in a Prisma file. Defining the schema as code in this way also allows for easy tracking in a VCS e.g. Git of any changes, allows for safer database migrations as the schema changes, and also allows other group members to speedily spin up a development database for their own use.

In addition, I have decided upon an Express.js stack, using TypeScript with Prisma as the ORM. Using Prisma and an ORM abstracts away the complexity of crafting SQL queries, and also protects the application from SQL injection attacks by design. It even provides type safety, which together with TypeScript provides for full type-safety across the backend. I have spun up a starting Express application with Prisma and the schema, allowing me to continue building out the application. I chose TypeScript for its type-safety guarantee and the extensive standard library provided by JavaScript, and I chose the Express framework because it has great support for API endpoint development, which will form the vast majority of the backend work.

In the coming week, I will implement and bring online the first few API endpoints, allowing for some rudimentary functionality and testing after integration.

Posted on

Team Status Report – 19 Feb 22

This week, we worked as a team primarily on finalizing and documenting our design as part of the design review presentation and report. In particular, we finalized our overall architecture and the way in which everything will integrate together. Notably, our original design of a front facing camera with a vertical platform got changed to one where the camera faces downward on a flat white surface. This was to facilitate a more intuitive and non-intrusive scanning process.

Architectural Overview:

> Architecture (Original Handwritten)

The following is a summary of our individual work (for more details, visit the individual status reports):

  • Samuel was able to write the code needed to train the neural network classifier (we now have a pretrained model with 98% accuracy), and is working on a C++ application that can use it to classify an image.
  • Oliver began work on the backend, and defined the database schema and column relations in Prisma. He is now working on implementing the API endpoints defined in the architecture.
  • Alex was able to complete a significant chunk of the front-end design, and get it running at https://capstone.astrasser.com:2096/

Next week, we will continue working independently on our individual portions of developing the CV, UI and backends (more details in the individual reports). As a group we will also start building our fake fridge and scanning platform

Posted on

Samuel’s Status Report – 19 Feb 22

This week, we worked on the design review slides, and as part of the process, we finalized our designs for the attachment system, CV algorithms, UI interface and backend. Notably, I made a contribution to a new scanner system design, where I suggested making the camera scan overhead onto the platform as opposed to a front-facing camera as originally designed. This will allow for a more intuitive and less intrusive scanning process.

In particular, as the one in charge of the CV algorithm, I wrapped up research on the various algorithms to use for classification. In particular, I decided to go with the ResNet based CNN instead of traditional SURF/SIFT methods because of the better accuracy and performance. I modified the code of this tutorial to train a classifier and was able to successfully train a model that achieved 98% accuracy after 10 epochs.

 

 

 

 

 

 

However, it remains to be seen if the classifier will work well with validation data (ie check for overfitting), and especially whether it will work with real-world data (our actual setup) . Next week, I will be working on the C++ PyTorch code to run said trained network, meant for optimized runningon the Jetson. I will also begin working on a basic webcam setup (the webcam just arrived this Wednesday!) and collect real-data images that I can use for testing.

Posted on

Oliver’s Status Report – 12 Feb 22

This week, I focused on back-end architecture and API design, and also came up with our testing strategy, task distribution, as well as task schedule (Gantt Chart) to guide us in our journey ahead.

Before beginning any back-end development, it is of vital importance to plan out the details such as the technology stack used, database system, and the specifications for API endpoints. The entire system will consist of up to 4 discrete components including the back-end. Hence, coming up with an exhaustive list of API endpoints, their functionality, and their input and outputs is of vital importance not just in guiding the design of the API endpoints and the back-end architecture, but also guiding the technical direction of all the discrete components, serving as a sort of technical “glue”. It is much easier to come up with a definitive list of use cases and develop around that structure, than it is to add arbitrary use cases as they arise during development. The architecture I have developed is linked here. Together with the API, I have also been exploring technology stacks such as Node.js with Express and SQLite, and selecting the most appropriate technology stack that will serve our use case best, including factors such as scalability and ease of integration with our other system components.

I have also worked on the task schedule, including the task distribution. The Gantt Chart is a visual, “waterfall”, breakdown of our tasks as well as phases of development. It enables us to keep track of whether we are on schedule and any upcoming tasks, in every stage of our project. Currently, we are on schedule and I see no significant roadblocks in the immediate future – let’s hope this keeps up!

Our current Gantt Chart
Posted on

Alex’s Status Report – 12 Feb 22

This week, I mainly worked on the presentation (I was the presenter), and the UI / Front end.

For the presentation: I worked on creating a lot of the content, such as the use case requirements and the solution approach. I practiced presenting and presented our presentation to the class. The presentation went very well and we are set to begin working on our project.

I started the UI off by designing our logo (shown as our website favicon). I set up basic bootstrap for the website.

Next week, I will create the basic calendar interface and the popup for correcting CV detections. Hopefully, when the API endpoints are created, I can start working on basic requests.