Category: status (alex)

This week, I worked on finishing some UI fixes and integrating everything together. First, I finished the nutrition information popup:

Then, Samuel and I spray painted the platform for better visibility. The computer vision algorithm improved drastically with the nicer white background.

Next, I set up the Jetson for the computer vision processing. We had a lot of issues with performance, but I discovered that when I disabled all of the GUI processes, the performance improved slightly. We wanted to improve performance using Tensor RT, but I found it very hard to compile (resource limited and arm processor) and no precompiled binaries were available. Fortunately, disabling the GUI was enough to improve performance, but it appears that there is a bug with torchscript that causes the performance to be extremely slow on the first detection. After “warming up” it seems to perform much better, so we will account for this in the code. My hypothesis is that the CPU needs to optimize what goes in the swapfile since it has to use about 1 GB of swap to run the process, which is significantly slower than RAM.

Finally, I tested the performance of the algorithm and we worked out a few bugs on the API and made sure that we are in good shape for the demo. I also collected detection times for our performance evaluation. I then worked on putting all this data into the final presentation.

Last week, we used the saws and equipment from carnival to cut up our wood into usable pieces. This week, we finally got the chance to actually assemble them into a folding platform to assist in our computer vision algorithm. This is necessary to create a nice, light background that is easy to mask out and better detect changes on. For example, when someone walks by, there won’t be significant change to the camera detection since it won’t see the foot, just small changes in lighting. The platform also helps to get more regular sizing of fruit. Users will know to hold it at a consistent height (next to the platform) which makes classification and segmentation much easier.

The folding feature is a small feature so that the fridge takes up less square footage when scanning is not in use.

This week I finished construction of the folding platform for the camera. using the pieces we had cut last week. We found various wood screws in tech spark to finish attaching the foldable hinges. The wood screws were slightly long, so I used hot glue to cover up anything dangerous that could catch.

We also purchased white spray paint for the platform to even further decrease any barriers of detection, but when Samuel ran the computer vision it appeared to work fine with just the light colored wood. I may still paint it next week for aesthetic purposes.

On the front end side, I have begun work on a nutrition information system in place of the recipe recommendation system. The recipe recommendation system was proving very difficult to integrate and test, since these timers are for very long periods of time and required extra frontend and backend integration. Instead, when clicking on items in the interface, it will display nutritional information about the item. For an apple, this will pop up: 

I’m still in the process of integration and will work to finish this up next week. Overall, we’re in very good shape. I just need to get the front end up and running on the tablet next week.

This week, we did an ethics study on our product and learned about how to make more ethical products.

I also worked on integrating the front end with the back end API. I’m a little behind schedule with this, but this was because I didn’t have the API earlier so I just need to work through this quickly. Unfortunately the API is still incomplete, so I will just have to work through what I can now. This week, we got the first endpoint up to tell us when food is going to expire and adds it to the calendar.

I worked on a significant portion of the front end, and was able to get a live demo hosted at https://capstone.astrasser.com:2096/

This week I finished the typescript for the site design. I used typescript since in the past it  has caught lots of bugs without needing to run and debug for me. Although it took a while to set up, it will pay off in the long run.

Additionally, setting up the domain forwarding was difficult for the demo, but will also speed up development a lot, as it allows me to quickly test on the tablet.

Next week, I will work on integrating the code with the API endpoints. I’m slightly ahead of schedule, but need to work hard since the rest of the semester will get busier.

This week, I mainly worked on the presentation (I was the presenter), and the UI / Front end.

For the presentation: I worked on creating a lot of the content, such as the use case requirements and the solution approach. I practiced presenting and presented our presentation to the class. The presentation went very well and we are set to begin working on our project.

I started the UI off by designing our logo (shown as our website favicon). I set up basic bootstrap for the website.

Next week, I will create the basic calendar interface and the popup for correcting CV detections. Hopefully, when the API endpoints are created, I can start working on basic requests.