(Please use this link to the pdf, not the one on the sidebar. That’s the old link to the docx with messed up formatting and I can’t figure out how to remove it ;_; )
Team B4: Smart Library
Smart Library is the intelligent service that allows you to see where socially distant seating is available in any public space, without having to visit the space yourself. Designed for CMU’s Sorrells library (but hosted in Pablo’s dining room due to Covid) , our node network will collect images and use a machine learning algorithm to determine which seats are available and safe to sit in. Smart Library does not save or collect any identifying information determine seat occupanct. Smart Library was created by Pablo Wilson, Krish Vaswani, and Arjun Raguram.
Video Link: https://www.youtube.com/watch?v=2dljjIvalZc
Team Status Update for 5/12
Since campus is now completely closed and Pablo is now based in Virginia, there was some reorganizing that needed to be done. First of all, with the library closed, we had to find a new location to capture images. The new location was Pablo’s dining room table, capturing the 4 chairs there. With this new set up, Pablo needed a couple of extra parts to get the image capture nodes set up(things like wires and breadboards), so he was a little behind, but data is now being captured and uploaded. The rest of the system is following close behind.
Arjun’s Status Update for 5/12
This week Pablo and I discussed integration steps of the Camera and Central Node code. The Central Node could still receive requests, but due the filesize of the images being larger than I expected (1 MB), the central node could not receive all of the image. This resulted in the code for receiving TCP messages to change a good bit. The connection resets early or the connection hangs in an infinite loop while trying to add larger file sizes. This is something I am currently still working on, but can find a solution for by the time of the final video and report. I am considering going back to a C solution due to pre-processing not necessary anymore. This would help due to receiving strings/buffers being able to use heap allocation effectively.
Pablo’s Status Update for 5/12
This week, the focus was on getting the system integrated and set up in my dining room. The breadboards and wires luckily arrived sooner than expected and I was able to have the image capture nodes set up! Image from the node below:
Still awaiting final integration with the server as I don’t have the server code yet, but an easy workaround is just automated batch uploading of the images to the model. I am task complete on my end and will now transition to helping out other areas to ensure we are fully functional by final presentation time.
Krish’s Status Update for 5/12
Last week, I finished making the model using my custom dataset. I came across a major problem when it came to working the YOLO model, which is what we had previously intended. Initially, I had thought that an object detection model like YOLO was ideal, since it is trained to pick up different objects in a scene. However, one shortcoming I did not foresee was that the images we would collect in our system were not like the natural images, because they were taken from an aerial view. The following two pictures display this disparity.
In the first image, we see the picture is taken from the side. The second image is taken from the top. For human beings, it is easier to identify the presence or absence of other humans, but for a machine learning model, this cannot be abstracted.
In order to fix this problem, I decided to use my own machine learning model, without any pretrained weights. The advantage of this is that it will solely focus on data that we have fed it, and so it doesn’t need to depend on natural images. However, the disadvantage is that I need to make a simpler model, since there is less data availability. For this reason I had to adapt the model, where instead of the model finding the location of the seats, I specify the location of the seats. Then, the algorithm would crop out each seat, resize the pictures and identify whether there was a person in each seat or not. I thought this was a fair compromise, given that the seats in Pablo’s dining room are in a relatively fixed position, and so is the camera that we set up. If we were to take this project further, there would be an extra cost associated with installing the system in a new location, but this cost would be negligible compared to the effort it would take to mount the camera and central nodes at the location.
Additionally, I worked on the frontend of the website. When we had planned on setting up in Sorrell’s Library, I did not know the layout of the seats, especially since we were not sure where we could have mounted the cameras. Once I got a few pictures of Pablo’s dining room, I could understand the layout and set up the website to mirror that layout. Right now, the website can read four occupation bits and display an appropriate html page with red and green colors based on availability, at different locations representing locations at the dining table.