This week was all about getting the torso recognition software to work. I decided to go with trt-pose due to its compatibility with Nvidia Jetsons as well as its simplicity compared to OpenPose. We ordered a new camera that was compatible with this as well to work with the software. However, the camera ended up not being compatible with trt-pose. The SAINSMART imx219 camera does not work with trt-pose and we have another camera ordered to be compatible with the software. In the beginning of the week, Jeremy and I fully assembled the mirror and frame as well as testing it so the UI is visible through the mirror. This upcoming week, when the camera arrives, we will be testing it and making sure that trt-pose works no matter what. Also, as a backup plan we will be reaching out to groups that are using OpenPose to figure out what is wrong with our system and get the torso recognition up and running for the interim demo. This obstacle has set us back a week in terms of schedule but I am confident that we will be able to get it up in running before we go on our break for carnival.
Yun’s Status Report for 4/2
This week, I touched a bit more upon the weather api as the one that I worked on last week was based on http request. It may cause unnecessary delay or complication in the future, so I worked on running everything completely locally, and was able to complete the task. On top of that I did more research on some basic recommendation algorithm, and also acquired some dataset regarding which outfits are appropriate for which weather.
I think we are bit behind of schedule, especially due to the unexpected delay in the integration of camera, jetson, and openpose. Two people are working on resolving that issue which delays the rest of the project. However, I am very optimistic about the rest of modular tasks as well, because I am almost done with the recommendation which means I can hop on the mobile app development and general integration as well.
Next week, I will work on interim demo presentation, and will work on integrating all modular tasks that my team has worked on so that we can have a demo. If I have some time left, then will work further on the basic recommendation system I have and improve it to be more sophisticated.
Ramzi’s Status Report for 4/2
Unfortunately, I was preoccupied with illnesses throughout the week which greatly hampered my progress. I brought forth concerns with my teammates about support for the bluetooth components of the project, and we ultimately decided to switch over to utilizing a wifi system in order to transfer data to and from the mirror. This is bound to create a multitude of security concerns, however we think it should be fine as long as we will address aforementioned security concerns in the future.
Team Status Report for 4/2
Early on in the week, we finished creating the frame for the mirror, and attaching the mirror itself to its frame. Jeremy and Wonho decided to use tape to hold the four mirrors of the frame in order to compensate for the flexibility of the mirror, which is made of acrylic. We were able to see the background through the mirror, which was not our objective, so we also used a cardboard box as a back to the frame to block any excess light from coming through, and the result is that only the monitor is visible through the mirror.
Jeremy and Wonho Posing with the Mirror
As such, we have moved on to the most significant challenge to the project, our software components. To be specific, we mostly worked on the Openpose troubles and getting the Nvidia Xavier and trtpose to work as we want it too. Unfortunately due to some team members being preoccupied, we were not able to approach the other software components this week. We switched from utilizing bluetooth to using a wifi connection for our application and mirror to communicate, which may provide a few security concerns to address, but will ultimately be easier to implement due to the additional support provided.
We seem to be a little behind schedule, but in the coming weeks we hope to make more progress on the software components, such as trtpose and the application, in order to be able to present them at the interim demo.
Jeremy’s Status Report for 4/2
This week Wonho and I attached the mirror to our mirror frame. We ended up simply taping the four corners of the mirror onto the four corners of our mirror frame because the mirror bends a lot as the mirror is made of acrylic. In a well-lit environment, it is possible to see through the front of the mirror which is not what we want, so we have decided (as a temporary solution) to use the cardboard box that the mirror came in to cover the back of our smart mirror and block any light from coming in through the back. I tried putting my phone behind the mirror and setting it at a full brightness level to test if the smart mirror would still be able to show the monitor display and it is visible as expected.
We are running into some trouble with the Xavier not being able to recognize that we have a camera input. A week ago, we had no issues with this but they appeared out of nowhere. We tried switching out the CSI cables, reading through troubleshooting threads online and it still would not work. This is the only obstacle that is preventing us from testing out trtpose and I am planning to go in on either Sunday/Monday to get this working again.
I would say that we are slightly behind schedule, but as long as we get trtpose working before the interim demo, then we should be in a very good spot to finish up during the last two weeks of this semester.
Wonho’s Status Report for 3/26
This week we decided to move on to our plan B of using trt-pose after struggling for a week to get OpenPose working on our Jetson Xavier. We spent endless hours on it trying to debug and reinstall drivers but it still did not seem to work. I’ve wiped our SD card and have installed trt-pose on the Xavier. I just need to go in the lab and test it with the camera to see if it works. Other than working on this crucial part of our project, we decided to get the hardware components (the frame of the mirror) out of the way by setting an internal goal to finish the mirror this week
As a team, we all went to Home Depot to purchase the wood necessary for the project and were able to finish building the frame for the mirror. In terms of the hardware, we seem to be on track but lacking slightly in the software progress. This will be the week where I will really start to make progress with our CV code as trt-pose should be up and running soon.
Ramzi’s Status report for 3/26
This week I reviewed the system we are using for our application, and tried to learn more of its functions so that we could integrate the application thoroughly. We want to find a way to upload images through the app to our mirror. We also spent a large amount of time working on our physical mirror, as we have been trying to get access to a woodshop and create a frame that would work for a while. I created a diagram that thoroughly detailed how we would design the physical dimensions of our mirror.
Myflection_Mirror_Frame_Dimensions
We went to home depot, and purchased two 2″x10″x8′ pieces of wood, before heading over to the woodshop at techspark to have the pieces properly cut into what we needed. We screwed together the pieces together, and managed to create the completed frame below:
For next week, we will be resuming working on the software components of our project, as we try to bring together the application and the video analysis software to create a coherent system.
Yun’s Status Report for 3/26
Since I worked on modular tasks related to color detection last week, I moved on and started to work on weather API. Using weatherapi.com, I was able to successfully run weather API locally using node js which is the same programming language used for color detection. On top of that, I also visited TechSpark, got some advice from Ryan, who is in charge of woodshop, and also arranged our wood work schedules with him. As a team, we all worked together at wood shop, building the frame holding mirror and monitor.
I think we are on schedule as the wood frame is done, and the rest of modular tasks are almost done as well. Next week, I am hoping to connect weather API and color detection API together to build a small system that recommends an outfit based on those information.
Team Status Report for 3/26
The most significant risks that pose a threat to the overall completion of our project is the issues that we have been facing with OpenPose. There are a bunch of Github threads that have been made by others who have had problems with setting up OpenPose on their NVIDIA Jetsons yet the documentation is either outdated or incorrect depending on each person who had the issue. Our team faced the same challenges and therefore created a contingency plan where we will be using trtpose now. We plan to wipe the SD card that was in our NVIDIA Xavier and start from scratch by installing trtpose instead of OpenPose.
In terms of the mirror frame that we built, we made a few changes to our initial blueprint of what the frame would look like. Our most recent diagram shows the exact dimensions of each side of our frame and this is what we referred to when constructing the frame in the woodshop downstairs.
None of the scrap wood in the woodshop matched the dimensions of our mirror frame so we had to go to Home Depot to buy wood. This incurred an extra cost of around $50, but it is still well under our budget limit. Another extra component to our mirror design that we thought of after constructing the frame was how we would hold the acrylic two-way mirror in place. The mirror itself is quite bendy and malleable for some reason, so we decided that we would attach triangular pieces to each corner of the frame which the mirror could stick to without falling forwards. A picture is shown below of our finalized frame.
During the construction of the mirror, we got assistance from an instructor in Techspark who was generous enough to give us a few tips about woodworking as well. One obstacle was the fact that Home Depot claimed they sold this wood plank with dimensions of 2in x 10in x 8ft, but the height of the wood was actually shorter than 2 in which resulted in an unforeseen consequence of having to cut the wood again to match the design we wanted to in our original blueprint.
We moved the frame back to our bench, so now that the frame is done, all we need to do now is get the software working and running. We are on schedule in terms of what we have built so far, but we are behind schedule with testing out the software because it hasn’t been set up properly yet. That will be our group’s main focus for the coming week.
Jeremy’s Status Report for 3/26
This week I worked on figuring out what was wrong with setting up OpenPose with our Xavier. After hours of endless debugging and reinstalling, Wonho and I decided that it may be easier to switch to our plan B of using trt-pose. I’ve been working on the app a little more, but the past few days, I have been working with my team to get to the woodshop and create the frame for our mirror. We went to Home Depot to get two 2 x 10 x 8ft wood planks which we cut at the woodshop down in techspark with the assistance of an instructor there. As a result, I don’t have any photos to post in my individual status report, but will post the mirror frame that we built as a team in the team status report. For the next week, I hope to finish the app and get trtpose working so that we have something to show for the interim demo in the first week of April. The app dev process should be a little easier now that I learned more about frontend UI and GUI development in a class I am taking right now which also uses React and Node.js to create a backend and frontend that communicate via APIs. This is essentially the route that we are taking to have the frontend use a button that will send an API request to a backend which stores the user’s wardrobe.