This week I worked mainly on training the model that we will be using for our clothing recognition and image warping. I had some difficulties training initially on my computer as CPU-only so I put everything on Google Colab and trained from there. I will experiment with different training methods once we deploy it on the Jetson. Judy and I built the TV stand and I helped her with some of the set up of Open Pose on the Xavier. I think that we will be in a good place for the demo on Wednesday and I’m excited to see everything start to come together a bit more. I feel like we are currently working at a good pace, and hopefully things will go smoothly next week.
Christina’s Weekly Status Report for April 10
This week, I worked on reworking the matching algorithm to accommodate the use of the Deep_Fashion_Try-on ML model which trains on a set of OpenPose json responses to match key points on uploaded images of clothing to a user image. This needs to be integrated with the model that Devon is currently training, so we hope to accomplish this before the demo. This should give us exactly the try-on effect we desire. The result will be as follows:
At this point, we will only allow users to upload pre-background-subtracted images.
Construction of the mirror is going at a really good pace, the hardware subsystem itself should all be complete.
References:
Team Status Report Week of April 10
This week we made some exciting strides as a team. We were able to successfully deploy OpenPose on the Xavier and start running tests. As expected, we had some initial hiccups with the deployment but we sorted through them. Unfortunately, our camera strip broke and we had to order a new one so we will continue testing for our demo when it comes in. Our TV stand was delivered and we built it and attached the monitor. It was a perfect height and we were very happy with the way the stand looks and functions. We also laser cut the mirror but we will be holding off on glueing it to the monitor until we have everything running smoothly. We also began working with the ACGPN model which solves our issues of clothing recognition and image warping. Our updated Gantt chart for the remainder of the class is attached below.
Judy’s Weekly Status Report for April 10
This week we were mainly preparing for the demo. Devon and I assembled the TV stand and place the TV monitor on the stand. We place it vertically so that the tv could show more of the torso. In addition to this, I wrote a script that allowed the camera to take images of the body, then send the pictures directly to openpose to analyze. However, the script that I wrote took over 90 seconds to take picture and analyze in OpenPose. Marios gave us the recommendation to use the GPU version of OpenPose so that it will run faster. I downloaded the GPU version and Openpose did indeed work a lot faster. However, while testing the script, the plastic that connected the camera to the Jetson smoked and burned a little, so the camera is not working anymore. I ordered more of the plastic strips so they are coming on Sunday so we can test before the demo next week. In addition to this, I brought the mirror to Ansys and got the mirror laser cut to fit the screen. We are going to try and glue the mirror to the tv. We left a one inch border around the tv so we could potentially cut a hole for the camera to peak through.
Christina’s Status Report for April 3
This week I have been working on the matching algorithm. I’ll be able to get the fixed point output json from the two images (clothing and user) and use the distance formula to figure out the relative distance. By using arctan function tan^(−1)(𝑦/𝑥)=𝜃, I can isolate the angle the line the two points create relative to the horizontal axis. With this information, I’ll be able to figure out where to place the image over the user by aligning the shoulder points, which will give a pretty accurate centering for the clothing, even though it doesn’t look perfect with superimposition.
On the hardware side, I worked with Judy and Devon to get OpenPose set up on the Jetson. On my end, we’ll need to figure out how to hook up the camera to get a video feed and get the subsequent fixed points in the format I expect for the matching module.
Team Status Report for Week of April 3
This week we worked on configuring openpose onto the Jetson. We all went over to Judy’s house to work on the mirror in person. We decided on how we want to actually display the mirror, so we ordered a tv mount. We want to glue the mirror onto the screen then put the screen vertically on the tv mount with the keyboard and mouse on the bottom. In addition to this, we got openpose installed onto the Jetson, however we ran into issues when running openpose. After talking to Marios, we realized that the issue was with the display openpose was using but openpose still works. We tested openpose on the sample images as well as on images of shirts and both work! We have also been researching how we want to cut out the background of images of shirts because we just want the shirt and not the person or the background. And lastly we have been working on the matching algorithm and the user interface.
Judy Status Report for Week of April 2
This week, we worked on getting our mirror demo ready. One of the biggest challenges we ran into was downloading openpose onto the Jetson Xavier NX. There wasn’t any documentation on how to do it on the openpose website, but we found a blog that detailed the steps to download. However, after we downloaded it, we would run openpose and we would just see a gray screen. I redownloaded openpose three times on the Jetson and we encountered the same issue each time. After meeting with Marios, we realized that the gray screen meant that the display wasn’t working, but openpose itself was working. Then, I worked on connecting openpose to the arducam, but I am also running into issues because openpose isn’t recognizing the camera. When I run it will say that the camera was detached. I am thinking that a work-around could be to record the body, then download that recording and send it to openpose to analyze. However, this may take a lot more time and we want to keep it to a 3.5 second latency so I may have to do more research on connecting the arducam to openpose.
Judy’s Status Report for Week of March 27th
Last week I worked on writing the design report. We divided up the report according to how we divided the proposal and design presentations. I worked on the project management aspects as well as the user interface of the presentations, so I wrote those sections of the proposal. I wrote the schedule, division of labor, introduction, budget, user interface, and IR frame design trade-off sections. After we each wrote our individual sections, we met together to combine everything and format it all correctly. In addition to this, this week we started to receive the parts for our mirror. We have already received the Jetson Xavier NX, Arducam, and monitor, and we are still waiting on the mirror.
Our parts came in this week so I worked on assembling the mirror. When I opened all of the parts, I realized there were many parts still missing. The Jetson NX needed a MicroSD card. In addition to this, we realized that we needed an HDMI cord and a keyboard and mouse to connect to the Jetson. We were also deciding how we wanted to mount the mirror to the tv, and we ultimately decided to buy a picture frame and use the mirror as the glass of the picture frame to attach to the tv. In addition to this, we had to do some research on how to laser cut the glass.
Christina’s Status Report for March 27
Last week, I worked on the Design Requirements, Software Architecture, Software Trade Studies, Software System Description, and Risk Management sections of the Design Report. I also created a more generalized block diagram of our entire system for the overall system structure to add to our Design Report as well. This attached below.
These past week has largely been dedicated to figuring out and learning how OpenPose works as well as playing with the configurations on my laptop before deploying to the Jetson. Unlike we previously thought, we probably won’t need to subtract functionality from the 17 fixed point setting since this already doesn’t include hands. By Marios’ suggestion of using GPU mode instead of CPU-ONLY mode in the cmake configuration of OpenPose, it runs much faster on my laptop now. The image below shows that OpenPose works with baggy clothing. I am also working on figuring out a background subtraction algorithm that we can use to isolate the clothing image.
Block Diagram:
OpenPose Testing:
Judy’s Status Report for March 20
This week I worked on writing the design report. We divided up the report according to how we divided the proposal and design presentations. I worked on the project management aspects as well as the user interface of the presentations, so I wrote those sections of the proposal. I wrote the schedule, division of labor, introduction, budget, user interface, and IR frame design trade-off sections. After we each wrote our individual sections, we met together to combine everything and format it all correctly. In addition to this, this week we started to receive the parts for our mirror. We have already received the Jetson Xavier NX, Arducam, and monitor, and we are still waiting on the mirror. Next week, we will begin assembling the mirror.